Secondary indexes or Riak search ?

Russell Brown russell.brown at icloud.com
Mon Feb 13 07:05:16 EST 2017


Hi Alex,
A quick look at riak_test it seems that the `return_body` parameter is generally supported for both HTTP and PB now. I wasn’t aware of that.

This test exercises the API.

https://github.com/basho/riak_test/blob/develop/tests/verify_2i_returnbody.erl

The `ref` returned from the link I gave you is a way to correlate messages received with the query. If you have a look at the code in the test here 

https://github.com/basho/riak_test/blob/develop/tests/secondary_index_tests.erl#L204

You can get an idea of the general pattern. I guess line 227

https://github.com/basho/riak_test/blob/develop/tests/secondary_index_tests.erl#L227

Is the pertinent one for you.

Cheers

Russell

On 13 Feb 2017, at 11:08, Alex Feng <sweden.feng at gmail.com> wrote:

> Hi Russell,
> 
> We have tried this, but it returns a reference(some number) only.  We have googled and haven't found any clue about how to use this API.
> The API user guide doesn't say much about the "reference".  What is the next step with the returned reference ?
> 
> 
> cs_bucket_fold(Pid::pid(), Bucket::bucket() | bucket_and_type(), Opts::cs_opts()) -> {ok, reference()} | {error, term()}
> secret function, do not use, or I come to your house and keeel you.
> 
> cs_opt() = {timeout, timeout()} | {continuation, binary()} | {max_results, non_neg_integer() | all} | {start_key, binary()} | {start_incl, boolean()} | {end_key, binary()} | {end_incl, boolean()}
> 
> 
> Br,
> Alex
> 
> 2017-02-13 16:34 GMT+08:00 Russell Brown <russell.brown at icloud.com>:
> If you look at the riak-erlang-client here https://github.com/basho/riak-erlang-client/blob/develop/src/riakc_pb_socket.erl#L1129 there is a client API call that was implemented for CS (riak s2, or whatever it is called, the large object store thingy that basho had) that will use the $keys index, but in the back end calls fold_objects rather than fold_keys on eleveldb, and so returns the whole object. NOTE: this is equivalent to r=1, you don’t get a quorum read here, but a single vnode per-value only. I don’t know if other clients added this API, but it would not be hard to add to any client that supports 2i. Like I say, originally it was for riakCS, but it’s open source and part of the release for 4 years now, so hardly a secret.
> 
> Cheers
> 
> Russell
> 
> On 13 Feb 2017, at 06:18, Alex Feng <sweden.feng at gmail.com> wrote:
> 
> > Hi Russell,
> >
> > In your reply, you mentioned this,
> >
> > >> There is also the feature that can return the actual riak objects for a $keys index search,
> > >>You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see >>return_terms in docs.)
> >
> > If I understood correctly, it is possible to fetch object (value) by 2i in one time. But, we have tried using "return_term = true", it returns the index with key comparing only key when not using "return_term=true".  It doesn't help much with extra index, what we want to achieve is to fetch the object in one time.
> >
> >
> > Our use case,  client search DB every 10 seconds by 2i, Riak will return a list of around 5000 results(Keys), then client will query DB to fetch value for each key, basically it is around 5000 times,  client is easy to run into some issues most of the time. Any suggestion here ?
> >
> > Many thanks in advance.
> >
> > Br,
> > Alex
> >
> > 2017-02-06 19:02 GMT+08:00 Alex Feng <sweden.feng at gmail.com>:
> > Hi Russell,
> >
> > It is really helpful, thank you a lot.
> > We are suffering from solr crash now, are considering to switch to 2i.
> >
> > Br,
> > Alex
> >
> > 2017-02-06 16:53 GMT+08:00 Russell Brown <russell.brown at icloud.com>:
> > It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.
> >
> > Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
> > You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
> > Secondary indexes are written atomically with the object they index.
> > Operationally they don’t require you run a JVM and Solr alongside your riak nodes.
> >
> > You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.
> >
> > Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.
> >
> > I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.
> >
> > Cheers
> >
> > Russell
> >
> > On 2 Feb 2017, at 09:43, Alex Feng <sweden.feng at gmail.com> wrote:
> >
> > > Hello Riak-users,
> > >
> > > I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> > > So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
> > >
> > > Many thanks in advance.
> > >
> > > Br,
> > > Alex
> > > _______________________________________________
> > > riak-users mailing list
> > > riak-users at lists.basho.com
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
> >
> 
> 





More information about the riak-users mailing list