Secondary indexes or Riak search ?

Alex Feng sweden.feng at gmail.com
Mon Feb 13 08:37:39 EST 2017


Hi Russell,

We will look into the links, thank you for the great help.

Br,
Aelx

2017-02-13 20:05 GMT+08:00 Russell Brown <russell.brown at icloud.com>:

> Hi Alex,
> A quick look at riak_test it seems that the `return_body` parameter is
> generally supported for both HTTP and PB now. I wasn’t aware of that.
>
> This test exercises the API.
>
> https://github.com/basho/riak_test/blob/develop/tests/
> verify_2i_returnbody.erl
>
> The `ref` returned from the link I gave you is a way to correlate messages
> received with the query. If you have a look at the code in the test here
>
> https://github.com/basho/riak_test/blob/develop/tests/
> secondary_index_tests.erl#L204
>
> You can get an idea of the general pattern. I guess line 227
>
> https://github.com/basho/riak_test/blob/develop/tests/
> secondary_index_tests.erl#L227
>
> Is the pertinent one for you.
>
> Cheers
>
> Russell
>
> On 13 Feb 2017, at 11:08, Alex Feng <sweden.feng at gmail.com> wrote:
>
> > Hi Russell,
> >
> > We have tried this, but it returns a reference(some number) only.  We
> have googled and haven't found any clue about how to use this API.
> > The API user guide doesn't say much about the "reference".  What is the
> next step with the returned reference ?
> >
> >
> > cs_bucket_fold(Pid::pid(), Bucket::bucket() | bucket_and_type(),
> Opts::cs_opts()) -> {ok, reference()} | {error, term()}
> > secret function, do not use, or I come to your house and keeel you.
> >
> > cs_opt() = {timeout, timeout()} | {continuation, binary()} |
> {max_results, non_neg_integer() | all} | {start_key, binary()} |
> {start_incl, boolean()} | {end_key, binary()} | {end_incl, boolean()}
> >
> >
> > Br,
> > Alex
> >
> > 2017-02-13 16:34 GMT+08:00 Russell Brown <russell.brown at icloud.com>:
> > If you look at the riak-erlang-client here
> https://github.com/basho/riak-erlang-client/blob/develop/
> src/riakc_pb_socket.erl#L1129 there is a client API call that was
> implemented for CS (riak s2, or whatever it is called, the large object
> store thingy that basho had) that will use the $keys index, but in the back
> end calls fold_objects rather than fold_keys on eleveldb, and so returns
> the whole object. NOTE: this is equivalent to r=1, you don’t get a quorum
> read here, but a single vnode per-value only. I don’t know if other clients
> added this API, but it would not be hard to add to any client that supports
> 2i. Like I say, originally it was for riakCS, but it’s open source and part
> of the release for 4 years now, so hardly a secret.
> >
> > Cheers
> >
> > Russell
> >
> > On 13 Feb 2017, at 06:18, Alex Feng <sweden.feng at gmail.com> wrote:
> >
> > > Hi Russell,
> > >
> > > In your reply, you mentioned this,
> > >
> > > >> There is also the feature that can return the actual riak objects
> for a $keys index search,
> > > >>You can pack the index terms with data and return the terms in a
> query so that you don’t need a further object fetch (see >>return_terms in
> docs.)
> > >
> > > If I understood correctly, it is possible to fetch object (value) by
> 2i in one time. But, we have tried using "return_term = true", it returns
> the index with key comparing only key when not using "return_term=true".
> It doesn't help much with extra index, what we want to achieve is to fetch
> the object in one time.
> > >
> > >
> > > Our use case,  client search DB every 10 seconds by 2i, Riak will
> return a list of around 5000 results(Keys), then client will query DB to
> fetch value for each key, basically it is around 5000 times,  client is
> easy to run into some issues most of the time. Any suggestion here ?
> > >
> > > Many thanks in advance.
> > >
> > > Br,
> > > Alex
> > >
> > > 2017-02-06 19:02 GMT+08:00 Alex Feng <sweden.feng at gmail.com>:
> > > Hi Russell,
> > >
> > > It is really helpful, thank you a lot.
> > > We are suffering from solr crash now, are considering to switch to 2i.
> > >
> > > Br,
> > > Alex
> > >
> > > 2017-02-06 16:53 GMT+08:00 Russell Brown <russell.brown at icloud.com>:
> > > It’s worth noting that secondary indexes (2i) has some other
> advantages over solr search. If you _can_ model your queries in 2i then I'd
> recommend it.
> > >
> > > Secondary indexes  have a richer API than is currently documented, if
> you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see
> that documents a feature that allows the index terms to be filtered via reg
> ex. There is also the feature that can return the actual riak objects for a
> $keys index search,
> > > You can pack the index terms with data and return the terms in a query
> so that you don’t need a further object fetch (see return_terms in docs.)
> > > Secondary indexes are written atomically with the object they index.
> > > Operationally they don’t require you run a JVM and Solr alongside your
> riak nodes.
> > >
> > > You have the tools with basho_bench to answer the question about
> performance and overhead for your workload. I suspect for “overhead” 2i
> wins, as there is no JVM-per-node.
> > >
> > > Modelling for 2i is perhaps harder, in the classical nosql way, you
> have to do more work upfront when designing your querying.
> > >
> > > I hope that helps a little. I worked quite a lot on 2i and never
> really understood why riak-search was seen as a replacment, imo they’re
> complementary, and you pick the one that best fits.
> > >
> > > Cheers
> > >
> > > Russell
> > >
> > > On 2 Feb 2017, at 09:43, Alex Feng <sweden.feng at gmail.com> wrote:
> > >
> > > > Hello Riak-users,
> > > >
> > > > I am currently using Riak search to do some queries, since my
> queries are very simple, it should be fulfilled by secondary indexes as
> well.
> > > > So, my question is which one has better performance and less
> overhead, let's say both can fulfill the query requirement.
> > > >
> > > > Many thanks in advance.
> > > >
> > > > Br,
> > > > Alex
> > > > _______________________________________________
> > > > riak-users mailing list
> > > > riak-users at lists.basho.com
> > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
> > >
> > >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20170213/cfb66bb2/attachment-0002.html>


More information about the riak-users mailing list