Using $bucket index for listing keys

Oleksiy Krivoshey oleksiyk at gmail.com
Fri Mar 11 08:46:58 EST 2016


I got the recursive behavior with other, larger buckets but I had no
logging so when I enabled debugging this was the first bucket to replicate
the problem. I have a lot of buckets of the same type, some have many
thousands keys some are small. My task is to iterate the keys (once only)
of all buckets. Either with 2i or with Yokozuna.
On Fri, Mar 11, 2016 at 15:32 Russell Brown <russell.brown at me.com> wrote:

> Not the answer, by why pagination for 200 keys? Why the cost of doing the
> query 20 times vs once?
>
> On 11 Mar 2016, at 13:28, Oleksiy Krivoshey <oleksiyk at gmail.com> wrote:
>
> > Unfortunately there are just 200 keys in that bucket. So with larger
> max_results I just get all the keys without continuation. I'll try to
> replicate this with a bigger bucket.
> > On Fri, Mar 11, 2016 at 15:21 Russell Brown <russell.brown at me.com>
> wrote:
> > That seems very wrong. Can you do me a favour and try with a larger
> max_results. I remember a bug with small results set, I thought it was
> fixed, I’m looking into the past issues, but can you try “max_results=1000”
> or something, and let me know what you see?
> >
> > On 11 Mar 2016, at 13:03, Oleksiy Krivoshey <oleksiyk at gmail.com> wrote:
> >
> > > Here it is without the `value` part of request:
> > >
> > > curl '
> http://127.0.0.1:8098/types/fs_chunks/buckets/0r0e5wahrhsgpolk9stbnrqmp77fjjye.chunks/index/$bucket/_?max_results=10&continuation=g20AAAAja1AzdzJwOXpYcVoyb0F4NDhTMVNnRUpBbGJ0ZkhVdkk6MjU=
> '
> > >
> > >
> {"keys":["4rpG2PwRTs3YqasGGYrhACBvZqTg7mQW:0","4rpG2PwRTs3YqasGGYrhACBvZqTg7mQW:2","FSEky50kr2TLkBuo1JKv6sphINYwnJfV:1","F3KcwtjG9VAtM5u8vbwBuCjuGBrPTnfq:0","RToMNlsnVKvXcawQK6BGnCAKx58pC9xX:1","UMiHx4qDR5pHWT9OgLAu1KMlFeEKbISm:0","F3KcwtjG9VAtM5u8vbwBuCjuGBrPTnfq:2","YQlRWkJPFYiLlAwhvgqOysJC3ycmQ9OA:0","kP3w2p9zXqZ2oAx48S1SgEJAlbtfHUvI:15","kP3w2p9zXqZ2oAx48S1SgEJAlbtfHUvI:25"],"continuation":"g20AAAAja1AzdzJwOXpYcVoyb0F4NDhTMVNnRUpBbGJ0ZkhVdkk6MjU="}
> > >
> > > On 11 March 2016 at 14:58, Oleksiy Krivoshey <oleksiyk at gmail.com>
> wrote:
> > > I'm actually using PB interface, but I can replicate the problem with
> HTTP as in my previous email. Request with '&continuation=AAAA' returns the
> result set with the same continuation AAAA.
> > >
> > > On 11 March 2016 at 14:55, Magnus Kessler <mkessler at basho.com> wrote:
> > > Hi Oleksiy,
> > >
> > > How are you performing your 2i-based key listing? Querying with
> pagination as shown in the documentation[0] should work.
> > >
> > > As an example here is the HTTP invocation:
> > >
> > > curl "
> https://localhost:8098/types/default/buckets/test/index/\$bucket/_?max_results=10&continuation=g20AAAACNTM=
> "
> > >
> > > Once the end of the key list is reached, the server returns an empty
> keys list and no further continuation value.
> > >
> > > Please let me know if this works for you.
> > >
> > > Kind Regards,
> > >
> > > Magnus
> > >
> > >
> > > [0]: http://docs.basho.com/riak/latest/dev/using/2i/#Querying
> > >
> > > On 11 March 2016 at 10:06, Oleksiy Krivoshey <oleksiyk at gmail.com>
> wrote:
> > > Anyone?
> > >
> > > On 4 March 2016 at 19:11, Oleksiy Krivoshey <oleksiyk at gmail.com>
> wrote:
> > > I have a bucket with ~200 keys in it and I wanted to iterate them with
> the help of $bucket index and 2i request, however I'm facing the recursive
> behaviour, for example I send the following 2i request:
> > >
> > > {
> > > bucket: 'BUCKET_NAME',
> > > type: 'BUCKET_TYPE',
> > > index: '$bucket',
> > > key: 'BUCKET_NAME',
> > > qtype: 0,
> > > max_results: 10,
> > > continuation: 'AAAA'
> > > }
> > >
> > > I receive 10 keys and continuation 'BBBB', I then repeat the request
> with continuation 'BBBB' and at this point I can receive a reply with
> continuation 'CCCC' or 'AAAA' or even 'BBBB' and its going in never ending
> recursion.
> > >
> > > I'm running this on a 5 node 2.1.3 cluster.
> > >
> > > What I'm doing wrong? Or is this not supported at all?
> > >
> > > Thanks!
> > >
> > >
> > > _______________________________________________
> > > riak-users mailing list
> > > riak-users at lists.basho.com
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
> > >
> > >
> > >
> > > --
> > > Magnus Kessler
> > > Client Services Engineer
> > > Basho Technologies Limited
> > >
> > > Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
> > >
> > >
> > > _______________________________________________
> > > riak-users mailing list
> > > riak-users at lists.basho.com
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160311/2d2e63e9/attachment-0002.html>


More information about the riak-users mailing list