Using $bucket index for listing keys

Magnus Kessler mkessler at basho.com
Fri Mar 11 08:53:59 EST 2016


Hi Oleksiy,

Could you please share the bucket or bucket-type properties for that small
bucket? If you open an issue on github, please add the properties there,
too.

Many Thanks,

On 11 March 2016 at 13:46, Oleksiy Krivoshey <oleksiyk at gmail.com> wrote:

> I got the recursive behavior with other, larger buckets but I had no
> logging so when I enabled debugging this was the first bucket to replicate
> the problem. I have a lot of buckets of the same type, some have many
> thousands keys some are small. My task is to iterate the keys (once only)
> of all buckets. Either with 2i or with Yokozuna.
> On Fri, Mar 11, 2016 at 15:32 Russell Brown <russell.brown at me.com> wrote:
>
>> Not the answer, by why pagination for 200 keys? Why the cost of doing the
>> query 20 times vs once?
>>
>> On 11 Mar 2016, at 13:28, Oleksiy Krivoshey <oleksiyk at gmail.com> wrote:
>>
>> > Unfortunately there are just 200 keys in that bucket. So with larger
>> max_results I just get all the keys without continuation. I'll try to
>> replicate this with a bigger bucket.
>> > On Fri, Mar 11, 2016 at 15:21 Russell Brown <russell.brown at me.com>
>> wrote:
>> > That seems very wrong. Can you do me a favour and try with a larger
>> max_results. I remember a bug with small results set, I thought it was
>> fixed, I’m looking into the past issues, but can you try “max_results=1000”
>> or something, and let me know what you see?
>> >
>> > On 11 Mar 2016, at 13:03, Oleksiy Krivoshey <oleksiyk at gmail.com> wrote:
>> >
>> > > Here it is without the `value` part of request:
>> > >
>> > > curl '
>> http://127.0.0.1:8098/types/fs_chunks/buckets/0r0e5wahrhsgpolk9stbnrqmp77fjjye.chunks/index/$bucket/_?max_results=10&continuation=g20AAAAja1AzdzJwOXpYcVoyb0F4NDhTMVNnRUpBbGJ0ZkhVdkk6MjU=
>> '
>> > >
>> > >
>> {"keys":["4rpG2PwRTs3YqasGGYrhACBvZqTg7mQW:0","4rpG2PwRTs3YqasGGYrhACBvZqTg7mQW:2","FSEky50kr2TLkBuo1JKv6sphINYwnJfV:1","F3KcwtjG9VAtM5u8vbwBuCjuGBrPTnfq:0","RToMNlsnVKvXcawQK6BGnCAKx58pC9xX:1","UMiHx4qDR5pHWT9OgLAu1KMlFeEKbISm:0","F3KcwtjG9VAtM5u8vbwBuCjuGBrPTnfq:2","YQlRWkJPFYiLlAwhvgqOysJC3ycmQ9OA:0","kP3w2p9zXqZ2oAx48S1SgEJAlbtfHUvI:15","kP3w2p9zXqZ2oAx48S1SgEJAlbtfHUvI:25"],"continuation":"g20AAAAja1AzdzJwOXpYcVoyb0F4NDhTMVNnRUpBbGJ0ZkhVdkk6MjU="}
>> > >
>> > > On 11 March 2016 at 14:58, Oleksiy Krivoshey <oleksiyk at gmail.com>
>> wrote:
>> > > I'm actually using PB interface, but I can replicate the problem with
>> HTTP as in my previous email. Request with '&continuation=AAAA' returns the
>> result set with the same continuation AAAA.
>> > >
>> > > On 11 March 2016 at 14:55, Magnus Kessler <mkessler at basho.com> wrote:
>> > > Hi Oleksiy,
>> > >
>> > > How are you performing your 2i-based key listing? Querying with
>> pagination as shown in the documentation[0] should work.
>> > >
>> > > As an example here is the HTTP invocation:
>> > >
>> > > curl "
>> https://localhost:8098/types/default/buckets/test/index/\$bucket/_?max_results=10&continuation=g20AAAACNTM=
>> "
>> > >
>> > > Once the end of the key list is reached, the server returns an empty
>> keys list and no further continuation value.
>> > >
>> > > Please let me know if this works for you.
>> > >
>> > > Kind Regards,
>> > >
>> > > Magnus
>> > >
>> > >
>> > > [0]: http://docs.basho.com/riak/latest/dev/using/2i/#Querying
>> > >
>> > > On 11 March 2016 at 10:06, Oleksiy Krivoshey <oleksiyk at gmail.com>
>> wrote:
>> > > Anyone?
>> > >
>> > > On 4 March 2016 at 19:11, Oleksiy Krivoshey <oleksiyk at gmail.com>
>> wrote:
>> > > I have a bucket with ~200 keys in it and I wanted to iterate them
>> with the help of $bucket index and 2i request, however I'm facing the
>> recursive behaviour, for example I send the following 2i request:
>> > >
>> > > {
>> > > bucket: 'BUCKET_NAME',
>> > > type: 'BUCKET_TYPE',
>> > > index: '$bucket',
>> > > key: 'BUCKET_NAME',
>> > > qtype: 0,
>> > > max_results: 10,
>> > > continuation: 'AAAA'
>> > > }
>> > >
>> > > I receive 10 keys and continuation 'BBBB', I then repeat the request
>> with continuation 'BBBB' and at this point I can receive a reply with
>> continuation 'CCCC' or 'AAAA' or even 'BBBB' and its going in never ending
>> recursion.
>> > >
>> > > I'm running this on a 5 node 2.1.3 cluster.
>> > >
>> > > What I'm doing wrong? Or is this not supported at all?
>> > >
>> > > Thanks!
>> > >
>> > >
>> > > _______________________________________________
>> > > riak-users mailing list
>> > > riak-users at lists.basho.com
>> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Magnus Kessler
>> > > Client Services Engineer
>> > > Basho Technologies Limited
>> > >
>> > > Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg
>> 07970431
>> > >
>> > >
>> > > _______________________________________________
>> > > riak-users mailing list
>> > > riak-users at lists.basho.com
>> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>>
>>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160311/360d332a/attachment-0002.html>


More information about the riak-users mailing list