2i at large scale?

Wagner Camarao wagner at crunchbase.com
Tue Oct 1 18:52:46 EDT 2013


Hey Reid,

Just wanted to say thanks for your last thoughts on my 2i thread. I am
sorry I didn't reply on a good timing, I just realized that now.

Best
~W


On Wed, Sep 25, 2013 at 3:44 PM, Reid Draper <reiddraper at gmail.com> wrote:

>
> On Sep 25, 2013, at 4:35 PM, Wagner Camarao <wagner at crunchbase.com> wrote:
>
> What size page are you using (ie. how many results per query)?
>
> A: 10
>
>
> If I understood correctly, you're paginating through a few million results
> total. If so, I'd try setting your page size much larger: try 1000.
>
>
>
> When you said the results would sometimes take three minutes, is that per
> page, or to paginate through all of the results?
>
> A: I don't observe any difference by paging the first or any next page.
> That wait time is only when I start riak and run the first request
> (whatever that request is, paging, key lookup, or just getting a reference
> to a bucket).
>
>
> So performance levels out after the first couple of requests? If so, this
> might be explained by the leveldb cache being built up.
>
>
>
> Are you observing CPU and memory utilization while these stalls happen?
>
> A: Yes, I see both CPU and memory but nothing above 50% of use. If I do
> bucket.keys (yes, not recommended) then it's the only way I get my CPU
> usage close to 100%. I don't do that as part of my benchmarking. I did it a
> couple times just to see it happening, and actually none of these times
> took riak down.
>
>
> Do you have swap disabled?
>
> A: Nope. Why you would recommend doing so? Could there be any downsides?
>
>
> You can read about our rationale for this here [1].
>
> [1] http://docs.basho.com/riak/latest/ops/tuning/linux/
>
>
>
> On Wed, Sep 25, 2013 at 2:18 PM, Reid Draper <reiddraper at gmail.com> wrote:
>
>>
>> On Sep 25, 2013, at 3:39 PM, Wagner Camarao <wagner at crunchbase.com>
>> wrote:
>>
>>  Reid,
>> Thanks for bringing that up and yes I'm using paginated 2i.
>>
>>
>> * What size page are you using (ie. how many results per query)?
>> * When you said the results would sometimes take three minutes, is that
>> per page, or to paginate through all of the results?
>> * Are you observing CPU and memory utilization while these stalls happen?
>> * Do you have swap disabled?
>>
>> Reid
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20131001/102ff8a2/attachment.html>


More information about the riak-users mailing list