Speed of linkwalking

Jonathan Langevin jlangevin at loomlearning.com
Tue Aug 30 15:31:40 EDT 2011


Map/reduce is known to be slower than a standard fetch, so what you're
experiencing is likely a direct result of m/r overhead.

I would be interested in seeing if your test case shows a notable speed
difference against the current official release of Riak (0.14.2 I believe).
Based on what I've read of the in-development changes, performance should be
worse in the 0.14.2 release (assuming that the new m/r functionality based
on Riak Pipe is merged into whichever branch you're using). I may be worth
testing just to make sure you're not seeing worse performance (which could
suggest a new bug has been introduced).
*

 <http://www.loomlearning.com/>
 Jonathan Langevin
Systems Administrator
Loom Inc.
Wilmington, NC: (910) 241-0433 - jlangevin at loomlearning.com -
www.loomlearning.com - Skype: intel352
*


On Tue, Aug 30, 2011 at 3:06 PM, Mark Steele <msteele at beringmedia.com>wrote:

> I'm actually testing with an object that has only one link as that's my
> use-case.
>
> In testing, 2 separate gets is way faster than using a map-reduce
> link-walk, which is disappointing :(
>
> I'm also testing on a one node cluster, and my bucket has a N/R/W=1. I was
> thinking that the reduction in network hops would outweigh the map reduce
> overhead, but apparently that's not the case.
>
> Simple get (one value) : 0.000741
> Multiget linkwalk (get one object, then link walk in php) : 0.001567
> Linkwalk (map reduce): 0.041994
>
> The map phase does not significantly change anything as far as performance.
>
> In case it matters, I'm testing against the latest riak code on github.
>
> Cheers,
>
> Mark Steele
> Bering Media Inc.
>
>  On Tue, Aug 30, 2011 at 3:00 PM, Jonathan Langevin <
> jlangevin at loomlearning.com> wrote:
>
>> Good catch Kev.
>>
>> Mark, If you run the same operation with map removed, what is the
>> performance at that point?*
>>
>>  <http://www.loomlearning.com/>
>>  Jonathan Langevin
>> Systems Administrator
>> Loom Inc.
>> Wilmington, NC: (910) 241-0433 - jlangevin at loomlearning.com -
>> www.loomlearning.com - Skype: intel352
>> *
>>
>>
>>
>> On Tue, Aug 30, 2011 at 2:45 PM, Kev Burns <kevburnsjr at gmail.com> wrote:
>>
>>> Mark,
>>>
>>> That's not just a link walk, you're also performing a map operation
>>> there.
>>>
>>> $client->add($bucketname, 'linkkey1')->
>>>     link()->
>>>     map(array("riak_kv_mapreduce", "map_object_value")) ->
>>>     run();
>>>
>>>
>>> If the expected number of returned objects is small, performing the map
>>> phase in PHP may be faster.
>>>
>>> - Kev
>>> c: +001 (650) 521-7791
>>>
>>>
>>> On Tue, Aug 30, 2011 at 7:31 AM, Mark Steele <msteele at beringmedia.com>wrote:
>>>
>>>> Hi folks,
>>>>
>>>> Just want to know if I'm doing something obviously dumb here.
>>>>
>>>> First the (PHP) code (Sorry of the API is different from the official
>>>> API, we're using a heavily modified version):
>>>>
>>>> <snip>
>>>> $obj1 = $bucket->newObject('linkkey1', array('link1'));
>>>> $obj2 = $bucket->newObject('linkkey2', 'dataforlinkkey2');
>>>> $obj1->addLink($obj2);
>>>> $obj2->addLink($obj1);
>>>> $obj1->store();
>>>> $obj2->store();
>>>>
>>>> $start = microtime(true);
>>>> $blargh = $bucket->get('linkkey2');
>>>> $end = microtime(true);
>>>> printf("Took : %04f\n",$end - $start);
>>>>
>>>> $start = microtime(true);
>>>>  $result =
>>>>     $client->add($bucketname, 'linkkey1')->
>>>>     link()->
>>>>     map(array("riak_kv_mapreduce", "map_object_value")) ->
>>>>     run();
>>>>   foreach ($result as $data) {
>>>>     //var_dump($data);
>>>>   }
>>>> $end = microtime(true);
>>>> printf("Took : %04f\n",$end - $start);
>>>> <snip>
>>>>
>>>> So here's what I'm seeing:
>>>>
>>>> The simple key fetch takes 0.000661 seconds to execute, whereas the
>>>> link-walk takes 0.042043. Ouch. Quite a bit slower. Any ways to speed
>>>> this up?
>>>>
>>>> Cheers,
>>>>
>>>> Mark Steele
>>>> Bering Media Inc.
>>>>
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110830/70f256d6/attachment.html>


More information about the riak-users mailing list