Extracting data during link walking using PBC

Russell Brown russell.brown at me.com
Mon May 2 03:34:59 EDT 2011


Hi OJ,

On 2 May 2011, at 00:29, OJ Reeves wrote:

> Hi Russell,
> 
> Thanks for the response. Pardon my tardiness in responding, I've had lots of fun at home with sick kids of late.

I heard, thinking of you and your family. Fingers crossed for a speedy recovery.

> 
>> So my question: Is this how link walking should work when using the PBC? Or is there some way of getting the PBC to return the content at the same time as doing the link walking?
> 
> I asked yesterday and the answer I received was: no.
> 
> Cool, so I'm not barking mad then! Excellent :) Did you get any feedback as to whether or not there would be future work done to enable this?

Heh, I didn't ask. I've now opened a ticket (https://issues.basho.com/show_bug.cgi?id=1089) in reference to this.

>    
>> Currently I link walk and do each step manually and I was wondering if there was a better or more optimal way of performing these actions.
>> 
> 
> At the moment the plan is running an initial m/r job for the link phases then making a second m/r job with all the intermediate results as inputs. So maximum 2 jobs is my target.
> 
> 
> Off the top of my head with the current implementation I can't see how you would do this any better. In fact, for link walks where the levels > 2 this is really the only option (IMHO). For 2 levels or less you can easily fetch based on ID and you end up with the data faster with the same number of calls.
> 
> I will think about it a bit more and see what I can come up with (as soon as this sickness clears!).

I implemented link walking via m/r on the PB Java client last week and it was pretty simple. 2 m/r queries: 

    1. run link phases to gather inputs
    2. reduce to de-dupe and map to get values

I added a key arg so that the "link step" number was attached to the returned objects so that I could recreate a REST link walk style result.

>  
> How do you satisfy your requirement with the REST link walker? Specifically how do you get the value of *a* returned as part of the result?
> 
> With the REST link walker I'm not able to do this, so I have to make another call to get to this info. It'd be nice if the link walking mechanism also had a flag that said "please give me the link source value too". I realise that might be pie in the sky though :)

http://issues.basho.com, there is no harm in asking ;)

> 
> Cheers mate, thanks for the response.
> OJ
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110502/2948eaa0/attachment.html>


More information about the riak-users mailing list