Schema Architecture, Map Reduce & Key Lists

Mat Ellis mat at tecnh.com
Thu Feb 10 12:35:08 EST 2011


We are converting a mysql based schema to Riak using Ripple. We're tracking a lot of clicks, and each click belongs to a cascade of other objects:

click -> placement -> campaign -> customer

i.e. we do a lot of operations on these clicks grouped by placement or sets of placements.

Reading this http://lists.basho.com/pipermail/riak-users_lists.basho.com/2010-July/001591.html gave me pause for thought. I was hoping the time needed to crunch each day's data would be proportional to the volume of clicks on that day but it seems that it would be proportional to the total number of clicks ever.

What's the best approach here? I can see a number of 'solutions' each of them complicated:

(1) Maintain an index of clicks by day so that we can focus our operations on a time bound set of clicks

(2) Delete or archive clicks once they have been processed or after a certain number of days

(3) Add many links to each placement, one per click (millions potentially)

On a related noob-note, what would be the best way of creating a set of the clicks for a given placement? Map Reduce or Riak Search or some other method?

Thanks in advance.

M.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110210/b33a16ab/attachment.html>


More information about the riak-users mailing list