multi-get (yet again)

Jeremy Dunck jdunck at gmail.com
Thu Aug 9 01:11:41 EDT 2012


I'm new to riak and need multi-get (that is, getting the value and/or
existence of keys in a single network-trip latency).

I was wondering what the latency of the map-reduce approach is?
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-February/003229.html

Alternatively, has anyone tried scaling concurrent gets (perhaps with
evented io) to do many concurrent requests and combining results on
the client?

I am toying with a python+gevent multiget function.  If the stance is
still that a multiget operation doesn't belong in core, I'm a little
surprised that there doesn't seem to at least be a nice client-lib API
func to do it.  It sure seems useful...

In my use-case, the immediate need is to know whether a db insert
needs to be done.  We're handling too many keys to want to store in
memory (so no redis, etc), and we don't want to go to the db more than
we need to, so it seems riak would be good here.  But we're getting
1000s of potential insert keys and want to whittle down all those to a
relative few db inserts.

So I was thinking riak key-per-id, and insert to the db iff the riak
key doesn't exist, then add the riak key.  We'll get some race
conditions on the insert, but that's OK in our case.

We do need low latency on the riak check, though, hence either
multiplexing w/ eventing or map-reduce (if that latency is actually
good).

Am I doing it wrong?




More information about the riak-users mailing list