Large numbers of keys

Eric Anderson anderson at copperegg.com
Wed Jun 27 10:19:04 EDT 2012


Hi all,

I did a quick computation, and figured that I need to be able to manage a minimum 5 billion keys roughly 80bytes each.  Since Bitcask needs all keys in memory, that's a lot of memory. How does it handle swapping?  I typically only need a portion of those keys available at a given time, so swapping them out would be fine if it was intelligent about that and loading it back in.  Meaning that if it has to pull in the whole index to add a new key, that would be an issue for me.  I typically need to add new keys pretty quickly and then access the most recent ones.  The older ones could swap out without too much worry.

Would LevelDB help with this?

Thanks!

Eric Anderson
Co-Founder
http://CopperEgg.com


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120627/5c48df2e/attachment.html>


More information about the riak-users mailing list