Bitcask - large keydirs

David Smith dizzyd at basho.com
Fri Mar 11 17:40:13 EST 2011


TLDR: Yes, good idea. We'd welcome any patches/pull requests you have.... ;)

There are significant tradeoffs here, particularly as the accesses
devolve into a uniformly random pattern. The rebalancing and secondary
disk hits would also have some interesting impacts to the
predictability of our latency profile. Speaking solely for myself, I
think if we want to really solve the problem of keys in memory, a more
traditiona, append-onlyl b-tree based structure would be more
desirable, have similar tradeoffs and yield the ability to more
cheaply traverse ranges of keys. However, I would note that there are
a number of deployments where "all the keys in memory!!" was a big
deal in theory, but a reasonable tradeoff in reality. It also was less
of an issue when people started looking at the very boring latency
profiles. :)

_Should_ we do something about the issue? Yes, that would be very,
very nice. I'm just adding some color to the conversation. :)

D.




More information about the riak-users mailing list