Migration from memcachedb to riak

Howard Chu hyc at symas.com
Wed Jul 10 11:19:23 EDT 2013


>> On 10 July 2013 10:49, Edgar Veiga <edgarmveiga at gmail.com
>> <mailto:edgarmveiga at gmail.com>> wrote:
>>
>>      Hello all!
>>
>>      I have a couple of questions that I would like to address all of
>>      you guys, in order to start this migration the best as possible.
>>
>>      Context:
>>      - I'm responsible for the migration of a pure key/value store that
>>      for now is being stored on memcacheDB.
>>      - We're serializing php objects and storing them.
>>      - The total size occupied it's ~2TB.
>>
>>      - The idea it's to migrate this data to a riak cluster with
>>      elevelDB backend (starting with 6 nodes, 256 partitions. This
>>      thing is scaling very fast).
>>      - We only need to access the information by key. *We won't need
>>      neither map/reduces, searches or secondary indexes*. It's a pure
>>      key/value store!
>>
>>      My questions are:
>>      - Do you have any riak fine tunning tip regarding this use case
>>      (due to the fact that we will only use the key/value capabilities
>>      of riak)?

If you only need a pure key/value store, you should consider memcacheDB using 
LMDB as its backing store. It's far faster than memcacheDB using BerkeleyDB.
	http://symas.com/mdb/memcache/

I doubt LevelDB accessed through any interpreted language will be anywhere 
near its performance either, though I haven't tested. (Is there a LevelDB 
backend for modular memcache yet?)

Also if you're serializing language objects, you should consider using LMDB as 
an embedded data store. With the FIXEDMAP option you can copy objects to the 
store and then execute the objects directly from the store on future 
retrievals, no deserialization required.

>>      - It's expected that those 2TB would be reduced due to the levelDB
>>      compression. Do you think we should compress our objects to on the
>>      client?

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/




More information about the riak-users mailing list