Riak Bitcask merging

Joseph Blomstedt joe at basho.com
Mon Aug 22 18:04:10 EDT 2011


> And while I'm pining for additional command line tools, any idea if
> transfers will ever work without disrupting the actual transfers?
> It's sort of annoying that it's listed as one of the steps for a
> rolling upgrade/restart but if you actually use it, it can cause
> upgrade or startup to take longer.  Also, it tends to timeout on
> a highly trafficked cluster.

Not guaranteeing anything, but there is a very high chance that
'riak-admin transfers' will not delay handoff in the next release of
Riak.

-Joe


> On Mon, Aug 22, 2011 at 11:18 AM, Anthony Molinaro
> <anthonym at alumni.caltech.edu> wrote:
>> While I didn't ask this time, I'll explain why I think manual
>> merging as an option would be great.
>>
>> As far as I know specifying a merge window doesn't guarantee the
>> merging happens, only that it might if other thresholds are met.
>>
>> With our cassandra cluster we've ended up scheduling twice weekly
>> full compactions (the close equivalent to merging I believe), via
>> cron.  The days and times are specificaly chosen based on traffic
>> patterns, and can be changed without restarting the servers.
>>
>> We don't have this convenience with riak.  We can set it up to only
>> merge during a window, but can't guarantee everything was merged
>> or even that any merges will occur.  If I wanted to change when I
>> do the merging, I have to restart the servers to pick up the new
>> config (at least I think I would).  I have no way to stagger the
>> merging (have different nodes merge at different times), unless
>> I have slightly different config on each node.
>>
>> If there were a riak-admin command called merge/compact/cleanup/expire
>> or something which triggered a manual merge I know we would use it.
>>
>> And while I'm pining for additional command line tools, any idea if
>> transfers will ever work without disrupting the actual transfers?
>> It's sort of annoying that it's listed as one of the steps for a
>> rolling upgrade/restart but if you actually use it, it can cause
>> upgrade or startup to take longer.  Also, it tends to timeout on
>> a highly trafficked cluster.
>>
>> Anyway, sorry about hijacking someone else's question, but
>> figured more information from users is usually welcome?
>>
>> -Anthony
>>
>> On Mon, Aug 22, 2011 at 09:14:06AM -0700, Dan Reverri wrote:
>>> There is no way to manually trigger a Bticask merge. What's the use case for
>>> needing to manually trigger the merge? Are you concerned about the size of
>>> the data files? Are you trying to avoid merging at a particular time?
>>>
>>> Not sure if this will help but you can restrict Bitcask merging to a
>>> specified window of time:
>>> http://wiki.basho.com/Bitcask-Configuration.html#Merge-Window
>>>
>>> Thanks,
>>> Dan
>>>
>>> Daniel Reverri
>>> Developer Advocate
>>> Basho Technologies, Inc.
>>> dan at basho.com
>>>
>>>
>>> On Sun, Aug 21, 2011 at 11:36 PM, raghwani sohil <sohil4you at gmail.com>wrote:
>>>
>>> >
>>> > Hi All,
>>> >
>>> > Is there any way to run  bitcask merging process manually ?
>>> >
>>> > thanks ,
>>> > Sohil Raghwani .
>>> >
>>> >
>>> > _______________________________________________
>>> > riak-users mailing list
>>> > riak-users at lists.basho.com
>>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>> >
>>> >
>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> --
>> ------------------------------------------------------------------------
>> Anthony Molinaro                           <anthonym at alumni.caltech.edu>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>




More information about the riak-users mailing list