build 393: adding server results in missing data

Jonathan Lee jonathan_lee at
Fri Nov 13 15:46:16 EST 2009

I also am seeing the same thing.  Sometimes, all of my objects
reappear after a few minutes, but usually only about 90% of my objects

In the handoff case, I imagine that a node that has freshly joined the
cluster should either be marked "offline" or pass on reads for keys it
doesn't have until handoff is complete, right?  Are the in-progress
improvements along these lines?


On Fri, Nov 13, 2009 at 2:28 PM, B. Todd Burruss <bburruss at> wrote:
> i let my instances run all night and the data still hasn't replicated
> completely.  my dataset is 1000 2k blocks of data.  it should be fairly
> quick i would assume.  but even if it isn't fast (say i had 100,000,000
> 2k blocks) the data should always be available during a handoff.  right?
> thx Justin for getting back to us
> On Fri, 2009-11-13 at 14:04 -0500, Justin Sheehy wrote:
>> My guess is that you're seeing the result of a large amount of handoff
>> in process.
>> When you add a new node, the old nodes will slowly notice the change
>> in ownership and begin sending the appropriate data to the new node as
>> it claims partitions and rebalances ownership.  If you issue these
>> requests after ownership has changed but before handoff has completed,
>> there will be some cases where some of the ideal owning nodes for
>> given keys will not yet have received the data they are being asked
>> for.
>> If that is the case, the problem should go away after some minutes
>> have passed.  Is that what you observe?
>> It is also good timing that you mention this, as we are working on
>> some significant improvements to the performance and behavior of
>> handoff right now.
>> -Justin
> _______________________________________________
> riak-users mailing list
> riak-users at

More information about the riak-users mailing list