Continuous HTTP POSTs to Riak

Vitaly E 13vitamins at gmail.com
Sun Mar 6 03:24:46 EST 2016


Hi Qiang,

You mentioned that my suggestion to synchronize the clocks of your Riak
nodes had solved the problem. Let me explain why -- it may clear things up
for you.

Riak uses vector clocks / version vectors
<http://docs.basho.com/riak/latest/theory/concepts/context/#Vector-Clocks>
to keep track of the sequence of updates to a particular key. These are
"logical clocks" as opposed to physical clocks that usually can't be relied
upon in a distributed system like Riak. Fetching the vector clock of a key
and passing it back on write (along with a new value), helps Riak decide
which modification is the "latest" one. Keep in mind that "concurrent
modification" does not necessary means simultaneous, but can happen due to
a network partition, for example.

Normally, when a concurrent update is detected (based on vector clocks),
Riak returns siblings and lets a client decide which version should be
used. Since you're running with allow_mult = false, you've actually told
Riak to resolve conflicts for you. In this case Riak still uses vector
clocks when possible, and then time-stamps if there's still a conflict.
Obviously, if you don't pass vector clocks, only time-stamps will be used.
Moreover, without vector clocks any update is considered concurrent, Riak
just doesn't expose it with allow_mult = false (remember, you opted for
automatic conflict resolution?).

Keep in mind that a time-stamp is assigned by the first Riak node a request
hits. The node then forwards the request based on the key this request is
trying to write
<http://docs.basho.com/riak/latest/theory/concepts/Clusters/>.

Now, imagine that a write to a key passes through node X, and within 30 sec
another update to the same key passes through node Y, which is 2 min behind
node X. Since time-stamps are being used to resolve conflicts, the second
update will be considered older than the first one, and eventually
discarded.

So, if this kind of conflict resolution is good enough for you, make sure
your Riak's clocks are in sync (NTP?). I would also suggest to pass vector
clocks anyway.

Hope this helps. Everyone is welcome to correct/clarify.

Vitaly


On Mar 5, 2016 11:07 PM, "Qiang Cao" <caoqiang.cs at gmail.com> wrote:

> Thanks, Russell. I'm just curious. My application handles that. Maybe
> that's just because of the vclock.
>
> -Qiang
>
> On Sat, Mar 5, 2016 at 2:09 PM, Russell Brown <russell.brown at me.com>
> wrote:
>
>> And you pass the vclock back after a GET with the next POST? Unless we
>> get a look at the vclocks then it's hard to say why or where you have
>> concurrency. But since concurrency is ultimately unavoidable using riak,
>> why the concern? Can your application/data model handle it is the main
>> question?
>>
>>
>> On 5 Mar 2016, at 19:04, Qiang Cao <caoqiang.cs at gmail.com> wrote:
>>
>> Thanks, Russell! I do a GET immediately after a POST is done. I use
>> apache httpclient, which handles requests synchronously. On the client,
>> POSTs and GETs are sent out sequentially.
>>
>> On Sat, Mar 5, 2016 at 1:57 PM, Russell Brown <russell.brown at me.com>
>> wrote:
>>
>>>
>>> On 5 Mar 2016, at 18:43, Qiang Cao <caoqiang.cs at gmail.com> wrote:
>>>
>>> > Just curious. The POSTs are sent out sequentially and a quorum is set
>>> up on Riak. I wonder how would it happen that Riak still considers the POST
>>> requests concurrent?
>>> Did you read the result of POST 1 before sending POST 2? If not, and you
>>> don’t send the causal context, Riak has to view them as concurrent.
>>>
>>> How does the quorum work here?
>>
>> Thanks,
>> Qiang
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160306/111e059b/attachment-0002.html>


More information about the riak-users mailing list