Weird RIAK behavior

Claudio Cesar Sanchez Tejeda demonccc.y at gmail.com
Mon Dec 29 10:27:44 EST 2014


Hi,

The clocks weren't synchronized because the servers don't have direct
access to internet or a NTP server. They can use a http proxy, so I
installed and configured htpdate on them in order to maintain their
clocks synchronized.

The issue was resolved by synchronizing the clocks.

Thanks!

On Tue, Dec 23, 2014 at 6:00 AM, Sargun Dhillon <sargun at sargun.me> wrote:
> It sounds like allow_mult is off.
> 1. What's your last_write_wins set to?
> 2. Are the clocks on your nodes accurately synced to one another (if
> they're all NTP peers from one another, what's the deltas?)? I know
> drift is a common problem in virtual machines on VMWare -- or clocks
> plain lying.
> 3. Are you using vector clocks for all operations?
> 4. Are you making concurrent updates?
>
> Are you familiar with AP Riak, the consistency model, and conflict
> resolution? There is some excellent documentation here:
> http://docs.basho.com/riak/2.0.2/dev/using/conflict-resolution/
>
>
>
> On Mon, Dec 22, 2014 at 7:15 PM, Claudio Cesar Sanchez Tejeda
> <demonccc.y at gmail.com> wrote:
>> Hi,
>>
>> Sorry, I forgot to mention that it is RIAK 1.4.10. They are configured
>> with multibackend. We are using, memory, bitcask and elevelDB
>> backends.
>>
>> On the buckets where we are having issues, the siblings are disabled
>> (allow_multi = false).
>>
>> Regards.
>>
>> On Mon, Dec 22, 2014 at 9:17 PM, Sargun Dhillon <sargun at sargun.me> wrote:
>>> What versions of Riak are you using? And are these CRDT sets?
>>>
>>> Sent from my iPhone
>>>
>>>> On Dec 22, 2014, at 16:04, Claudio Cesar Sanchez Tejeda <demonccc.y at gmail.com> wrote:
>>>>
>>>> I'm a sysadmin and I managing 5 cluster of RIAK:
>>>>
>>>> - two of them are LXC containers on the same physical machine (3 nodes
>>>> per cluster)
>>>> - one of them are LXC containers located on different physical
>>>> machines (6 nodes)
>>>> - one of them are LXC containers located on different physical
>>>> machines and XEN VMs (6 nodes)
>>>> - and the last of them are VMware ESX VMs (3 nodes)
>>>>
>>>> Our application works correctly on the first four clusters, but it
>>>> doesn't work as we expected on the last one.
>>>>
>>>> When we update a key and we retrieve this key in order to write it
>>>> again, it has an old value (it doesn't have the first value that we
>>>> wrote), for example:
>>>>
>>>> The key has: lalala
>>>> We retrieve the key, and add lololo, so it should be lalala,lololo
>>>> We retrieve the key again, and try to add lelele, so it should be now:
>>>> lalala,lololo,lelele, but when we retrieve it again, we only have:
>>>> lalala,lelele
>>>>
>>>> In the second write action, when we retrieve the key, we obtained a
>>>> key with the old value. We set r, w, pr and rw to 3 to the REST
>>>> requests, but it doesn't help.
>>>>
>>>> All the configuration files are very similiar and we don't have any
>>>> major differences in the disk I/O and network performance of the nodes
>>>> of the clusters.
>>>>
>>>> Did anyone have a similar issue?
>>>>
>>>> Regards.
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list