Basho Product Alert Update re DataTypes Disk Format Incompatibility

Seema Jethani seema at
Wed Jan 21 18:20:36 EST 2015

This update adds some more details on the data types incompatibility issue
posted to the user list and alerted by Russell Brown earlier today

On January 20th 2015, a user reported issues [1] with CRDTs on upgrading
from Riak 2.0.2 to 2.0.4.

Keys that store maps inside maps, or sets inside maps created by Riak 2.0.0
to 2.0.2 are unreadable after upgrading to 2.0.4.  Keys that just contain a
set or contain a single level map are unaffected.

The root cause is a change to the on disk format as part of a performance
improvement to the Map and Set DataTypes introduced in 2.0.4.


*Applicability*: Riak 2.0.4

*Symptoms:* Request timeouts, inability to access data in maps with nested
maps or sets created in a previous version of Riak 2.0.

*Cause:* Change in the on-disk format of CRDTs

*Identification:* The Riak log can be used to assess impact and will
contain this entry for each failed get:

2015-01-21 13:01:00.441 [error]
<0.1033.0>@riak_core_vnode:vnode_command:348 riak_kv_vnode command
failed {{badrecord,dict},[{dict,filter_dict,2,[{file,"dict.erl"},{line,464}]},{riak_dt_map,'-filter_unique/4-fun-1-',4,[{file,"src/riak_dt_map.erl"},{line,466}]},{sets,fold_bucket,3,[{file,"sets.erl"},{line,313}]},{sets,fold_seg,4,[{file,"sets.erl"},{line,309}]},{sets,fold_segs,4,[{file,"sets.erl"},{line,305}]},{riak_dt_map,merge,2,[{file,"src/riak_dt_map.erl"},{line,454}]},{riak_kv_crdt,'-merge_value/2-fun-0-',5,[{file,"src/riak_kv_crdt.erl"},{line,204}]},{orddict,update,4,[{file,"orddict.erl"},{line,170}]}]}

Additionally, the client making the request will experience a timeout.
Affected Users

Users will be affected if they:


   Use Riak DataTypes to store Sets inside Maps or Maps inside Maps.



   Have already upgraded to Riak 2.0.4.

If you do not use Riak DataTypes (aka CRDTs) you are not affected and may
upgrade to 2.0.4 as normal.
Mitigation Strategy

If you use (or would like to use) Riak DataTypes and you have not upgraded
to 2.0.4, use versions 2.0.0 to 2.0.2.

If you use Riak DataTypes and have upgraded to 2.0.4, you will be unable to
read data written before 2.0.4.  There are two choices:


   Downgrade to 2.0.2 to access your old data. Data written by 2.0.4 will
   be unreadable until 2.0.5 is available (or patches for 2.0.4).

   Stay at 2.0.4 and only access newly written data. Data written by
   2.0.0-2.0.2 will be readable once 2.0.5 is available (or patches for 2.0.4).

In both cases it is possible to delete the unreadable data and then
recreate it.
Moving Forward

Basho has prepared a fix [2] for the issue to enable reading of nested data
in the old and new formats and is reviewing and testing it for the 2.0.5
release. Once testing is complete we will provide a set of patch files that
can be added to the lib/basho-patches directory in the event this advisory
is discovered after upgrade. We have also downgraded the version of Riak
available in repositories served by (apt/yum) back to
2.0.2, in order to prevent unattended upgrades being affected by this issue.


Seema Jethani
Director of Product Management, Basho <>
4083455739 | @seemaj <>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the riak-users mailing list