Massive json schema update

Guillaume Boddaert guillaume at lighthouse-analytics.co
Wed Jun 8 07:14:22 EDT 2016


Well,

We are talking about ~30 million items, in a single bucket. Those are 
standard tweets, I need to alter all of them because of an invalid 
structure.
My cluster is not in production, but I am willing to start validating a 
procedure that will be used at some point on a production cluster, 
that's a pre-release rehearsal.

I can write my own script that read/update every entries in big scan of 
the bucket, but It may suffer from network lag, or other external 
stuffs. I'd really like to learn how to alter a set of document in the 
erlang console for instance, in order to validate that I can patch data 
easily in something that would looks more like an SQL command (Yes, 
that's not how it works, understand a map-reduce script in a mongo shell 
for instance) in the Riak core than a programmatic fetch/update loop 
that I already know.

Guillaume

On 08/06/2016 08:49, Damien Krotkine wrote:
> Hi Guillaume,
>
> If I understand correctly you need to change all the values of your JSON data.
>
> How many keys are we talking about, how big are the data, and in how many buckets are the keys?
>
> Also, is your cluster in production yet?
>
>
>
>> Le 7 juin 2016 à 18:43, Guillaume Boddaert <guillaume at lighthouse-analytics.co> a écrit :
>>
>> Hi,
>>
>> I'd like to patch my current riak collection to rename a field inside a JSON schema, how can I achieve that from command line on the riak server itself ? Is there some kind of map/reduce mechanism that allow any json record to be updated then saved to the riak cluster ?
>>
>> Guillaume
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list