Riak search, post schema change reindexation
guillaume at lighthouse-analytics.co
Mon Aug 29 03:56:22 EDT 2016
I recently needed to alter my Riak Search schema for a bucket type that
contains ~30 millions rows. As a result, my index was wiped since we are
waiting for a Riak Search 2.2 feature that will sync Riak storage with
Solr index on such an occasion.
I adapted a since script suggested by Evren Esat Özkan there
It is a simple python script that will stream keys and trigger a store
action for any items. Unfortunately it failed past 178k items due to
time out on the key stream. I calculated that this kind of reindexation
mechanism would take up to 5 days without a crash to succeed.
I was wondering if there would be a pure Erlang mean to achieve a
complete forced rewrite of every single element in my bucket type rather
that an error prone and very long python process.
How would you guys reindex a 30 million item bucket type in a fast and
reliable way ?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users