riak memstore clarification on enomem error

Shane McEwan shane at mcewan.id.au
Tue Oct 9 05:29:34 EDT 2012


G'day Sangeetha.

On 09/10/12 07:40, Sangeetha.PattabiRaman2 at cognizant.com wrote:
> Dear Team ,
>
>                   I have a 64 GB RAM ,during the Load of 35 million
> dataset (1.8 GB) it consumes nearly 40-45 GB of RAM durial the startup
> of the erlang script ,but
>
> While trying to load 40 million dataset (2.1 GB) I am getting  the
> following error
>
> *escript: exception error: no match of right hand side value
> {error,enomem}**,*

The error message is coming from escript and not Riak. It's just a guess 
but could it be that the script you're using to load your data into Riak 
is trying to load all the data into memory before sending it to Riak? 
Can you break your dataset into smaller chunks and load them separately? 
Or send the data to Riak as you read it from the dataset without storing 
it all in memory?

> *2.**Is there a provision to make use of the swap memory in riak config?*

Using swap in this situation is almost always a bad idea. Your script 
will end up running so slowly you will be waiting for days, maybe 
months, for your data to load.

Shane.




More information about the riak-users mailing list