Store() Performance

matthew hawthorne mhawthorne at gmail.com
Thu Apr 7 22:58:26 EDT 2011


Hi Gui,

I recently pushed 70 million records of size 1K each into a 5-node
Riak cluster (which was replicating to another 5-node cluster) at
around 1000 writes/second using basho_bench and the REST interface.  I
probably could have pushed it further, but I wanted to confirm that it
could maintain the load for the entire data set, which it did.

My point being that your speed-limit of 200 writes/second is likely
specific to your configuration.

I wonder:
1) what's your average write latency?
2) how big is your connection pool?

Because it's possible that you don't have enough connections available
to handle your desired load.

-matt


On Thu, Apr 7, 2011 at 6:01 PM, Gui Pinto <gpinto at chitika.com> wrote:
> Hey guys,
> I'm attempting to importing 300M+ objects into a Riak cluster, but have
> quickly reached the REST API's speed-limit at 200-store()'s per second..
> At the rate of 200/s, I'm looking at 20-days to import this data set! That
> can't be the fastest method to do this..
>
> Any recommendations?
>
> Thanks!
> Gui Pinto
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>




More information about the riak-users mailing list