mhawthorne at gmail.com
Thu Apr 7 22:58:26 EDT 2011
I recently pushed 70 million records of size 1K each into a 5-node
Riak cluster (which was replicating to another 5-node cluster) at
around 1000 writes/second using basho_bench and the REST interface. I
probably could have pushed it further, but I wanted to confirm that it
could maintain the load for the entire data set, which it did.
My point being that your speed-limit of 200 writes/second is likely
specific to your configuration.
1) what's your average write latency?
2) how big is your connection pool?
Because it's possible that you don't have enough connections available
to handle your desired load.
On Thu, Apr 7, 2011 at 6:01 PM, Gui Pinto <gpinto at chitika.com> wrote:
> Hey guys,
> I'm attempting to importing 300M+ objects into a Riak cluster, but have
> quickly reached the REST API's speed-limit at 200-store()'s per second..
> At the rate of 200/s, I'm looking at 20-days to import this data set! That
> can't be the fastest method to do this..
> Any recommendations?
> Gui Pinto
> riak-users mailing list
> riak-users at lists.basho.com
More information about the riak-users