Best way to bulk load 1M rows

John D. Rowell me at jdrowell.com
Fri Jul 23 15:48:36 EDT 2010


Hi Robert,

I get about 200 inserts/s with the Ruby client on really low-end hardware
(athlon xp2000). This means that in just over 1h you can get 1M tweets up,
which is way less time than you'll have to wait for a response from the
list. Also you can insert in parallel on different nodes, so if you have 4
(really old) nodes you can do it in 15min.

2010/7/23 <golsombe at gmail.com>

> All,
> I'm trying to find an optimized method for loading 1Million trending tweets
> into a single Riak node for map-reduce analysis. I've played with the erlang
> protobuffer client as well as Ruby Riak-Client, both are too slow. Any
> ideas?
>
> Robert Hall aka Golsombe
> Sent on the Sprint® Now Network from my BlackBerry®
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20100723/f8189751/attachment.html>


More information about the riak-users mailing list