New to Riak, hitting a connection limit

Mark Phillips mark at basho.com
Fri Aug 3 12:53:13 EDT 2012


Hi Doug,

In line...

On Fri, Aug 3, 2012 at 7:48 AM, Douglas Muth <doug.muth at gmail.com> wrote:
> Hi Mark,
>
> On Thu, Aug 2, 2012 at 2:02 AM, Mark Phillips <mark at basho.com> wrote:
>>
>> I think you're problem here is that you're front-loading requests.
>> You're better off to issue them serially and wait for one to finish
>> before issuing the next. Rumor has there's an example of how one might
>> do this somewhere in Riaktant [1], a somewhat dusty but useful sample
>> app. I'll dig through it tomorrow if I remember.
>>
>
> Your comment gave me some ideas, so I went and started using the
> node-pool module (https://github.com/coopernurse/node-pool) in my test
> script.  That artificially limited the number of concurrent
> connections to whatever number I specified, and solved the problem.
>
> Unfortunately found another issue in node_riak which makes it
> unsuitable for production use: unbounded use of network sockets.  It
> seems that socks aren't being closed/reused properly.  At ~250
> queries/second, I could start getting EADDRNOTAVAIL errors after about
> 15k total queries. :-(
>

Hmm. I would say you should ping Matt Ranney and Voxer about this but
it looks like you've already dug into that. [1]

> Since my workplace is a node.js shop, and I really want to use Riak,
> I'm going to take a whack at writing my own node.js driver for Riak.
> Would anyone find it useful if I could talk my employer into releasing
> it under the GPL?
>

So, as much as I love to see more code written (especially from the
community), you might want to hold off. To date, the most heavily-used
node.js code with Riak has been riak-js (which used to live at
riakjs.org). Francisco Treacy was leading the charge on this but had
to put it down on due to some other opportunities. Sadly, it hasn't
seen too much love over the past few months. However, I have it from a
few very solid authorities that there will soon be consistent work
being done on Riak-js. There's a fork of it here (but there should be
a different location for the new work that's happening) :

https://github.com/mostlyserious/riak-js

So, I would say hold off on writing your own and ping me offline if
you're interested in lending a hand on riak-js. Or sit tight for a few
days until they make their plans known. :)


Mark

[1] https://github.com/mranney/node_riak/issues/5

> -- Doug
>
> --
> http://www.dmuth.org/
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list