CorrugatedIron & Idle Connection

Jeremiah Peschka jeremiah.peschka at gmail.com
Thu Jul 11 20:02:37 EDT 2013


We're aware of problems in our implementation of connection disposal on
time out and we're working to implement fixes for common error scenarios
before we ship 1.4.0 and a hot fix for 1.3.x - while not a complete
solution for the unbounded connection pool that you're seeing, this should
alleviate some of the pain, especially on idle socket timeouts.

I have a few questions for you so we can better direct our fixes and
functionality (both right now and going forward):

What are the data access patterns you're primarily using?
Are you using the streaming APIs?
Are you getting rid of/fully enumerating collections before moving on?
How are you handling errors processing individual items in the IEnumerable?
(This is primarily a concern if you're using the streaming interfaces)

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Thu, Jul 11, 2013 at 7:46 AM, Amir Halatzi <Amir.Halatzi at oovoo.com>wrote:

>  We’ve tried that. The result was 3K open connections per Riak node, and
> as you can guess – this is not an improvement.****
>
> ** **
>
> Do you have other recommendations?****
>
> ** **
>
> Thanks,****
>
> Amir****
>
> ** **
>
> ** **
>
> *From:* OJ Reeves [mailto:oj at buffered.io]
> *Sent:* 11 July 2013 17:31
> *To:* Amir Halatzi
> *Cc:* Jeremiah Peschka; riak-users at lists.basho.com
>
> *Subject:* Re: CorrugatedIron & Idle Connection****
>
> ** **
>
> This is a discussion we really need to have. Thanks for opening the Github
> issue.****
>
> ** **
>
> In the short term, you can "turn off" pooling by setting the pool size to
> zero. This will mean all the connections are done "on the fly".****
>
> ** **
>
> On Fri, Jul 12, 2013 at 12:26 AM, Amir Halatzi <Amir.Halatzi at oovoo.com>
> wrote:****
>
> Just did: https://github.com/DistributedNonsense/CorrugatedIron/issues/153
> ****
>
>  ****
>
> Still, if we’d like to manage it ourselves - how would you recommend going
> about it? Should we interfere with the way you handle the connection, or
> just retry the action on ‘network failure’?****
>
>  ****
>
> BR,****
>
> Amir****
>
>  ****
>
> *From:* Jeremiah Peschka [mailto:jeremiah.peschka at gmail.com]
> *Sent:* 11 July 2013 17:11
> *To:* Amir Halatzi
> *Cc:* riak-users at lists.basho.com
> *Subject:* Re: CorrugatedIron & Idle Connection****
>
>  ****
>
> We're in the process of figuring out how to best handle killing off idle
> connections inside of CorrugatedIron. If you can head over to
> https://github.com/DistributedNonsense/CorrugatedIron/issues and file a
> new bug report, that would be fantastic.****
>
>  ****
>
> We've avoided killing off connections until now because we don't want to
> introduce locking into our connection management. Truthfully, we haven't
> run into a bug report where the network team was killing off connections.*
> ***
>
>  ****
>
> So, anyway, if you can submit a bug report we'll get on it. It won't make
> it out for the 1.4.0 release, but we can put that in a point release.****
>
>
> ****
>
> ---****
>
> Jeremiah Peschka - Founder, Brent Ozar Unlimited****
>
> MCITP: SQL Server 2008, MVP****
>
> Cloudera Certified Developer for Apache Hadoop****
>
>  ****
>
> On Thu, Jul 11, 2013 at 6:59 AM, Amir Halatzi <Amir.Halatzi at oovoo.com>
> wrote:****
>
> Hi all,
>
> Trying to optimize our application connection pool, we encountered and
> issue with Corrugated Iron and identifying idle/reset connections. Namely,
> if our network infrastructure (firewall/proxy/etc) closes an idle
> connection - we're getting a 'cluster is offline' error code.
>
> I saw that this is a feature in the making (for the past 8 months), but I
> was wondering if any of you found a way around it?
>
> Thank you,
> Amir Halatzi,
> ooVoo
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com****
>
>  ****
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com****
>
>
>
> ****
>
> ** **
>
> --
>
> OJ Reeves****
>
> +61 431 952 586
> http://buffered.io/****
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130711/cc692709/attachment.html>


More information about the riak-users mailing list