Read Before Writes on Distributed Counters

Russell Brown russell.brown at mac.com
Thu Oct 17 10:40:04 EDT 2013


I have some from a while back, if I can find my graphs I'll put them up somewhere.

Cheers

Russell

On 17 Oct 2013, at 16:35, Weston Jossey <weston.jossey at gmail.com> wrote:

> Great everyone, thank you.  
> 
> @Russell:  I specifically work with either Go (https://github.com/tpjg/goriakpbc) or Ruby (basho client).  I haven't tested the ruby client, but I'd assume it will perform the write without the read (based on my reading of the code).  The Go library, on the other hand, currently always performs a read prior to the write.  It's an easy patch that I've already applied locally for benchmarking, I just didn't want to submit the PR till I was sure this was the correct behavior.
> 
> Somewhat off topic, but I dont' want to open up another thread if it's unnecessary.  This questions arose because I've been doing extensive benchmarking around distributed counters.  Are there pre-existing benchmarks out there that I can measure myself against?  I haven't stumbled across many at this point, probably because of how new it is.
> 
> Cheers,
> Wes
> 
> 
> On Thu, Oct 17, 2013 at 10:21 AM, Russell Brown <russell.brown at mac.com> wrote:
> Hi Wes,
> 
> The client application does not need to perform a read before a write, the riak server must read from disk before updating the counter. Or at least it must with our current implementation.
> 
> What PRs did you have in mind? I'm curious.
> 
> Oh, it looks like Sam beat me to it…to elaborate on his "not idempotent" line, that means when riak tells you "error" for some counter increment, it may only be a partial failure, and re-running the operation may lead to over counting.
> 
> Cheers
> 
> Russell
> 
> On 17 Oct 2013, at 16:03, Weston Jossey <weston.jossey at gmail.com> wrote:
> 
> > In the context of using distributed counters (introduced in 1.4), is it strictly necessary to perform a read prior to issue a write for a given key?  A la, if I want to blindly increment a value by 1, regardless of what its current value is, is it sufficient to issue the write without previously having read the object?
> >
> > I ask because looking at some of the implementations for counters in the open source community, it's common to perform a read before a write, which impacts performance ceilings on clusters with high volume reads / writes.  I want to verify before issuing some PRs that this is in fact safe behavior.
> >
> > Thank you!
> > -Wes Jossey
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> 





More information about the riak-users mailing list