Read Before Writes on Distributed Counters

Weston Jossey weston.jossey at gmail.com
Thu Oct 17 10:35:42 EDT 2013


Great everyone, thank you.

@Russell:  I specifically work with either Go (
https://github.com/tpjg/goriakpbc) or Ruby (basho client).  I haven't
tested the ruby client, but I'd assume it will perform the write without
the read (based on my reading of the code).  The Go library, on the other
hand, currently always performs a read prior to the write.  It's an easy
patch that I've already applied locally for benchmarking, I just didn't
want to submit the PR till I was sure this was the correct behavior.

Somewhat off topic, but I dont' want to open up another thread if it's
unnecessary.  This questions arose because I've been doing extensive
benchmarking around distributed counters.  Are there pre-existing
benchmarks out there that I can measure myself against?  I haven't stumbled
across many at this point, probably because of how new it is.

Cheers,
Wes


On Thu, Oct 17, 2013 at 10:21 AM, Russell Brown <russell.brown at mac.com>wrote:

> Hi Wes,
>
> The client application does not need to perform a read before a write, the
> riak server must read from disk before updating the counter. Or at least it
> must with our current implementation.
>
> What PRs did you have in mind? I'm curious.
>
> Oh, it looks like Sam beat me to it…to elaborate on his "not idempotent"
> line, that means when riak tells you "error" for some counter increment, it
> may only be a partial failure, and re-running the operation may lead to
> over counting.
>
> Cheers
>
> Russell
>
> On 17 Oct 2013, at 16:03, Weston Jossey <weston.jossey at gmail.com> wrote:
>
> > In the context of using distributed counters (introduced in 1.4), is it
> strictly necessary to perform a read prior to issue a write for a given
> key?  A la, if I want to blindly increment a value by 1, regardless of what
> its current value is, is it sufficient to issue the write without
> previously having read the object?
> >
> > I ask because looking at some of the implementations for counters in the
> open source community, it's common to perform a read before a write, which
> impacts performance ceilings on clusters with high volume reads / writes.
>  I want to verify before issuing some PRs that this is in fact safe
> behavior.
> >
> > Thank you!
> > -Wes Jossey
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20131017/6f4ecaff/attachment.html>


More information about the riak-users mailing list