riak-python-client2, a rewrite of the official client
gstein at gmail.com
Fri Mar 16 14:13:53 EDT 2012
On Thu, Mar 15, 2012 at 14:08, Armon Dadgar <armon.dadgar at gmail.com> wrote:
>> Message: 4
>> Date: Thu, 15 Mar 2012 11:13:03 -0400
>> From: Shuhao Wu <admin at thekks.net>
>> To: "Andrey V. Martyanov" <realduke at gmail.com>
>> Cc: riak-users at lists.basho.com
>> Subject: Re: riak-python-client2, a rewrite of the official client
>> <CAP1J66d0VmsV0VKx+W849tE2KM8_S7wMW5ta=a9+mUnncXuWCA at mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>> I'm looked into just modifying and contributing to the existing library, and
>> found several issues with it. Here's my main motivation for a rewrite:
>> ? 1. The current structure riak-python-client is somewhat messy. Everything
>> depends on each other. Just look at things like RiakLink and
>> They're unnecessary and overcomplicates the code. Furthermore, if you look
>> the transports, it's very much dependent on things like RiakObject, and
>> RiakObject is pretty much nothing without the transport. It's almost like a
>> circular dependency. So instead, I redesigned the transports to operate
>> independently from things like RiakObject. To do this, simply modifying
>> is not feasible and it will result in an almost complete rewrite anyway due
>> to the dependencies problems I described.
>> 2. There's a lot of "bloat" in the current riak-python-client. A simple
>> example would be get_ and set_, as well as things like RiakLink and
>> RiakIndexEntry. To get rid of those would pretty much require a rewrite as
>> 3. Basho currently do not have a dedicated python developer working on this.
>> I don't know this for sure but I think their resources, in terms
>> of clients,
>> My primary goal of having a rewrite is hopefully simplify the code base as well
>> as improve some aspects of the python client (such as not using deprecated
>> functions such as apply) and (hopefully) increase the speed of the client.
>> After examining the code (which I had to do while rewriting), I don't think
>> simply modifying the current codebase could fix its issues (there are more
>> issues then what I've stated), and I don't think it will take as long as people
>> think. The current code base has about 4k lines of python and 0.5k lines of
>> Erlang. In my client, a chunk of the code actually comes from the original
>> client as they work with a few adaptations.
>> As far as road map goes, I'm currently just rewriting all the functionalities
>> provided by the current python client, and here's a list of things that Sean
>> would like to see accomplished, which I will work on once I have all the
>> functionalities of the current client complete:
>> I hope I've answered all the questions. If there is any more questions/comments,
>> feel free to shoot it my way.
> I agree that the Python client does need some work to clean it up,
> and make it more idiomatic Python, but I'm not sure that a total rewrite
> is necessary.
> Most of the abstractions are good, but they just need a bit of cleanup.
> I think there is always a tendency to assume things are unnecessarily messy,
> but once you get to rewriting it you end up running into the pain points that
> drove those design decisions.
> If there is interest, we could just formulate a roadmap for the a set of breaking
> changes to the existing client, and release it as a new version of the same project.
> Things I would like to see:
> * Cleanup the transport interface
> * RiakObject needs to be simplified
> * Support a RiakJSONObject subclass which has the encoding logic that is inside RiakObject now
> * Indexes need to be cleaned up
> * MapReduce interfaces feel a bit dirty
> * Much improved exception hierarchy
> * RiakClient / RiakBucket interface needs to cleaned up
Right. Evolution of a client that everybody uses.
It isn't like Basho is refusing to accept pull requests from the
community. If they were a total black hole, then there may be an
argument for forking the project and evolving it. But that would at
least be from a fork. I don't see a valid argument for
Yet that isn't the case. Basho's responsiveness on the Python client
might not be "awesome", but they *are* willing to engage. I met up
with them last fall, have exchanged numerous emails with them, and
even had a conference call about the Python client. Not to mention all
the discussion and the pull requests that I provided, which got
merged. They *are* working with the community.
In my experience, communities exist to develop/maintain/focus on a
code base. Multiple/competing codebases end up fracturing the
community. That rarely leads to a long-term, sustainable, healthy
More information about the riak-users