riak-python-client2, a rewrite of the official client

Greg Stein gstein at gmail.com
Fri Mar 16 14:13:53 EDT 2012


On Thu, Mar 15, 2012 at 14:08, Armon Dadgar <armon.dadgar at gmail.com> wrote:
>> Message: 4
>> Date: Thu, 15 Mar 2012 11:13:03 -0400
>> From: Shuhao Wu <admin at thekks.net>
>> To: "Andrey V. Martyanov" <realduke at gmail.com>
>> Cc: riak-users at lists.basho.com
>> Subject: Re: riak-python-client2, a rewrite of the official client
>> Message-ID:
>>       <CAP1J66d0VmsV0VKx+W849tE2KM8_S7wMW5ta=a9+mUnncXuWCA at mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> I'm looked into just modifying and contributing to the existing library, and
>> found several issues with it. Here's my main motivation for a rewrite:
>>
>> ? 1. The current structure riak-python-client is somewhat messy. Everything
>>     depends on each other. Just look at things like RiakLink and
>> RiakIndexEntry.
>>     They're unnecessary and overcomplicates the code. Furthermore, if you look
>>     the transports, it's very much dependent on things like RiakObject, and
>>     RiakObject is pretty much nothing without the transport. It's almost like a
>>     circular dependency. So instead, I redesigned the transports to operate
>>     independently from things like RiakObject. To do this, simply modifying
>>     is not feasible and it will result in an almost complete rewrite anyway due
>>     to the dependencies problems I described.
>>  2. There's a lot of "bloat" in the current riak-python-client. A simple
>>     example would be get_ and set_, as well as things like RiakLink and
>>     RiakIndexEntry. To get rid of those would pretty much require a rewrite as
>>     well.
>>  3. Basho currently do not have a dedicated python developer working on this.
>>     I don't know this for sure but I think their resources, in terms
>> of clients,
>>     go mainly to java, ruby and javascript, though that's just my observations.
>>
>> My primary goal of having a rewrite is hopefully simplify the code base as well
>> as improve some aspects of the python client (such as not using deprecated
>> functions such as apply) and (hopefully) increase the speed of the client.
>> After examining the code (which I had to do while rewriting), I don't think
>> simply modifying the current codebase could fix its issues (there are more
>> issues then what I've stated), and I don't think it will take as long as people
>> think. The current code base has about 4k lines of python and 0.5k lines of
>> Erlang. In my client, a chunk of the code actually comes from the original
>> client as they work with a few adaptations.
>>
>> As far as road map goes, I'm currently just rewriting all the functionalities
>> provided by the current python client, and here's a list of things that Sean
>> would like to see accomplished, which I will work on once I have all the
>> functionalities of the current client complete:
>>
>>    https://gist.github.com/1959278
>>
>> I hope I've answered all the questions. If there is any more questions/comments,
>> feel free to shoot it my way.
>>
>> Shuhao
>>
>
> I agree that the Python client does need some work to clean it up,
> and make it more idiomatic Python, but I'm not sure that a total rewrite
> is necessary.
>
> Most of the abstractions are good, but they just need a bit of cleanup.
> I think there is always a tendency to assume things are unnecessarily messy,
> but once you get to rewriting it you end up running into the pain points that
> drove those design decisions.
>
> If there is interest, we could just formulate a roadmap for the a set of breaking
> changes to the existing client, and release it as a new version of the same project.
>
> Things I would like to see:
>  * Cleanup the transport interface
>  * RiakObject needs to be simplified
>  * Support a RiakJSONObject subclass which has the encoding logic that is inside RiakObject now
>  * Indexes need to be cleaned up
>  * MapReduce interfaces feel a bit dirty
>  * Much improved exception hierarchy
>  * RiakClient / RiakBucket interface needs to cleaned up

Right. Evolution of a client that everybody uses.

It isn't like Basho is refusing to accept pull requests from the
community. If they were a total black hole, then there may be an
argument for forking the project and evolving it. But that would at
least be from a fork. I don't see a valid argument for
start-from-scratch.

Yet that isn't the case. Basho's responsiveness on the Python client
might not be "awesome", but they *are* willing to engage. I met up
with them last fall, have exchanged numerous emails with them, and
even had a conference call about the Python client. Not to mention all
the discussion and the pull requests that I provided, which got
merged. They *are* working with the community.

In my experience, communities exist to develop/maintain/focus on a
code base. Multiple/competing codebases end up fracturing the
community. That rarely leads to a long-term, sustainable, healthy
outcome.

Cheers,
-g




More information about the riak-users mailing list