riak-python-client2, a rewrite of the official client

Shuhao Wu admin at thekks.net
Thu Mar 15 11:13:03 EDT 2012


I'm looked into just modifying and contributing to the existing library, and
found several issues with it. Here's my main motivation for a rewrite:

  1. The current structure riak-python-client is somewhat messy. Everything
     depends on each other. Just look at things like RiakLink and
RiakIndexEntry.
     They're unnecessary and overcomplicates the code. Furthermore, if you look
     the transports, it's very much dependent on things like RiakObject, and
     RiakObject is pretty much nothing without the transport. It's almost like a
     circular dependency. So instead, I redesigned the transports to operate
     independently from things like RiakObject. To do this, simply modifying
     is not feasible and it will result in an almost complete rewrite anyway due
     to the dependencies problems I described.
  2. There's a lot of "bloat" in the current riak-python-client. A simple
     example would be get_ and set_, as well as things like RiakLink and
     RiakIndexEntry. To get rid of those would pretty much require a rewrite as
     well.
  3. Basho currently do not have a dedicated python developer working on this.
     I don't know this for sure but I think their resources, in terms
of clients,
     go mainly to java, ruby and javascript, though that's just my observations.

My primary goal of having a rewrite is hopefully simplify the code base as well
as improve some aspects of the python client (such as not using deprecated
functions such as apply) and (hopefully) increase the speed of the client.
After examining the code (which I had to do while rewriting), I don't think
simply modifying the current codebase could fix its issues (there are more
issues then what I've stated), and I don't think it will take as long as people
think. The current code base has about 4k lines of python and 0.5k lines of
Erlang. In my client, a chunk of the code actually comes from the original
client as they work with a few adaptations.

As far as road map goes, I'm currently just rewriting all the functionalities
provided by the current python client, and here's a list of things that Sean
would like to see accomplished, which I will work on once I have all the
functionalities of the current client complete:

    https://gist.github.com/1959278

I hope I've answered all the questions. If there is any more questions/comments,
feel free to shoot it my way.

Shuhao


On Thu, Mar 15, 2012 at 5:30 AM, Andrey V. Martyanov <realduke at gmail.com> wrote:
>
> Hi Shuhao,
>
> What is the reason to create a fork? What is your primary goal? Do you have any particular plans or roadmap? It would be very helpful.
>
> Best regards,
> Andrey Mart
>
> On Thu, Mar 15, 2012 at 5:30 AM, Shuhao Wu <admin at thekks.net> wrote:
>>
>> That would be most appreciated.
>>
>> I just completed a very bare minimum higher level API that's similar to the current Python client. However, there's still a lot of work to be done. Once again, I'm just one lone student working in my spare time and I can't possibly think of everything. If we want to bring Python up to speed with the rest of the clients, I'll need more people helping me!
>>
>> Also, I have a general plan of bringing riakkit to riak-python-client2 as well.
>>
>> Shuhao
>>
>>
>> On Mon, Mar 12, 2012 at 12:50 AM, Jim Adler <jim.adler at comcast.net> wrote:
>>>
>>> I'll certainly take a look at your repo and help where I can.
>>>
>>> Jim
>>>
>>> Sent from my phone. Please forgive the typos.
>>>
>>> On Mar 11, 2012, at 11:25 PM, Shuhao Wu <admin at thekks.net> wrote:
>>>
>>>
>>> I'm not too familiar with PBC . I'm looking for individuals to help me as I'm just a student and only working on it in my spare time.
>>>
>>> Shuhao
>>>
>>> On Mar 11, 2012 10:43 PM, "Jim Adler" <jim.adler at comcast.net> wrote:
>>>>
>>>> This is great Shuhao!
>>>>
>>>> I've had problems using the python client using multiple threads with protocol buffers. The exact same multithreaded code works fine with http protocol. So, any attention you can give to the pbc transport would be hugely appreciated.
>>>>
>>>> Good luck! Can't wait to give your new client a test drive.
>>>>
>>>> Jim
>>>>
>>>> Sent from my phone. Please forgive the typos.
>>>>
>>>> On Mar 11, 2012, at 12:31 PM, Shuhao Wu <admin at thekks.net> wrote:
>>>>
>>>> Hey guys,
>>>>
>>>> I've been working with riak and python for the past couple of months, and I'm at the point where I'm ready to completely reimplement riak-python-client. So.. I did/started. The project is hosted on github and under the same apache 2 license as the original client.
>>>>
>>>> https://github.com/ultimatebuster/riak-python-client2
>>>>
>>>> Couple of key highlights right now:
>>>>
>>>>  1. 2 layers of API, a very basic layer (Transports), which handles communicating with the server, and a high level API, which consists of Bucket, Object, and so on.
>>>>  2. Eliminated things like `set_` and `get_`. Made them just variables where possible
>>>>
>>>> I'm still taking suggestions for how to reimplement RiakObject (now called RObject). Any feedback will be appreciated.
>>>>
>>>> Cheers,
>>>>
>>>> Shuhao
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>




More information about the riak-users mailing list