Python client performance issue
nico.meyer at adition.com
Wed Feb 16 05:34:09 EST 2011
Ah, I see it now. The thing is, we don't usually install libraries on
each machine manually but build Debian packages for them. In our case we
missed the automatically installed protobuf-2.3. I have to check the
difference in performance.
But back to the original problem. The other obvious culprit is the json
encoder/decoder. By default riak-client uses the built in json module,
which is also notoriously slow, especially at decoding as some sources
http://shane.willowrise.com/archives/python-json-performance/), but I'm
to lazy to check myself right now ;-).
Since Mike uses the default of return_body=True for the store(), his
date has to be encoded AND decoded on every put.
Mike: can you first try to replace 'entry.store()' with
'entry.store(return_body=False)'? That should already be much faster. If
it is still not fast enough you can replace the default encoder/decoder
by using the client.set_encoder/set_decoder methods.
Am Dienstag, den 15.02.2011, 10:05 -0800 schrieb Andy Gross:
> Sorry, I should have been more clear. The Python client depends on
> "protobuf>=2.3.0" in setup.py, so people are already most likely using
> - Andy
> On Tuesday, February 15, 2011 at 3:09 AM, Nico Meyer wrote:
> > Hi Andy.
> > I am not quite sure what you mean, is the protobuf library included
> > with
> > riak-python-client? Or are you talking about the version of the
> > protobuf
> > compiler that was used to create riakclient_pb2.py from
> > riakclient.proto?
> > Cheers,
> > Nico
> > Am Dienstag, den 15.02.2011, 02:23 -0800 schrieb Andy Gross:
> > > python-riak-client already uses version 2.3.0. Adventurous types
> > > might want to check out https://github.com/Greplin/fast-python-pb,
> > > which wraps the C/C++ protocol buffers library.
> > >
> > > --
> > > Andy Gross
> > > Principal Architect
> > > Basho Technologies, Inc.
More information about the riak-users