Protobuf latency on large keys

Pavel Kirienko pavel.kirienko.list at gmail.com
Tue Apr 16 13:34:58 EDT 2013


Hi everyone,

I encountered a noticeable difference in latency between PBC and HTTP when
querying Riak for large data.

Consider the following setup: 3-node Riak 1.3 cluster with default settings
on LevelDB is connected to an application server via 100Mbit LAN. KV
contains a key of size 25 megabytes.

I request that key using both HTTP and PBC in order to compare the delays:
http://pastebin.com/dr8N82t4

It is interesting that I'm getting completely different timings when I swap
the request order:

HTTP first, then PBC:
-----------------
$ ./riak_latency_test.py
HTTP
24555775
4.82462286949 sec

PBC
24555775
7.63437104225 sec
-----------------

Vice versa:
-----------------
$ ./riak_latency_test.py
PBC
24555775
18.1577270031 sec

HTTP
24555775
4.79673409462 sec
-----------------

Note that these results are stable in the sense of repeatability, i.e.
successive runs of every combination yield pretty the same timings.

However, that little observation is hardly relevant to my question, which
is: Why protobuf is so slow being compared with HTTP? Is it possible to
speed it up via fine tuning, or shall I resort to using HTTP?

Regards,
Pavel.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130416/d107485e/attachment.html>


More information about the riak-users mailing list