About Protocol Buffers

Alex De la rosa alex.rosa.box at gmail.com
Tue Jun 24 10:42:04 EDT 2014


Sean,

Thank you very much for your answer; somehow I imagined the Python library
may not be updated with last code. Can't wait to be out as I can not fully
test everything I need for my project.

Thanks,
Alex


On Tue, Jun 24, 2014 at 3:35 PM, Sean Cribbs <sean at basho.com> wrote:

> Alex,
>
> Not all Riak 2.0 features have been merged to master on the Python client,
> including CRDTs. We are in final testing of that feature and will have a
> release candidate package of the client out soon.
>
>
> On Tue, Jun 24, 2014 at 5:26 AM, Alex De la rosa <alex.rosa.box at gmail.com>
> wrote:
>
>> Hi Luc,
>>
>> I tried your new solution and gives the same error. The problem is that
>> .bucket_type() doesn't exist:
>>
>> Traceback (most recent call last):
>>   File "test.py", line 18, in <module>
>>     bucket = client.bucket_type('counters').bucket('likes')
>> AttributeError: 'RiakClient' object has no attribute 'bucket_type'
>>
>> I have Riak 2.0.0beta1 and riak-2.0.3-py2.7.egg as Python library.
>>
>> Thanks,
>> Alex
>>
>>
>> On Tue, Jun 24, 2014 at 11:44 AM, Luc Perkins <lperkins at basho.com> wrote:
>>
>>> Alex,
>>>
>>> There's a mistake in that tutorial that I need to fix. Early in the
>>> tutorial I instruct you to create a bucket type called *counters*, and
>>> then later on I use the name *counter_bucket*. Silly mistake on my
>>> part. If you establish that as the bucket type, it should work:
>>>
>>> *bucket = client.bucket_type('counters').bucket('likes')*
>>>
>>> Also be aware that you can call a bucket type that you set up to use
>>> counters anything that you'd like. The only reserved term for bucket types
>>> is *default*.
>>>
>>> Luc
>>>
>>>
>>> On Tue, Jun 24, 2014 at 12:31 AM, Alex De la rosa <
>>> alex.rosa.box at gmail.com> wrote:
>>>
>>>> By the way, I have the following error using PBC on Riak 2.0:
>>>>
>>>> test.py
>>>>
>>>> ------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------
>>>> import riak
>>>> client = riak.RiakClient(host ='127.0.0.1', http_port = 8098, pb_port =
>>>> 8087, protocol = 'pbc')
>>>> bucket = client.bucket_type('counter_bucket').bucket('likes')
>>>> counter = bucket.new('Barcelona')
>>>> counter.increment(5)
>>>>
>>>> ------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------
>>>>
>>>> OUTPUT:
>>>>
>>>> Traceback (most recent call last):
>>>>   File "test.py", line 18, in <module>
>>>>     bucket = client.bucket_type('counter_bucket').bucket('likes')
>>>> AttributeError: 'RiakClient' object has no attribute 'bucket_type'
>>>>
>>>> What is wrong? I was following the instructions at:
>>>> https://raw.githubusercontent.com/basho/basho_docs/2.0.0/source/languages/en/riak/dev/using/data-types.md
>>>>
>>>> Thanks,
>>>> Alex
>>>>
>>>>
>>>> On Mon, Jun 23, 2014 at 10:03 PM, Alex De la rosa <
>>>> alex.rosa.box at gmail.com> wrote:
>>>>
>>>>> Hi Bret,
>>>>>
>>>>> Thanks for your reply, I had used Riak since version 0.14 via HTTP...
>>>>> for 2.0 I decided to do the switch to PBC and I'm glad to hear all HTTP
>>>>> calls has been migrated to PBC too :)
>>>>>
>>>>> Does anybody know an estimate date for version 2.0 to be out? I'm
>>>>> starting a new project and I would love to be able to use 2.0 as "list"
>>>>> buckets is a great feature for my needs.
>>>>>
>>>>> Thanks,
>>>>> Alex
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jun 23, 2014 at 9:57 PM, Brett Hazen <brett at basho.com> wrote:
>>>>>
>>>>>> Alex -
>>>>>>
>>>>>> I suspect you may be looking at some older documentation.  AFAIK all
>>>>>> HTTP functionality is now (Riak 2.0) supported via PBC so you can pick
>>>>>> either transport or the other for all of your communication.  In the past
>>>>>> there was some functionality available only via HTTP.  There are now some
>>>>>> direct queries to the Solr API which are available only via HTTP, but
>>>>>> that’s outside of Riak’s core [1].
>>>>>>
>>>>>> I’m sure those more in the know will correct me if I’ve misspoken.
>>>>>>
>>>>>> Brett
>>>>>>
>>>>>> [1] http://docs.basho.com/riak/2.0.0beta1/dev/advanced/search/
>>>>>>
>>>>>> On June 23, 2014 at 10:47:45 AM, Alex De la rosa (
>>>>>> alex.rosa.box at gmail.com) wrote:
>>>>>>
>>>>>>  Hi Luc,
>>>>>>
>>>>>> Cool, thanks! I can see how to do it now:
>>>>>>
>>>>>> bucket = client.bucket_type('counter_bucket').bucket('counters')
>>>>>>
>>>>>> Will be checking it more to see what can I do that is not yet
>>>>>> documented.
>>>>>>
>>>>>> The Python documentation says at one point: "Some requests are only
>>>>>> valid over 'http', and will always be sent via those transports, regardless
>>>>>> of which protocol is preferred.". Can I know which requests are HTTP only?
>>>>>> is it for something in particular? or they will be made into PBC at some
>>>>>> point?
>>>>>>
>>>>>> Thanks,
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 23, 2014 at 5:29 PM, Luc Perkins <lperkins at basho.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Alex,
>>>>>>>
>>>>>>> It sounds like you're working mostly with the lower-level Python API
>>>>>>> at the moment, but if you want to use the higher-level parts of the client
>>>>>>> for interacting with Riak Data Types (sets, maps, and counters), there is a
>>>>>>> tutorial here:
>>>>>>>
>>>>>>>
>>>>>>> https://raw.githubusercontent.com/basho/basho_docs/2.0.0/source/languages/en/riak/dev/using/data-types.md
>>>>>>>
>>>>>>> You'll see the Python samples interspersed throughout. I know it's
>>>>>>> annoying because it's just raw Markdown at the moment, but this will be
>>>>>>> deployed to our normal docs site
>>>>>>> <http://docs.basho.com/riak/2.0.0beta1/> soon.
>>>>>>>
>>>>>>> Hope that helps!
>>>>>>>
>>>>>>> Luc
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 23, 2014 at 4:54 PM, Alex De la rosa <
>>>>>>> alex.rosa.box at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Sean,
>>>>>>>>
>>>>>>>> Thanks for your quick reply, is what i suspected, moreover after
>>>>>>>> checking riak.proto file in the source. So mainly the contents to store
>>>>>>>> (json or not json) is not PB encoded semantically, but is encoded as a
>>>>>>>> "body" field in the Message. That's good though.
>>>>>>>>
>>>>>>>> Another question... is there any estimated date for Riak 2.0 to be
>>>>>>>> released? I'm playing with 2.0 beta 1 and the "list" object is a must-have
>>>>>>>> for me (same as the "counter" object that can be found in Riak 1.4).
>>>>>>>>
>>>>>>>> Is crucial for me to decide which technologies to use in my new
>>>>>>>> project; so far Riak 2.0 seems a great option, but I can not build under a
>>>>>>>> beta.
>>>>>>>>
>>>>>>>> Another question now that I talk about "counters"... how do you do
>>>>>>>> it with Python's PBC? I can see the functions: RiakClient.get_counter()
>>>>>>>> and RiakClient.update_counter()... but... how do you create a new counter?
>>>>>>>> how do you specify the special connotations to the bucket?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jun 23, 2014 at 4:38 PM, Sean Cribbs <sean at basho.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Alex,
>>>>>>>>>
>>>>>>>>> By default, the client assumes new objects use JSON as the
>>>>>>>>> encoding. The protocol buffers are only used on the wire to encode Riak's
>>>>>>>>> protocol. If you want to use your own protobuffs messages, I suggest two
>>>>>>>>> things:
>>>>>>>>>
>>>>>>>>> 1. Decide on a `content_type` property for each of your data types
>>>>>>>>> and be sure to assign that to each key as you create it. I've used
>>>>>>>>> 'application/pb-person' as an example below.
>>>>>>>>> 2. Register encoder and decoder functions [1] on the RiakClient
>>>>>>>>> object, like so:
>>>>>>>>>
>>>>>>>>> client.set_decoder('application/pb-person', Person.ParseFromString)
>>>>>>>>> client.set_encoder('application/pb-person', lambda x:
>>>>>>>>> x.SerializeToString())
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> http://riak-python-client.readthedocs.org/en/latest/client.html#serialization
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  On Mon, Jun 23, 2014 at 9:19 AM, Alex De la rosa <
>>>>>>>>> alex.rosa.box at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>  Hi there,
>>>>>>>>>>
>>>>>>>>>> I just installed RIAK 2.0 beta 1 and was playing with the Python
>>>>>>>>>> library using 'pbc' (Protocol Buffers).
>>>>>>>>>>
>>>>>>>>>> test.py
>>>>>>>>>>
>>>>>>>>>> --------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------
>>>>>>>>>>  import riak
>>>>>>>>>>  client = riak.RiakClient(host ='127.0.0.1', http_port = 8098,
>>>>>>>>>> pb_port = 8087, protocol = 'pbc')
>>>>>>>>>>  bucket = client.bucket('people')
>>>>>>>>>>
>>>>>>>>>> key = bucket.new('alex', data={"username":"Alex","age":33})
>>>>>>>>>> key.store()
>>>>>>>>>>
>>>>>>>>>> print bucket.get('alex').encoded_data
>>>>>>>>>>
>>>>>>>>>> --------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------
>>>>>>>>>>
>>>>>>>>>> it works perfectly and it prints {"username": "Alex", "age": 33}
>>>>>>>>>> as it should print.
>>>>>>>>>>
>>>>>>>>>> However, I'm not using a proto file (person.proto) and compiling
>>>>>>>>>> it into binary for Python to use it... something like:
>>>>>>>>>>
>>>>>>>>>> person.proto
>>>>>>>>>>
>>>>>>>>>> --------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------
>>>>>>>>>>  message Person {
>>>>>>>>>>   required string username = 1;
>>>>>>>>>>   required int32 age = 2;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> --------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------
>>>>>>>>>>
>>>>>>>>>> xxx.py
>>>>>>>>>>
>>>>>>>>>> --------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------
>>>>>>>>>>  import person_pb2
>>>>>>>>>> person = person_pb2.Person()
>>>>>>>>>> person.username = "Alex"
>>>>>>>>>> person.age = 33
>>>>>>>>>> data = person.SerializeToString()
>>>>>>>>>>
>>>>>>>>>> --------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------+--------
>>>>>>>>>>
>>>>>>>>>> Am I using the 'pbc' interface wrong? however, on Riak's examples
>>>>>>>>>> page is done also without the "proto" file... I'm a bit confused at the
>>>>>>>>>> moment as it changed the way I normally would use Protocol Buffers.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>>  _______________________________________________
>>>>>>>>>> riak-users mailing list
>>>>>>>>>> riak-users at lists.basho.com
>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sean Cribbs <sean at basho.com>
>>>>>>>>> Software Engineer
>>>>>>>>> Basho Technologies, Inc.
>>>>>>>>> http://basho.com/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> riak-users mailing list
>>>>>>>> riak-users at lists.basho.com
>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> riak-users mailing list
>>>>>> riak-users at lists.basho.com
>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
>
> --
> Sean Cribbs <sean at basho.com>
> Software Engineer
> Basho Technologies, Inc.
> http://basho.com/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20140624/6e7a3cca/attachment.html>


More information about the riak-users mailing list