Why does sending X-Riak-ClientId prevent subsequent updates?

Jay Doane jay.s.doane at gmail.com
Sun Feb 7 02:13:55 EST 2010


On Feb 6, 2010, at 5:18 PM, Rusty Klophaus wrote:

> Interesting. While investigating this, I get different output for  
> each store statement:
>
> >>> client.store('b', 'k', {'foo':0})
> {'object': {'foo': 0}, '__riak_vclock__':  
> 'a85hYGBgzGDKBVIsDL6y6zOYEhnzWBls2x8d4csCAA=='}

That's because there's a bug in the header handling when using pycurl  
which is causing my __riak_vclock__ to not get set.  I've attached a  
patch, which also incidentally moves the response.reset() into the  
_pycurl_request method, for improved encapsulation.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: riak.diff
Type: application/octet-stream
Size: 789 bytes
Desc: not available
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20100206/03770d04/attachment.diff>
-------------- next part --------------



> In the end though, I still see an object with foo=1 rather than  
> foo=2. I imagine what we're both seeing is that the foo=1 update is  
> still running when the foo=2 update starts. The default quorum  
> settings in the Python client _should_ prevent this, so I'm going to  
> continue investigating, and I'll let you know what I find.

Great!

> In the meantime, can you answer some questions for me?
> - Can you double check that you are running a fresh 0.8 install? Run  
> an 'hg tip', you should be at revision 839, and maybe do a make  
> clean/make compile for good measure.

I did a fresh clone/make rel yesterday.

$ hg tip
changeset:   840:c4486329e4af
tag:         tip
user:        justin at basho.com
date:        Wed Feb 03 15:51:26 2010 -0500
summary:     Added tag riak-0.8 for changeset 65b66ac0fc83

> - Are you running these tests on a single machine or in a cluster?

Single machine.

> - Can you run "io:format("~p", [riak_ring_manager:get_my_ring()])."  
> in the Riak console and send me the output? (I'd like to double  
> check that there are no ghost nodes in the ring.)

(riak at 127.0.0.1)1> io:format("~p", [riak_ring_manager:get_my_ring()]).
{ok,{chstate,'riak at 127.0.0.1',[],
              {64,
               [{0,'riak at 127.0.0.1'},
                {22835963083295358096932575511191922182123945984,
                 'riak at 127.0.0.1'},
                {45671926166590716193865151022383844364247891968,
                 'riak at 127.0.0.1'},
                {68507889249886074290797726533575766546371837952,
                 'riak at 127.0.0.1'},
                {91343852333181432387730302044767688728495783936,
                 'riak at 127.0.0.1'},
                {114179815416476790484662877555959610910619729920,
                 'riak at 127.0.0.1'},
                {137015778499772148581595453067151533092743675904,
                 'riak at 127.0.0.1'},
                {159851741583067506678528028578343455274867621888,
                 'riak at 127.0.0.1'},
                {182687704666362864775460604089535377456991567872,
                 'riak at 127.0.0.1'},
                {205523667749658222872393179600727299639115513856,
                 'riak at 127.0.0.1'},
                {228359630832953580969325755111919221821239459840,
                 'riak at 127.0.0.1'},
                {251195593916248939066258330623111144003363405824,
                 'riak at 127.0.0.1'},
                {274031556999544297163190906134303066185487351808,
                 'riak at 127.0.0.1'},
                {296867520082839655260123481645494988367611297792,
                 'riak at 127.0.0.1'},
                {319703483166135013357056057156686910549735243776,
                 'riak at 127.0.0.1'},
                {342539446249430371453988632667878832731859189760,
                 'riak at 127.0.0.1'},
                {365375409332725729550921208179070754913983135744,
                 'riak at 127.0.0.1'},
                {388211372416021087647853783690262677096107081728,
                 'riak at 127.0.0.1'},
                {411047335499316445744786359201454599278231027712,
                 'riak at 127.0.0.1'},
                {433883298582611803841718934712646521460354973696,
                 'riak at 127.0.0.1'},
                {456719261665907161938651510223838443642478919680,
                 'riak at 127.0.0.1'},
                {479555224749202520035584085735030365824602865664,
                 'riak at 127.0.0.1'},
                {502391187832497878132516661246222288006726811648,
                 'riak at 127.0.0.1'},
                {525227150915793236229449236757414210188850757632,
                 'riak at 127.0.0.1'},
                {548063113999088594326381812268606132370974703616,
                 'riak at 127.0.0.1'},
                {570899077082383952423314387779798054553098649600,
                 'riak at 127.0.0.1'},
                {593735040165679310520246963290989976735222595584,
                 'riak at 127.0.0.1'},
                {616571003248974668617179538802181898917346541568,
                 'riak at 127.0.0.1'},
                {639406966332270026714112114313373821099470487552,
                 'riak at 127.0.0.1'},
                {662242929415565384811044689824565743281594433536,
                 'riak at 127.0.0.1'},
                {685078892498860742907977265335757665463718379520,
                 'riak at 127.0.0.1'},
                {707914855582156101004909840846949587645842325504,
                 'riak at 127.0.0.1'},
                {730750818665451459101842416358141509827966271488,
                 'riak at 127.0.0.1'},
                {753586781748746817198774991869333432010090217472,
                 'riak at 127.0.0.1'},
                {776422744832042175295707567380525354192214163456,
                 'riak at 127.0.0.1'},
                {799258707915337533392640142891717276374338109440,
                 'riak at 127.0.0.1'},
                {822094670998632891489572718402909198556462055424,
                 'riak at 127.0.0.1'},
                {844930634081928249586505293914101120738586001408,
                 'riak at 127.0.0.1'},
                {867766597165223607683437869425293042920709947392,
                 'riak at 127.0.0.1'},
                {890602560248518965780370444936484965102833893376,
                 'riak at 127.0.0.1'},
                {913438523331814323877303020447676887284957839360,
                 'riak at 127.0.0.1'},
                {936274486415109681974235595958868809467081785344,
                 'riak at 127.0.0.1'},
                {959110449498405040071168171470060731649205731328,
                 'riak at 127.0.0.1'},
                {981946412581700398168100746981252653831329677312,
                 'riak at 127.0.0.1'},
                {1004782375664995756265033322492444576013453623296,
                 'riak at 127.0.0.1'},
                {1027618338748291114361965898003636498195577569280,
                 'riak at 127.0.0.1'},
                {1050454301831586472458898473514828420377701515264,
                 'riak at 127.0.0.1'},
                {1073290264914881830555831049026020342559825461248,
                 'riak at 127.0.0.1'},
                {1096126227998177188652763624537212264741949407232,
                 'riak at 127.0.0.1'},
                {1118962191081472546749696200048404186924073353216,
                 'riak at 127.0.0.1'},
                {1141798154164767904846628775559596109106197299200,
                 'riak at 127.0.0.1'},
                {1164634117248063262943561351070788031288321245184,
                 'riak at 127.0.0.1'},
                {1187470080331358621040493926581979953470445191168,
                 'riak at 127.0.0.1'},
                {1210306043414653979137426502093171875652569137152,
                 'riak at 127.0.0.1'},
                {1233142006497949337234359077604363797834693083136,
                 'riak at 127.0.0.1'},
                {1255977969581244695331291653115555720016817029120,
                 'riak at 127.0.0.1'},
                {1278813932664540053428224228626747642198940975104,
                 'riak at 127.0.0.1'},
                {1301649895747835411525156804137939564381064921088,
                 'riak at 127.0.0.1'},
                {1324485858831130769622089379649131486563188867072,
                 'riak at 127.0.0.1'},
                {1347321821914426127719021955160323408745312813056,
                 'riak at 127.0.0.1'},
                {1370157784997721485815954530671515330927436759040,
                 'riak at 127.0.0.1'},
                {1392993748081016843912887106182707253109560705024,
                 'riak at 127.0.0.1'},
                {1415829711164312202009819681693899175291684651008,
                 'riak at 127.0.0.1'},
                {1438665674247607560106752257205091097473808596992,
                 'riak at 127.0.0.1'}]},
              {dict,0,16,16,8,80,48,
                    {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
                    {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[], 
[]}}}}}ok

> Also, side note, you would normally want to re-use an object across  
> updates, which would prevent this problem, like this:
>
> obj = client.store('b', 'k', {'foo':0})
> obj["object"]["foo"] = 1
> client.store('b', 'k', obj)
> obj["object"]["foo"] = 2
> client.store('b', 'k', obj)

Good to know, although this apparently requires the __riak_vclock__  
property being set in order to work (see patch above).

Thanks,
Jay

> Best,
> Rusty
>
> On Sat, Feb 6, 2010 at 6:48 PM, Jay Doane <jay.s.doane at gmail.com>  
> wrote:
> I'm using the new (raw) riak.py client library, and testing multiple  
> updates to the same object with the following doctest file (also  
> attached):
>
> $ cat riak_test.py
>
> """
> >>> import riak
> >>> client = riak.Riak('127.0.0.1', 8098)
> >>> client.delete('b', 'k')
> >>> client.store('b', 'k', {'foo':0})
> {'object': {u'foo': 0}}
> >>> client.store('b', 'k', {'foo':1})
> {'object': {u'foo': 1}}
> >>> client.store('b', 'k', {'foo':2})
> {'object': {u'foo': 2}}
> """
>
> import doctest
> doctest.testmod()
>
>
> When run against the 0.8 release, the final store operation always  
> fails:
>
> $ python riak_test.py
> **********************************************************************
> File "riak_test.py", line 9, in __main__
> Failed example:
>    client.store('b', 'k', {'foo':2})
> Expected:
>    {'object': {u'foo': 2}}
> Got:
>    {'object': {u'foo': 1}}
> **********************************************************************
> 1 items had failures:
>   1 of   6 in __main__
> ***Test Failed*** 1 failures.
>
>
> Usually, that's all there is to the failure, but sometimes,  
> unpredictably, I see the following on the riak console:
>
> =ERROR REPORT==== 6-Feb-2010::14:53:48 ===
> webmachine error: path="/raw/b/k"
> {error,
>    {error,function_clause,
>        [{raw_http_resource,select_doc,
>             [{ctx,<<"b">>,<<"k">>,
>                  {riak_client,'riak at 127.0.0.1',<<1,11,16,235>>},
>                  2,2,2,2,"raw",local,
>                  {error,notfound},
>                  undefined,undefined,[]}]},
>         {raw_http_resource,produce_doc_body,2},
>         {raw_http_resource,accept_doc_body,2},
>         {webmachine_resource,resource_call,3},
>         {webmachine_resource,do,3},
>         {webmachine_decision_core,resource_call,1},
>         {webmachine_decision_core,accept_helper,0},
>         {webmachine_decision_core,decision,1}]}}
>
>
> However, if I alter riak.py so that it doesn't send the X- 
> Riak_ClientId header, the tests all pass.  Here's the trivial diff  
> (also attached)
>
> diff -r c4486329e4af client_lib/riak.py
> --- a/client_lib/riak.py        Wed Feb 03 15:51:26 2010 -0500
> +++ b/client_lib/riak.py        Sat Feb 06 15:31:36 2010 -0800
> @@ -135,8 +135,7 @@
>
>     @expect(200)
>     def store(self, bucket, key, obj, links=[], w=2, dw=2):
> -        uphead = {'Content-Type': 'application/json',
> -                  'X-Riak-ClientId': self.clientid}
> +        uphead = {'Content-Type': 'application/json'}
>         try:
>             uphead['X-Riak-Vclock'] = obj['__riak_vclock__']
>         except KeyError:
>
>
> Any ideas what's going on?
>
> Thanks,
> Jay
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>



More information about the riak-users mailing list