repair-2i stops with "bad argument in call to eleveldb:async_write"

Effenberg, Simon seffenberg at team.mobile.de
Tue Jul 29 08:06:08 EDT 2014


Said to say but the issue stays the same.. even after the upgrade to
1.4.10.

Any ideas what is happening here?

Cheers
Simon

On Tue, Jul 29, 2014 at 08:46:42AM +0000, Effenberg, Simon wrote:
> Already started to prepare everything for it.. :)
> 
> On Tue, Jul 29, 2014 at 09:43:22AM +0100, Guido Medina wrote:
> > Hi Simon,
> > 
> > There are some (maybe related) Level DB fixes in 1.4.9 and 1.4.10, I don't
> > think there isn't any harm for you to do a rolling upgrade since nothing
> > major changed, just bug fixes, here is the release notes' link for
> > reference:
> > 
> > https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md
> > 
> > Best regards,
> > 
> > Guido.
> > 
> > On 29/07/14 09:35, Effenberg, Simon wrote:
> > >Hi,
> > >
> > >we have some issues with 2i queries like that:
> > >
> > >seffenberg at kriak46-1:~$ while :; do curl -s localhost:8098/buckets/conversation/index/createdat_int/0/23182680 | ruby -rjson -e "o = JSON.parse(STDIN.read); puts o['keys'].size"; sleep 1; done
> > >
> > >13853
> > >13853
> > >0
> > >557
> > >557
> > >557
> > >13853
> > >0
> > >
> > >
> > >...
> > >
> > >So I tried to start a repair-2i first on one vnode/partition on one node
> > >(which is quiet new in the cluster.. 2 weeks or so).
> > >
> > >The command is failing with the following log entries:
> > >
> > >seffenberg at kriak46-7:~$ sudo riak-admin repair-2i 22835963083295358096932575511191922182123945984
> > >Will repair 2i on these partitions:
> > >         22835963083295358096932575511191922182123945984
> > >Watch the logs for 2i repair progress reports
> > >seffenberg at kriak46-7:~$ 2014-07-29 08:20:22.729 UTC [info] <0.5929.1061>@riak_kv_2i_aae:init:139 Starting 2i repair at speed 100 for partitions [22835963083295358096932575511191922182123945984]
> > >2014-07-29 08:20:22.729 UTC [info] <0.5930.1061>@riak_kv_2i_aae:repair_partition:257 Acquired lock on partition 22835963083295358096932575511191922182123945984
> > >2014-07-29 08:20:22.729 UTC [info] <0.5930.1061>@riak_kv_2i_aae:repair_partition:259 Repairing indexes in partition 22835963083295358096932575511191922182123945984
> > >2014-07-29 08:20:22.740 UTC [info] <0.5930.1061>@riak_kv_2i_aae:create_index_data_db:324 Creating temporary database of 2i data in /var/lib/riak/anti_entropy/2i/tmp_db
> > >2014-07-29 08:20:22.751 UTC [info] <0.5930.1061>@riak_kv_2i_aae:create_index_data_db:361 Grabbing all index data for partition 22835963083295358096932575511191922182123945984
> > >2014-07-29 08:25:22.752 UTC [info] <0.5929.1061>@riak_kv_2i_aae:next_partition:160 Finished 2i repair:
> > >         Total partitions: 1
> > >         Finished partitions: 1
> > >         Speed: 100
> > >         Total 2i items scanned: 0
> > >         Total tree objects: 0
> > >         Total objects fixed: 0
> > >With errors:
> > >Partition: 22835963083295358096932575511191922182123945984
> > >Error: index_scan_timeout
> > >
> > >
> > >2014-07-29 08:25:22.752 UTC [error] <0.4711.1061> gen_server <0.4711.1061> terminated with reason: bad argument in call to eleveldb:async_write(#Ref<0.0.10120.211816>, <<>>, [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}], []) in eleveldb:write/3 line 155
> > >2014-07-29 08:25:22.753 UTC [error] <0.4711.1061> CRASH REPORT Process <0.4711.1061> with 0 neighbours exited with reason: bad argument in call to eleveldb:async_write(#Ref<0.0.10120.211816>, <<>>, [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}], []) in eleveldb:write/3 line 155 in gen_server:terminate/6 line 747
> > >2014-07-29 08:25:22.753 UTC [error] <0.1031.0> Supervisor {<0.1031.0>,poolboy_sup} had child riak_core_vnode_worker started with {riak_core_vnode_worker,start_link,undefined} at <0.4711.1061> exit with reason bad argument in call to eleveldb:async_write(#Ref<0.0.10120.211816>, <<>>, [{put,<<131,104,2,109,0,0,0,20,99,111,110,118,101,114,115,97,116,105,111,110,95,115,101,99,114,...>>,...}], []) in eleveldb:write/3 line 155 in context child_terminated
> > >
> > >
> > >Anything I can do about that? What's the issue here?
> > >
> > >I'm using Riak 1.4.8 (.deb package).
> > >
> > >Cheers
> > >Simon
> > >_______________________________________________
> > >riak-users mailing list
> > >riak-users at lists.basho.com
> > >http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > 
> > 
> > _______________________________________________
> > riak-users mailing list
> > riak-users at lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> -- 
> Simon Effenberg | Site Op | mobile.international GmbH
> 
> Phone:    + 49. 30. 8109. 7173
> M-Phone:  + 49. 151. 5266. 1558
> Mail:     seffenberg at team.mobile.de
> Web:      www.mobile.de
> 
> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> 
> ______________________________________________________
> Geschäftsführer: Malte Krüger
> HRB Nr.: 18517 P, Amtsgericht Potsdam
> Sitz der Gesellschaft: Kleinmachnow
> ______________________________________________________
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-- 
Simon Effenberg | Site Op | mobile.international GmbH

Phone:    + 49. 30. 8109. 7173
M-Phone:  + 49. 151. 5266. 1558
Mail:     seffenberg at team.mobile.de
Web:      www.mobile.de

Marktplatz 1 | 14532 Europarc Dreilinden | Germany

______________________________________________________
Geschäftsführer: Malte Krüger
HRB Nr.: 18517 P, Amtsgericht Potsdam
Sitz der Gesellschaft: Kleinmachnow
______________________________________________________




More information about the riak-users mailing list