A story about unexpected changes in riak-erlang-client dependencies

Yuri Lukyanov snaky at aboutecho.com
Tue Apr 9 10:49:50 EDT 2013


Hi,

To begin with, I almost fucked up production servers with these
changes in riak_pb:
https://github.com/basho/riak_pb/commit/2505ff1fa3975b93150d7445b6f7b91940ecb970

Well, this issue boils down to commonly known rebar-related problem. I
was even hesitating which mailing list I should've sent this massage
to, riak-users@ or rebar at .
The thing is that rebar does not allow you to fix dependencies on a
certain version.

When you do this in your rebar.config:

{deps, [
    {riakc, "1.3.1", {git,
"https://github.com/basho/riak-erlang-client.git", "c377347bb0"}},
]}.

you actually do _nothing_ to prevent you from unexpected changes to come.

Any dependency may have its own dependencies and so does riakc. And
guess what we see in riakc rebar.config in the commit "c377347bb0"?

Right:
{deps, [
        {riak_pb, ".*", {git, "git://github.com/basho/riak_pb", "master"}}
]}.

Welcome master.

And here is the welcoming message on your production servers when you
do hot code reloading:

  crasher:
    initial call: riakc_pb_socket:init/1
    pid: <0.3364.6566>
    registered_name: []
    exception exit:
{{{badrecord,rpbgetreq},[{riak_kv_pb,iolist,2,[{file,"src/riak_kv_pb.erl"},{line,48}]},{riak_kv_pb,encode,2,[{file,"src/r
iak_kv_pb.erl"},{line,40}]},{riak_pb_codec,encode,1,[{file,"src/riak_pb_codec.erl"},{line,77}]},{riakc_pb_socket,send_request,2,[{file,"src/r
iakc_pb_socket.erl"},{line,1280}]},{riakc_pb_socket,handle_call,3,[{file,"src/riakc_pb_socket.erl"},{line,759}]},{gen_server,handle_msg,5,[{f
ile,"gen_server.erl"},{line,588}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},[{gen_server,terminate,6,[{file,"gen_ser
ver.erl"},{line,747}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}


I see that now the situation is better and the last master version of
the riakc rebar.config looks like this:

{deps, [
        {riak_pb, "1.4.0.2", {git, "git://github.com/basho/riak_pb",
{tag, "1.4.0.2"}}}
]}.

That helps. But only a bit.
Let's look at the riak_pb rebar.config...
https://github.com/basho/riak_pb/blob/master/rebar.config

And voilà:
{deps, [
        {protobuffs, "0.8.*", {git,
"git://github.com/basho/erlang_protobuffs.git", "master"}}
]}.

New interesting things to come in the future. Just wait for it.


And if we go through any erlang project on github using rebar we will
see a lot of the same stuff. No one seems to worry about this much.

The question is why? Is this me only who facing those issues? It
doesn't look this. I see messages appearing again and again in rebar
mailing list and in rebar issues list on github about the same matter.
People invent hacks and patches to somehow improve the situation.
For example, the following rebar plugin recursively locks all deps by
creating a separate rebar.config:
https://github.com/seth/rebar_lock_deps_plugin
(and my fork of this: https://github.com/lukyanov/rebar-lock-deps)

But all of those attempts are just partial solutions. The real
solution would be to have a single centered dependencies repository,
like ruby has, for instance. While we, as Erlang community, still do
not have one, why don't we start _never_ using masters in our
rebar.config's and make this a habit?

I strongly urge Basho to be a good example of this, as you guys always do.

Thanks.




More information about the riak-users mailing list