Riak packaging for RPM-based distros

Jared Morrow jared at basho.com
Wed Jan 9 14:02:22 EST 2013


First off, thanks for the email and suggestions.  Also thanks for working
on RPM's for OpenSUSE and providing your repo.  I'll try to hit each topic
in the rough order you mentioned them.

Let me comment first on our packaging of Erlang with Riak.  I agree, this
is not typically how distributions work.  In our case, making a distributed
database that people depend on to be stable and performant also requires us
to control the Erlang VM we use.  In a particular case, we've been fighting
Erlang scheduler issues in regards to our use of NIF's that has changed
behavior between versions of Erlang.  Because of this issue, we haven't
moved forward on using new versions past R15B02 of erlang.  Because we are
responsible for any problems with Riak, I do not see anytime in the future
we will give up control of the Erlang we use at the expense of a slightly
larger rpm / deb / tgz size.  That was just one example, another is the use
of HiPE or no HiPE where we've found segfaults when using HiPE on SunOS or
*BSD due to fixed memory buffer sizes.  So, this is a lot to say that I
agree with you it seems like we are doing the wrong thing by including
Erlang with Riak, but I'm also saying that what we are doing is the right
thing in regards to stability, performance, and the customer experience as
a whole.

Regarding packaging components as separate RPM's.  This is another thing
that distributions have tried to get us to do, make a riak_core, riak_kv,
erlang_js, etc., etc. rpm.  Again, I see no benefit at all to people who
use riak.  This is a benefit solely to fit into packaging guidelines made
by distributions.  When testing such a complicated system such as Riak,
breaking it up into 30 pieces would simply make testing harder and the
product less stable.  Unless you are writing a riak_core app, who would
just *just* want a riak_core RPM anyway?  Yes it might help a few people
here and there, but for those people the source *is* available and it isn't
worth sacrificing the experience of the other 99% of users.  Right now if
someone has a problem, I can say, "tell me what ./riak version says" and
know exactly what is running.  If they have 30 RPM's installed that makes
that much much more difficult.  So again, theoretically I agree we seem to
be doing it wrong, but I'd argue distributions have that completely
backwards prioritizing the ability to customize every install at the
expense of a cohesive and stable system.

- honor filesystem hierarchy of underlying OS.

This one I'd like to hear more about.  I've made some changes along the way
for systems I didn't know as much about at first pass (mainly SunOS
flavors), but I know for a fact we haven't got everything right.  If you
have some examples of where we are doing it wrong in regards to our
install, please let me know or file a bug at
https://github.com/basho/riak  This is especially true for any new
guidelines that have come out
recently that we have missed.

So again, my answer is probably not what you hoped for, but I do appreciate
the criticism.  Please keep it coming and argue with me if you think we're
still doing it wrong after my justifications above.


On Wed, Jan 9, 2013 at 5:53 AM, Aleksey Morarash <tuxofil at gmail.com> wrote:

> Hi All!
> Someone can possibly perceive the states below as a heresy, so, first of
> all, I want to declare: I have no intention to start new holywar here, I
> just propose a solution.
> I have noticed that packages available at
> http://docs.basho.com/riak/latest/downloads/ includes a kind of so-called
> Erlang "release". This release includes all Riak-related Erlang
> applications and Erlang VM in one package.
> Using such package produces several problems:
> - Erlang VM upgrade. Erlang VM, used by Riak installed, can be upgraded
> only with Riak code;
> - integration with 3d party Erlang code. I mean filesystem hierarchy usage
> - this "release" uses its own location for Erlang applications;
> - and again: integration with 3d party Erlang code on Erlang VM level - to
> include custom functionality I must patch existing "release".
> I know that such a "release" is generated by rebar, which used to build
> almost all Riak-related Erlang applications and libraries.
> My proposal is to make Riak packaging more system-friendly:
> - use Erlang OTP, provided by target distribution;
> - package each Riak-related Erlang application in separate RPM/DEB etc;
> - honor filesystem hierarchy of underlying OS.
> This includes (for each distro):
> - create a bundle of package specs;
> - create a package repo, based on packages built.
> I have already packaged most of Riak-related Erlang applications and
> libraries for OpenSUSE 11.4. RPM package repository with packages generated
> we already use in production. To package some applications I had replace
> usual "rebar compile" recipes, because there is no way to disable recursive
> dependency fetch from remote GIT repositories. All these packages was
> successfully built in chrooted clean environment which ensures RPM
> repository consistency.
> All these RPM specs are available at forked
> https://github.com/strikead/riak repo:
> https://github.com/strikead/riak/tree/master/pkg.d/opensuse/specs
> P.S. Sorry, there is no possibility to public use of our RPM repo at all :(
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130109/2c633e25/attachment.html>

More information about the riak-users mailing list