Riak packaging for RPM-based distros

Andrew Thompson andrew at hijacked.us
Wed Jan 9 14:38:18 EST 2013


To amplify Jared's reply a little...
 
The packaging choices we have made are with good reason. I'll try to
counter the most important points in your email below:
 
- Erlang VM upgrade. Erlang VM, used by Riak installed, can be upgraded
only with Riak code;
 
This is actually what we want. Each Riak release is tested with a
specific Erlang release, and on occasion we've shipped a patched VM to
work around bugs that have not yet been fixed in the upstream Erlang
distribution. There's also been cases where particular Erlang releases
have security bugs (eg. SSL validation code not being called, without
error).
 
Additionally, if Riak did force an exact Erlang version, that could hold
up other Erlang packages that use new Erlang VM features. At one point
Homebrew for OSX was having a lot of trouble because they could not
upgrade their Erlang package recipe without breaking Riak, but other
Homebrew packages would break *without* the Erlang update. Erlang itself
can sometimes introduce backwards and (forwards incompatible) changes
(eg. removing system stats, adding arguments that error on older
version, etc).
 
- integration with 3d party Erlang code. I mean filesystem hierarchy
usage
 
Riak explicitly discourages you running other Erlang code on a Riak
node. We recommend that you stand up a separate Erlang node and run your
custom code there. The exception to this rule is custom map/reduce code,
which has a well defined way of loading 3rd party Erlang code.
 
- this "release" uses its own location for Erlang applications;
 
Again, by design. We sometimes ship with our own forks of common Erlang
libraries (folsom, poolboy, protobuffs, etc) which may be incompatible
with upstream, or just include some fixes that we were not able to merge
upstream by the release window. If Riak was forced to use, for example,
the 'system' folsom, users could see performance degradation or Riak
crashes as a result. Also recall that Erlang lacks a decent way to
require a certain version of an application, unlike something like
rubygems.
 
- package each Riak-related Erlang application in separate RPM/DEB etc;
 
This is a LOT of work. We only have a single release manager and cutting
a riak release is already a non-trivial task. Having to create RPMs for
every single library that composes riak would only add complexity.
Again, this would also make it hard for ALL Erlang applications, because
of the inability to 'require' a certain version of an Erlang
application. For example, if 2 Erlang applications were installed, but
one depended on lager 1.2 and one on lager 2.0 (which are incompatible
versions), and lager was installed to /usr/lib/erlang/lib (or wherever),
one application would be unusuable. Erlang's 3rd party libraries are, in
general, in quite a bit of flux, and this kind of system wide dependency
management would be a nightmare.

Also, remember that we don't only support RPM and DEB based systems,
we'd have to create 30+ packages for Solaris, FreeBSD, SmartOS, etc as
well. So the added complexity is:

NumberOfApplications * NumberOfSupportedPlatforms

Which, in my quick estimation, is something like 150+ packages.

Andrew




More information about the riak-users mailing list