native erlang client on non-Riak node?

Alexander Sicular siculars at gmail.com
Thu Apr 5 15:52:45 EDT 2012


I think that being able to have a native erlang interface to a riak cluster would be a "Good Thing." I can imagine some compute applications that could benefit from having access to data at the speed of network but not actually be a member of the storage pool.

Just watched Rich Hickey discussing similar concepts employed in his new system, Datomic, here, http://www.infoq.com/interviews/hickey-datomic . Worth the 30min.

-Alexander Sicular

@siculars

On Apr 5, 2012, at 3:30 PM, Ryan Zezeski wrote:

> Mike,
> 
> The native Erlang client was never designed to be used in this fashion, as you are finding out first hand.  You are correct in that some operations assume they are running on the Riak VM, e.g. mapred_dynamic_inputs_stream.
> 
> Normally I would say not to establish a Erlang dist connection to the Riak cluster but given you need a work around for the performance issue it seems to be your only good choice ATM.  You can use `rpc:call` to invoke the mapred queries.
> 
> -Z
> 
> On Wed, Apr 4, 2012 at 2:40 PM, Michael Radford <mrad at blorf.com> wrote:
> In case anyone else is wondering, the answer is no, the native client
> isn't fully usable from a non-Riak node.
> 
> I started a hidden node and connected to one of my Riak nodes using
> riak:client_connect. The returned client is able to successfully
> execute Client:get(Bucket, Key) (and probably other simple things like
> put/delete), but it fails on mapred with a {modfun, ...} input. 
> 
> The problem is that the native client tries to run the input function
> on the local node, rather than the node the client points to, which of
> course won't work unless the local node is participating in the Riak
> cluster.
> 
> Mike
> 
> On Tue, Apr 3, 2012 at 9:43 AM, Michael Radford <mrad at blorf.com> wrote:
> > Is it OK to use Riak's native erlang client from an erlang node that's
> > connected to a Riak cluster via normal erlang distribution
> > (net_adm:ping etc.), but isn't participating in the Riak cluster?  I'm
> > wondering if e.g. Riak makes any assumptions about nodes listed in
> > nodes(), or the nodes of processes running client code, that would
> > break if a node isn't running Riak.
> >
> > I'm trying to figure out a fallback strategy in case I can't resolve
> > performance issues I'm seeing with the protocol buffers client (see
> > other thread about slow searches).  The scariest option would be to
> > run my application on the same nodes as Riak, which seems like it
> > would complicate operations too much.  Or if
> > connected-but-not-participating nodes aren't a good idea, maybe my
> > application nodes could each start a slave node with -hidden, which
> > would then connect to Riak and act as a proxy?
> >
> > Mike
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120405/0cc3fcb6/attachment.html>


More information about the riak-users mailing list