native erlang client on non-Riak node?

John E. Vincent lusis.org+riak-users at gmail.com
Thu Apr 5 16:12:58 EDT 2012


ElasticSearch has this feature as well.

When using the Java transport client (as opposed to the HTTP/JSON
interface), you actually become a cluster member as a non-data node.
You can also add any number of nodes to the cluster as non-data nodes
that then participate in general processing.

http://www.elasticsearch.org/guide/reference/modules/node.html
http://www.elasticsearch.org/guide/reference/java-api/client.html

On Thu, Apr 5, 2012 at 3:52 PM, Alexander Sicular <siculars at gmail.com> wrote:
> I think that being able to have a native erlang interface to a riak cluster
> would be a "Good Thing." I can imagine some compute applications that could
> benefit from having access to data at the speed of network but not actually
> be a member of the storage pool.
>
> Just watched Rich Hickey discussing similar concepts employed in his new
> system, Datomic, here, http://www.infoq.com/interviews/hickey-datomic .
> Worth the 30min.
>
> -Alexander Sicular
>
> @siculars
>
> On Apr 5, 2012, at 3:30 PM, Ryan Zezeski wrote:
>
> Mike,
>
> The native Erlang client was never designed to be used in this fashion, as
> you are finding out first hand.  You are correct in that some operations
> assume they are running on the Riak VM, e.g. mapred_dynamic_inputs_stream.
>
> Normally I would say not to establish a Erlang dist connection to the Riak
> cluster but given you need a work around for the performance issue it seems
> to be your only good choice ATM.  You can use `rpc:call` to invoke the
> mapred queries.
>
> -Z
>
> On Wed, Apr 4, 2012 at 2:40 PM, Michael Radford <mrad at blorf.com> wrote:
>>
>> In case anyone else is wondering, the answer is no, the native client
>> isn't fully usable from a non-Riak node.
>>
>> I started a hidden node and connected to one of my Riak nodes using
>> riak:client_connect. The returned client is able to successfully
>> execute Client:get(Bucket, Key) (and probably other simple things like
>> put/delete), but it fails on mapred with a {modfun, ...} input.
>>
>>
>> The problem is that the native client tries to run the input function
>> on the local node, rather than the node the client points to, which of
>> course won't work unless the local node is participating in the Riak
>> cluster.
>>
>> Mike
>>
>> On Tue, Apr 3, 2012 at 9:43 AM, Michael Radford <mrad at blorf.com> wrote:
>> > Is it OK to use Riak's native erlang client from an erlang node that's
>> > connected to a Riak cluster via normal erlang distribution
>> > (net_adm:ping etc.), but isn't participating in the Riak cluster?  I'm
>> > wondering if e.g. Riak makes any assumptions about nodes listed in
>> > nodes(), or the nodes of processes running client code, that would
>> > break if a node isn't running Riak.
>> >
>> > I'm trying to figure out a fallback strategy in case I can't resolve
>> > performance issues I'm seeing with the protocol buffers client (see
>> > other thread about slow searches).  The scariest option would be to
>> > run my application on the same nodes as Riak, which seems like it
>> > would complicate operations too much.  Or if
>> > connected-but-not-participating nodes aren't a good idea, maybe my
>> > application nodes could each start a slave node with -hidden, which
>> > would then connect to Riak and act as a proxy?
>> >
>> > Mike
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list