Node cannot join

John E. Vincent lusis.org+riak-users at gmail.com
Tue Aug 21 10:23:05 EDT 2012


It might be worth looking at the Chef cookbook for how it does it. As
I see it on a fresh install with no data, there's probably not much
major risk in concurrent joins. On an existing install, however, I'd
think you would want to go serially.

We stand up all of our riak clusters from scratch using the chef
cookbook from Basho and it works fine but we do it serially with 3
nodes as the base. Haven't gotten to the point where we need to bump
up to 5 nodes (and thus not had to tackle the addition of those
nodes).

On Tue, Aug 21, 2012 at 10:17 AM, Shane McEwan <shane at mcewan.id.au> wrote:
> Speaking of joining clusters . . .
>
> I'm in the middle of writing a CFEngine promise to automatically install
> Riak, configure it and join a cluster.
>
> Is it best to have all nodes issue a 'riak-admin cluster join' command
> before a single node issues the final 'riak-admin cluster commit' command?
> (Potentially hard to do with CFEngine) Or is it OK to have each node issue
> its own 'join' and 'commit' commands? (Reasonably easy)
>
> Or is it a bad idea to build the cluster automatically and I should really
> be doing it manually?
>
> Secondly, is there a riak-admin command or status value that definitively
> says "This node is part of a cluster" (even if all other nodes in the
> cluster are down) so that CFEngine knows not to re-issue the cluster join
> command a second time?
>
> Thanks!
>
>
> On 21/08/12 14:36, Ryan Zezeski wrote:
>>
>> Daniel,
>>
>> Cluster operations (invoked by `riak-admin cluster`) are now
>> multi-phase.  Instead of calling an operation and it taking effect
>> immediately you need to stage your changes, view your changes, and then
>> commit them.  E.g.
>>
>> riak-admin cluster join blah
>>
>> riak-admin cluster plan
>>
>> riak-admin cluster commit
>>
>> There is an `-f` option to keep the old behavior but we recommend
>> against this.  Staged clustering was put in place to keep users from
>> hurting their clusters and to make multiple changes more efficient.
>>
>> -Z
>>
>> On Tue, Aug 21, 2012 at 9:28 AM, Daniel Iwan <iwan.daniel at gmail.com
>> <mailto:iwan.daniel at gmail.com>> wrote:
>>
>>     Hi
>>
>>     In my setup everything worked fine until I upgraded to riak 1.2
>>     (although this may be a coincidence)
>>     Nodes are installed from scratch with changes only to db backend (I'm
>>     using eLevelDB)
>>     and names.
>>     For some reason node cannot join to another.
>>     What am I doing wrong?
>>
>>     I'm using Ubuntu 10.04 but I've seen the same behaviour on Ubuntu
>> 12.04
>>     I may miss openssl dependency. I don't know if that matters
>>
>>
>>     user at node2:~$ riak-admin cluster join riak at 10.173.240.1
>>     <mailto:riak at 10.173.240.1>
>>
>>     Attempting to restart script through sudo -H -u riak
>>     Success: staged join request for 'riak at 10.173.240.2
>>     <mailto:riak at 10.173.240.2>' to 'riak at 10.173.240.1
>>     <mailto:riak at 10.173.240.1>'
>>
>>
>>
>>     user at node1:~$ riak-admin member-status
>>     Attempting to restart script through sudo -H -u riak
>>     ================================= Membership
>>     ==================================
>>     Status     Ring    Pending    Node
>>
>> -------------------------------------------------------------------------------
>>     joining     0.0%      --      'riak at 10.173.240.2
>>     <mailto:riak at 10.173.240.2>'
>>     valid     100.0%      --      'riak at 10.173.240.1
>>     <mailto:riak at 10.173.240.1>'
>>
>>
>> -------------------------------------------------------------------------------
>>
>>     Valid:1 / Leaving:0 / Exiting:0 / Joining:1 / Down:0
>>     user at node1:~$
>>     user at node1:~$ riak-admin transfers
>>
>>     Attempting to restart script through sudo -H -u riak
>>     No transfers active
>>
>>     Active Transfers:
>>
>>     user at node1:~$
>>
>>     riak-admin status
>>     Attempting to restart script through sudo -H -u riak
>>     1-minute stats for 'riak at 10.173.240.1 <mailto:riak at 10.173.240.1>'
>>
>>     -------------------------------------------
>>     vnode_gets : 0
>>     vnode_gets_total : 0
>>     vnode_puts : 0
>>     vnode_puts_total : 0
>>     vnode_index_reads : 0
>>     vnode_index_reads_total : 0
>>     vnode_index_writes : 0
>>     vnode_index_writes_total : 0
>>     vnode_index_writes_postings : 0
>>     vnode_index_writes_postings_total : 0
>>     vnode_index_deletes : 0
>>     vnode_index_deletes_total : 0
>>     vnode_index_deletes_postings : 0
>>     vnode_index_deletes_postings_total : 0
>>     node_gets : 0
>>     node_gets_total : 0
>>     node_get_fsm_siblings_mean : 0
>>     node_get_fsm_siblings_median : 0
>>     node_get_fsm_siblings_95 : 0
>>     node_get_fsm_siblings_99 : 0
>>     node_get_fsm_siblings_100 : 0
>>     node_get_fsm_objsize_mean : 0
>>     node_get_fsm_objsize_median : 0
>>     node_get_fsm_objsize_95 : 0
>>     node_get_fsm_objsize_99 : 0
>>     node_get_fsm_objsize_100 : 0
>>     node_get_fsm_time_mean : 0
>>     node_get_fsm_time_median : 0
>>     node_get_fsm_time_95 : 0
>>     node_get_fsm_time_99 : 0
>>     node_get_fsm_time_100 : 0
>>     node_puts : 0
>>     node_puts_total : 0
>>     node_put_fsm_time_mean : 0
>>     node_put_fsm_time_median : 0
>>     node_put_fsm_time_95 : 0
>>     node_put_fsm_time_99 : 0
>>     node_put_fsm_time_100 : 0
>>     read_repairs : 0
>>     read_repairs_total : 0
>>     coord_redirs_total : 0
>>     executing_mappers : 0
>>     precommit_fail : 0
>>     postcommit_fail : 0
>>     pbc_active : 0
>>     pbc_connects : 0
>>     pbc_connects_total : 0
>>     cpu_nprocs : 340
>>     cpu_avg1 : 182
>>     cpu_avg5 : 136
>>     cpu_avg15 : 92
>>     mem_total : 4154609664
>>     mem_allocated : 1475477504
>>     disk : [{"/",54417668,2},
>>              {"/dev",2024188,1},
>>              {"/dev/shm",2028616,0},
>>              {"/var/run",2028616,1},
>>              {"/var/lock",2028616,0},
>>              {"/lib/init/rw",2028616,0},
>>              {"/var/lib/ureadahead/debugfs",54417668,2},
>>              {"/media/ARRAY1",10739982336,1},
>>              {"/boot",186663,22}]
>>     nodename : 'riak at 10.173.240.1 <mailto:riak at 10.173.240.1>'
>>     connected_nodes : ['riak at 10.173.240.2 <mailto:riak at 10.173.240.2>']
>>
>>     sys_driver_version : <<"2.0">>
>>     sys_global_heaps_size : 0
>>     sys_heap_type : private
>>     sys_logical_processors : 4
>>     sys_otp_release : <<"R15B01">>
>>     sys_process_count : 1392
>>     sys_smp_support : true
>>     sys_system_version : <<"Erlang R15B01 (erts-5.9.1) [source] [64-bit]
>>     [smp:4:4] [async-threads:64] [kernel-poll:true]">>
>>     sys_system_architecture : <<"x86_64-unknown-linux-gnu">>
>>     sys_threads_enabled : true
>>     sys_thread_pool_size : 64
>>     sys_wordsize : 8
>>     ring_members : ['riak at 10.173.240.1
>>     <mailto:riak at 10.173.240.1>','riak at 10.173.240.2
>>     <mailto:riak at 10.173.240.2>']
>>     ring_num_partitions : 64
>>     ring_ownership : <<"[{'riak at 10.173.240.1
>>     <mailto:riak at 10.173.240.1>',64}]">>
>>
>>     ring_creation_size : 64
>>     storage_backend : riak_kv_eleveldb_backend
>>     erlydtl_version : <<"0.7.0">>
>>     riak_control_version : <<"1.2.0">>
>>     cluster_info_version : <<"1.2.2">>
>>     riak_api_version : <<"1.2.0">>
>>     riak_search_version : <<"1.2.0">>
>>     merge_index_version : <<"1.2.0">>
>>     riak_kv_version : <<"1.2.0">>
>>     riak_pipe_version : <<"1.2.0">>
>>     riak_core_version : <<"1.2.0">>
>>     lager_version : <<"1.2.0">>
>>     syntax_tools_version : <<"1.6.8">>
>>     compiler_version : <<"4.8.1">>
>>     bitcask_version : <<"1.5.1">>
>>     basho_stats_version : <<"1.0.2">>
>>     luke_version : <<"0.2.5">>
>>     webmachine_version : <<"1.9.2">>
>>     mochiweb_version : <<"1.5.1">>
>>     inets_version : <<"5.9">>
>>     erlang_js_version : <<"1.2.0">>
>>     runtime_tools_version : <<"1.8.8">>
>>     os_mon_version : <<"2.2.9">>
>>     riak_sysmon_version : <<"1.1.2">>
>>     ssl_version : <<"5.0.1">>
>>     public_key_version : <<"0.15">>
>>     crypto_version : <<"2.1">>
>>     sasl_version : <<"2.2.1">>
>>     stdlib_version : <<"1.18.1">>
>>     kernel_version : <<"2.15.1">>
>>     memory_total : 26853928
>>     memory_processes : 6744452
>>     memory_processes_used : 6744438
>>     memory_system : 20109476
>>     memory_atom : 429569
>>     memory_atom_used : 406875
>>     memory_binary : 58504
>>     memory_code : 9277444
>>     memory_ets : 774080
>>
>>
>>     D.
>>
>>     _______________________________________________
>>     riak-users mailing list
>>     riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list