1. Riak Crashing on Data Load from CSV and 2. How to get Ripple to load js functions

Gustav Gennrich ggennrich at gmail.com
Fri May 18 12:03:59 EDT 2012


Hi,

Below is our setup to better diagnose:

Ring creation size: 128
Backend: LevelDB

All other settings in app.config are using the default values

We have 3 servers (nodes) each of which have 8GB of ram.

Question 1: We are attempting to load data into Riak using CSVs that we
generated from data previously stored is our MySQL database. We are using
Ripple to create the documents and associations (links). The problem we are
having is Riak continue to grow in memory consumption until it finally
crashed while loading this data. We are only able to add around a couple
hundred thousand rows from the CSV's before this happens. Riak then
responds with 500 errors and eventually all of the nodes crash and the
nodes need to be restarted.

Question 2 (unrelated): I am attempting to use some of the already defined
js functions in ripple.js in a mapreduce, however, I keep getting the
following error:

Riak::MapReduceError:
{"phase":0,"error":"[{<<\"lineno\">>,1},{<<\"message\">>,<<\"ReferenceError:
Ripple is not defined\">>},{<<\"source\">>,<<\"unknown\">>}]"

In ripple.yml I have the following (***** represent hidden values):

# Configure Riak connections for the Ripple library.
development:
  # http_port: 8091
  # pb_port: 8087
  # host: 127.0.0.1
  basic_auth: *****
  http_port: 80
  pb_port: 8087
  host: *****
  http_backend: Excon
  solr: "/solr"
  js_source_dir: <%= Rails.root + "app/mapreduce" %>

# The test environment has additional keys for configuring the
# Riak::TestServer for your test/spec suite:
#
# * bin_dir specifies the path to the "riak" script that you use to
#           start Riak (just the directory)
# * js_source_dir specifies where your custom Javascript functions for
#           MapReduce should be loaded from. Usually app/mapreduce.
test:
  http_port: 9000
  pb_port: 9002
  host: localhost
  source: /usr/local/bin   # Default for Homebrew.
  js_source_dir: <%= Rails.root + "/app/mapreduce" %>

production:
  nodes:
  - host: *****
    http_port: 8098
    pb_port: 8087
  - host: *****
    http_port: 8098
    pb_port: 8087
  - host: *****
    http_port: 8098
    pb_port: 8087
  http_backend: Excon
  solr: "/solr"
  js_source_dir: <%= Rails.root + "app/mapreduce" %>

I don't see why I am getting this error when I have defined where the
js_source_dir is.

Any help would be greatly appreciated!

Thanks,

Gustav
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20120518/e0feaece/attachment.html>


More information about the riak-users mailing list