Looking for a replacement datastore

Parnell Springmeyer ixmatus at gmail.com
Fri Aug 17 14:28:44 EDT 2012


I use Riak at my company primarily for time series data; I quickly learned that key filters were a bad idea (when I designed our data model, I had the uid's of the objects in MySQL plus the timestamp of the data piece) and once I moved to Map/Reduce using Erlang module/functions, it dramatically improved response time and stability. It works really well for me IMHO.

OpenTSDB is also a great option; but we use Riak for quite a few other things INCLUDING MySQL and Redis - so I didn't want 4 DBs running around (3 is enough). Riak is working very well for these needs.

On Aug 17, 2012, at 8:01 AM, Alexander Zhuravlev wrote:

> On Thu, Aug 16, 2012 at 04:20:50PM -0600, Shawn Parrish wrote:
>> Howdy Riak folk,
>> 
>> We're looking for a possible datastore replacement for our server
>> monitoring check results.  Maybe some of you can offer feedback if
>> Riak is a possible good solution.
>> 
>> Each ping, http request, etc has a result with various metadata that
>> we store.  We're looking at about 250 million results a month and that
>> number continues to grow.
>> 
>> We query this data for:
>> 1. last result (is the server up or down?)
>> 2. if it's up, when was the last 'down' and inversely when it's down,
>> when was the last up?
>> 3. Full detail of the last 5 results (to show recent results)
>> 4. Last 24 hours results (usually ~1440 results) to graph
>> 5. Results in a date range (example: all results from July 1 through
>> July 31)... this can be very large.
>> 
>> We currently use bigcouch (Couchdb) but the views and built in
>> _all_docs slow down with so many results and especially when we call
>> them with 'include_docs', cause we need the details of the results as
>> well.
>> 
>> We're trying to trim down the total results stored by summarizing
>> older data and deleting it but that slows down Couchdb views even
>> farther.
>> 
>> Questions:
>> 1. Is Riak a possible datastore for this use case?  Can I get so many
>> results, including all the details quickly enough?
>> 2. Do you know of another datastore that might be better?
> 
> I would recommend to take a look at OpenTSDB (http://opentsdb.net/).
> -- 
> Alexander Zhuravlev
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





More information about the riak-users mailing list