riak as log storage

Kresten Krab Thorup krab at trifork.com
Thu Apr 14 10:09:38 EDT 2011


1. How much data do you anticipate?
2. Linking to the previous as you suggest really limits the log because it can then only write one thing at a time, which would be a pity for a Riak :-)  I'd go for a model that does not imply such constraints.

I had some experience recently you can read about here: http://markmail.org/message/sbwueb5qyj3gk4i5?q=riak-users+kresten+krab+thorup+log

... of which the highlights are:

- It may be a good idea to use a "YYYYMMDDHHSS:<random>" as your keys, because then you can key-filter on time-related log entries; 
- If you have *alot* of data, use innostore, and the above key-scheme will make innostore almost as fast as bitcask (well 2/3 speed, YMMW) because of data locality.

You could also use a bucket for some prefix of the timestamp, which may make searching faster for some backends.

Good luck with your project.

Kresten

On Apr 14, 2011, at 10:48 , Alessandro Testa wrote:

> hi guys,
> i'm try to design a log storage based on riak.
> the requiment are:
> 1 the system needs to be able to store any number of these log event
> 2 the system needs to be able to apply a label to some of these event
> 3 the user can access the most recent log, and then can go back in time (cannot 
> decide to start in the middle of the list or look for a specific event)
> 4 the user can filter the log based on the label and access the filtered list in 
> with the same parameter as point 3
> 
> my idea is to store each event as a json document, the key of the document will 
> be a uuid (type1). each document will be linked to the previous one, so i can 
> follow the link to view the previus log.
> 
> i want to add also other links that point to the previous event marked with the 
> same label.
> 
> i hope i explain enough the problem and the design.
> 
> now my questions:
> - do you think this is a good architecture?
> - i read about luwak, do you think it can be used in a scenario like this?
> 
> thank, i really hope to get some feedback from you!
> Alex
> 
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110414/039bccbf/attachment.html>


More information about the riak-users mailing list