Data model for microblogging-like service

Alexander Sicular siculars at
Mon Nov 29 12:31:26 EST 2010


On Mon, Nov 29, 2010 at 12:15, Gleb Peregud <gleber.p at> wrote:
> Hello all
> I'm considering Riak as a datastore for a service, which has a part
> which looks like a microblog - i.e. each user will have his own feed
> of events, status updates, links and liked items from other feeds.
> Events on a feed are usually pretty small (avg. 500 bytes).
> What is the best way to store user's stream in Riak?
> Operations I'm most interested in are the following:
> - append to "top" of the feed

There is no top or bottom in riak. The closest thing would be adding
links to a key via your  client of choice. Depending on how that
client operates those links would prepend or append to the list of
links and will be returned in that order. Keep in mind that links,
which are part of the "header" info must be written with the value of
your key. Write your links without your value you lose your value.

> - get items from the feed with offset and limit (i.e. pagination)

There is no offset in riak. Limit could be implemented in a reduce
after a sort, perhaps.

> As far as I can see after reading documentation (one user -> multiple
> events) links are the way to deal with such task. Would this scale
> well? Service has to be able to serve tens of thousands of users with
> ever-growing feeds for each user. Deletions will be pretty seldom.

The consideration with links is how many per key. A few hundred or
even a thousand could work, more or less. But beyond that you need to
change your model. For instance a twitter user with 100k followers
would not translate well to one user with 100k links hanging off the
key. But a user with links to say their last 200 messages in a capped
collection, filo sort of way would work, managed by your application
of course.

My recommendation to everyone who asks is to implement redis as
middleware for key management and periodically persist those keys to
riak. Read the keys back (either redis or riak) and pass those keys to
a m/r to get your values back from riak.


More information about the riak-users mailing list