Nightly Prune

Stephan Kepser stephan.kepser at codecentric.de
Fri Feb 8 03:36:32 EST 2013


Hi Chad,

yes, you're right: the use of secondary indexes requires LevelDB  as
backend. How much of an performance penalty this imposes to you I really
don't know. I'd still consider the use of secondary indexes because it is
an architecturally clean solution and hence simple to implement and
maintain. If you experience performance issues you still have the option to
scale out, i.e, use more nodes.

There may yet be another option to speed up write operations, depending on
your demands. You could use the memory backend for about a third of your
nodes and set the write quorum to 1. This turns the memory-only nodes into
something like a cache without any further need to administer this cache.
On the other hand I have to say I'm not sure you can enforce that at least
one replica of each data item goes to a node with LevelDB backend.

Best,

Stephan

2013/2/7 Chad Engler <Chad.Engler at patlive.com>

> I'm writing the prune script in Node, and the dates are stored as int
> timestamps so that isn't an issue.****
>
> ** **
>
> I was under the impression that secondary indexes only worked on the
> LevelDB backend, we have a solid write throughput (maybe 10ish writes per
> second) and very little read (aside from the prune). Would we see
> significant performance degradation by switching from Bitcask?****
>
> ** **
>
> -Chad****
>
> ** **
>
> *From:* riak-users [mailto:riak-users-bounces at lists.basho.com] *On Behalf
> Of *Stephan Kepser
> *Sent:* Thursday, February 07, 2013 1:50 PM
> *To:* riak-users at lists.basho.com
> *Subject:* Re: Nightly Prune****
>
> ** **
>
> Hi Chad,
>
> I recommend looking at secondary indexes. You can set up a secondary index
> with the relevant date from your entry. Note that there is no data type
> date, only string or integer. But you can easily convert a date into a
> string or an integer for your query purposes. Secondary indexes even
> provide you with a way to query for date ranges. And they are fast. So, I
> think they'd serve your purpose.
>
> Best,
>
> Stephan
>
> --
> Dr. Stephan Kepser | Senior IT-Consultant
>
> codecentric AG | Merscheider Straße 1 | 42699 Solingen | Deutschland
> tel: +49 (0) 212.23362845 | fax: +49 (0) 212.23362879 | mobil: +49 (0)
> 151.52883635
> www.codecentric.de | blog.codecentric.de | www.meettheexperts.de |
> www.more4fi.de****
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130208/4aa22322/attachment.html>


More information about the riak-users mailing list