2i performance and ring-creation-size
martin.sumner at adaptip.co.uk
Fri Jun 1 17:13:49 EDT 2012
We've just been re-running some performance tests today following a number
of changes - and we've found our use of secondary indexes to be much slower
than previously, and much slower than we expected.
I think I know why, but could someone confirm if I have understood the
Between the tests we changed the ring-creation-size from 64 to 256 - we
don't need that size now, but we're cautious that we should have room to
expand. If we now do a secondary index query for a sparsely populated
term, should it be expected that this will take a lot longer: as the search
is document-partitioned, so there will now be 4 times as many places to
look (and probably not find) the existence of the term? So we're doing a
lot more expensive disk seeks than with the default ring size.
Would I also be right in assuming that for sparsely populated terms on the
index, once the bloom filter functionality is implemented in leveldb/riak -
that such queries should get a massive speed advantage. However, for less
sparsely populated terms (i.e. ones where there is a document match on most
levels for most partitions) performance will continue to suffer if the ring
size is out of touch with the number of nodes (e.g. running a small cluster
on a large ring-creation-size).
Other than physical changes (such as moving the searchable buckets onto SSD
to remove disk-seeking, and adding more memory to provide deeper cache
coverage of the levels), is there anything else I can do for 2i performance
if I have a small cluster but want to build it with a large ring size to
support future scalability?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the riak-users