Yokozuna indexes too slow

Jason Voegele jvoegele at basho.com
Thu Oct 1 10:12:00 EDT 2015


Hello Ilyas, I’ve got a few questions for you to help diagnose the performance problem.

* What Riak version are you running?

* Are you using the default solrconfig on the index, or a custom configuration?

* You say that there is a 200x slow down when Yokozuna indexing is enabled. Do you have any specific numbers? A graph would be helpful if you can generate one.

* When you say that Yokozuna takes up 100% of one core, have you been able to narrow that down to specific process within Yokozuna or Riak? Using “riak-admin top” might help here.

* Can you provide your JVM configuration (heap settings, etc.)?


> On Oct 1, 2015, at 2:18 AM, ilyas <i.sergeev at keepsolid.com> wrote:
> 
> 
> Hello to the Riak community! 
> I have a cluster of 5 nodes (hardware servers with SSDs, 8 CPU cores, 16GB of RAM, most of it is free). When I store data to the database (up to 1 000 000 keys) without indexes, everything happens very quickly. If I create an index for yokozuna and do the same, the store rate decreases greatly (200 times). The process yokozuna takes up 100% of the time one core of CPU, other cores are not loaded. Keys are short strings (16 symbols), values are json strings (length is about 200 characters). 
> How can I find out the reason of such performance and fix it?
> 
> creation of bucket:
> 
> #!/bin/bash
> 
> RIAK_HOST="127.0.0.1:8098"
> 
> #define search scheme
> curl -XPUT $RIAK_HOST/search/schema/dbsearch -H 'Content-Type:application/xml' --data-binary @dbSchema.xml
> 
> # create search index for scheme
> curl -XPUT $RIAK_HOST/search/index/user_idx_bin -H 'Content-Type: application/json' -d '{"schema":"dbsearch"}'
> 
> # create bucket type for leleldb backend
> sudo /usr/sbin/riak-admin bucket-type create leveldb '{"props":{"backend":"leveldb_mult"}}'
> sudo /usr/sbin/riak-admin bucket-type activate leveldb
> 
> #create buckets with known backends and indexes
> curl -XPUT $RIAK_HOST/types/leveldb/buckets/SomeBucket/props -H 'Content-Type: application/json' -d '{"props":{"search_index":"user_idx_bin"}}'
> 
> index scheme:
> 
> <?xml version="1.0" encoding="UTF-8" ?>
> <schema name="dbschema" version="1.5">
> 
>  <fields>
>    <field name="username"   type="string"  indexed="true" stored="true" />
>    <field name="deviceid"   type="string"  indexed="true" stored="true" />
>    <field name="appid"      type="int"  indexed="true" stored="true" />
>    <field name="ip_address" type="int"  indexed="true" stored="true" />
> 
>    <!-- All of these fields are required by Riak Search -->
>    <field name="_yz_id"   type="_yz_str" indexed="true" stored="true"  multiValued="false" required="true"/>
>    <field name="_yz_ed"   type="_yz_str" indexed="true" stored="false" multiValued="false"/>
>    <field name="_yz_pn"   type="_yz_str" indexed="true" stored="false" multiValued="false"/>
>    <field name="_yz_fpn"  type="_yz_str" indexed="true" stored="false" multiValued="false"/>
>    <field name="_yz_vtag" type="_yz_str" indexed="true" stored="false" multiValued="false"/>
>    <field name="_yz_rk"   type="_yz_str" indexed="true" stored="true"  multiValued="false"/>
>    <field name="_yz_rt"   type="_yz_str" indexed="true" stored="true"  multiValued="false"/>
>    <field name="_yz_rb"   type="_yz_str" indexed="true" stored="true"  multiValued="false"/>
>    <field name="_yz_err"  type="_yz_str" indexed="true" stored="false" multiValued="false"/>
> 
>    <dynamicField name="*" type="ignored"  />
>  </fields>
> 
> <types>
>    <!-- YZ String: Used for non-analyzed fields -->
>    <fieldType name="_yz_str" class="solr.StrField" sortMissingLast="true" />
>    <fieldType name="string" class="solr.StrField" sortMissingLast="true" />
>    <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true"/>
>    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
>    <fieldtype name="ignored" stored="false" indexed="false" multiValued="true" class="solr.StrField" />
> </types>
> 
>  <uniqueKey>_yz_id</uniqueKey>
> 
> </schema>
> 
> <dbSchema.xml><make_db_idx.sh>_______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20151001/e2714e35/attachment-0002.html>


More information about the riak-users mailing list