Trouble with Riak Search JSON Extractor

Zeeshan Lakhani zlakhani at basho.com
Mon Mar 23 12:14:08 EDT 2015


Hey Matt, 

Remember, with the reload, it’s only new objects in *new* fields that will take the updates. Depending on how big your dataset is, the quick fix may be to upload the *updated* schema under a new name and tie the index to that. The K/V pairs should then work correctly when PUT.

Zeeshan Lakhani
programmer | 
software engineer at @basho | 
org. member/founder of @papers_we_love | paperswelove.org
twitter => @zeeshanlakhani

> On Mar 23, 2015, at 12:07 PM, Matt Brooks <mtbrooks33 at gmail.com> wrote:
> 
> Zeeshan, 
> 
> Is there a way to reload the index manually? I tried running "rp(yz_index:reload(<<"index_name">>))." on each node as suggested on a GitHub issue, and though I got an "ok" response back listing each nodes in the cluster, I still could not query using the updated schema. I even deleted all of my K/V pairs and re-added them with changed keys. 
> 
> Thank you, 
> Matt.
> 
> On Sun, Mar 22, 2015 at 10:21 PM Matt Brooks <mtbrooks33 at gmail.com <mailto:mtbrooks33 at gmail.com>> wrote:
> I see. I'll be looking forward to the future release! Thank you, Zeeshan.
> 
> Matt.
> 
> On Sun, Mar 22, 2015 at 9:57 PM Zeeshan Lakhani <zlakhani at basho.com <mailto:zlakhani at basho.com>> wrote:
> Hey Matt,
> 
> Thanks for clarifying the situation. Yes, I’m working on auto-reloading and re-syncing data along with changes to schemas; it’ll be in a future release. Currently, you can add fields to the schema, for example, for new sets of data, and reload the index if you `riak attach` into one of the running nodes and reload the index across the cluster (based on https://github.com/basho/yokozuna/pull/260 <https://github.com/basho/yokozuna/pull/260>), e.g. https://github.com/basho/yokozuna/issues/130#issuecomment-43802826 <https://github.com/basho/yokozuna/issues/130#issuecomment-43802826>.
> 
> Zeeshan Lakhani
> programmer | 
> software engineer at @basho | 
> org. member/founder of @papers_we_love | paperswelove.org <http://paperswelove.org/>
> twitter => @zeeshanlakhani
> 
>> On Mar 22, 2015, at 9:37 PM, Matt Brooks <mtbrooks33 at gmail.com <mailto:mtbrooks33 at gmail.com>> wrote:
>> 
>> Just discovered this on GitHub https://github.com/basho/yokozuna/issues/130 <https://github.com/basho/yokozuna/issues/130>. Looks like auto-reloading is a work in progress.
>> 
>> On Sun, Mar 22, 2015 at 9:24 PM Matt Brooks <mtbrooks33 at gmail.com <mailto:mtbrooks33 at gmail.com>> wrote:
>> Not getting any errors... but I did notice something interesting after deleting the nodes on my system and then reconfiguring them. 
>> 
>> If the first time I set up a schema I add the multivalued field in question, everything works. If I then change the schema and then update it over HTTP, the changes do not take affect. 
>> 
>> For example, when I reconfigured the nodes on my system I added kept the multivalued "groups" field I mentioned above. I then added some users and was able to successfully retrieve them using q=groups:* or something more specific. I then changed the schema from "groups" to "groups.id <http://groups.id/>" and PUT it on the server successfully (a subsequent GET returned the new schema). I also updated the users so that the "groups" array was changed to an array of JSON objects, each with an "id" field that would match the "groups.id <http://groups.id/>" field in the schema. I then tried to query the users using q=groups.id:*. This failed. 
>> 
>> Do changed to the schema not cause the solr indexes to update? 
>> 
>> PS: I dug around in $RIAK/data/yz/user/conf and noticed a user.xml file. The file held the ORIGINAL schema, not the updated one.
>> 
>> 
>> On Sun, Mar 22, 2015 at 6:45 PM Zeeshan Lakhani <zlakhani at basho.com <mailto:zlakhani at basho.com>> wrote:
>> Hello Matt,
>> 
>> Are you receiving any indexing errors in your logs (check the Solr logs as well)? To be sure, I just wrote a test (using our erlang pb api, https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251 <https://github.com/basho/yokozuna/pull/474/files#diff-5d0015658ff90b94cb1a00329ffafe00R251>) using a custom schema, and I didn’t run into any troubles; I also checked via the http api. If you have the required _yz* fields, you should get those back with the document. If you want the groups items themselves, you’d have to make `stored` true.
>> 
>> Zeeshan Lakhani
>> programmer | 
>> software engineer at @basho | 
>> org. member/founder of @papers_we_love | paperswelove.org <http://paperswelove.org/>
>> twitter => @zeeshanlakhani
>> 
>> 
>>> On Mar 22, 2015, at 4:13 PM, Matt Brooks <mtbrooks33 at gmail.com <mailto:mtbrooks33 at gmail.com>> wrote:
>>> 
>> 
>>> Hello,
>>> 
>>> I have a quick question about a search schema that would index an array of strings in a JSON object. I am storing user data in JSON that looks something like this:
>>> 
>>> {
>>>    "name" : "John Smith",
>>>    "email" : "jsmith at gmail.com <mailto:jsmith at gmail.com>",
>>>    "groups" : [
>>>       "3304cf79",
>>>       "abe155cf"
>>>    ]
>>> }
>>> The custom schema I use for users includes this field entry: 
>>> 
>>> <field name="groups" type="string" indexed="true" stored="false" multiValued="true"/>
>>> I also have the following field type entry: 
>>> 
>>> <fieldType name="string" class="solr.StrField" sortMissingLast="true"/>
>>> I assumed that these entries would allow me to search for users using a query like: 
>>> 
>>> curl $RIAK/search/query/user?wt=json&q=groups:3304cf79
>>> But no docs are returned. I'm almost certain I have an issue with my schema. How can I query users by string entries in the "groups" array?
>>> 
>>> Thank you, 
>>> Matt.
>>> 
>> 
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20150323/d19797fc/attachment-0002.html>


More information about the riak-users mailing list