Does Riak support Range Queries over binary safe strings?

Sean Cribbs sean at basho.com
Wed Jul 17 08:48:10 EDT 2013


On Wed, Jul 17, 2013 at 5:31 AM, gbrits <gbrits at gmail.com> wrote:

> Sounds good to me. Just to confirm: the value which I want to get a slice
> from can be a bytearray encoded as a string? (base 255)?
>
>
Bytes are bytes to Riak. Your value is completely opaque.


> Oh and almost forgot, I DO need to be able to parameterize the m/r (with
> param startSliceAt, endSliceAt + some other parameters that manage how
> aggregation of values is performed) .
> Being an absolute noob to m/r is this possible?
>
>
Each phase of a MapReduce job can receive arbitrary additional arguments. I
highly suggest you check out our tutorials on the matter:
http://docs.basho.com/riak/latest/tutorials/querying/MapReduce/


>
>
> 2013/7/17 Alexander Sicular [via Riak Users] <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028366&i=0>
> >
>
>> To the extent you limit your sliced data via an m/r you will reap those
>> savings on the wire when transferring back to the client
>>
>> You can feed an m/r from riak search, 2i or enumerated keys thereby
>> skipping a costly bucket scan.
>>
>>
>> @siculars
>> http://siculars.posthaven.com
>>
>> Sent from my iRotaryPhone
>>
>> On Jul 16, 2013, at 19:46, Jeremiah Peschka <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028365&i=0>>
>> wrote:
>>
>> Not a problem.
>>
>> MapReduce across an entire keyspace is slow.
>>
>> MapReduce when provided with a few bucket/key pairs is the same as a
>> multi-get + processing.
>>
>> You can combine 2i + MR to get quick processing of data. Although, at
>> that point, you might as well just process your data on the client side.
>> Especially if you're just pulling out a slice of bytes.
>>
>> ---
>> Jeremiah Peschka - Founder, Brent Ozar Unlimited
>>  MCITP: SQL Server 2008, MVP
>> Cloudera Certified Developer for Apache Hadoop
>>
>>
>> On Tue, Jul 16, 2013 at 4:13 PM, gbrits <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028365&i=1>
>> > wrote:
>>
>>> Wow, high speed on this list!
>>>
>>> I wanted it for near realtime anyway so Map/reduce is out of the
>>> question. Thought somehow it could be done through Riak Search or directly
>>> on secondary indices instead of map/reduce.
>>> Guess not. Oh well, can't have it all.
>>>
>>> Thanks
>>>
>>>
>>> 2013/7/17 Jeremiah Peschka [via Riak Users] <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028360&i=0>
>>> >
>>>
>>>> Following up on Alex's comments -
>>>>
>>>> If you know which bytes you need to slice, you can store this in a
>>>> secondary index. You can perform range queries across secondary indices (as
>>>> well as keys).
>>>>
>>>> As long as you're storing your data in a way that allows it to be read
>>>> by either Erlang or JavaScript, you should be able to query over it in
>>>> MapReduce. This is typically regarded as a Bad Idea™ since an MR query will
>>>> need to scan all keys in a bucket (which effectively means scanning the
>>>> entire cluster) and is best done as an infrequent activity to transform
>>>> data.
>>>>
>>>> ---
>>>> Jeremiah Peschka - Founder, Brent Ozar Unlimited
>>>>  MCITP: SQL Server 2008, MVP
>>>> Cloudera Certified Developer for Apache Hadoop
>>>>
>>>>
>>>> On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028359&i=0>
>>>> > wrote:
>>>>
>>>>> I would say no. Riak is generally oblivious as to the content of your
>>>>> data. Any ranges or other method you would use to query needs to be
>>>>> explicitly indexed via riak search or secondary indexes. Once you have
>>>>> found your data you could operate over that data in a map reduce, but I
>>>>> can't speak to "binary safe" blob operations in either erlang or JavaScript
>>>>> although I'm inclined to say yes, you would be able to operate over it in
>>>>> m/r.
>>>>>
>>>>> So searching for keys with certain data in the binblob is probably not
>>>>> gonna happen but once you have a key to feed an m/r you could get a slice
>>>>> of that value.
>>>>>
>>>>> Make sense?
>>>>> -Alexander
>>>>>
>>>>> @siculars
>>>>> http://siculars.posthaven.com
>>>>>
>>>>> Sent from my iRotaryPhone
>>>>>
>>>>> On Jul 16, 2013, at 18:17, gbrits <[hidden email]<http://user/SendEmail.jtp?type=node&node=4028359&i=1>>
>>>>> wrote:
>>>>>
>>>>> > First, hello all!
>>>>> >
>>>>> > Coming from Redis, I love that you can just put any binary blob in
>>>>> Redis
>>>>> > which is just treated as a string. This is possible because Redis
>>>>> strings
>>>>> > are what they call 'binary safe'. This makes it possible to return
>>>>> slices of
>>>>> > string-encoded binary data, which is super useful for
>>>>> bitset-operations,
>>>>> > etc.
>>>>> >
>>>>> > I'm investigating Riak and I like it a lot so far. Riak seems to
>>>>> have range
>>>>> > queries (on values, as it seems I must make that distinction with
>>>>> > column-stores), but I'm not sure if strings in Riak are "Binary
>>>>> safe" in the
>>>>> > above sense. If not, is there another way to store binary data in
>>>>> Riak and
>>>>> > still do range queries over them quickly?
>>>>> >
>>>>> > To be exact: I want to do multi-key lookups in Riak, where each
>>>>> returned
>>>>> > result should be of format: <key,slice(featureX,start,end)>
>>>>> >
>>>>> > Thanks,
>>>>> > Geert-Jan
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > View this message in context:
>>>>> http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
>>>>> > Sent from the Riak Users mailing list archive at Nabble.com.
>>>>> >
>>>>> > _______________________________________________
>>>>> > riak-users mailing list
>>>>> > [hidden email]<http://user/SendEmail.jtp?type=node&node=4028359&i=2>
>>>>>
>>>>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>
>>>>> _______________________________________________
>>>>> riak-users mailing list
>>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4028359&i=3>
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4028359&i=4>
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>>
>>>> ------------------------------
>>>>  If you reply to this email, your message will be added to the
>>>> discussion below:
>>>>
>>>> http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028359.html
>>>>  To unsubscribe from Does Riak support Range Queries over binary safe
>>>> strings?, click here.
>>>> NAML<http://riak-users.197444.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: Does Riak support Range Queries over
>>> binary safe strings?<http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028360.html>
>>>
>>> Sent from the Riak Users mailing list archive<http://riak-users.197444.n3.nabble.com/>at
>>> Nabble.com.
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4028365&i=2>
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4028365&i=3>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4028365&i=4>
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> ------------------------------
>>  If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028365.html
>>  To unsubscribe from Does Riak support Range Queries over binary safe
>> strings?, click here.
>> NAML<http://riak-users.197444.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: Does Riak support Range Queries over
> binary safe strings?<http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028366.html>
> Sent from the Riak Users mailing list archive<http://riak-users.197444.n3.nabble.com/>at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


-- 
Sean Cribbs <sean at basho.com>
Software Engineer
Basho Technologies, Inc.
http://basho.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20130717/aadbeb1f/attachment.html>


More information about the riak-users mailing list