Javascript errors in reduce phases seen as {error, javascript_reduce_timeout}?

Jeremy Raymond jeraymond at gmail.com
Wed May 11 17:37:06 EDT 2011


Implementing the m/r functions in Erlang did the trick.

- Jeremy

On 5/10/2011 9:45 AM, Sean Cribbs wrote:
> You might see an error in the log, but there are a lot of things about 
> the JS VM that are too opaque. It's on our list of things to improve, 
> for sure.
>
> Sean Cribbs <sean at basho.com <mailto:sean at basho.com>>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
>
> On May 10, 2011, at 9:30 AM, Jeremy Raymond wrote:
>
>> Okay thanks. I figure something like this was going on. Besides the 
>> timeout return value is there any way to tell what happened or at 
>> what phase of some chained map reduce operations things failed? Would 
>> I see errors in the logs that indicated a crash of the Javascript 
>> engine or something along those lines?
>>
>> - Jeremy
>>
>>
>> On Tue, May 10, 2011 at 9:27 AM, Sean Cribbs <sean at basho.com 
>> <mailto:sean at basho.com>> wrote:
>>
>>     For large numbers of objects, your JavaScript functions will
>>     become bottlenecks, largely because the data has to be converted
>>     to and from JSON for every application of the function. If your
>>     reduce function isn't actually coalescing a value (e.g. sort
>>     doesn't), then you will run into this problem.  See if you can
>>     reimplement the functions in Erlang or decrease the quantity of
>>     values being passed to and returned from the reduce functions.
>>
>>     Sean Cribbs <sean at basho.com <mailto:sean at basho.com>>
>>     Developer Advocate
>>     Basho Technologies, Inc.
>>     http://basho.com/
>>
>>     On May 10, 2011, at 9:20 AM, Jeremy Raymond wrote:
>>
>>>     Hello,
>>>     In a MapReduce job I have filtering through about 6500 items if
>>>     I filter out enough items my m/r job completes successfully. If
>>>     I loosen the filtering such that a large number of items would
>>>     be included in the results the m/r fails with error
>>>     {error,javascript_reduce_timeout}. This error occurs after about
>>>     10 seconds of processing. Would Javascript errors such as the
>>>     Javascript process running out of memory and crashing result in
>>>     this timeout error? I also noticed that after this error is
>>>     returned, the riak node I made the request on chugs along at
>>>     ~100% CPU for about another minute but the other two nodes quite
>>>     down almost immediately after the error is returned. How might I
>>>     go about debugging this?
>>>     - Jeremy
>>>     ---------------------------------------------------------------------
>>>     This transmission (including any attachments) may contain
>>>     confidential information, privileged material (including
>>>     material protected by the solicitor-client or other applicable
>>>     privileges), or constitute non-public information. Any use of
>>>     this information by anyone other than the intended recipient is
>>>     prohibited. If you have received this transmission in error,
>>>     please immediately reply to the sender and delete this
>>>     information from your system. Use, dissemination, distribution,
>>>     or reproduction of this transmission by unintended recipients is
>>>     not authorized and may be
>>>     unlawful._______________________________________________
>>>     riak-users mailing list
>>>     riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>     _______________________________________________
>>     riak-users mailing list
>>     riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110511/e8b0b015/attachment.html>


More information about the riak-users mailing list