Javascript errors in reduce phases seen as {error, javascript_reduce_timeout}?

Sean Cribbs sean at basho.com
Tue May 10 09:45:21 EDT 2011


You might see an error in the log, but there are a lot of things about the JS VM that are too opaque. It's on our list of things to improve, for sure.

Sean Cribbs <sean at basho.com>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On May 10, 2011, at 9:30 AM, Jeremy Raymond wrote:

> Okay thanks. I figure something like this was going on. Besides the timeout return value is there any way to tell what happened or at what phase of some chained map reduce operations things failed? Would I see errors in the logs that indicated a crash of the Javascript engine or something along those lines?
> 
> - Jeremy
> 
> 
> On Tue, May 10, 2011 at 9:27 AM, Sean Cribbs <sean at basho.com> wrote:
> For large numbers of objects, your JavaScript functions will become bottlenecks, largely because the data has to be converted to and from JSON for every application of the function. If your reduce function isn't actually coalescing a value (e.g. sort doesn't), then you will run into this problem.  See if you can reimplement the functions in Erlang or decrease the quantity of values being passed to and returned from the reduce functions.
> 
> Sean Cribbs <sean at basho.com>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
> 
> On May 10, 2011, at 9:20 AM, Jeremy Raymond wrote:
> 
>> Hello,
>>  
>> In a MapReduce job I have filtering through about 6500 items if I filter out enough items my m/r job completes successfully. If I loosen the filtering such that a large number of items would be included in the results the m/r fails with error {error,javascript_reduce_timeout}. This error occurs after about 10 seconds of processing. Would Javascript errors such as the Javascript process running out of memory and crashing result in this timeout error? I also noticed that after this error is returned, the riak node I made the request on chugs along at ~100% CPU for about another minute but the other two nodes quite down almost immediately after the error is returned. How might I go about debugging this?
>>  
>> - Jeremy
>>  
>> --------------------------------------------------------------------- 
>> This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful._______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> 
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110510/aea13b31/attachment.html>


More information about the riak-users mailing list