Riak-S2 javascript aws-sdk failing on multi-part uploads

John Fanjoy jfanjoy at inetu.net
Wed Jan 13 17:02:01 EST 2016

Correction: The aws-sdk version we are using is aws-sdk-2.2.29 currently (were using 2.2.19 with no real difference). I have applied the patch from this PR which fixed a 403 I was getting prior to the change:


Thanks again for any help,

- John

From: John <jfanjoy at inetu.net<mailto:jfanjoy at inetu.net>>
Date: Wednesday, January 13, 2016 at 4:24 PM
To: "riak-users at lists.basho.com<mailto:riak-users at lists.basho.com>" <riak-users at lists.basho.com<mailto:riak-users at lists.basho.com>>
Subject: Riak-S2 javascript aws-sdk failing on multi-part uploads

Hello Everyone,

I have been doing some searching around to see if anyone has come across the specific issue I am having and can’t find anything. I have a project that provides a file upload interface through a website and puts the uploaded object into riak-cs. The front end uses the javascript aws-sdk 2.1.29 (tried .19 as well) with a patch I found using the list archives to fix up some url encoding issues with uploadIDs. Everything seems to work fine when the image is less than 5mb (max part size), however when the object is larger the ManagedUpload functionality of the S3 object is failing after the first part. I’ve tested the code I’m using against a vanilla AWS bucket and a 400+ mb file was uploaded without issue. I tested the same file with riak-cs using the aws-sdk and CyberDuck. Cyberduck was able to upload the file without any failure, but it has not been successful even once using the javascript library. I have turned the riak-cs debug logging on, and I am getting some errors logged, but I’m not really sure how useful they are:

2016-01-13 15:34:51.648 [error] <0.1255.0> Webmachine error at path "/buckets/three2016com-devopsdevel/objects/DeSMan_Ep1_Create_Website_720p_DRAFT%2520%25281%2529.mov/uploads/kL9dRGLaQCiZBNnPSpu5mw==" : {error,{error,{badmatch,{error,closed}},[{webmachine_request,recv_unchunked_body,3,[{file,"src/webmachine_request.erl"},{line,490}]},{riak_cs_wm_object_upload_part,accept_streambody,4,[{file,"src/riak_cs_wm_object_upload_part.erl"},{line,308}]},{riak_cs_wm_object_upload_part,accept_body,2,[{file,"src/riak_cs_wm_object_upload_part.erl"},{line,224}]},{riak_cs_wm_common,accept_body,2,[{file,"src/riak_cs_wm_common.erl"},{line,342}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_re..."},...]},...]}} in webmachine_request:recv_unchunked_body/3 line 490

This error does not show up when using Cyberduck or performing an upload that is <5mb using javascript. I’ve tried riak-cs 1.5 and now risk-s2 2.1.0 with out any success. Any help you can offer would be greatly appreciated. If you have any questions for me or need additional information let me know and I’ll fill in any holes I left.

Thank you in advance,

John Fanjoy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20160113/999d88f7/attachment-0002.html>

More information about the riak-users mailing list