Riak-S2 javascript aws-sdk failing on multi-part uploads

Luke Bakken lbakken at basho.com
Wed Jan 13 17:33:58 EST 2016


Hi John,

Does CyberDuck automatically enable multi-part uploads for files
greater than a certain size? Does the aws-sdk support multi-part
uploads?
--
Luke Bakken
Engineer
lbakken at basho.com


On Wed, Jan 13, 2016 at 1:24 PM, John Fanjoy <jfanjoy at inetu.net> wrote:
> Hello Everyone,
>
> I have been doing some searching around to see if anyone has come across the
> specific issue I am having and can’t find anything. I have a project that
> provides a file upload interface through a website and puts the uploaded
> object into riak-cs. The front end uses the javascript aws-sdk 2.1.29 (tried
> .19 as well) with a patch I found using the list archives to fix up some url
> encoding issues with uploadIDs. Everything seems to work fine when the image
> is less than 5mb (max part size), however when the object is larger the
> ManagedUpload functionality of the S3 object is failing after the first
> part. I’ve tested the code I’m using against a vanilla AWS bucket and a 400+
> mb file was uploaded without issue. I tested the same file with riak-cs
> using the aws-sdk and CyberDuck. Cyberduck was able to upload the file
> without any failure, but it has not been successful even once using the
> javascript library. I have turned the riak-cs debug logging on, and I am
> getting some errors logged, but I’m not really sure how useful they are:
>
> ```
> 2016-01-13 15:34:51.648 [error] <0.1255.0> Webmachine error at path
> "/buckets/three2016com-devopsdevel/objects/DeSMan_Ep1_Create_Website_720p_DRAFT%2520%25281%2529.mov/uploads/kL9dRGLaQCiZBNnPSpu5mw=="
> :
> {error,{error,{badmatch,{error,closed}},[{webmachine_request,recv_unchunked_body,3,[{file,"src/webmachine_request.erl"},{line,490}]},{riak_cs_wm_object_upload_part,accept_streambody,4,[{file,"src/riak_cs_wm_object_upload_part.erl"},{line,308}]},{riak_cs_wm_object_upload_part,accept_body,2,[{file,"src/riak_cs_wm_object_upload_part.erl"},{line,224}]},{riak_cs_wm_common,accept_body,2,[{file,"src/riak_cs_wm_common.erl"},{line,342}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_re..."},...]},...]}}
> in webmachine_request:recv_unchunked_body/3 line 490
> ```
>
> This error does not show up when using Cyberduck or performing an upload
> that is <5mb using javascript. I’ve tried riak-cs 1.5 and now risk-s2 2.1.0
> with out any success. Any help you can offer would be greatly appreciated.
> If you have any questions for me or need additional information let me know
> and I’ll fill in any holes I left.
>
>
>
> Thank you in advance,
>
> John Fanjoy
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list