Riak-S2 javascript aws-sdk failing on multi-part uploads

Luke Bakken lbakken at basho.com
Wed Jan 13 17:55:51 EST 2016


John -

The following error indicates that the connection was unexpectedly
closed by something outside of Riak while the chunk is uploading:

{badmatch,{error,closed}}

Is it possible to remove haproxy to test using the the aws-sdk?

That is my first thought as to the cause of this issue, especially
since writing to S3 works with the same code.

--
Luke Bakken
Engineer
lbakken at basho.com

On Wed, Jan 13, 2016 at 2:46 PM, John Fanjoy <jfanjoy at inetu.net> wrote:
> Luke,
>
> Yes on both parts. To confirm cyberduck was using multi-part I actually tailed the console.log while it was uploading the file, and it uploaded the file in approx. 40 parts. Afterwards the parts were reassembled as you would expect. The AWS-SDK for javascript has an object called ManagedUpload which automatically switches to multi-part when the input is larger than the maxpartsize (default 5mb). I have confirmed that it is splitting the files up, but so far I’ve only ever seen one part get successfully uploaded before the others failed at which point it removes the upload (DELETE call) automatically. I also verified that the javascript I have in place does work with an actual AWS S3 bucket to rule out coding issues on my end and the same >400mb file was successfully uploaded to the bucket I created there without issue.
>
> A few things worth mentioning that I missed before. I am running riak-s2 behind haproxy. Haproxy is handling ssl and enabling CORS for browser based requests. I have tested smaller files (~4-5mb) and GET requests using the browser client and everything works with my current haproxy configuration, but the larger files are failing, usually after 1 part is successfully uploaded. I can also list bucket contents and delete existing contents. The only feature that is not working appears to be the multi-part uploads. We are running centOS 7 (kernel version 3.10.0-327.4.4.el7.x86_64). Please let me know if you have any further questions.
>
> --
> John Fanjoy
> Systems Engineer
> jfanjoy at inetu.net




More information about the riak-users mailing list