RiakCS large file uploads fail with 403/AccessDenied and 400/InvalidDigest

Shunichi Shinohara shino at basho.com
Mon Mar 9 23:15:48 EDT 2015


Hi Niels,

Thank you for your interest on Riak CS.

Some questions about 400 - InvalidDigest:

- Can you confirm which MD5 was correct for the log
  2015-02-11 16:34:17.854 [debug]
<0.23568.18>@riak_cs_put_fsm:is_digest_valid:326
     Calculated = <<"pIFX5fpeo7+sPPNjtSBWBg==">>,
     Reported = "0BsQLab2tMEzr8IWoS2m5w=="
- What was the transfer-encoding? I want to confirm chunked encoding
was NOT used.
- Hopefully, packet capture (e.g. by pcap format) will be helpful to debug

Thanks,
Shino

On Tue, Mar 10, 2015 at 10:25 AM, Kota Uenishi <kota at basho.com> wrote:
> Sorry, being late. I thought I've replied to you, but it was a very
> close one where I think you're hitting the same problem as this:
>
> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-February/016845.html
>
> Riak CS often includes `=' in uploadId of multipart uploads while S3
> doesn't (where no specs described in official documents).
>
> On Thu, Feb 12, 2015 at 12:41 AM, Niels O <nielsoo at gmail.com> wrote:
>> Hello everyone,
>>
>> I have just installed riakcs and have the s3cmd and nodejs (the official
>> amazon) plugin working.
>>
>> with the same credentials (accesskey&secret) I CAN upload big files with
>> S3CMD but I CANNOT with the AWS/S3 nodejs plugin? (downloading very big
>> files is no problem b.t.w.)
>>
>>
>> with the nodejs plugin
>>
>> - until 992k, (I tested with 32 KiB increases) everything works
>> - starting at 1024 KiB I get [400 InvalidDigest: The Content-MD5 you
>> specified was invalid.]
>> - from 8192 KiB and beyond I get [403 AccessDenied] back from riakcs.
>>
>> this while -again- with s3cmd I am able to upload files of over 1 GiB size
>> easily  .. same machine, same creds
>>
>> any ideas?
>>
>>
>>
>>
>>
>> (below some riakcs debug logging from both the 400 and 403)  ...
>>
>>
>> 400 - InvalidDigest:
>>
>> 2015-02-11 16:34:16.911 [debug]
>> <0.17889.18>@riak_cs_s3_auth:calculate_signature:129 STS:
>> ["PUT","\n","0BsQLab2tMEzr8IWoS2m5w==","\n","application/octet-stream","\n","\n",[["x-amz-date",":",<<"Wed,
>> 11 Feb 2015 15:34:16 GMT">>,"\n"]],["/testje/4096k",[]]]
>> 2015-02-11 16:34:17.854 [debug]
>> <0.23568.18>@riak_cs_put_fsm:is_digest_valid:326 Calculated =
>> <<"pIFX5fpeo7+sPPNjtSBWBg==">>, Reported = "0BsQLab2tMEzr8IWoS2m5w=="
>> 2015-02-11 16:34:17.860 [debug] <0.23568.18>@riak_cs_put_fsm:done:303
>> Invalid digest in the PUT FSM
>>
>>
>> 403 - AccessDenied
>>
>> 2015-02-11 16:36:00.448 [debug]
>> <0.22889.18>@riak_cs_s3_auth:calculate_signature:129 STS:
>> ["POST","\n",[],"\n","application/octet-stream","\n","\n",[["x-amz-date",":",<<"Wed,
>> 11 Feb 2015 15:36:00 GMT">>,"\n"]],["/testje/8192k","?uploads"]]
>> 2015-02-11 16:36:00.484 [debug]
>> <0.23539.18>@riak_cs_s3_auth:calculate_signature:129 STS:
>> ["PUT","\n","sq5d2PIhC7I1xxT8Rp9cVg==","\n","application/octet-stream","\n","\n",[["x-amz-date",":",<<"Wed,
>> 11 Feb 2015 15:36:00
>> GMT">>,"\n"]],["/testje/8192k","?partNumber=1&uploadId=TXR2AuCeRDWwc2bviLPcOg=="]]
>> 2015-02-11 16:36:00.484 [debug]
>> <0.23539.18>@riak_cs_wm_common:post_authentication:471 bad_auth
>> 2015-02-11 16:36:00.494 [debug]
>> <0.23543.18>@riak_cs_s3_auth:calculate_signature:129 STS:
>> ["DELETE","\n",[],"\n","application/octet-stream","\n","\n",[["x-amz-date",":",<<"Wed,
>> 11 Feb 2015 15:36:00
>> GMT">>,"\n"]],["/testje/8192k","?uploadId=TXR2AuCeRDWwc2bviLPcOg=="]]
>> 2015-02-11 16:36:00.494 [debug]
>> <0.23543.18>@riak_cs_wm_common:post_authentication:471 bad_auth
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
>
> --
> Kota UENISHI / @kuenishi
> Basho Japan KK
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




More information about the riak-users mailing list