Brokenness specific to nginx and jets3t with Riak CS
toby at dryft.net
Thu Sep 25 03:42:47 EDT 2014
Following up further again, with a solution.
Turns out that Nginx canonicalises the URLs when proxying them, if and
only if, the proxy_pass parameter includes a path component, including
a plain slash.
Meanwhile, Jets3t URL-encodes the object path so that slashes become
%2F, but Nginx converts them to actual slashes before forwarding to
Fix was to change
On 25 September 2014 16:24, Toby Corkindale <toby at dryft.net> wrote:
> Quite update to mention that by removing some extra (unneeded) custom
> headers from the proxy configuration and fiddling with some other
> nginx options, I'm at a point where Riak CS throws an actual error
> back rather than disconnecting prematurely.
> The error now returned is below. I'm wondering if something else nginx
> is doing is causing the request to get mangled enough that it doesn't
> work.. but hard to tell what, or why :/
> ResponseCode: 403, ResponseStatus: Forbidden, XML Error Message: <?xml
> On 25 September 2014 15:59, Toby Corkindale <toby at dryft.net> wrote:
>> I've hit an issue that only seems to occur when using the Jets3t
>> client library and Nginx as a load balancer in front of Riak CS.
>> The issue is NOT present when using other S3 libraries, nor is it
>> present if I switch out Nginx for haproxy.
>> Unfortunately, in this instance it is desirable to use both, unless it
>> turns out to be *really* troublesome.
>> I wondered if anyone here can offer advice?
>> The problem is that reads and writes to objects results in Riak CS,
>> apparently, disconnecting the client prematurely.
>> Nginx reports "connection reset by peer" and retries upstream Riak CS
>> servers several times before giving up.
>> The Riak CS access logs indicate what look like valid URLs, with a 403
>> status indicated.
>> Accessing the same buckets with the same auth and transferring the
>> same files is fine when done with s3cmd, and we have other apps that
>> have been running through nginx for a while using other S3 client
>> A simple "check if bucket exists" command through jets3t seems to work OK.
>> Any thoughts on what could be going on, or is this all just a bit too vague?
> Turning and turning in the widening gyre
> The falcon cannot hear the falconer
> Things fall apart; the center cannot hold
> Mere anarchy is loosed upon the world
Turning and turning in the widening gyre
The falcon cannot hear the falconer
Things fall apart; the center cannot hold
Mere anarchy is loosed upon the world
More information about the riak-users