Random questions

Ben Tilly btilly at gmail.com
Mon Apr 18 18:58:31 EDT 2011


On Mon, Apr 18, 2011 at 3:12 PM, Jon Brisbin <jon at jbrisbin.com> wrote:
>
> On Apr 18, 2011, at 2:47 PM, Ben Tilly wrote:
>
>> The first is that I've heard that there is a limit to the size of the
>> headers that can be sent using the http interface and therefore the
>> number of links that you can use.  Is this true, and if so does anyone
>> know what the limit is?
>
> I suppose it depends on the length of your key. I've heard in the
> thousands. If you're using command-line clients or curl or something,
> you might find it gets untenable really quickly.
>
>>
>> My second question is this.  It seems that if accesses are configured
>> so that writes go to a quorum, and reads also comes from a quorum,
>> then once a write is reported as complete, a read is close to
>> guaranteed to come back with that write.  How reliable is this
>> behavior?  I'm sure that there are boundary cases around a machine
>> joining in the same window that the write/read is happening, but I'd
>> like to know what they are.
>
> This is completely unscientific, but in a large test suite with
> many hundreds of write and immediate read operations, a random number
> of them will fail for no apparent reason. I traced this down to the
> quorum thing. If you have a whole lot of writes and immediate reads
> that need to happen (e.g. in a test suite), you have to specify a
> full write (i.e. to every single vnode with no exceptions).

In your test suite what are N, R, W and DW in your unit test?  (N =
number of partitions data is stored on, R is the number that need to
be read, W is the number to respond to a write before a write is
complete, and DW is the number that have to say that they have written
data to disk.)

My understanding is that the default values say that N = 3, R = 1, W =
2, and DW = 0.  With that configuration race conditions are easy to
create.  They should be much harder to create with N = 3, R = 2, W =
2, and DW = 2.  With that configuration my understanding is that every
read has to see at least one node that got written to unless we are in
the process of failing over a node in which case I don't know what
guarantees exist.
>>
>> My third question is how bad an idea people think it is to try to get
>> some form of atomic behavior from Riak by locking important read/write
>> pairs.
>
> There's some very interesting work being done lately to implement the
> Zookeeper protocol for doing atomic checkpointing (which would be
> necessary to achieve atomic operations in a distributed system) but it
> seems way too complicated for most of what people need this kind of
> thing for.
>
> If true atomicity is really a concern, then use Redis and write a
> pub/sub handler to update your Riak documents whenever things change.
>
> You know we could probably take the Riak RabbitMQ postcommit hook and
> adapt it to use Redis for something along these lines... :)
>
> Thanks!
>
> Jon Brisbin
>
> http://jbrisbin.com
> Twitter: @j_brisbin
>
>
>




More information about the riak-users mailing list