luwak backend and misc.

Drew Whitehouse Drew.Whitehouse at anu.edu.au
Tue Aug 23 17:32:52 EDT 2011


It would seem that a new file is readable (sort of) before the tree is
flushed, no ?

   https://gist.github.com/1160026

-Drew


...deleted...

> The simple answer is *new* Luwak files cannot be read while being
>> written.  This has to do with the fact that the HTTP interface does
>> not expose a way to flush the file's tree to Riak before finishing the
>> upload.  The data for the file is being persisted, but the root
>> pointer is not modified until the end.
>>
>> This also means that *existing* Luwak file *can* be read while being
>> modified, but modifications made after the root pointer is found will
>> be invisible.
>>
>>  4) Are there any latency numbers available, I am specifically looking at
>>> the
>>> time difference between the first byte read and the last byte write for
>>> an
>>> object.
>>>
>> I'm interpreting this question as, "After I finish writing, how long
>> will it be before I can begin reading what I just wrote?" because of
>> the tree-flushing behavior I described above.
>>
>> The answer depends on the backlog to the Luwak writer process (on the
>> Riak/server side), and the depth of the resulting file tree.  Once the
>> writer has written the final block to an object, it must then flush
>> the tree pointing to that object.  Flushing the tree requires writing,
>> at least, the root node and the "tld" object (where the metadata about
>> the file is stored).  The block_size and tree_order parameters
>> (1,000,000 bytes and 250, by default) determine how many other nodes
>> must be written between the root and the block.  Each node is an
>> additional Riak object write.
>>
>> I hope that helps,
>> Bryan
>>
>
> ______________________________**_________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/**mailman/listinfo/riak-users_**lists.basho.com<http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com>
>



-- 
Drew Whitehouse
ANU Supercomputer Facility Vizlab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20110824/80fe30e7/attachment.html>


More information about the riak-users mailing list