luwak backend and misc.

Kunal Nawale knawale at verivue.com
Thu Jul 28 12:00:21 EDT 2011


Hi,
  I am evaluating luwak to be used as a redundant file storage server. I am trying to find out which backend will better suit for my purpose. Each of my server has sixteen 1TB drives, 4 total servers, 48GB ram each, 1x10Gb network interface
The file sizes that will be stored range from 1GB-20GB, with an average size of 3 GB.

Here are some observations/questions I had regarding this.

1) With bitcask backend, I tried uploading a 6 GB file. The upload and download worked fine for this file. But when I tried to upload a 17 GB file it took a very long time (more than 20 mins). Tried to download it but did not succeed, the download always come back with a size of 1,000,000 bytes.

2) I also tried fs_backend, but it turned out to be quite slow, the
upload of a 6 GB took considerably longer. The download never succeeded
it always returned me a chunk of that file not the whole file.

3) Are there any performance measurements available about the read/write bandwidths

4) Are there any latency numbers available, I am specifically looking at the time difference between the first byte read and the last byte write for an object.

5) Can an object be read simultaneously while it is being written. With a lag between the read write pointer being in the range of 60 MBytes.

Thanks
-Kunal









More information about the riak-users mailing list