Chunked data retreival

Mojito Sorbet mojitotech at gmail.com
Sat Oct 9 00:12:20 EDT 2010


I understand that the way to do access control to data is with a web
server in front of Riak.   I am using YAWS for that, and the Erlang code
will be making direct Riak API calls and then sending the results out in
HTTP responses.   The access control is non-trivial, can involve a large
number of users, and the authorization can be time-limited;  so I don't
see any other way to handle that.  ok.

Most of my data records are images, and they can get into the few
hundred KB size range.  If I have to fetch the entire thing from Riak
into my app code, then send that in one operation out in HTTP, the
response time to the end client will be increased.

Is there a way I can fetch large data from Riak in pieces and do chunked
transfer to the end user, perhaps 64KB at a time?   It might save on
memory usage too, not having to buffer entire images at a time.





More information about the riak-users mailing list