Faster deletes?

matthew hawthorne mhawthorne at gmail.com
Fri Nov 25 16:45:05 EST 2011


I think that writes to Riak will always be faster than other
operations.  It has to do with the fact that writes are append only,
whereas deletes, for example, must write a tombstone into all of the
replicas.

I'm basically paraphrasing this message since I don't understand how
this works 100%:
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-October/006048.html

However I do think concurrency is your best bet - we needed a bulk
delete job awhile back and the steps were:
1) generate a keylist, split into multiple files
2) spawn a process/thread for each file

the key is finding the proper number of processes/files to keep your
delete rate acceptable.

-matt


On Fri, Nov 25, 2011 at 1:06 PM, Keith Irwin <keith at zentrope.com> wrote:
> I'm still interested in this problem (deletes lagging way behind inserts). Is there anything about the technology of Riak itself that might explain this, or is it instead something to do with the map/reduce phase in order to find keys to delete?
>
> Andy indicated that the map part of what he was doing was really fast, but iterating deletes were the problem.
>
> Perhaps spinning up a lot of concurrent deletes is the key?
>
> Keith
>
> On Nov 17, 2011, at 1:29 PM, Bartling, Aron wrote:
>
>> It's using a map job with key filters to list the keys to delete. This part of the process happens in a very reasonable amount of time.
>>
>> Thanks
>>
>> -----Original Message-----
>> From: riak-users-bounces at lists.basho.com [mailto:riak-users-bounces at lists.basho.com] On Behalf Of Andy Skelton
>> Sent: Thursday, November 17, 2011 11:24 AM
>> To: riak-users at lists.basho.com
>> Subject: Re: Faster deletes?
>>
>> Aron Bartling wrote:
>>> The deletion is based on the timestamp incorporated into the key
>>
>> Does this mean your deletion process is listing all of the keys in a bucket?
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list