Riak: reliable object deletion

al so volkswak at gmail.com
Fri Feb 24 01:14:58 EST 2017


What is the best way to root cause it.

Any metadata/etc to query the lifecycle of this deleted object given that
Key and Bucket are known.

On Thu, Feb 23, 2017 at 9:53 PM, al so <volkswak at gmail.com> wrote:

> Present both in Solr and Riak. It's a 1-way replication in MDC. MDC is not
>> the cause unless there is a bug there as well.
>>
>>
>> On Thu, Feb 23, 2017 at 6:02 AM, Fred Dushin <fdushin at basho.com> wrote:
>>
>>> Running a Solr query has no impact on writes -- Riak search queries are
>>> direct pass throughs to Solr query and don't touch any of the salient Riak
>>> systems (batching writes to Solr, YZ AAE, etc).  I believe the timing of
>>> the reappearance is a coincidence.
>>>
>>> Is it possible the object reappeared via MDC?  Do you have record of the
>>> reappeared object in your other cluster?
>>>
>>> Also, just to confirm, when you say the object reappeared -- it
>>> re-appeared in Riak (and subsequently in Solr), correct?  Or are you saying
>>> you just find the object in Solr (via a search)?
>>>
>>> -Fred
>>>
>>> On Feb 23, 2017, at 1:43 AM, al so <volkswak at gmail.com> wrote:
>>>
>>>  Here is brief env:
>>>>  Riak v 2.0.8 + Solr/ 5 node cluster/ MDC
>>>>
>>>>  Problem:
>>>>   Deleted object suddenly resurrected after few days. Solr search query(
>>>> "*:*") was executed around the time of reappearance.
>>>>
>>>>  Bucket property for this reappeared object
>>>>  {
>>>>   "props": {
>>>>     "name": "UsaHype",
>>>>     "allow_mult": false,
>>>>     "basic_quorum": false,
>>>>     "big_vclock": 50,
>>>>     "chash_keyfun": {
>>>>       "mod": "riak_core_util",
>>>>       "fun": "chash_std_keyfun"
>>>>     },
>>>>     "dvv_enabled": true,
>>>>     "dw": "quorum",
>>>>     "last_write_wins": false,
>>>>     "linkfun": {
>>>>       "mod": "riak_kv_wm_link_walker",
>>>>       "fun": "mapreduce_linkfun"
>>>>     },
>>>>     "n_val": 3,
>>>>     "notfound_ok": true,
>>>>     "old_vclock": 86400,
>>>>     "postcommit": [
>>>>       {
>>>>         "mod": "riak_repl_leader",
>>>>         "fun": "postcommit"
>>>>       },
>>>>       {
>>>>         "mod": "riak_repl2_rt",
>>>>         "fun": "postcommit"
>>>>       }
>>>>     ],
>>>>     "pr": 0,
>>>>     "precommit": [
>>>>
>>>>     ],
>>>>     "pw": 0,
>>>>     "r": "quorum",
>>>>     "repl": true,
>>>>     "rw": "quorum",
>>>>     "small_vclock": 50,
>>>>     "w": "quorum",
>>>>     "young_vclock": 20
>>>>   }
>>>> }
>>>>
>>>>
>>>> * delete_mode is not configured at all. Defaults to 3 sec? Do Tombstone still get created when no delete_mode at all in the config? Can we query metadata(*X-Riak-Deleted)* for this reappeared object.*
>>>>
>>>>  How would one go about finding root cause. I do know the date when the
>>>> object was deleted from our app logs. I also seem to know when it
>>>> reappeared. Seems like a solr search ("*:*") in code was executed just
>>>> before this object reappeared? Look at AAE? Is there a way to find if Solr
>>>> and Riak backends are out-of-sync (AAE?).
>>>>
>>>>  I understand tuning other params (allow_mult = true, ...) will have
>>>> its own implications.
>>>>
>>>>  In summary:
>>>>   How do I find root cause of this issue.
>>>>   How to reliable delete an object.
>>>>
>>>> -Volk
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20170223/ab51ac99/attachment-0002.html>


More information about the riak-users mailing list