Logging chat messages

Shashwat Srivastava darklrd at gmail.com
Tue Oct 23 06:47:53 EDT 2012


On Tue, Oct 23, 2012 at 3:35 PM, Guido Medina <guido.medina at temetra.com>wrote:

>    I know you want to follow a criteria for sorting messages, you could
> stamp each message up to the second precision (storing milliseconds would
> be a problem) and index them by day (YYYYMMDD), so that way you can find by
> day using 2i search, and sort by time, so the ID won't be needed for
> sorting, that way your design will be more loose, no need for
> synchronization of any type.
>
>   Now if you really need a sequence, if you feel compelled to use a
> sequence, just use a dummy database sequence, any PostgreSQL or MySQL
> service can do that for you, but if you want your app to be loosely couple
> and just exploit the Riak cluster as much as possible, UUID will be your
> winning ticket for message ID. If you are unsure UUID will be unique, add
> another criteria and prefix the UUID with something like sender, etc.
>
> Guido.
>
>
Thank you so much!

Sounds reasonable and feasible :) One question, why storing milliseconds
would be a problem?

The only reason to use an incremental id is to be able to build pagination
easily i.e. last 50 msgs, next 50, and so on. I guess I can do it without
using id, with the help of datestamp and timestamp.

Thanks,
Shashwat


>
> On 23/10/12 10:56, Joshua Muzaaya wrote:
>
> http://couchbase.com CouchBase server has this as a configuration and
> works exactly how riak would. However, using a different storage for
> incremental ids will present challenges. Have you carefully considered
> Couchbase , CouchDB or Big Couch ?
>
>    <http://www.linkedin.com/pub/muzaaya-joshua/39/2ba/202>
> Designed with WiseStamp -
> <http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350986076607&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>Get
> yours<http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350986076607&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>
>
>
>
> On Tue, Oct 23, 2012 at 7:32 AM, Rapsey <rapsey at gmail.com> wrote:
>
>> There is also another trick you can use. Pick a number. Assign every app
>> server you have a number between 1 and N. The number assigned to the server
>> is your starting ID, then increment by N every time you generate an ID from
>> that server. The only limitation is that you have to know in advance how
>> big N can get (it has to be larger than the number of your app servers).
>>
>>
>>  Sergej
>>
>>
>> On Tue, Oct 23, 2012 at 6:00 AM, Shashwat Srivastava <darklrd at gmail.com>wrote:
>>
>>> Thank you Guido. Yes, a secondary index based on date would be immensely
>>> helpful for me to navigate via date. I will do this. An incremental message
>>> id would be helpful for me to get last 50 messages and so forth. I will use
>>> another db for this. Thanks for all your help.
>>>
>>>  Shashwat
>>>
>>>
>>> On Mon, Oct 22, 2012 at 2:46 PM, Guido Medina <guido.medina at temetra.com>wrote:
>>>
>>>>  Don't overkill it with technology, you could use Riak with a simple 2i
>>>> index (integer index YYYYMMDD for the message date so you can search day by
>>>> day backward), and for the message sequence or identifier you could either
>>>> user ANY SQL database sequence or a UUID generator.
>>>>
>>>> HTH,
>>>>
>>>> Guido.
>>>>
>>>>
>>>> On 22/10/12 10:04, Rapsey wrote:
>>>>
>>>>
>>>> On Mon, Oct 22, 2012 at 10:29 AM, Shashwat Srivastava <
>>>> darklrd at gmail.com> wrote:
>>>>
>>>>>
>>>>>  Now, each bucket would have conversation between two users or of a
>>>>> room of a site. The conversation rate for (some) rooms is very high, some
>>>>> 20,000 - 30,000 messages per hour. We have observed that users usually
>>>>> don't access conversations past one week. So, if a bucket has conversation
>>>>> of 3 years, then mostly users would access the recent conversation upto a
>>>>> week or month. Can riak handle this easily? Also, would riak use RAM wisely
>>>>> in this scenario? Would it only keep keys and indexes, corresponding to
>>>>> recent messages per bucket, in RAM?
>>>>>
>>>>>
>>>>  Leveldb backend should.
>>>>
>>>>
>>>>> Finally, what is the best approach for creating keys in a bucket?
>>>>> Earlier, I was planning to use timestamp (in milliseconds). But in a room
>>>>> there can be multiple messages at the same time. As I understand I cannot
>>>>> have a unique incremental message id per bucket (as riak has write
>>>>> capability in all nodes in a cluster so consistency is not guareented).
>>>>> Please correct me if I am wrong. One other way could be to let riak
>>>>> generate key and I use timestamp as a secondary index. But this seems to be
>>>>> a bad design. Also, what would be the best way to achieve pagination for
>>>>> this use case?
>>>>>
>>>>>
>>>>  You could use redis for incremental id's.
>>>>
>>>>
>>>>
>>>>  Sergej
>>>>
>>>>
>>>>  _______________________________________________
>>>> riak-users mailing listriak-users at lists.basho.comhttp://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> riak-users at lists.basho.com
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> riak-users at lists.basho.com
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
>
> --
> *Muzaaya Joshua
> Systems Engineer
> +256774115170*
> *"Through it all, i have learned to trust in Jesus. To depend upon His
> Word"
> *
>
>
>
>
>
> _______________________________________________
> riak-users mailing listriak-users at lists.basho.comhttp://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20121023/a508df08/attachment.html>


More information about the riak-users mailing list