Logging chat messages

Guido Medina guido.medina at temetra.com
Tue Oct 23 07:01:29 EDT 2012


I haven't been able to store millis, that is a Riak restriction with 
Integers and Longs. If someone from Basho team could clarify this or 
give any more ideas, it would be great, I ran out of them...

Guido.

On 23/10/12 11:47, Shashwat Srivastava wrote:
>
> On Tue, Oct 23, 2012 at 3:35 PM, Guido Medina 
> <guido.medina at temetra.com <mailto:guido.medina at temetra.com>> wrote:
>
>       I know you want to follow a criteria for sorting messages, you
>     could stamp each message up to the second precision (storing
>     milliseconds would be a problem) and index them by day (YYYYMMDD),
>     so that way you can find by day using 2i search, and sort by time,
>     so the ID won't be needed for sorting, that way your design will
>     be more loose, no need for synchronization of any type.
>
>       Now if you really need a sequence, if you feel compelled to use
>     a sequence, just use a dummy database sequence, any PostgreSQL or
>     MySQL service can do that for you, but if you want your app to be
>     loosely couple and just exploit the Riak cluster as much as
>     possible, UUID will be your winning ticket for message ID. If you
>     are unsure UUID will be unique, add another criteria and prefix
>     the UUID with something like sender, etc.
>
>     Guido.
>
>
> Thank you so much!
>
> Sounds reasonable and feasible :) One question, why storing 
> milliseconds would be a problem?
>
> The only reason to use an incremental id is to be able to build 
> pagination easily i.e. last 50 msgs, next 50, and so on. I guess I can 
> do it without using id, with the help of datestamp and timestamp.
>
> Thanks,
> Shashwat
>
>
>     On 23/10/12 10:56, Joshua Muzaaya wrote:
>>     http://couchbase.com CouchBase server has this as a configuration
>>     and works exactly how riak would. However, using a different
>>     storage for incremental ids will present challenges. Have you
>>     carefully considered Couchbase , CouchDB or Big Couch ?
>>
>>     <http://www.linkedin.com/pub/muzaaya-joshua/39/2ba/202>
>>     Designed with WiseStamp -
>>     <http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350986076607&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>Get
>>     yours
>>     <http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350986076607&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>
>>
>>
>>
>>     On Tue, Oct 23, 2012 at 7:32 AM, Rapsey <rapsey at gmail.com
>>     <mailto:rapsey at gmail.com>> wrote:
>>
>>         There is also another trick you can use. Pick a number.
>>         Assign every app server you have a number between 1 and N.
>>         The number assigned to the server is your starting ID, then
>>         increment by N every time you generate an ID from that
>>         server. The only limitation is that you have to know in
>>         advance how big N can get (it has to be larger than the
>>         number of your app servers).
>>
>>
>>         Sergej
>>
>>
>>         On Tue, Oct 23, 2012 at 6:00 AM, Shashwat Srivastava
>>         <darklrd at gmail.com <mailto:darklrd at gmail.com>> wrote:
>>
>>             Thank you Guido. Yes, a secondary index based on date
>>             would be immensely helpful for me to navigate via date. I
>>             will do this. An incremental message id would be helpful
>>             for me to get last 50 messages and so forth. I will use
>>             another db for this. Thanks for all your help.
>>
>>             Shashwat
>>
>>
>>             On Mon, Oct 22, 2012 at 2:46 PM, Guido Medina
>>             <guido.medina at temetra.com
>>             <mailto:guido.medina at temetra.com>> wrote:
>>
>>                 Don't overkill it with technology, you could use Riak
>>                 with a simple 2i index (integer index YYYYMMDD for
>>                 the message date so you can search day by day
>>                 backward), and for the message sequence or identifier
>>                 you could either user ANY SQL database sequence or a
>>                 UUID generator.
>>
>>                 HTH,
>>
>>                 Guido.
>>
>>
>>                 On 22/10/12 10:04, Rapsey wrote:
>>>
>>>                 On Mon, Oct 22, 2012 at 10:29 AM, Shashwat
>>>                 Srivastava <darklrd at gmail.com
>>>                 <mailto:darklrd at gmail.com>> wrote:
>>>
>>>
>>>                     Now, each bucket would have conversation between
>>>                     two users or of a room of a site. The
>>>                     conversation rate for (some) rooms is very high,
>>>                     some 20,000 - 30,000 messages per hour. We
>>>                     have observed that users usually don't access
>>>                     conversations past one week. So, if a bucket has
>>>                     conversation of 3 years, then mostly users would
>>>                     access the recent conversation upto a week or
>>>                     month. Can riak handle this easily? Also, would
>>>                     riak use RAM wisely in this scenario? Would it
>>>                     only keep keys and indexes, corresponding to
>>>                     recent messages per bucket, in RAM?
>>>
>>>
>>>                 Leveldb backend should.
>>>
>>>                     Finally, what is the best approach for creating
>>>                     keys in a bucket? Earlier, I was planning to use
>>>                     timestamp (in milliseconds). But in a room there
>>>                     can be multiple messages at the same time. As I
>>>                     understand I cannot have a unique incremental
>>>                     message id per bucket (as riak has write
>>>                     capability in all nodes in a cluster so
>>>                     consistency is not guareented). Please correct
>>>                     me if I am wrong. One other way could be to let
>>>                     riak generate key and I use timestamp as a
>>>                     secondary index. But this seems to be a bad
>>>                     design. Also, what would be the best way
>>>                     to achieve pagination for this use case?
>>>
>>>
>>>                 You could use redis for incremental id's.
>>>
>>>
>>>                 Sergej
>>>
>>>
>>>                 _______________________________________________
>>>                 riak-users mailing list
>>>                 riak-users at lists.basho.com  <mailto:riak-users at lists.basho.com>
>>>                 http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>                 _______________________________________________
>>                 riak-users mailing list
>>                 riak-users at lists.basho.com
>>                 <mailto:riak-users at lists.basho.com>
>>                 http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>>
>>
>>             _______________________________________________
>>             riak-users mailing list
>>             riak-users at lists.basho.com
>>             <mailto:riak-users at lists.basho.com>
>>             http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>>         _______________________________________________
>>         riak-users mailing list
>>         riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>>         http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>>
>>     -- 
>>     *Muzaaya Joshua
>>     Systems Engineer
>>     +256774115170 <tel:%2B256774115170>*
>>     *"Through it all, i have learned to trust in Jesus. To depend
>>     upon His Word"
>>     *
>>
>>
>>
>>
>>
>>     _______________________________________________
>>     riak-users mailing list
>>     riak-users at lists.basho.com  <mailto:riak-users at lists.basho.com>
>>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>     _______________________________________________
>     riak-users mailing list
>     riak-users at lists.basho.com <mailto:riak-users at lists.basho.com>
>     http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.basho.com/pipermail/riak-users_lists.basho.com/attachments/20121023/4a687911/attachment.html>


More information about the riak-users mailing list