Memory Utilization

Jason J. W. Williams jasonjwwilliams at gmail.com
Thu May 27 11:51:33 EDT 2010


Terrific! Thank you. I realize y'all get a lot of flak for the
comparisons on the wiki, but this is a good differentiator re:
MongoDB.

Mongo is a very nice product, but if you want to start a project small
on a single server and grow if it makes it, you're exposed on the data
corruption side until you can justify a second server to replicate
against.

-J

On Thu, May 27, 2010 at 7:07 AM, Jon Meredith <jmeredith at basho.com> wrote:
> Hi Jason/Sean,
>
> I'll chime in on this one.  Embedded Inno uses exactly the same
> transactional logging as it does under MySQL so it should recover in the
> same way without needing any user intervention.  The tools Sean mentioned
> can be used in dire circumstances (e.g. log files got overrun or double
> buffering was disabled on a filesystem that doesn't ensure atomic writes).
>
> --Jon
>
> On 5/27/10 5:42 AM, Sean Cribbs wrote:
>>
>> Yes, you can use low-level commands to recover lost data from the binlogs.
>>  It's embedded InnoDB, not MySQL's driver, so not everything is exactly the
>> same, but the technique is similar.
>>
>> Sean Cribbs<sean at basho.com>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>>
>> On May 26, 2010, at 11:42 PM, Jason J. W. Williams wrote:
>>
>>
>>>
>>> That's terrific. We're very familiar with InnoDBs buffer pool and
>>> that's exactly the kind of control we're looking for. With the
>>> InnoStore backend is the crash recovery/durability similar to InnoDB
>>> under MySQL?
>>>
>>> -J
>>>
>>> On Wed, May 26, 2010 at 8:13 PM, Sean Cribbs<sean at basho.com>  wrote:
>>>
>>>>
>>>> Riak's usage of memory primarily depends on the backend you choose.
>>>>  Innostore, for example, has a configurable buffer pool (cache) which can
>>>> help you limit the memory footprint.  Bitcask, our most recently released
>>>> backend, keeps only a hash mapping keys to file/offset in memory (with the
>>>> value only on disk), so you could store thousands of keys per node without
>>>> noticing much.
>>>>
>>>> Outside of the backend, Riak generally only needs enough RAM to have
>>>> copies of objects in transit -- either being written or read.  If you see it
>>>> using too much RAM, there are some flags to the Erlang VM that can be
>>>> tweaked.
>>>>
>>>> Sean Cribbs<sean at basho.com>
>>>> Developer Advocate
>>>> Basho Technologies, Inc.
>>>> http://basho.com/
>>>>
>>>> On May 26, 2010, at 6:23 PM, Jason J. W. Williams wrote:
>>>>
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> We have a couple of projects we want to start small and to this point
>>>>> we've been considering MongoDB or Cassandra. Mongo's main drawback for
>>>>> us is it's extensive use of mmap, which can make it a bad neighbor
>>>>> vis-a-vis RAM usage if it has to co-exist with other parts of our
>>>>> stack. Cassandra's has it's own drawbacks for our use cases.
>>>>>
>>>>> Riak looks very interesting, but we're curious about it's memory
>>>>> profile. That is what drives memory consumption in Riak, is it
>>>>> possible to limit the memory consumption, and how does Riak behave
>>>>> when memory exhaustion occurs?
>>>>>
>>>>> -J
>>>>>
>>>>> _______________________________________________
>>>>> riak-users mailing list
>>>>> riak-users at lists.basho.com
>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>>>
>>>>
>>>>
>>
>> _______________________________________________
>> riak-users mailing list
>> riak-users at lists.basho.com
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users at lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>




More information about the riak-users mailing list