6 ms·

10gen doesn't publish any benchmarks. See http://www.mongodb.org/display/DOCS/Benchmarks http://www.mongodb.org/display/DOCS/Benchmarks for the official positi

10gen doesn't publish any benchmarks. See http://www.mongodb.org/display/DOCS/Benchmarks http://www.mongodb.org/display/DOCS/Benchmarks for the official position. I transcribed the MongoDB vs. Riak part of the Changelog webcast (available at http://thechangelog.com/post/3742814720/episode-0-5-1-mongodb-nosql-and-web-scale-with-eliot-hor http://thechangelog.com/post/3742814720/episode-0-5-1-mongod...): ------------------------ Riak and all the dynamo-style databases are really distributed key/value stores and I think, you know, I've never used Riak in production, but I have no reason not to believe it's not a very good, highly scalable distributed key/value store. The difference between something like Riak and Mongo is that Mongo tries to solve a more generic problem. A couple of key points: one is consistency. Mongo is fully consistent, and all dynamo implementations are eventually consistent and for a lot of developers and a lot of applications, eventual consistency just is not an option. So I think for the default data store for a web site, you need something that's fully consistent. The other major difference is just data model and query-ability and being able to manipulate data. So for example with Mongo you can index on any fields you want, you can have compound indexes, you can sort, you know, all the same types of queries you do with a relational database work with Mongo. In addition, you can update individual fields, you can increment counters, you can do a lot of the same kinds of update operations you would do with a relational database. It maps much closer to a relational database than to a key/value store. Key/value stores are great if you've got billions of keys and you need to store them, they'll work very well, but if you need to replace a relational database with something that is pretty feature-comparable, they're not designed to do that. ----------------------- It starts at minute 17. edited: formatting.

batasrki 15y ago
>Mongo is fully consistent Can you please explain this for a case where there are multiple replica sets, the database is sharded and nodes are across data centers? What's sacrificed? Something must be.
- mihasya 15y ago
  The implication is that the people for whom eventual consistency is not an option will never reach a data set size or availability requirement that'll require them to use replication and experience the lag (and eventual consistency) involved.
  - batasrki 15y ago
    Among major features touted are auto-sharding and replica sets. I don't know if the implication is that it's only for web apps/websites that won't need those
  - alexpopescu 15y ago
    That's not completely true. Take a look at Google's Megastore paper: http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf James Hamilton has a good summary of the ideas in the paper: http://perspectives.mvdirona.com/2011/01/09/GoogleMegastoreTheDataEngineBehindGAE.aspx http://perspectives.mvdirona.com/2011/01/09/GoogleMegastoreT...
    - mihasya 15y ago
      think you're viewing my statement out of the necessary context..
- ethangunderson 15y ago
  When we talk about consistency, we're talking about taking the database from one consistant state to another. With replica sets, we're still only dealing with one master. We can get inconsistant reads from the replicas, but we're always writing to a single master, which allows that master to determine the integrity of a write. With sharding, we're still only dealing with one canonical home for a specific key(defined by the shard key). (besides latency, I'm not sure how datacenters would affect this) What we're giving up in this case is availability. If an entire replica set goes down, we can't read or write any data for the key ranges contained on those machines. This is where Riak shines. With Riak, any node can accept writes, and nodes contain copys of several other nodes data. What that means is, as long as we have one node up, we can write to the database. Because of this, there is the possibility of nodes having different views of the data. This is handled in a number of ways(read repairs, vector clocks, etc). Check out the Amazon Dynamo paper for more info, great read. I'm sure I'm missing some stuff, but I think that covers the gist of it. EDIT: One thing that I want to make clear, I don't think that one architecture is better than the other. They each have their own pros and cons, and are really suited to solve different problems.
  - batasrki 15y ago
    None of this is guaranteed by default. By default, writes are flushed every 60 seconds. By default, there's no journaling. How can one claim full consistency if the the former two points are true? Don't get me wrong, I love mongo. I'm building a web app backed by it. But the marketing talk is grating, which whT this post nails.
    - mathias_10gen 15y ago
      I think those two issues are orthogonal to consistency. In ACID, consistency and durability are two different letters and CAP doesn't even mention durability. Are you referring to another definition of consistency?
      - batasrki 15y ago
        How is flushing a write every 60 seconds orthogonal to consistency? If there's a server crash between the write to RAM and the subsequent flush, the data is lost, is it not? How do you guarantee the data is there in that case?
- kchodorow 15y ago
  MongoDB is partition tolerant and consistent. You can never have multi-master with MongoDB, which is required for "always writable." However, it can be readable. Our CEO did a series of posts on distributed consistency, see http://blog.mongodb.org/post/475279604/on-distributed-consistency-part-1 http://blog.mongodb.org/post/475279604/on-distributed-consis....
  - benblack 15y ago
    If a slave can continue serving reads whilst partitioned from a master that continues to accept writes then you cannot guarantee consistency. If a slave cannot serve reads when partitioned then you aren't available. If a master cannot accept writes when partitioned then you aren't available. See this excellent post from Coda Hale on why it is meaningless to claim a system is partition tolerant http://codahale.com/you-cant-sacrifice-partition-tolerance/ http://codahale.com/you-cant-sacrifice-partition-tolerance/. One love. - Lil' B
    - jorgeortiz85 15y ago
      In a replica set configuration, all reads and writes are routed to the master by default. In this scenario, consistency is guaranteed. (You can optionally mark reads as "slaveOk", but then you admit inconsistency.) This does sacrifice availability (in the CAP sense), but I haven't heard anyone claim otherwise.
      - benblack 15y ago
        "In a replica set configuration, all reads and writes are routed to the master by default. In this scenario, consistency is guaranteed." One would hope that reading and writing a single node database was consistent. This is table stakes for something calling itself a persistent store. Claiming partition tolerance in the above is the same as claiming availability. The former claim has been made. Rest left as exercise for the reader. Namasté. - Lil' B
        jorgeortiz85 15y ago
- mathias_10gen 15y ago
  In the sharded case, at any given moment each object will still live on exactly one replica set, which will have at most one master. You can do operations (such as findAndModify http://bit.ly/ilomQo http://bit.ly/ilomQo) that require a "current" version of an object because all writes are always sent to the master for that object. You can also choose to accept a weaker form of consistency for some reads by directing them to slaves for performance. This decision can be made per-operation from most languages. As for trade-offs: Relative to a relational db, there is no way to guarantee a consistent view of multiple objects because they could live on different servers which disagree about when "now" is. Relative to an eventually consistent system, you are unable to do writes if you can't contact the master or a majority of nodes are down.

batasrki 15y ago

>Mongo is fully consistent Can you please explain this for a case where there are multiple replica sets, the database is sharded and nodes are across data centers? What's sacrificed? Something must be.

mihasya 15y ago
The implication is that the people for whom eventual consistency is not an option will never reach a data set size or availability requirement that'll require them to use replication and experience the lag (and eventual consistency) involved.
- batasrki 15y ago
  Among major features touted are auto-sharding and replica sets. I don't know if the implication is that it's only for web apps/websites that won't need those
- alexpopescu 15y ago
  That's not completely true. Take a look at Google's Megastore paper: http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf James Hamilton has a good summary of the ideas in the paper: http://perspectives.mvdirona.com/2011/01/09/GoogleMegastoreTheDataEngineBehindGAE.aspx http://perspectives.mvdirona.com/2011/01/09/GoogleMegastoreT...
  - mihasya 15y ago
    think you're viewing my statement out of the necessary context..
ethangunderson 15y ago
When we talk about consistency, we're talking about taking the database from one consistant state to another. With replica sets, we're still only dealing with one master. We can get inconsistant reads from the replicas, but we're always writing to a single master, which allows that master to determine the integrity of a write. With sharding, we're still only dealing with one canonical home for a specific key(defined by the shard key). (besides latency, I'm not sure how datacenters would affect this) What we're giving up in this case is availability. If an entire replica set goes down, we can't read or write any data for the key ranges contained on those machines. This is where Riak shines. With Riak, any node can accept writes, and nodes contain copys of several other nodes data. What that means is, as long as we have one node up, we can write to the database. Because of this, there is the possibility of nodes having different views of the data. This is handled in a number of ways(read repairs, vector clocks, etc). Check out the Amazon Dynamo paper for more info, great read. I'm sure I'm missing some stuff, but I think that covers the gist of it. EDIT: One thing that I want to make clear, I don't think that one architecture is better than the other. They each have their own pros and cons, and are really suited to solve different problems.
- batasrki 15y ago
  None of this is guaranteed by default. By default, writes are flushed every 60 seconds. By default, there's no journaling. How can one claim full consistency if the the former two points are true? Don't get me wrong, I love mongo. I'm building a web app backed by it. But the marketing talk is grating, which whT this post nails.
  - mathias_10gen 15y ago
    I think those two issues are orthogonal to consistency. In ACID, consistency and durability are two different letters and CAP doesn't even mention durability. Are you referring to another definition of consistency?
    - batasrki 15y ago
      How is flushing a write every 60 seconds orthogonal to consistency? If there's a server crash between the write to RAM and the subsequent flush, the data is lost, is it not? How do you guarantee the data is there in that case?
kchodorow 15y ago
MongoDB is partition tolerant and consistent. You can never have multi-master with MongoDB, which is required for "always writable." However, it can be readable. Our CEO did a series of posts on distributed consistency, see http://blog.mongodb.org/post/475279604/on-distributed-consistency-part-1 http://blog.mongodb.org/post/475279604/on-distributed-consis....
- benblack 15y ago
  If a slave can continue serving reads whilst partitioned from a master that continues to accept writes then you cannot guarantee consistency. If a slave cannot serve reads when partitioned then you aren't available. If a master cannot accept writes when partitioned then you aren't available. See this excellent post from Coda Hale on why it is meaningless to claim a system is partition tolerant http://codahale.com/you-cant-sacrifice-partition-tolerance/ http://codahale.com/you-cant-sacrifice-partition-tolerance/. One love. - Lil' B
  - jorgeortiz85 15y ago
    In a replica set configuration, all reads and writes are routed to the master by default. In this scenario, consistency is guaranteed. (You can optionally mark reads as "slaveOk", but then you admit inconsistency.) This does sacrifice availability (in the CAP sense), but I haven't heard anyone claim otherwise.
    - benblack 15y ago
      "In a replica set configuration, all reads and writes are routed to the master by default. In this scenario, consistency is guaranteed." One would hope that reading and writing a single node database was consistent. This is table stakes for something calling itself a persistent store. Claiming partition tolerance in the above is the same as claiming availability. The former claim has been made. Rest left as exercise for the reader. Namasté. - Lil' B
      - jorgeortiz85 15y ago
mathias_10gen 15y ago
In the sharded case, at any given moment each object will still live on exactly one replica set, which will have at most one master. You can do operations (such as findAndModify http://bit.ly/ilomQo http://bit.ly/ilomQo) that require a "current" version of an object because all writes are always sent to the master for that object. You can also choose to accept a weaker form of consistency for some reads by directing them to slaves for performance. This decision can be made per-operation from most languages. As for trade-offs: Relative to a relational db, there is no way to guarantee a consistent view of multiple objects because they could live on different servers which disagree about when "now" is. Relative to an eventually consistent system, you are unable to do writes if you can't contact the master or a majority of nodes are down.