13 ms·
“MongoDB is dead. Long live Postgresql”
- egeozcan 13y agoI'm nearly sure one day someone will write a MongoDB compatibility layer on top of PostgreSQL
- jeffdavis 13y agohttps://github.com/umitanuki/mongres https://github.com/umitanuki/mongres
- seiji 13y agoPostgreSQL has native json support now. What else is missing? Just a protocol implementation? I'd love to see MongoDB give up and become a PostgreSQL consultancy. Everybody I talk to in the field has the exact same Mongo story: "We love JSON! We use JSON everywhere! We just wanted a DB with native JSON support. We didn't look at the implementation details. We only looked at their marketing. Now we wake up at 3am to fix it every night and lose data every day. Somebody help us. We love JSON."
- busterarm 13y agoI really would love it if somebody could go back in HN-Time and track the data on Mongo posts & comments. I've always been slightly skeptical about it, but it always seemed to me that there was a long love affair with it overall. Then people started voicing their frustrations and the community was divided and now it looks acrimonious for everyone.
- this_user 13y agoIsn't that true for most of those 'flavour of the month' technologies? First a shiny new piece of technology shows up that promises to solve all the issues you have with a mature and widely adopted solution. People get exited and at some point the media picks up on it und starts the hype cycle. More and more decision makers hear about the technology (probably aided by marketing) and decide to adopt it. Implementation takes place and the new technology is deployed into the live environment. Some time goes by and the first issues appear, workarounds and tweaks are devised to mitigate those. After even more time the technology's inherent flaws become apparent. At this point either someone else develops a new iteration of that technology with the promise of solving those issues or the technology is abandoned altogether as it is unable to deliver sufficient value and cannot be fixed.
- busterarm 13y agoTotes. I just wanna see pretty graphics of the trend. These things interest me.
- yen223 13y agocoughbitcoincough
- jacques_chester 13y agohttp://en.wikipedia.org/wiki/Hype_cycle http://en.wikipedia.org/wiki/Hype_cycle
- egeozcan 13y agoIs there an algorithm which measures negativity in a given text? Considering the level of grammatical errors and sarcasm involved, one would need a really complicated system, I'd bet. A google search brought me to Sentiment Analysis page on Wikipedia[0], which, after the initial skimming, doesn't seem to link to any implementations. [0]: http://en.wikipedia.org/wiki/Sentiment_analysis http://en.wikipedia.org/wiki/Sentiment_analysis
- TheZenPsycho 13y agoI know of one implementation: http://nltk.org http://nltk.org
- egeozcan 13y ago> What else is missing? Just a protocol implementation? Yes. Actually that's why I said that I'm nearly sure. As a side note: We may also need some rumors on being "web scale" (Actually I don't even know much about the events/comments/whatever which lead to that famous video but I still find it funny)
- jcampbell1 13y agoIt is probably not as simple as "supports json now". Imagine if HN comments were stored as a JSON document: Client A: Read JSON. Client B: Read JSON. Client A: Append new comment to json document. Client B: Append new comment to json document. Client A: Save JSON Client B: Save JSON A's comment will get deleted. My understanding is that Mongo DB does have a way to append a record within a document, but Postgres does not. I am in no way advocating for MongoDB (I dislike it). I am just saying that I understand that MongoDB has much more sophisticated updates capability than Postgres.
- viraptor 13y agoIn that case you can (should) just "SELECT ... FOR UPDATE" in your transaction. This should prevent the issue. Client B will wait with the read until client A commits.
- AlisdairO 13y agoOr you can use serialisable transaction isolation and retry on conflict.
- jeltz 13y agoI am pretty sure that just running UPDATE is enough. If an UPDATE happens to get stuck waiting for a lock it is re-evaluated. EDIT: Found where it is explained in the documentation. I do not see why this should not also apply to JSON. http://www.postgresql.org/docs/9.3/static/transaction-iso.html#XACT-READ-COMMITTED http://www.postgresql.org/docs/9.3/static/transaction-iso.ht...
- csmuk 13y agoThat's sensible when you consider that postgresql used to be a SQL layer over postgres. Not sure if that is the case now.
- dragonwriter 13y ago> That's sensible when you consider that postgresql used to be a SQL layer over postgres. Were they ever actually separate layers? I thought that PostgreSQL was a rename of Postgres that happened shortly (one-two versions) after they swapped query languages from the Ingres-derived QUEL to SQL.
- csmuk 13y agoPostgres95 had both if I remember. Sorry was nearly 20 years ago now - bit rusty :)
- deleted 13y ago[deleted]
- willvarfar 13y agoOr was it PostgresME?
- lucisferre 13y agoI hope so. I've become so used to Mongoid's API and I've never really liked the "invisible" properties of Active Record.
- sanderjd 13y agoYeah, it would be cool if AR came with a `field` method that could be made non-optional through configuration, and enforce that only those fields are accessible.
- mrud 13y agoNot exactly what you were looking for, but you can connect from PostgreSQL to MongoDB via Foreign Data Wrappers - https://github.com/citusdata/mongo_fdw https://github.com/citusdata/mongo_fdw
- egeozcan 13y agoI know about Foreign Data Wrappers and use them extensively. To clarify, what I want is to have a quick engine swap for some legacy apps without touching anything but config files.
- deleted 13y ago[deleted]
- ddorian43 13y agohttps://github.com/JerrySievert/mongolike https://github.com/JerrySievert/mongolike https://github.com/citusdata/mongo_fdw https://github.com/citusdata/mongo_fdw
- gaius 13y agoThere is already one on top of DB/2. http://www.theregister.co.uk/2013/06/05/ibm_db2_mongodb/ http://www.theregister.co.uk/2013/06/05/ibm_db2_mongodb/
- nasalgoat 13y agoFunny you mention this, we've written one as we transition from MongoDB to PostgreSQL. It was actually much easier than I expected, because MongoDB doesn't really do anything - all the JOIN logic is in code and it mostly consists of "find" and "findOne" calls with minor filtering that is easily translatable to SQL. The hardest part is re-training all the devs to stop thinking like Mongo devs (ie. "I must make five queries and join the info in code") and let the DB do the heavy lifting it was designed to do.
- mml 13y agoThe hstore enhancements coming in psql 9.4 will pretty much put mongo out to pasture. "Mongodb" already nearly exists as a single column type, 9.4 will complete it.
- threeseed 13y agoRight. Just like MySQL/Oracle was put out to pasture. And if you think MongoDB is only popular because it is a JSON store then it shows just little you know about the database landscape and about how developers actually use databases.
- mml 13y agoFor those of us that use all 3 of those, your statement is in large part true. I'd also hazard that Mongodb isn't actually that popular outside the HN buzz bubble.
- functional_test 13y agoSeriously, another case of using Mongo incorrectly? I want to believe all the Mongo hate, but I can't because I always find out that the actual problem was one or more of: * didn't read the manual * poor schema * didn't maintain the database (compactions, etc.) In this case, they hit several: " Its volume on disk is growing 3-4 times faster than the real volume of data it store;" They should be doing compactions and are not. Using PostgreSQL does not avoid administration; it simply changes the administration to be done. "it eats up all the memory without the possibility to limit this" That's the idea -- that memory isn't actually used though; it's just memory mapping the file. It will swap out for something else that needs the space unless you are actively using all the data, in which case you really are using all your memory. Which is why you should put it on its own server... "it begins to slow down the application because of frequent disk access" "Finally we sleep quietly, and don’t fear that mongodb will drive out redis to swap once again." You should be running Mongo on a server by itself. At the very least, if you're having disk contention issues, don't run it on the same server as your other database. I'm not sure you always need to read the manual for everything, but for your production database, it's probably worth it.
- dev360 13y agoIn all fairness, the compaction is a major pain in Mongo. I get a little worked up about this because I cant think of another database that handles compaction this poorly, but feel free to correct me if Im wrong.
- ehwizard 13y agoHave you tried turning on power of 2 allocation? In general, it makes compaction much less important. Though online compaction is definitely needed.
- dev360 13y agoYes we switched to it a month ago, which improved it but like you said, we are still having to compact frequently and having the hassle of switching the primary. Cassandra has turned out to be much more performant and easier to maintain for our use case.
- iand 13y agoTypo in title.
- pilif 13y agoAll the philosophical issues and /(No)?SQL/ discussions aside, as a heavy user of Postgres and a user of Errbit, this is very good news to me. I have not much experience with running Mongo, but I have a ton of experience with running Postgres. Even better: The application I'm using Errbit the most for is already running in front of a nicely replicated and immensely powerful postgres install. Being able to put the Errbit data there is amazing. This is some of the best news I've read today :-)
- trekky1700 13y agoThis just makes me wonder why they chose Mongo in the first place. It sounds like they didn't really consider their needs when initially choosing databases. Mongo has some benefits that when properly implemented far outweigh the negatives. At the same time, it's still relatively young, and doesn't have the "maturity of process" that makes older SQL engines so easy to manage/implement. Eventually, I'm sure, Mongo will solve these issues and be a great database for those who need to utilize its many virtues.
- jmngomes 13y agoBecause 1) a lot of startups seem to choose "startup technology", i.e. whatever famous startups are using, just because it seems fashionable and/or they don't consider if it'll actually solve their specific problem; or 2) they're technically curious and end up using it just for fun, even if it's not a good fit for their problem. I've seen people using Redis for their MVPs, which is hardly necessary to serve 100 or 1000 or 10000 users. When you have a hammer at hand, everything looks like a nail.
- aidenn0 13y agoAs far as Redis goes, is there really much of anything in the space between bdb style KV stores and Redis? If you have design reasons for wanting your KV store in a separate process, why not use Redis?
- jmngomes 13y agoI agree with your point, my example was about using Redis for performance reasons when designing/building an MVP, a stage where you are unlikely to have a (real) performance problem.
- Xorlev 13y agoMongoDB is easy. I'll be the first one to spit-roast MongoDB with war stories, but the biggest benefit I keep coming back to is ease of use for a developer. It's very easy to change your data model and rapidly iterate. As soon as your project starts to solidify, the main benefit of MongoDB is gone. It still lives in some of my personal projects (e.g. <100mb of data, because even flat files can't mess that up).
- memracom 13y agoLets just say that PostgreSQL answers the criticisms of relational databases that led to NoSQL. The complaints all boiled down to saying that the RDBMS forced you to do things one way and that it was cumbersome. PostgreSQL evolved and fixed the most annoying issues like JSON support and schemaless key-value store support. That's the way open source is supposed to work. Now folks are learning that throwing out the baby with the bathwater leads to more complexity than just learning how to use a relational database. The pendulum has swung back.
- yid 13y ago> PostgreSQL evolved and fixed the most annoying issues like JSON support and schemaless key-value store support. As I recall, automatic sharding was on that list, and pg doesn't attempt to tackle that afaik.
- deleted 13y ago[deleted]
- lucian1900 13y agoMongo doesn't really do that in a way you can reliably use in practice either, though. Its sharding offers a subset of operations over an inconsistent view of your dbs. You can do that with Postgres trivially, and even automatically with postgres_fdw and writeable views.
- remon 13y agoThis is not true at all. The actual realization the past years is that strictly enforced relationality (is that a word?) and transactions are constructs that are not always or even rarely actually needed. Eventual consistency, schemaless data modelling and so on picked up steam and for good reasons. Every technology that survives the "Oh, new toy!" stage has a place or it wouldn't still exist. It is up to developers to choose the appropriate technology for them and their projects. That isn't to say that a lot of persistence problems cannot be solved by an RDBMS, a k/v store and a document store. In that case just base your decision on other drivers (comfort level, cost, and so on)
- cullenking 13y agoMaybe I am just incredibly lucky, but mongodb has worked fine for ridewithgps.com - we are sitting at 670gb of data in mongo (actual DB size, indexes included) and haven't had a problem. Replica sets have been fantastic, I wish there was another DB out there that did auto-failover as cleanly/easily as mongo does. We've had a few server crashes of our primary, and aside from 1-2 seconds or so of errors as requests come in before the secondary is promoted, it's transparent. With that being said, we are using it to store our JSON geo track data, most everything else is in a mysql database. As a result we haven't run into limitations around the storage/query model that some other people might be experiencing. Additionally, we have some serious DB servers so haven't felt the pain of performance when exceeding working memory. 192gb of ram with 8 RAID10 512gb SSDs probably masks performance issues that other people are feeling. Final note: I'll probably be walking away from mongo, due to the natural evolution of our stack. We'll store high fidelity track data as gzipped flat files of JSON, and a reduced track inside of postgis. tl;dr - using mongo as a very simple key/value store for data that isn't updated frequently, which could easily be replaced by flat file storage, is painless. YMMV with other use cases.
- blablabla123 13y ago>data that isn't updated frequently How often did you update your data then? In my current project I am seeing locking issues in my way soon...
- cullenking 13y agoI think I peaked at 5% lock utilization this year, so I haven't seen any real issues. Our actual track data isn't updated frequently. Mostly it serves as an archive for a user, and is only seen by 1-2 other people. Most people use our service to store all their activities, which for the most part are really boring. They are interested in aggregate metrics like "I've ridden 200 miles this month". A smaller portion of our data is from planning a route using google maps, which has much more modest storage requirements, since it's optimized data (one point every mile if it's a straight line) instead of 1hz logging from a GPS unit. This stuff is edited, but I'd say only 10% of planned routes are ever modified, so actual updates on the track data are small.
- pilif 13y agoThe title is a bit misleading. This is basically an announcement of a fork of Errbit that has Postgres support. Additionally, the fork was announced as an issue on errbit with no discussion or as an official pull request. I would not consider this good etiquette. If you fork your project (especially without discussing the intention first), adding a bug to the original project isn't a very nice thing to do. An official pull request would be nicer or, even better, don't bother the original project, but just announce your fork over other channels. Even better would be to at least discuss the issue with the original project - maybe they agree and you can work together.
- spellboots 13y agoI disagree, I think that opening an issue on github is a good way to start a discussion about a feature. Many projects accept feature requests this way and if anyone did the same for one of my projects, this would be the way I would prefer them to handle it.
- felideon 13y agoThe thing is it sounds like he is just promoting his own fork he started 11 months ago, rather than "starting a discussion." > We suggest to put errbit on PG. For those who want to try - the code here: https://github.com/Undev/errbit/tree/pg-upstream https://github.com/Undev/errbit/tree/pg-upstream The problem here is that the bad English grammar could have given the wrong impression. Maybe he is just saying: "hey guys, you should consider migrating to postgresql. here's some code you can check out that has worked for us." Rather than: "hey guys, screw Errbit/MongoDB, use our fork!"
- eonil 13y agoIt seems that latter one is far better description of how people feels.
- deleted 13y ago[deleted]
- 13y ago
- jvvlimme 13y agoIf you want to use MongoDB in a project and you don't intend to rely heavily on the aggregation framework, the consider TokuMX (http://www.tokutek.com/products/tokumx-for-mongodb/ http://www.tokutek.com/products/tokumx-for-mongodb/) as it alleviates many of the shortcomings of MongoDB (data compression, document level locking for writes, ...) + it adds transactions. It's a drop in replacement so it will work with current drivers. (if you have a running mongo cluster however expect quite some work if you want to migrate) (I have no affiliation with TokuTek whatsoever except that I use their product)
- stonewhite 13y agoIt is basic an ops-friendly mongo fork with _obviously_ better engineering decisions. I hope mongodb will support pluggable storage engines soon.
- jeffdavis 13y ago"Its volume on disk is growing 3-4 times faster than the real volume of data it store[sic]" Are they saying that it has a high constant overhead to the data, or are they saying the storage grows in a super-linear fashion?
- remon 13y agoIt's constant overhead with some spikiness to it. See my reply in this thread for details.
- 1945 13y agoThis title is why I've been reading Hacker News less and less.
- filipedeschamps 13y agoThis is what I call a click bait title.
- kldavenport 13y agoTheir use case didn't seem especially Mongo-centric, I wonder why they chose to go down the road. We used MongoDB TokuMX to improve performance: http://www.tokutek.com/resources/benchmark-results/tokumx-benchmark-hdd/ http://www.tokutek.com/resources/benchmark-results/tokumx-be...
- endijs 13y agoI'm no MongoDB expert, but recently started to look into this db. Can anyone tell me (from experience, not from promo materials) - for which use cases MongoDB is good fit and for which ones it's not? It's clear that it can't fit for everyone. That's why it would be good to know in advance, for what it most likely to find and for what it's most likely not to fit.
- sigzero 13y agohttp://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
- endijs 13y agoThis is more like how it is in theory. I was hoping for more real life stories. "We had problem X, tried MongoDB, but failed because of ...", "We had problem Z, MongoDB works better than R, because..."
- mikegioia 13y agoIt has the benefits and ease of use of a json document store, it allows you to do SQL style where clauses, it takes about a minute to install and start using, there are a wide range of drivers available for many languages, and it has a simple javascript map/reduce. on the flip side, it implements database level locking, uses more disk/RAM than it probably should, and can start to give you headaches if you try to do a lot of writes at once. edit: to give you a real world example, we use mariadb for storing everything persistently. however, a lot of data like "number of teachers in school A" is aggregated and too difficult to run in real time when we render paged results. to get around that, we use mongo as a document store and use its SQL like querying to generate the paged search results. this lets us sort/filter on the data without having to do everything in SQL.
- jeltz 13y ago> to give you a real world example, we use mariadb for storing everything persistently. however, a lot of data like "number of teachers in school A" is aggregated and too difficult to run in real time when we render paged results. to get around that, we use mongo as a document store and use its SQL like querying to generate the paged search results. this lets us sort/filter on the data without having to do everything in SQL. This use case should be possible to solve with the JSON type in PostgreSQL. The indexing in PostgreSQL is just as advanced in 9.3 and will be better than MongoDB in 9.4 if a couple of patches land.
- r0muald 13y agoIn case you missed it, this submission is not about PostgreSQL vs MongoDB. It's about the crazy GIF parade in the comments interleaved with thumbs up emojis. You don't see such stuff often on github :)
- mtolan 13y agoIs PostgreSQL web scale? I will use it if it is web scale.
- deleted 13y ago[deleted]
- WoodenChair 13y ago"Nobody ever got fired for buying IBM." Nobody ever got fired for using PostgreSQL.
- coolrhymes 13y agoRDS now supports Postgres. It supports both hot & cold swaps. Hopefully in future it will support read replicas.
- WalterSear 13y agoDoes anyone have a recommendation for an authoritative guide to either Postgres or Mongodb? One that does more than show you where the levers are, that is.
- deleted 13y ago[deleted]
- weixiyen 13y ago> Finally we sleep quietly, and don’t fear that mongodb will drive out redis to swap once again. Well duh, Mongo was designed to live on its own server as it tries to claim all of the free memory available. Putting it on the same server with Redis makes no sense. The case that caused you sleepless nights does not apply to 99% of projects out there.
- poseid 13y agoAnyone compared MongoDB with other document stores, e.g. with https://github.com/triAGENS/ArangoDB https://github.com/triAGENS/ArangoDB ?