6 ms·
This kind of post (Postgres! It's all you need!) is getting pretty tiresome. Postgres does not even come close to a full replacement for Elastic, and that's jus
by devin 28d ago
This kind of post (Postgres! It's all you need!) is getting pretty tiresome. Postgres does not even come close to a full replacement for Elastic, and that's just the first bullet.
Looking down the list it is pretty easy to go: Yes, postgres can be used instead of that for extremely basic use cases, but it all goes out the window you actually need any of the power of these other tools.
- airocker 28d agoPostgres its all you need means to me(IMHO) postgres for all Olap (DB + message) , not all analytical databases.
- andriy_koval 28d agoIt can run analytics too, natively on some volumes of data, but also there are more specialized extensions.
- airocker 28d agosorry all OLTP
- onesandofgrain 28d agoif you need elastic youre doing something wrong
- switchbak 28d agoOr something big. Which is often not wrong.
- deleted 28d ago[deleted]
- jjordan 28d agoI'm partial to Typesense, especially for smaller data sets, since it runs primarily in memory, is easy to use and is hella fast. For bigger data sets, I hear good things about Meilisearch.
- anarazel 28d agoFwiw, I, as someone who has worked on Postgres for a long time, also find it quite tiresome. Like there's plenty stuff I wouldn't use Postgres for, and I can probably get get more out of it than most.
- switchbak 28d agoExactly - these recommendations often come with no context or scale provisions. Yes Postgres can work in the small for a lot of things, it can even work at surprising scale if you use it according to its strengths. But if you use it for things it doesn't shine at, at inappropropriate scale - you'll almost certainly run into issues. And resolving those can often be a bigger challenge than choosing a more suitable solution in the first place. But often I think younger/less experienced engineers just have to burn themselves, thus why this never seems to die.
- wolttam 28d agoThat's just it - most use-cases are pretty basic, and if you don’t know what you need then Postgres is probably a great place to start. If you’re just starting out, keep things simple. Otherwise, you probably already know exactly why you need something more than Postgres.
- dewey 28d agoThe point is in general for people to just consider it, often people start out on their side projects or internal company projects and commission Elastic, Redis, Postgres, Kafka before even getting started. In reality they could fit it all into Postgres for a very long time. Nobody is saying that a huge ecommerce store with complicated filtered search logic should throw away their Elasticsearch cluster and switch to Postgres.
- devin 28d agoIf you actually start looking into these things, you often start looking at custom pg extensions, which means you just made the decision to "simplify" your stack by maintaining your own postgres cluster with custom extensions. This is just papering over the fact that you're increasing the complexity and saying "well it's still just postgres!" as you do it.
- dewey 28d agoNot really, most popular extensions are available on GCP/AWS (https://docs.cloud.google.com/sql/docs/postgres/extensions https://docs.cloud.google.com/sql/docs/postgres/extensions) out of the box and there's many things that you can easily run in Postgres without extensions (Queue, KV store).
- pphysch 28d agoInstalling & maintaining a Postgres extension is vastly, vastly simpler than running Elasticsearch and Kafka. Like, how could you even compare these things if you know what you are talking about? Or maybe you are looking at it from "just swipe your credit card at AWS" perspective, in which case "just use Postgres" articles are for a different audience.
- devin 27d agoRunning a small ES with dual save alongside your relational store is trivial. Kafka I will grant you is a lot to operate, but I would not choose Kafka unless I have a Kafka-shaped problem, and if it looks like that I would never, ever choose Postgres. Equating Kafka with “I need a queue” is a laughable comparison.
- otherme123 28d agoI have a lot of troubles with a small private instance of Rocket chat, all due to MongoDb stuff, versions, migrations and backups. I bet almost all private instances of Rocket chat would be perfectly served with Postgres. Posts like this can be tiresome, yet the general consensus among developers seems to be "yeah, Postgre/SQLite is ok for 99% of the cases, but MY case is going to be in the 1%, because I am going to be the next Facebook".
- kumarvvr 28d agoThe power of the other tools mostly shines in large scales. For most applications, though, performance of postgres more than suffices. I tried to use rabbitmq for a small app, installed it, configured it and then it didn't work. Spent a day jumping through hoops getting it right. Dumped it and used postgres, in half an hour. Worked like a charm.
- aaaronic 28d agoSure, best not to overcomplicate early if you don't need it. PG is great and I work with it daily, but it's also not a problem to think about scale early and at least have a notional plan for what to and how to know when scale is becoming an issue in your system as you're designing it. Even PG is overkill and sqlite is more than enough for some of my projects. There are a lot of specialized tools available, but you definitely don't need to put every one in your toolbox. Experience and observation help you make those edits -- and of course there's almost always room for improvement, but "good enough" definitely exists (until it doesn't anymore :D).
- ethbr1 28d agoThis is the way. Notionally building a space/path to scale into architecture early, but delaying implementation of that scaling component until actually needed. Then a system gets most of the benefits of not accidentally making it torturous to rearchitect for scale, without paying the headcount / complexity cost until it's needed.
- osener 28d agoWould your app run equally well with sqlite?
- 0cf8612b2e1e 28d agoI think it would be helpful if some of these posts included scale. There are almost always two groups talking past each other - I run my B2B application, Postgres only, and it is perfect for my 50k MAU. No complaints, sleeping soundly with the low complexity and a two man team. - I work at FAANG, where we have 1 billion DAU, and this is a joke. Would fall over immediately. The dedicated ops teams for Kubernetes, Elastic, and Redis have never complained about scaling issues.
- joshuamoyers 28d agoi think 1 billion DAU is the exception here, so I would not expect everyone to constantly caveat personally.
- 0cf8612b2e1e 28d agoAgreed, but many of the criticisms I am reading here are assuming high scaling requirements and invalidating the approach entirely. When there are many business domains that will comfortably fit within a modestly specced database instance.
- deleted 28d ago[deleted]
- rtpg 28d agoEven just B2B vs B2C is a huge split. Plenty of specialized ERPs in the world that have relatively few "butts in seats" users but have a lot of data to work through. Size of the data, read/write ratio, number of "requests"... lots of axes that change how much pain or not you're in by just pointing to pg
- devin 27d agoThis is a ridiculous cutoff. Are you kidding?
- lelanthran 28d agoI broadly agree, but would tweak those numbers a little for small B2B apps: I ran a small B2B application on a cheap VPS using PostgreSQL as a primitive messaging interface, and even on a small VPS 50k DAU won't even cause the machine to break a sweat.
- philippemnoel 28d agoYou're right that vanilla Postgres doesn't come close to replacing Elastic. There are efforts to resolve this, though, like ParadeDB: https://github.com/paradedb/paradedb https://github.com/paradedb/paradedb (disclaimer: I work for ParadeDB)
- majewsky 28d ago"Disclaimer" means "don't take this seriously because I'm not an expert". You mean "disclosure".
- philippemnoel 28d agoMy English fails me again :'). Thank you for the correction!
- aitchnyu 28d agoI see Tantivy mentioned in your readme but AFAICT there is no PG-Tantivy sync. I also see "native vector support is coming to our search index soon". Could you clarify? What do you suggest for a language like Malayalam which has no native support, preferably with low RAM requirements?
- vb-8448 28d ago> Postgres does not even come close to a full replacement for Elastic Size matters! For most of the application out there elastic (or kafka or any other specialized tool) is just too much(and too costly). They can do fine with postgres or mysql. Actually, I'd argue that in a lot of cases even postgres is too much, probably sqlite is enough.
- devin 27d agoWhat do you know about full text search? Faceted search? Custom tokenization? It is wild to see people talk about Postgres fulltext like it’s all you could need, even for small apps.
- vb-8448 21d agoI didn't say that is "all you could need", I said is "just fine for a lot of use cases out there", which implies that there are use cases where you could need specialized tools ...