6 ms·
Show HN: PostgreSQL performance and cost across 23 EC2 instance types
Hey! I'm Andrei.
I got frustrated by how people tend to build overcomplicated backend systems, being "motivated" by big tech case studies and popular books.
So, I started exploring lean architecture, and building my digital garden of ideas, approaches and data that align with this direction.
Here I want to present one of the tools – Sizing tool for PostgreSQL. I've benchmarked PostgreSQL on different EC2 instances and disks, with different initial data sets to see performance that these instances can give you. And I've built a tool to visualize this data, which I welcome you to explore.
So, you can put your usual input parameters, like needed RPS and disk size as input, and find out which instance will be the most cost-efficient for your needs.
You can read about the methodology here: https://postgres.saneengineer.com/about https://postgres.saneengineer.com/about
I've tested one workload – mixed 90/10 read/write, and only selected configurations. But it is extensible, and I (and you – benchmark is open source: https://github.com/anivaniuk/sanebench https://github.com/anivaniuk/sanebench) can run more configurations to have more data represented.
Does it look interesting? What workload should I benchmark next?
- Masho_17 2mo ago[flagged]
- valentynkit 2mo ago[dead]
- deleted 2mo ago[deleted]
- rnagulapalle 2mo ago[flagged]
- ballislife30 2mo agoWould love to see a comparison between Aurora PostgreSQL and self-host PostgreSQL on the same EC2 instance type.
- anivan_ 2mo agoGood point! I kept the configuration of the Postgres pretty close to the defaults, and it would be interesting to compare it with the same default Aurora Postgres. And it should be easy to add - I'll check it, thanks!
- toredash 2mo agoI would really see this compared to what AWS is offering via RDS
- anivan_ 2mo agoYes! This was my initial dilemma - whether to test RDS or self-hosted Postgres on EC2. I decided to start with EC2 to be a bit more "pure", and remove cost overhead of RDS. But support for RDS is my next candidate for development. Plus, comparison would also be interesting.
- nijave 2mo agoWould be interesting to see huge pages and io2 impact. I did a smaller version on Azure and disk latency had a massive impact much more so than max IOPs (although their crappy storage offering needed like 64-128 iodepth to get advertised iops). Results seem mostly in line with expectations. Iirc vcpu is threads so on arm64 you get 4 smt1 cores vs Intel/AMD you get 2 smt2 cores.
- anivan_ 2mo agoGood points, thanks. On huge pages: this is also about RDS vs self-managed EC2 Postgres. RDS effectively has "on" by default, but default self-managed (that I benchmarked) is "try" which is effectively "off". I'll update the methodology page to cover that, and, yeah, it makes sense to cover that separately. io2 is on my future-work list. And agree, I have the same feelings about IOPS.
- Rafuino 2mo agoMost of the recent AMD instances on AWS have SMT off so 1 core per vCPU. None of those seem to be tested here though
- crudgen 2mo agoInteresting, is there something like this for azure
- anivan_ 2mo agoI was initially inspired by the https://instances.vantage.sh/ https://instances.vantage.sh/, so, like them, I want to add other providers later. Like Azure and GCP. It would also be interesting to have cross-provider comparison. I think it's doable. Thanks!
- mattlong 2mo agoI'd be very curious to see you add the Optimized Reads instance types, e.g. r8gd or m8gd, to your benchmark. They add a local NVMe-based SSD block storage that serves as a cache in front of the network-based disks among other use cases. They have been a huge win for us for a read-heavy workload where the dataset is significantly larger than memory. Edit: Apologies, on a closer read, I realize you were not testing RDS but managing Postgres on EC2 directly.
- anivan_ 2mo agoYes, I want to cover RDS with all of its specifics as the next step. Thanks for highlighting this!
- Rafuino 2mo agoGotta get the AMD instances in there. Not seeing M8a, R8a, C8a, for example
- anivan_ 2mo agoYeah, good point - I've limited the scope to make the first release more lightweight. I'll add them in the next one, thanks!
- handfuloflight 2mo agoWhat are your thoughts on how design changes if writes become much heavier, ie. when recording agent operations?
- anivan_ 2mo agoI think we have a lot of DBs specialised for heavy write, like anything with LSM-tree in their base. My vision is that there will be more movement in this direction, but still, we need first to understand limits of "easy to work with" databases like Postgres. It's easy to underestimate what Postgres can do. And only when it's not enough, move to something LSM-tree-based.
- TurdF3rguson 2mo agoAn info badge next to rps would be nice, I'm trying to guess what r is. Rows? Records? Requests?
- anivan_ 2mo agoIt's requests per second, and I briefly described that on https://postgres.saneengineer.com/about https://postgres.saneengineer.com/about Small badges will be helpful, though. I added hovers in some places, like for latency numbers, but I think more visible badges will work better, thanks!
- TurdF3rguson 2mo agoIs requests the same as queries? Maybe this is an AWS thing and that's why I don't get it. I have a 80GB database serving 10 queries per second on a $8 VPS. It depends on the query though.
- sysguru2046 2mo agoSuperbase?
- anivan_ 2mo agoThat might be an interesting comparison :) Especially, cost-efficiency. I'll think how to put it, thanks :)
- vanyo09 2mo ago[flagged]