6 ms·

Agree. But even fighting for RDMS in 2022 is getting daunting as so many people will immediately start pushing for document DB's, GraphQL, etc. Seriously - I f

Agree. But even fighting for RDMS in 2022 is getting daunting as so many people will immediately start pushing for document DB's, GraphQL, etc. Seriously - I feel like SWE's have no idea the powerhouse a modest MySQL/Postgres instance can crank out while perfectly maintaining their data (ACID, replication, etc). Everyone wants to use tools that were designed for FAANG scale regardless of org size. Anecdotally I've fought this hard because it's almost always at incredible detriment to the business's needs. I find that modern devs balk at "single RDMS database + stateless middle tier" but only for selfish reasons like resume-driven development. This pattern is really burning me out more than anything else of my career. Again, all this is personal anecdotes.

jghn 4y ago
I've found the largest culprit here is that people don't understand how many orders of magnitude are between them and FAANG scale. It's akin to how people don't understand the difference between a millionaire, a billionaire, and Elon Musk. So they'll see some numbers that are on the upper end or beyond what they're used to seeing, and to them that's big scale. How do you solve that? Well, you reach for the tools other people have used when they have "big scale". All the literature says you can't just increase your instance size when you have big scale, so why would you try that? And yes, that's wrong. But it drives a lot of the mentality.
- peterangular 4y ago
  [dead]
- deleted 4y ago
  [deleted]
- bsder 4y ago
  "If you have to ask if you have big data, you don't have big data. When you have big data, you know you have big data." Seriously, you can launch a server that has almost a terabyte of RAM! Is your business doing enough to make that database even blink? "My sources say no".
- dvirsky 4y ago
  There is another aspect to this - if you're working on a startup that will scale, if successful - not to FAANG scale, but let's say many millions of DAUs and thousands of QPS on the servers - you also want to build some kind of future proofing into your architecture. And then begins the dance of balancing what we currently need (we have maybe 100K DAU and launched recently) and what we predict we might need if things go right. So you don't need to go full on K8S with some crazy sharded database and 50 microservices on day 1, but you also don't want a PHP/MySQL monolith if you're building something that works like, say, Twitter, because you'll have to scrap ALL of that under huge pressure if things start to take off.
  - peterangular 4y ago
    > 100K DAU > but you also don't want a PHP/MySQL monolith if you're building something that works like, say, Twitter This is a lot of whiplash in regards to discussing size. I just want to say... 100K DAU with a PHP/MySQL monolith is 100% possible - I'm certain a lot of people here have achieved this without much problem. Things like Nginx, PHP-FPM, reverse-proxy-caching, load balancing to stateless application servers, read replica, offloading static assets to CDN, blah blah... I digress but you can 100% hit 100K DAU with a traditional PHP/MySQL monolith.
    - dvirsky 4y ago
      Yes, my point is it will hold, but it won't hold 50M users, and if you take off you'll be scrambling to rewrite it from scratch. So a flexible architecture that is simple but can scale (as an architecture) might be a better choice than the simplest possible design you can make.
  - ativzzz 4y ago
    Could not disagree more. If you are a startup that is struggling to survive, it is an absolute waste of resources to overengineer your app for potential scale. It's the VC way to burn all of the money hiring talented infra engineers to manage this, but it is not pragmatic at all. The exception is if your startup is literally a high volume data processing platform or something similar. That simple PHP/MySQL or rails monolith will scale much further than you think by throwing more servers at it without having to hire an army of devops engineers. Solving problems you don't currently have when your company is not profitable is a waste of money.
- dalyons 4y ago
  i have worked on several not-faang-sized-but-largish b2c startups, all of which were forced to spend a very large amount of money, time and people trying to migrate things off the one central RDBMS when it could no longer be scaled vertically. This is a real problem that you likely will run into well before FAANG size, and it really does hurt, and i think it gets minimized by the "just use one postgres for everything" trope. You should probably think about it at least a little if your planned business trajectory is into the 1 million+ users realm. Im not saying you have to use nosql or whatever, but some sort of small future proofing (eg introducing different schemas/permissions for your different data domains inside your monolith) can help make future database seperation much easier. If you're doing a B2B app its a different story, much harder to hit the limits of a big RDS instance. The real answer, as in most software engineering is "it depends"
  - mwcampbell 4y ago
    How long ago did these startups hit the limits of vertical scaling? Those limits are always increasing. Expensify's famous post from 2018 [1] suggests that it should be feasible for one monster machine to handle millions of concurrent users. [1]: https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-qps-on-a-single-server/ https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...
    - dalyons 4y ago
      theres one that im still working on today :) for practical and all the other usual reasons most people are going to rent cloud dbs (like RDS), not run bare metal. Also, theoretical/benchmark performance is a very long way away from the actually achievable performance when you have 100s of developers actively evolving a codebase against the database. Optimal query usage etc in that scenario is an impossible pipe dream, best you can do is "not crazy", unless you grind all development to a halt behind gatekeeping dba-types. There are many other operational challenges to single very large database instances. Upgrades, backups, migrations, etc all become way more risky and hard when you get into the mega-db range. The number of total outages ive seen caused by migrations gone awry, query plans gone awry... RDBMS are incredibly complicated beasts and at huge sizes can be very unpredictable in a way that many smaller dbs are not.

jghn 4y ago

I've found the largest culprit here is that people don't understand how many orders of magnitude are between them and FAANG scale. It's akin to how people don't understand the difference between a millionaire, a billionaire, and Elon Musk. So they'll see some numbers that are on the upper end or beyond what they're used to seeing, and to them that's big scale. How do you solve that? Well, you reach for the tools other people have used when they have "big scale". All the literature says you can't just increase your instance size when you have big scale, so why would you try that? And yes, that's wrong. But it drives a lot of the mentality.

peterangular 4y ago
[dead]
deleted 4y ago
[deleted]
bsder 4y ago
"If you have to ask if you have big data, you don't have big data. When you have big data, you know you have big data." Seriously, you can launch a server that has almost a terabyte of RAM! Is your business doing enough to make that database even blink? "My sources say no".
dvirsky 4y ago
There is another aspect to this - if you're working on a startup that will scale, if successful - not to FAANG scale, but let's say many millions of DAUs and thousands of QPS on the servers - you also want to build some kind of future proofing into your architecture. And then begins the dance of balancing what we currently need (we have maybe 100K DAU and launched recently) and what we predict we might need if things go right. So you don't need to go full on K8S with some crazy sharded database and 50 microservices on day 1, but you also don't want a PHP/MySQL monolith if you're building something that works like, say, Twitter, because you'll have to scrap ALL of that under huge pressure if things start to take off.
- peterangular 4y ago
  > 100K DAU > but you also don't want a PHP/MySQL monolith if you're building something that works like, say, Twitter This is a lot of whiplash in regards to discussing size. I just want to say... 100K DAU with a PHP/MySQL monolith is 100% possible - I'm certain a lot of people here have achieved this without much problem. Things like Nginx, PHP-FPM, reverse-proxy-caching, load balancing to stateless application servers, read replica, offloading static assets to CDN, blah blah... I digress but you can 100% hit 100K DAU with a traditional PHP/MySQL monolith.
  - dvirsky 4y ago
    Yes, my point is it will hold, but it won't hold 50M users, and if you take off you'll be scrambling to rewrite it from scratch. So a flexible architecture that is simple but can scale (as an architecture) might be a better choice than the simplest possible design you can make.
- ativzzz 4y ago
  Could not disagree more. If you are a startup that is struggling to survive, it is an absolute waste of resources to overengineer your app for potential scale. It's the VC way to burn all of the money hiring talented infra engineers to manage this, but it is not pragmatic at all. The exception is if your startup is literally a high volume data processing platform or something similar. That simple PHP/MySQL or rails monolith will scale much further than you think by throwing more servers at it without having to hire an army of devops engineers. Solving problems you don't currently have when your company is not profitable is a waste of money.
dalyons 4y ago
i have worked on several not-faang-sized-but-largish b2c startups, all of which were forced to spend a very large amount of money, time and people trying to migrate things off the one central RDBMS when it could no longer be scaled vertically. This is a real problem that you likely will run into well before FAANG size, and it really does hurt, and i think it gets minimized by the "just use one postgres for everything" trope. You should probably think about it at least a little if your planned business trajectory is into the 1 million+ users realm. Im not saying you have to use nosql or whatever, but some sort of small future proofing (eg introducing different schemas/permissions for your different data domains inside your monolith) can help make future database seperation much easier. If you're doing a B2B app its a different story, much harder to hit the limits of a big RDS instance. The real answer, as in most software engineering is "it depends"
- mwcampbell 4y ago
  How long ago did these startups hit the limits of vertical scaling? Those limits are always increasing. Expensify's famous post from 2018 [1] suggests that it should be feasible for one monster machine to handle millions of concurrent users. [1]: https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-qps-on-a-single-server/ https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...
  - dalyons 4y ago
    theres one that im still working on today :) for practical and all the other usual reasons most people are going to rent cloud dbs (like RDS), not run bare metal. Also, theoretical/benchmark performance is a very long way away from the actually achievable performance when you have 100s of developers actively evolving a codebase against the database. Optimal query usage etc in that scenario is an impossible pipe dream, best you can do is "not crazy", unless you grind all development to a halt behind gatekeeping dba-types. There are many other operational challenges to single very large database instances. Upgrades, backups, migrations, etc all become way more risky and hard when you get into the mega-db range. The number of total outages ive seen caused by migrations gone awry, query plans gone awry... RDBMS are incredibly complicated beasts and at huge sizes can be very unpredictable in a way that many smaller dbs are not.