7 ms·

Modern cloud architecture on AWS: server fleets and databases

87 points by colemorrison 7y ago

kamilafsar 7y ago
I keep reading all these horror stories about Aurora (especially PostgreSQL). Is there anyone out there with an alternative story?
- vidar 7y ago
  Links?
  - kamilafsar 7y ago
    https://www.reddit.com/r/aws/comments/bv70k8/aurora_postgres_disastrous_experience/ https://www.reddit.com/r/aws/comments/bv70k8/aurora_postgres...
- Roritharr 7y ago
  Using Aurora MySQL for over a year now in prod, purrs like a kittycat. Just don't use the AWS Database Migration Service if you can help it, that thing has a couple of badly documented pitfalls. (Fe. tables can't have ENUM fields)
  - etaioinshrdlu 7y ago
    AWS Database Migration Service had a shockingly large gap between how it's marketed and how well it actually performed. It had so many gotchas and broken features. I'd be amazed if anyone got it really working on large applications without weeks of time invested. It would be so cool though if it worked seamlessly. It addresses some of the hardest tasks in DB management, namely zero downtime server migration.
- popotamonga 7y ago
  10 separate xlarge instances pgsql and never had a single issue in 3 yrs of heavy use. What stories are those?
- Trisell 7y ago
  Used it a year at my last job and now almost a year at my current job. Never had an issue. It just runs no matter how much I throw at it. Only have had to change the instance sizes to deal with data ingest.
- BrentOzar 7y ago
  We run a 2TB database with 30-60 days of data (only keeps 30-60 days on-hand, and we're roughly ingesting 50GB/day.) We've been using Aurora Postgres since it came out, and it's been pretty good. (Good enough to the point that it's never crossed my mind to think about moving to something else.) Source: Microsoft SQL Server performance tuner who runs an app that centralizes SQL Server performance data from thousands of servers. You'd think I would be running MSSQL on the back end, but the licensing costs just didn't make sense compared to Aurora Postgres.
Jonnax 7y ago
At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths before, and I managed somehow to crash it requiring a restore from snapshot. Also their smallest instance was quite expensive for the performance.
- paxy 7y ago
  RDS/Aurora does do autoscaling, backups, georeplication, encryption etc. It's more a matter of time & convenience rather than skill. Sure you could do all of it yourself on an EC2 instance, but at some point it becomes a big chunk of your job, and you would rather be spending your time on other things.
- zedpm 7y ago
  It isn't a matter of getting to a certain scale before you use RDS, if that's what you're asking. The value of RDS is streamlining and automating db administrative tasks. If you want to update from Postgres 11.5 to 11.6, for example, you just change that setting on your RDS instance and it happens, either immediately or at some scheduled maintenance window. I f you want a hot standby in a separate availability zone, it's trivial to add one. Read replica? Trivial to add. In general, I've found it makes sense to pay the premium for RDS and spend my and my team's time on more valuable work than db admin tasks.
- pestaa 7y ago
  There are advantages when using RDS other than scaling. The performance dashboard is especially nice. The reusable sets of configuration is convenient and the UI makes it easy to compare original vs. changed values. The instance upgrade is not seamless, though, it is normally scheduled for the next maintenance window, unless you decide to apply and reboot immediately. I agree the service is expensive, but setting up a db server for production takes a lot of time and expertise.
- pdeva1 7y ago
ramoz 7y ago
"Modern" architectures can get quite complex and fast at scale & in complex cases. This is merely a simple introduction to simple components of modern cloud architecture.
- pm90 7y ago
  I do agree with you, and I was lured into reading it because of that. However, this seems like a nice introduction for beginners. Maybe it should be tagged as such.
root-z 7y ago
As someone who has spent a fair amount of time working with AWS. I appreciate how approachable this tutorial is, as the official docs are usually way more arcane.
malisper 7y ago
Although there are a ton of AWS servers, there's only a few core services that I recommend: EC2 - You need a server. RDS - You need a database. S3 - You need to store files. Lambda - You are building an API with short lived requests. These services are all very high quality and are excellent at what they do. Once you get outside of these core services, the quality quickly drops. You're probably better off using the non-AWS versions of those services. For a few quick examples, you should be using Datadog over CloudWatch, Snowflake over Redshift or Athena, and Terraform over CloudFormation.
- reese_john 7y ago
  Just curious, why would you not recommend SQS ?
  - malisper 7y ago
    I've never used SQS but IMO it seems inferior to Kinesis or Kafka. The two big reasons are that you can't have multiple consumers read from a single queue and once data leaves the queue, it's gone forever. Both Kinesis and Kafka let you have multiple consumers and configure a retention period for your messages.
    - CoolGuySteve 7y ago
      How so? I've run thousands of consumers on SQS for batch jobs and it seems to work. There's also dead letter queue and retries for messages that aren't properly serviced.
      - malisper 7y ago
        See my reply to appwiz: https://news.ycombinator.com/item?id=22280200 https://news.ycombinator.com/item?id=22280200
    - appwiz 7y ago
      You can have millions of consumers read concurrently from a single SQS queue. Messages that are read remain in the queue up to the configured retention period or until a consumer calls DeleteMessage. Source: I’ve built very high volume services that continue to run production workloads and use SQS as the buffer between components.

87 points by colemorrison 7y ago

I keep reading all these horror stories about Aurora (especially PostgreSQL). Is there anyone out there with an alternative story?

vidar 7y ago
Links?
- kamilafsar 7y ago
  https://www.reddit.com/r/aws/comments/bv70k8/aurora_postgres_disastrous_experience/ https://www.reddit.com/r/aws/comments/bv70k8/aurora_postgres...
Roritharr 7y ago
Using Aurora MySQL for over a year now in prod, purrs like a kittycat. Just don't use the AWS Database Migration Service if you can help it, that thing has a couple of badly documented pitfalls. (Fe. tables can't have ENUM fields)
- etaioinshrdlu 7y ago
  AWS Database Migration Service had a shockingly large gap between how it's marketed and how well it actually performed. It had so many gotchas and broken features. I'd be amazed if anyone got it really working on large applications without weeks of time invested. It would be so cool though if it worked seamlessly. It addresses some of the hardest tasks in DB management, namely zero downtime server migration.
popotamonga 7y ago
10 separate xlarge instances pgsql and never had a single issue in 3 yrs of heavy use. What stories are those?
Trisell 7y ago
Used it a year at my last job and now almost a year at my current job. Never had an issue. It just runs no matter how much I throw at it. Only have had to change the instance sizes to deal with data ingest.
BrentOzar 7y ago
We run a 2TB database with 30-60 days of data (only keeps 30-60 days on-hand, and we're roughly ingesting 50GB/day.) We've been using Aurora Postgres since it came out, and it's been pretty good. (Good enough to the point that it's never crossed my mind to think about moving to something else.) Source: Microsoft SQL Server performance tuner who runs an app that centralizes SQL Server performance data from thousands of servers. You'd think I would be running MSSQL on the back end, but the licensing costs just didn't make sense compared to Aurora Postgres.

Jonnax 7y ago

At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths before, and I managed somehow to crash it requiring a restore from snapshot. Also their smallest instance was quite expensive for the performance.

paxy 7y ago
RDS/Aurora does do autoscaling, backups, georeplication, encryption etc. It's more a matter of time & convenience rather than skill. Sure you could do all of it yourself on an EC2 instance, but at some point it becomes a big chunk of your job, and you would rather be spending your time on other things.
zedpm 7y ago
It isn't a matter of getting to a certain scale before you use RDS, if that's what you're asking. The value of RDS is streamlining and automating db administrative tasks. If you want to update from Postgres 11.5 to 11.6, for example, you just change that setting on your RDS instance and it happens, either immediately or at some scheduled maintenance window. I f you want a hot standby in a separate availability zone, it's trivial to add one. Read replica? Trivial to add. In general, I've found it makes sense to pay the premium for RDS and spend my and my team's time on more valuable work than db admin tasks.
pestaa 7y ago
There are advantages when using RDS other than scaling. The performance dashboard is especially nice. The reusable sets of configuration is convenient and the UI makes it easy to compare original vs. changed values. The instance upgrade is not seamless, though, it is normally scheduled for the next maintenance window, unless you decide to apply and reboot immediately. I agree the service is expensive, but setting up a db server for production takes a lot of time and expertise.
pdeva1 7y ago

ramoz 7y ago

"Modern" architectures can get quite complex and fast at scale & in complex cases. This is merely a simple introduction to simple components of modern cloud architecture.

pm90 7y ago
I do agree with you, and I was lured into reading it because of that. However, this seems like a nice introduction for beginners. Maybe it should be tagged as such.

root-z 7y ago

As someone who has spent a fair amount of time working with AWS. I appreciate how approachable this tutorial is, as the official docs are usually way more arcane.

malisper 7y ago

Although there are a ton of AWS servers, there's only a few core services that I recommend: EC2 - You need a server. RDS - You need a database. S3 - You need to store files. Lambda - You are building an API with short lived requests. These services are all very high quality and are excellent at what they do. Once you get outside of these core services, the quality quickly drops. You're probably better off using the non-AWS versions of those services. For a few quick examples, you should be using Datadog over CloudWatch, Snowflake over Redshift or Athena, and Terraform over CloudFormation.

reese_john 7y ago
Just curious, why would you not recommend SQS ?
- malisper 7y ago
  I've never used SQS but IMO it seems inferior to Kinesis or Kafka. The two big reasons are that you can't have multiple consumers read from a single queue and once data leaves the queue, it's gone forever. Both Kinesis and Kafka let you have multiple consumers and configure a retention period for your messages.
  - CoolGuySteve 7y ago
    How so? I've run thousands of consumers on SQS for batch jobs and it seems to work. There's also dead letter queue and retries for messages that aren't properly serviced.
    - malisper 7y ago
      See my reply to appwiz: https://news.ycombinator.com/item?id=22280200 https://news.ycombinator.com/item?id=22280200
  - appwiz 7y ago
    You can have millions of consumers read concurrently from a single SQS queue. Messages that are read remain in the queue up to the configured retention period or until a consumer calls DeleteMessage. Source: I’ve built very high volume services that continue to run production workloads and use SQS as the buffer between components.