7 ms·

A Comprehensive Guide to Building a Scalable Web App on AWS

542 points by jkresner 12y ago

jkresner 12y ago
Apologies in advance for the table of contents going way off the screen. This is the biggest post published so far. We'll be doing some UX work on the table of contents widget in the next week.
- deleted 12y ago
  [deleted]
- coolgeek 12y ago
  While you're at it, how about a reasonable print stylesheet? I had to do a lot of surgery on the node.js tips article, but still gave up because I could only get a max of 19 lines of the code snippets to print
  - jkresner 12y ago
    We've scheduled a lot of UX design polish on our posts schedule in the next 2 weeks along with a big announcement first week of feb for you to watch out for.
- subrat_rout 12y ago
  Josh. This article is great and very helpful. Also I would like to know if you have any plan to write more articles on AWS that are more in depth. Few examples such as: 1. What are the security measures somebody needs to put in place to host a HIPAA compliant Web app etc. 2. What are the popular stacks that can be set up on AWS without much hassles and with less expertise. 3. What are day to day activities required to maintain few web apps on AWS. etc
  - jkresner 12y ago
    Josh has a part 2 in the works :)
  - rwfilice 12y ago
    Agree would love to see the HIPAA compliant article.
- wallflower 12y ago
  Please create an eBook and sell this on the Kindle store. Thank you! Amazing dive deep and fly over high overview.
deleted 12y ago
[deleted]
artifaxx 12y ago
Out of curiousity, why go with AWS when Linode, Digitalocean, etc appear to be so much more cost effective? Is the simplicity of spinning up AWS instances really great enough to counterbalance what appears to be a significantly greater cost? Is it the flexibility of different AWS services?
- taf2 12y ago
  It's more than just cost savings it's Flexibility of networking, Regional coverage, Tooling , Performance, On the cost side with reserved instances it's less expensive.
  - artifaxx 12y ago
    Really? I know reserved instances lower the price but they still have shown themselves less performant for the dollar in the benchmarks I have seen. Are there any good articles you could recommend with benchmarks showing AWS to be more cost effective?
    - bkeroack 12y ago
      My experience is similar. EC2 performance is mediocre in general and quite poor compared to dedicated bare metal. Disk I/O is abysmal (yes, even on "SSD" EBS).
      - fletchowns 12y ago
        How about for ephemeral instance storage? I'm curious how the performance compares there to SSD backed EBS volumes.
        IgorPartola 12y ago
        I only have a general comment about this, but that kind of performance is almost not worth thinking about in most cases. Just like NBA players need to be "tall enough", your cloud needs to be "fast enough", and AWS is just that. Having said that, nothing beats getting real hardware in terms of performance/$. I used to work for a company that exclusively worked on SoftLayer's hardware servers. These were $500+/month each, but they were fast. The point is that if you can devote a dual octocore machine to what you are doing, and are willing to pass up on the SAN, flexible networking, etc. then you get a very fast box. If what you are doing requires lots of very fast hardware, then yea AWS or anything like it is not really for you. But if you are like most people, your AWS bill is not going to break the bank and you can just spend one less billable hour figuring out why the server is slow, and just double its size.
deleted 12y ago
[deleted]
jwilliams 12y ago
This links to a HN discussion on the relative merits of VPN access versus bastion/jump boxes (https://news.ycombinator.com/item?id=8637154 https://news.ycombinator.com/item?id=8637154). This conversation didn't ever seem to bottom out to conclusion? In particular I was wondering how servers connected _outwards_ in the VPN scenario.
- kainosnoema 12y ago
  In either case—assuming you run a VPC—you usually configure one or more NAT instances to allow EC2 instances to communicate with the internet (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NA...). I suppose a bastion could do double-duty as a NAT, but in most cases you want one per availability zone to add isolation and redundancy.
  - dbarlett 12y ago
    You can also configure HA NAT with an ICMP heartbeat: https://aws.amazon.com/articles/2781451301784570 https://aws.amazon.com/articles/2781451301784570
  - jwilliams 12y ago
    Doesn't this -somewhat- mitigate the advantage of having zero-surface area? I'm using a Bastion setup, so don't get me wrong, just want to understand how strong the pros are for the VPN route. On your last note. I just run one Bastion as a general rule. They're quick enough to spin up another instance (in a different AZ if necessary). Generally our services won't die if the Bastion or NAT is down.
- maslam 12y ago
  We (Appuri) use VPCs exclusively. There are pros and cons. I'll try to list the top: Pros: - Logical isolation. You can put instances (and RDS, Redshift) etc. inside logical subnets that are not addressable from the outside world. - VPNs. If you really want extra security, you can wire up VPN so one of your VPC subnets shows up on your corporate subnet. Cons: - A complete pain to manage with SSH-based tools. Most deployment tools (Ansible, for example) and even lower-level tools like fleetctl don't play well (if at all) with jump boxes. Example - Ansible Tower requires instances that are publicly addressable OR placing a Tower instance inside a VPC (which means we can't use it to manage multiple VPCs) - We have had to write our own workarounds for the above con. - Complexity. There are more concepts to learn about. - Lack of portability. I don't know if all cloud providers (Azure, DO etc.) even support VPCs the same way AWS does. This makes our infrastructure less portable than I'd like
chuckcode 12y ago
Great to see article putting it all together. Also going to be great to send this to people who think that using AWS cloud means that you can skip hiring people with skills in systems administration.
tieTYT 12y ago
The scariest thing to me about AWS is that I might accidentally bankrupt myself while I learn to use it. I've seen horror stories on HN before. Articles titled "How I spent $32k with AWS, a for loop, and a simple typo" or something like that. Normally when I learn something new, I learn by tinkering and breaking stuff. I don't feel comfortable doing that with AWS. I'm hoping people will tell me I'm way off base because this fear has stopped me from getting the ball rolling.
- withdavidli 12y ago
  Here's a story for reference: http://www.devfactor.net/2014/12/30/2375-amazon-mistake/ http://www.devfactor.net/2014/12/30/2375-amazon-mistake/ More about bots scanning github for api keys, that's pretty scary in itself because I know pushing keys by accident happens a lot.
  - juliangregorian 12y ago
    Yeah, Amazon's key model is a pretty big weak point, both Google Cloud and Azure handle it better. It would be safer to use different sets of keys, like one to create new machines, and app-specific keys that can update only, but that's more work and more headache to manage. Google Cloud just makes me SSH tunnel, which I like.
- skuhn 12y ago
  AWS does have a way to warn you when your bill crosses a certain threshold: http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/create-billing-alarm.html http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/...
  - oafitupa 12y ago
    All the warnings in the world can't replace a configurable $ limit. We don't want to have to rely on them "forgiving" us if we make a mistake.
    - 12y ago
deleted 12y ago
[deleted]
deleted 12y ago
[deleted]
chacham15 12y ago
These guides are really hard to read (at least for me) because there isnt really a point (until the end) where I can stop and go try out some of what I've learned. Perhaps, if it were formatted in the way of "steps to setup a scalable web app on AWS" it would be more palatable.
Rapzid 12y ago
This article appears to include a lot of very good advice(speaking as an AWS solutions architect). I might suggest a emphasising a few things such as not having keys on login accounts(they negate multi-factor auth if leaked), and to ALWAYS pick or create a new IAM role if you aren't sure an existing one fits for the EC2 instances.. But perhaps this sort of advice is not appropriate for the article. Much respect for the amount of work that went into this. I'll try to get through it all here at some point :)
jacques_chester 12y ago
Missing from the list of PaaS offerings is Cloud Foundry, which already runs on AWS. Obvious disclaimer: I work on CF for my dayjob.
AliCollins 12y ago
Does anyone know of a similar article for the Google Cloud infrastructure?
minikomi 12y ago
Amazing post! Thank you very much. Would love to be able to read this on kindle - you should make an epub! haha
skc 12y ago
This is a great article. I knew I'd see a bunch of people stating that AWS is expensive and you should use a dedicated server or a VPS. But there are many applications built by people like me who are lone developers or small teams of developers who either don't have the admin skills or simply don't want to admin their own servers and the fact that AWS handles quite a lot of this for you is sometimes worth the the added cost.
lazyfunctor 12y ago
Nice post! Thanks for writing it. Do you know of any similar resource for google compute platform?

542 points by jkresner 12y ago

Apologies in advance for the table of contents going way off the screen. This is the biggest post published so far. We'll be doing some UX work on the table of contents widget in the next week.

deleted 12y ago
[deleted]
coolgeek 12y ago
While you're at it, how about a reasonable print stylesheet? I had to do a lot of surgery on the node.js tips article, but still gave up because I could only get a max of 19 lines of the code snippets to print
- jkresner 12y ago
  We've scheduled a lot of UX design polish on our posts schedule in the next 2 weeks along with a big announcement first week of feb for you to watch out for.
subrat_rout 12y ago
Josh. This article is great and very helpful. Also I would like to know if you have any plan to write more articles on AWS that are more in depth. Few examples such as: 1. What are the security measures somebody needs to put in place to host a HIPAA compliant Web app etc. 2. What are the popular stacks that can be set up on AWS without much hassles and with less expertise. 3. What are day to day activities required to maintain few web apps on AWS. etc
- jkresner 12y ago
  Josh has a part 2 in the works :)
- rwfilice 12y ago
  Agree would love to see the HIPAA compliant article.
wallflower 12y ago
Please create an eBook and sell this on the Kindle store. Thank you! Amazing dive deep and fly over high overview.

deleted 12y ago

[deleted]

artifaxx 12y ago

Out of curiousity, why go with AWS when Linode, Digitalocean, etc appear to be so much more cost effective? Is the simplicity of spinning up AWS instances really great enough to counterbalance what appears to be a significantly greater cost? Is it the flexibility of different AWS services?

taf2 12y ago
It's more than just cost savings it's Flexibility of networking, Regional coverage, Tooling , Performance, On the cost side with reserved instances it's less expensive.
- artifaxx 12y ago
  Really? I know reserved instances lower the price but they still have shown themselves less performant for the dollar in the benchmarks I have seen. Are there any good articles you could recommend with benchmarks showing AWS to be more cost effective?
  - bkeroack 12y ago
    My experience is similar. EC2 performance is mediocre in general and quite poor compared to dedicated bare metal. Disk I/O is abysmal (yes, even on "SSD" EBS).
    - fletchowns 12y ago
      How about for ephemeral instance storage? I'm curious how the performance compares there to SSD backed EBS volumes.
      - IgorPartola 12y ago
        I only have a general comment about this, but that kind of performance is almost not worth thinking about in most cases. Just like NBA players need to be "tall enough", your cloud needs to be "fast enough", and AWS is just that. Having said that, nothing beats getting real hardware in terms of performance/$. I used to work for a company that exclusively worked on SoftLayer's hardware servers. These were $500+/month each, but they were fast. The point is that if you can devote a dual octocore machine to what you are doing, and are willing to pass up on the SAN, flexible networking, etc. then you get a very fast box. If what you are doing requires lots of very fast hardware, then yea AWS or anything like it is not really for you. But if you are like most people, your AWS bill is not going to break the bank and you can just spend one less billable hour figuring out why the server is slow, and just double its size.

deleted 12y ago

[deleted]

jwilliams 12y ago

This links to a HN discussion on the relative merits of VPN access versus bastion/jump boxes (https://news.ycombinator.com/item?id=8637154 https://news.ycombinator.com/item?id=8637154). This conversation didn't ever seem to bottom out to conclusion? In particular I was wondering how servers connected _outwards_ in the VPN scenario.

kainosnoema 12y ago
In either case—assuming you run a VPC—you usually configure one or more NAT instances to allow EC2 instances to communicate with the internet (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NA...). I suppose a bastion could do double-duty as a NAT, but in most cases you want one per availability zone to add isolation and redundancy.
- dbarlett 12y ago
  You can also configure HA NAT with an ICMP heartbeat: https://aws.amazon.com/articles/2781451301784570 https://aws.amazon.com/articles/2781451301784570
- jwilliams 12y ago
  Doesn't this -somewhat- mitigate the advantage of having zero-surface area? I'm using a Bastion setup, so don't get me wrong, just want to understand how strong the pros are for the VPN route. On your last note. I just run one Bastion as a general rule. They're quick enough to spin up another instance (in a different AZ if necessary). Generally our services won't die if the Bastion or NAT is down.
maslam 12y ago
We (Appuri) use VPCs exclusively. There are pros and cons. I'll try to list the top: Pros: - Logical isolation. You can put instances (and RDS, Redshift) etc. inside logical subnets that are not addressable from the outside world. - VPNs. If you really want extra security, you can wire up VPN so one of your VPC subnets shows up on your corporate subnet. Cons: - A complete pain to manage with SSH-based tools. Most deployment tools (Ansible, for example) and even lower-level tools like fleetctl don't play well (if at all) with jump boxes. Example - Ansible Tower requires instances that are publicly addressable OR placing a Tower instance inside a VPC (which means we can't use it to manage multiple VPCs) - We have had to write our own workarounds for the above con. - Complexity. There are more concepts to learn about. - Lack of portability. I don't know if all cloud providers (Azure, DO etc.) even support VPCs the same way AWS does. This makes our infrastructure less portable than I'd like

chuckcode 12y ago

Great to see article putting it all together. Also going to be great to send this to people who think that using AWS cloud means that you can skip hiring people with skills in systems administration.

tieTYT 12y ago

The scariest thing to me about AWS is that I might accidentally bankrupt myself while I learn to use it. I've seen horror stories on HN before. Articles titled "How I spent $32k with AWS, a for loop, and a simple typo" or something like that. Normally when I learn something new, I learn by tinkering and breaking stuff. I don't feel comfortable doing that with AWS. I'm hoping people will tell me I'm way off base because this fear has stopped me from getting the ball rolling.

withdavidli 12y ago
Here's a story for reference: http://www.devfactor.net/2014/12/30/2375-amazon-mistake/ http://www.devfactor.net/2014/12/30/2375-amazon-mistake/ More about bots scanning github for api keys, that's pretty scary in itself because I know pushing keys by accident happens a lot.
- juliangregorian 12y ago
  Yeah, Amazon's key model is a pretty big weak point, both Google Cloud and Azure handle it better. It would be safer to use different sets of keys, like one to create new machines, and app-specific keys that can update only, but that's more work and more headache to manage. Google Cloud just makes me SSH tunnel, which I like.
skuhn 12y ago
AWS does have a way to warn you when your bill crosses a certain threshold: http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/create-billing-alarm.html http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/...
- oafitupa 12y ago
  All the warnings in the world can't replace a configurable $ limit. We don't want to have to rely on them "forgiving" us if we make a mistake.
  - 12y ago

deleted 12y ago

[deleted]

deleted 12y ago

[deleted]

chacham15 12y ago

These guides are really hard to read (at least for me) because there isnt really a point (until the end) where I can stop and go try out some of what I've learned. Perhaps, if it were formatted in the way of "steps to setup a scalable web app on AWS" it would be more palatable.

Rapzid 12y ago

This article appears to include a lot of very good advice(speaking as an AWS solutions architect). I might suggest a emphasising a few things such as not having keys on login accounts(they negate multi-factor auth if leaked), and to ALWAYS pick or create a new IAM role if you aren't sure an existing one fits for the EC2 instances.. But perhaps this sort of advice is not appropriate for the article. Much respect for the amount of work that went into this. I'll try to get through it all here at some point :)

jacques_chester 12y ago

Missing from the list of PaaS offerings is Cloud Foundry, which already runs on AWS. Obvious disclaimer: I work on CF for my dayjob.

AliCollins 12y ago

Does anyone know of a similar article for the Google Cloud infrastructure?

minikomi 12y ago

Amazing post! Thank you very much. Would love to be able to read this on kindle - you should make an epub! haha

skc 12y ago

This is a great article. I knew I'd see a bunch of people stating that AWS is expensive and you should use a dedicated server or a VPS. But there are many applications built by people like me who are lone developers or small teams of developers who either don't have the admin skills or simply don't want to admin their own servers and the fact that AWS handles quite a lot of this for you is sometimes worth the the added cost.

lazyfunctor 12y ago

Nice post! Thanks for writing it. Do you know of any similar resource for google compute platform?