7 ms·

In most cases, very few companies have products that need to scale to extreme load day 1 or even year 1. IMO, instead of reaching for the latest shiny cloud pro

by dr01d 8y ago

In most cases, very few companies have products that need to scale to extreme load day 1 or even year 1. IMO, instead of reaching for the latest shiny cloud product, try building initially with traditional databases, load balancing, and caching first. You can actually go very far on unsexy old stuff. Overall, this approach will make migration easier in the cloud and you can always evolve parts of your stack based on your actual needs later. Justify switching out to proprietary products like lambdas, etc once your system actually requires it and then weigh your options carefully. Everyone jumping on the bandwagon these days needs to realize: a LOT of huge systems are still rocking PHP and MySQL and chasing new cloud products is a never ending process.

scarface74 8y ago
And while we spend all of our energy and money doing that, we aren’t actually creating the product that is suppose to be bringing us revenue....
- wUabkSG6L5Bfa5 8y ago
  This is the correct answer. AFAIC, Intro to Economics should be the first CS course any software engineer takes.
nickthemagicman 8y ago
I have mixed feelings about this and I don't have enough experience to label on method better than the other. Lambdas basically require zero maintenance. SQS requires zero maintenance. EC2 load balancer is zero maintenance. And the setup is trivial too and there's no migration time down the line.If you start off with native cloud for everything you can keep your maintenance and setup costs down drastically. However, a lot can be done with the old school unsexy tech. So I'm mixed.
- dr01d 8y ago
  Zero maintenance? What about standing up multiple environments for staging, prod? Sharing secrets ,keys, and env vars? Deployments? Logging? No migration time down the line? I'm pretty sure GCP, Azure don't have Lambda, SQS, or EC2 load balancers so you absolutely will have migration time if you have to retool your implementation to switch cloud providers or products.
  - nickthemagicman 8y ago
    You make really good points about that. The classic things do migrate providers the easiest of all. I've just found in my experience maintaining a web server or a database server, keeping security in mind, upgrades, scaling, etc. is alot more work than simply spinning up RDS and a Lambda with API gateway. Or even hosting static sites on s3 or Netlify. Like I said I don't know enough to say one is better. Also, is your name Jason?
  - nickthemagicman 8y ago
    I think we may have worked together in the past. Lol
justasitsounds 8y ago
In my case switching to an AWS API Gateway + Lambda stack means I have zero-downtime, canary deployments that take less than 5 minutes to deploy from version control. Api Gateway is configured using the same swagger doc that autogenerates my request/response models (go-swagger) and (almost) completely removes routing, request logging, throttling and authentication concerns from the request handlers. Combined with a staticly hosted front-end and SNS/SQS+lambda pub-sub for out-of-process workers I never have to worry about auto-scaling, load-balancing or cache-invalidation and we only pay for what we use. It may not suit every use case, but in our case, we have bursty, relatively low-volume traffic and the hosting bill for our public-facing site/service that comprises most of the main business revenue is the same as a rounding error on our BI service bill.
- Aeolun 8y ago
  How do you deploy in less than 5 minutes? Our serverless deployments are running into 20 minutes now, and it’s becoming a bit unfunny.
  - justasitsounds 8y ago
    We use golang lambdas, binaries are built in our CI pipeline. Build stage takes ~10 seconds, tests (integration + unit) take ~30 seconds. We use AWS SAM for generating our CFN templates, we package and deploy using the AWS Cloudformation CLI and this takes the remaining 3-4 minutes. I didn't include post-deployment end-to-end tests in the 5 minute figure, but technically speaking, we do deploy that quickly
  - redisman 8y ago
    How does it take 20 minutes? Do you deploy everything at once? Most of our APIs take a few minutes (depending on what kind of a deploy it is).
    - bazza451 8y ago
      Obviously not using cloudfront...might be the slowest service I’ve ever seen to deploy
halbritt 8y ago
I'm generally a proponent of DIY, this doesn't make sense. When an org needs to scale, it makes sense to cultivate skill in the underlying infrastructure. On day 1, serverless makes a lot of sense because it encourages development patterns that eventually will scale nicely.
- dwaltrip 8y ago
  Placing stateless web servers on simple VPS instances with load balancers in front doesn't scale nicely? Or perhaps I'm misunderstanding the parent.
  - deleted 8y ago
    [deleted]
  - redisman 8y ago
    Serverless does force you to think for higher concurrency than bigger server instances which is worth something.
com2kid 8y ago
Serverless is also easier to develop for. With Google Firebase Functions I was able to start writing REST APIs in minutes. Compare that to setting up a VM somewhere, getting a domain name + certs + express setup + deployment scripts, and then handling login credentials for all of the above. I had never done any of that (eventually I grew until I had to), so serverless let me get up and running really quickly. Now I prefer my own express instance, since deployment is much faster and debugging is much easier. But even for the debugging scenario, expecting everyone who wants to Just Write Code to get the horrid mess of JS stuff up and running in order to debug, ugh. (If it wasn't for HTTPS, Firebase's function emulator would be fine for debugging, as it is, a few nice solutions exist anyway.) But, to be clear, on day 1 the option for me to write a JS rest endpoint was: 1. Follow a 5-10 minute tutorials on setting up Firebase Functions. OR 1. Pick a VM host (Digital Ocean rocks) and setup an account 2. Learn how to provision a VM 3. Get a domain 4. Get domain over to my host 5. SSH into machine as root, setup non-root accounts with needed permissions 6. Setup certbot 7. Learn how to setup an Express server 8. Setup an nginx reverse proxy to get HTTPS working on my Express server 9. Write deployment scripts (ok SCP) to copy my working code over to my machine 10. Setup PM2 to watch for script changes 11. Start writing code! (12. Keep track, in a secure fashion, of all the credentials I just created for the above steps!) I am experienced in a lot of things, and thankfully I had some experience messing around with VMs and setting up my own servers before, but despite what everyone on HN may think, not every dev in the world also wants to run a bunch of VMs and manage their setup/configuration just to write a few REST endpoints! So yeah, instead I can type 'firebase deploy' in a folder that has some JS functions exported in an index.js file and a minute later out pops some HTTPS URLs.
- fyfy18 8y ago
  If you don't want to learn DevOps why not use a PaaS like Heroku? That way when you want to learn DevOps, you can move your application without rewriting large swathes of it. It's funny but when I learned to code basically all ISPs provided you with free hosting and a database, and you just needed to drag and drop a PHP file to make it live. It's like we have gone backwards not just in terms of openness but also in terms of complexity.
  - 8y ago
lsorber 8y ago
> In most cases, very few companies have products that need to scale to extreme load day 1 or even year 1. That wouldn't be a great reason to choose serverless indeed. However, that doesn't mean serverless isn't still the right choice. We've tried both the traditional approach you describe and serverless, and from experience the latter is 10x less infrastructure code than the former (we compared both Terraform implementations). If serverless fits your use case, saving time and effort is a very good reason to go for it IMHO.
illumin8 8y ago
Honestly, I've developed the last few projects in Serverless framework and deployed to AWS Lambda. My biggest project is a custom Python application monitoring solution that has 4 separate Lambda function/handlers, populates an RDS PostgresSQL database with monitoring data (runs in a VPC), then reads from that database using complex joins across multiple application metric tables to send time-series data to Cloudwatch metrics. Then, I configured Cloudwatch alarms to have thresholds for certain metrics that send a web hook to PagerDuty. The benefit is that my monitoring system never goes down, never needs to be patched manually (AWS even patches my Postgres database, and fails over to the warm standby during patching), and never needs any system administration. Have you ever worked at a company where you had a serious outage that you didn't detect quickly enough because a monitoring system was down? Having a Serverless monitoring system means this has happened 0 times despite our app running in production for almost a year now.

by dr01d 8y ago

scarface74 8y ago

And while we spend all of our energy and money doing that, we aren’t actually creating the product that is suppose to be bringing us revenue....

wUabkSG6L5Bfa5 8y ago
This is the correct answer. AFAIC, Intro to Economics should be the first CS course any software engineer takes.

nickthemagicman 8y ago

I have mixed feelings about this and I don't have enough experience to label on method better than the other. Lambdas basically require zero maintenance. SQS requires zero maintenance. EC2 load balancer is zero maintenance. And the setup is trivial too and there's no migration time down the line.If you start off with native cloud for everything you can keep your maintenance and setup costs down drastically. However, a lot can be done with the old school unsexy tech. So I'm mixed.

dr01d 8y ago
Zero maintenance? What about standing up multiple environments for staging, prod? Sharing secrets ,keys, and env vars? Deployments? Logging? No migration time down the line? I'm pretty sure GCP, Azure don't have Lambda, SQS, or EC2 load balancers so you absolutely will have migration time if you have to retool your implementation to switch cloud providers or products.
- nickthemagicman 8y ago
  You make really good points about that. The classic things do migrate providers the easiest of all. I've just found in my experience maintaining a web server or a database server, keeping security in mind, upgrades, scaling, etc. is alot more work than simply spinning up RDS and a Lambda with API gateway. Or even hosting static sites on s3 or Netlify. Like I said I don't know enough to say one is better. Also, is your name Jason?
- nickthemagicman 8y ago
  I think we may have worked together in the past. Lol

justasitsounds 8y ago

In my case switching to an AWS API Gateway + Lambda stack means I have zero-downtime, canary deployments that take less than 5 minutes to deploy from version control. Api Gateway is configured using the same swagger doc that autogenerates my request/response models (go-swagger) and (almost) completely removes routing, request logging, throttling and authentication concerns from the request handlers. Combined with a staticly hosted front-end and SNS/SQS+lambda pub-sub for out-of-process workers I never have to worry about auto-scaling, load-balancing or cache-invalidation and we only pay for what we use. It may not suit every use case, but in our case, we have bursty, relatively low-volume traffic and the hosting bill for our public-facing site/service that comprises most of the main business revenue is the same as a rounding error on our BI service bill.

Aeolun 8y ago
How do you deploy in less than 5 minutes? Our serverless deployments are running into 20 minutes now, and it’s becoming a bit unfunny.
- justasitsounds 8y ago
  We use golang lambdas, binaries are built in our CI pipeline. Build stage takes ~10 seconds, tests (integration + unit) take ~30 seconds. We use AWS SAM for generating our CFN templates, we package and deploy using the AWS Cloudformation CLI and this takes the remaining 3-4 minutes. I didn't include post-deployment end-to-end tests in the 5 minute figure, but technically speaking, we do deploy that quickly
- redisman 8y ago
  How does it take 20 minutes? Do you deploy everything at once? Most of our APIs take a few minutes (depending on what kind of a deploy it is).
  - bazza451 8y ago
    Obviously not using cloudfront...might be the slowest service I’ve ever seen to deploy

halbritt 8y ago

I'm generally a proponent of DIY, this doesn't make sense. When an org needs to scale, it makes sense to cultivate skill in the underlying infrastructure. On day 1, serverless makes a lot of sense because it encourages development patterns that eventually will scale nicely.

dwaltrip 8y ago
Placing stateless web servers on simple VPS instances with load balancers in front doesn't scale nicely? Or perhaps I'm misunderstanding the parent.
- deleted 8y ago
  [deleted]
- redisman 8y ago
  Serverless does force you to think for higher concurrency than bigger server instances which is worth something.

com2kid 8y ago

Serverless is also easier to develop for. With Google Firebase Functions I was able to start writing REST APIs in minutes. Compare that to setting up a VM somewhere, getting a domain name + certs + express setup + deployment scripts, and then handling login credentials for all of the above. I had never done any of that (eventually I grew until I had to), so serverless let me get up and running really quickly. Now I prefer my own express instance, since deployment is much faster and debugging is much easier. But even for the debugging scenario, expecting everyone who wants to Just Write Code to get the horrid mess of JS stuff up and running in order to debug, ugh. (If it wasn't for HTTPS, Firebase's function emulator would be fine for debugging, as it is, a few nice solutions exist anyway.) But, to be clear, on day 1 the option for me to write a JS rest endpoint was: 1. Follow a 5-10 minute tutorials on setting up Firebase Functions. OR 1. Pick a VM host (Digital Ocean rocks) and setup an account 2. Learn how to provision a VM 3. Get a domain 4. Get domain over to my host 5. SSH into machine as root, setup non-root accounts with needed permissions 6. Setup certbot 7. Learn how to setup an Express server 8. Setup an nginx reverse proxy to get HTTPS working on my Express server 9. Write deployment scripts (ok SCP) to copy my working code over to my machine 10. Setup PM2 to watch for script changes 11. Start writing code! (12. Keep track, in a secure fashion, of all the credentials I just created for the above steps!) I am experienced in a lot of things, and thankfully I had some experience messing around with VMs and setting up my own servers before, but despite what everyone on HN may think, not every dev in the world also wants to run a bunch of VMs and manage their setup/configuration just to write a few REST endpoints! So yeah, instead I can type 'firebase deploy' in a folder that has some JS functions exported in an index.js file and a minute later out pops some HTTPS URLs.

fyfy18 8y ago
If you don't want to learn DevOps why not use a PaaS like Heroku? That way when you want to learn DevOps, you can move your application without rewriting large swathes of it. It's funny but when I learned to code basically all ISPs provided you with free hosting and a database, and you just needed to drag and drop a PHP file to make it live. It's like we have gone backwards not just in terms of openness but also in terms of complexity.
- 8y ago

lsorber 8y ago

> In most cases, very few companies have products that need to scale to extreme load day 1 or even year 1. That wouldn't be a great reason to choose serverless indeed. However, that doesn't mean serverless isn't still the right choice. We've tried both the traditional approach you describe and serverless, and from experience the latter is 10x less infrastructure code than the former (we compared both Terraform implementations). If serverless fits your use case, saving time and effort is a very good reason to go for it IMHO.

illumin8 8y ago

Honestly, I've developed the last few projects in Serverless framework and deployed to AWS Lambda. My biggest project is a custom Python application monitoring solution that has 4 separate Lambda function/handlers, populates an RDS PostgresSQL database with monitoring data (runs in a VPC), then reads from that database using complex joins across multiple application metric tables to send time-series data to Cloudwatch metrics. Then, I configured Cloudwatch alarms to have thresholds for certain metrics that send a web hook to PagerDuty. The benefit is that my monitoring system never goes down, never needs to be patched manually (AWS even patches my Postgres database, and fails over to the warm standby during patching), and never needs any system administration. Have you ever worked at a company where you had a serious outage that you didn't detect quickly enough because a monitoring system was down? Having a Serverless monitoring system means this has happened 0 times despite our app running in production for almost a year now.