6 ms·

This is my worst nightmare as a bootstrapped founder. And that there's no way to put a limit on spend is ridiculous. Someone that doesn't want me to do well can

by ji_zai 3y ago

This is my worst nightmare as a bootstrapped founder. And that there's no way to put a limit on spend is ridiculous. Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere. Just went through Vercel's docs: --- "Vercel helps to mitigate against L3 and L4 DDoS attacks at the platform level. Usage will be incurred for requests that are successfully served prior to us automatically mitigating the event. Mitigation usually takes place within one minute. Usage will be incurred for requests that are not recognized as a DDoS event, such as bot and crawler traffic. You should monitor your usage and utilize Edge Middleware to protect against undesired traffic based on its IP, User-Agent header value, or other identifiers." --- That doesn't help me sleep well. I feel that by now, these hosting providers should simply adopt best ddos protection practices and take responsibility for failure to protect. "You should monitor your usage and utilize Edge Middleware to protect against undesired traffic based on its IP" - there should be some really good defaults for this right? Clearly it's possible - Cloudflare's ddos protection is worded more strongly. I'm willing to pay more for the service for peace of mind. Like, even $10/mo more to insure against getting smacked out of nowhere.

baq 3y ago
Host on a provider which bills per hour. This caps your cost. It also makes your users pissed because you will go down, but if you’re small, you can afford that. If you’re big, you already have scaling options and should have a team to handle ddos.
- teaearlgraycold 3y ago
  Yeah. Any host that won't infinitely scale out will solve this concern for you.
  - ehnto 3y ago
    I think most people pick Netlify for it's Infra as a Service offering, so it would be nice if they had a way to throttle and budget in that offering. I would even imagine Netlify's target market is small to mid size businesses who really don't need ridiculous burstable scaling capacity at all. Seems like a bit of a trap door for that customer base. I agree though, I wouldn't host on them as a small business due to that risk, but I am also happy running my own server so I might be an edge case.
- cotillion 3y ago
  My experience is that customers don't really care that much about small amounts of downtime no matter what size you are, people mostly get that unexpected stuff happens as long as you don't get hacked or misplace their data. Customers might complain a bit but seldom leave because of a few hours downtime. This seems to mostly hold true to developers also, GitHub manages to survive just fine after all.
  - chgs 3y ago
    Depends on your service. 20 second downtime on loading HN? Nobody cares. 20 second downtime on the last play of the Super Bowl - big problems. For most internet consumers we’re accustomed to poor service so if a page doesn’t load we’ll assume it’s a local problem and try again 20 seconds later, same with buffering, it’s just something that happens occasionally. This is increasing the case for phone calls too. Legacy live tv and radio going silent though is still a major issue, especially on live events.
    - belthesar 3y ago
      Sure, but now you're talking about sites with completely different service level objectives, and conversely, different budgets for their hosting. The problem here, to play off of your analogy, is that Netlify is treating every customer, many with SLOs likely less strict than HN as if they are the Super Bowl. This is an assumption that, according to the most recent policy discoverable by looking through their forum posts, is a constraint of their platform, and something they tout as a feature, not a bug. When users expressed concerns for a similar scenario that the OP experienced on their community forum, Netlify's staff responded with "how likely is this, really?" Only has to happen once to put someone in significant financial harm.
ignoramous 3y ago
> Cloudflare's ddos protection Yeah, we got hammered once with over 10TB/mo and noped out of Netlify as fast as we could: https://twitter.com/rethinkdns/status/1370342245841342466 https://twitter.com/rethinkdns/status/1370342245841342466 Had to pay the bill in full. Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0. Makes it hard to move to any other platform. As a small tech shop, this is my Hotel California I'm happy to never leave.
- pacifika 3y ago
  That’s a free tier that doesn’t sound sustainable then, so that raises alarm bells to me.
  - lifthrasiir 3y ago
    I have heard that they rather drastically constrain QoS instead, which does sound reasonable. So you are still not charged for abusive traffic, but your service will be much slower than what is actually possible with paid tiers.
    - throwaway290 3y ago
      So you'd be either slow or pay them "for protection". Something that reminds me of;)
      - stavros 3y ago
        Capitalism? Mob-style "protection" would be if Cloudflare were the ones who DDoSed you if you didn't pay.
        throwaway290 3y ago
        How naive if you think the mob would disclose when it's affiliates trash your shop.
        gkbrk 3y ago
jefozabuss 3y ago
If you want to sleep tight just get a dedicated server or VPS from something like Hetzner and/or combine with CDN providers like BunnyCDN - set up alerts just in case though. It takes more time and resources to manage it but you could save a lot on it in this case.
- amerkhalid 3y ago
  > It takes more time and resources to manage it For most of the new web projects, setting up your brand new server is pretty well documented process and should not take more than couple of hours. It get complicated when you grow and add more servers or components. But at that point, you should be able to afford a part-time consultant to handle complicated tasks or just use Cloud then.
- mro_name 3y ago
  I'd even say build your system so as it can run on shared hosting. This way you even save the management.
- herbst 3y ago
  This so much. My hetzner (best choice for a media server within Europe) has 0 downtime in 1.5 years. And exactly as you said I am using bunny as well, which costs me a few $ per year.
- raxxorraxor 3y ago
  That is my setup after leaving AWS for some of my services (low user amount b2b). I put in far less resources and maintenance after I had the system running. Especially if you need to manage the software running anyway.
WyvernDrexx 3y ago
Imagine you lost your job. So you are here enjoying creating and hosting your hobby projects in theses services. Now, suddenly one fine morning you get slapped with $104K bill because someone decided to randomly ddos your one page dog lover website. Now, who in the would would be thinking of having ddos protection for their hobby project? This is just absurd thinking.
- signaru 3y ago
  Can't hosts just make a site unavailable once it reaches its plan's bandwidth limit, DDoS or not? I think being offline is a lesser headache than a large bill, especially for those who are inclined to a free tier to begin with.
  - cxr 3y ago
    Folks regularly show up in HN comments during these discussions stating the opposite—that it's categorically better for all sites/projects, now matter how inconsequential, to stay online. It's weird. This includes some of the TPTB, too. Occasionally, though, someone'll say the quiet part out loud. E.g. re fly.io: > putting work into features specifically to minimize how much people spend seems like a good way to fail a company <https://news.ycombinator.com/item?id=24699292 https://news.ycombinator.com/item?id=24699292>
- b112 3y ago
  This may seem weird, but I believe ToS ae the real problem here. I call it the "car rental" problem. When I rent a car in person, I am often given a contract. And this contract is filled with tiny print, and pages of it. There are often people behind you, waiting, and bored/annoyed people behind the counter, waiting. This is beyond unreasonable. A point of sale contract should be short, in readable text, and understandable. For example, renting a car? Under a page, easily parseable, and if the person behind the counter cannot explain it, it is null and void. From a legal side, you can do this. And you can explain legal terms. Of course this means you are describing intent, which limits one in court, oh boo hoo Mr Lawyer. Cry me a river. Well the same should be true of any retail contract. Sign up for a service? One page with costs listed. At least then, there is hope of an end-user sort of understanding. And as one could claim that a DoS was actually targetting the provider, and not the website, that should be described too. So back to the topic at hand. I would write a demand letter, insistong Netify explain the charges, and ask them if they and their IP ranges were DoS, and if so that the charges be reversed. Because you shpuld not be paying, if someone attacks Netify. This letter should also be sent by mail, sig required, to the corporate address too.
niceice 3y ago
How do these hosting providers sleep?
- chgs 3y ago
  In enormous mansions atop large piles of money.
- MikeDelta 3y ago
  I guess on eiderdown pillows.
rozenmd 3y ago
This might be a good time to point out Cloudflare Pages: https://pages.cloudflare.com/ https://pages.cloudflare.com/ Under the free tier: > Unlimited bandwidth
- itake 3y ago
  I'm moving everything to Cloudflare.
  - cod1r 3y ago
    Just looking at pages.cloudflare.com now and I think I'm going to be using cloudflare from now on.
  - rubymamis 3y ago
    I'm trying to sign up but it keeps saying "Verification is taking longer than expected. Check your Internet connection and refresh the page if the issue persists." Does anyone else experience this as well?
    - Arnavion 3y ago
      That's the Cloudflare user experience in a nutshell. Your users will see the same thing when they visit your Cloudflare-hosted site.
      - piperswe 3y ago
        As a heads up, you can disable challenges almost entirely if you don't want any visitors challenged. Security -> Settings -> Security Level -> Essentially off
    - xrisk 3y ago
      Might be iCloud private relay if you use that
  - ks2048 3y ago
    Ditto. "Migrating from Netlify to Pages" : https://developers.cloudflare.com/pages/migrations/migrating-from-netlify/ https://developers.cloudflare.com/pages/migrations/migrating...
jart 3y ago
Use a token bucket on your web server to catch abusive IPs and then blackhole them using `iptables -t raw -I PREROUTING -s ip -j DROP`. I know. I run https://ipv4.games/ https://ipv4.games/ which invites hackers to unleash their botnets, and the service runs on a small VM with only a few cores. It's been attacked by botnets with 49,131,669 IP addresses. There's no Cloudflare frontend or anything like that, because back when I used Cloudflare, the people who attacked the service would actually bring down the Cloudflare nodes before they brought down my web server. I doubt I've ever paid more than $100/month to operate the service. Please note that your service provider needs to have free ingress in order for this strategy to be effective.
- wielebny 3y ago
  This strategy may work for a (D)DoS that is targeted to an application layer, but won't work if the attack is designed to exhaust your bandwidth. Once you're receiving more traffic than you network cards can handle, it does not matter if you'll drop the packets with iptables or not. I was the target of attacks that caused Hetzner to terminate my contract. I was leasing physical servers there, so I assume the attacks were overwhelming their infrastructure.
  - jart 3y ago
    I rent a GCE VM and there's not many if any people out there who can exhaust Google's network infrastructure. The only thing I have to worry about is making sure my server doesn't respond to abusive traffic.
  - zettabomb 3y ago
    These days it seems that DDoS attacks are often not targeted at bandwidth either, but rather packets per second. It is (apparently) much easier to exhaust routing capacity with an inordinate number of tiny packets than with a still large number of large packets. Cloudflare has some fun ways to deal with this [0]. [0] https://blog.cloudflare.com/mitigating-a-754-million-pps-ddos-attack-automatically https://blog.cloudflare.com/mitigating-a-754-million-pps-ddo...
codewithcheese 3y ago
Vervel charges $400/TB for excess bandwidth, it's not even DDoS you should worry about, just moderate success.
- chgs 3y ago
  That’s a crazy high bandwidth. Bandwidth isn’t free, but $400 will get you a month of 10gig in my local peering point, that’s 1TB in 15 minutes.
Mandatum 3y ago
Wait until you learn that Vercel only supports blocking IP CIDR ranges on the Enterprise plan.
immibis 3y ago
Those services exist, and you have the option to use them. Netlify is not one. Apparently, you chose that the un-insured solution was best for you.
CalRobert 3y ago
Vercel seems to exist only to promote lock-in.
dstroot 3y ago
Can anyone share an example of edge middleware that might protect you on Vercel?
- leerob 3y ago
  You can also turn on soft and hard spend limits on Vercel (including SMS alerts) https://vercel.com/blog/introducing-spend-management-realtime-usage-alerts-sms-notifications https://vercel.com/blog/introducing-spend-management-realtim...
  - Jaaneek 3y ago
    This is not a hard limit, this is just a webhook and you need to handle disabling everything on your own. 1. You can make mistakes in your code. 2. Some junior developer can by mistake change the code and make it ineffective. 3. Webhook is not instant so you can get billed more than the limit. 4. There is no information which project hit the limit, so you need to Fetch all of your project ID's and then disable all of them. You need to basically disable all the projects assigned to your team/organization. 5. Vercel doesn't guarantee in any way that you won't get billed more, they are just sending an information to you (with a delay). Hard limit should be a deal between user and company that the user won't ever get billed more than X$
redbell 3y ago
> Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere. An interesting story that expands on the above concept but a different vector entitled, "Illegal Life Pro Tip: Want to ruin your competitors business?" : https://news.ycombinator.com/item?id=36566634 https://news.ycombinator.com/item?id=36566634
op00to 3y ago
“We leave your safe deposit box unlocked. You might want to forge your own lock and key. If we happen to notice someone stealing out of your box, we will let them grab as much as they can for one minute, then maybe install our own lock if our revenue is close to target.”

by ji_zai 3y ago

baq 3y ago

Host on a provider which bills per hour. This caps your cost. It also makes your users pissed because you will go down, but if you’re small, you can afford that. If you’re big, you already have scaling options and should have a team to handle ddos.

teaearlgraycold 3y ago
Yeah. Any host that won't infinitely scale out will solve this concern for you.
- ehnto 3y ago
  I think most people pick Netlify for it's Infra as a Service offering, so it would be nice if they had a way to throttle and budget in that offering. I would even imagine Netlify's target market is small to mid size businesses who really don't need ridiculous burstable scaling capacity at all. Seems like a bit of a trap door for that customer base. I agree though, I wouldn't host on them as a small business due to that risk, but I am also happy running my own server so I might be an edge case.
cotillion 3y ago
My experience is that customers don't really care that much about small amounts of downtime no matter what size you are, people mostly get that unexpected stuff happens as long as you don't get hacked or misplace their data. Customers might complain a bit but seldom leave because of a few hours downtime. This seems to mostly hold true to developers also, GitHub manages to survive just fine after all.
- chgs 3y ago
  Depends on your service. 20 second downtime on loading HN? Nobody cares. 20 second downtime on the last play of the Super Bowl - big problems. For most internet consumers we’re accustomed to poor service so if a page doesn’t load we’ll assume it’s a local problem and try again 20 seconds later, same with buffering, it’s just something that happens occasionally. This is increasing the case for phone calls too. Legacy live tv and radio going silent though is still a major issue, especially on live events.
  - belthesar 3y ago
    Sure, but now you're talking about sites with completely different service level objectives, and conversely, different budgets for their hosting. The problem here, to play off of your analogy, is that Netlify is treating every customer, many with SLOs likely less strict than HN as if they are the Super Bowl. This is an assumption that, according to the most recent policy discoverable by looking through their forum posts, is a constraint of their platform, and something they tout as a feature, not a bug. When users expressed concerns for a similar scenario that the OP experienced on their community forum, Netlify's staff responded with "how likely is this, really?" Only has to happen once to put someone in significant financial harm.

ignoramous 3y ago

> Cloudflare's ddos protection Yeah, we got hammered once with over 10TB/mo and noped out of Netlify as fast as we could: https://twitter.com/rethinkdns/status/1370342245841342466 https://twitter.com/rethinkdns/status/1370342245841342466 Had to pay the bill in full. Cloudflare's free tier is ridiculous: We do over 30TB+ of genuine traffic for $0. Makes it hard to move to any other platform. As a small tech shop, this is my Hotel California I'm happy to never leave.

pacifika 3y ago
That’s a free tier that doesn’t sound sustainable then, so that raises alarm bells to me.
- lifthrasiir 3y ago
  I have heard that they rather drastically constrain QoS instead, which does sound reasonable. So you are still not charged for abusive traffic, but your service will be much slower than what is actually possible with paid tiers.
  - throwaway290 3y ago
    So you'd be either slow or pay them "for protection". Something that reminds me of;)
    - stavros 3y ago
      Capitalism? Mob-style "protection" would be if Cloudflare were the ones who DDoSed you if you didn't pay.
      - throwaway290 3y ago
        How naive if you think the mob would disclose when it's affiliates trash your shop.
      - gkbrk 3y ago

jefozabuss 3y ago

If you want to sleep tight just get a dedicated server or VPS from something like Hetzner and/or combine with CDN providers like BunnyCDN - set up alerts just in case though. It takes more time and resources to manage it but you could save a lot on it in this case.

amerkhalid 3y ago
> It takes more time and resources to manage it For most of the new web projects, setting up your brand new server is pretty well documented process and should not take more than couple of hours. It get complicated when you grow and add more servers or components. But at that point, you should be able to afford a part-time consultant to handle complicated tasks or just use Cloud then.
mro_name 3y ago
I'd even say build your system so as it can run on shared hosting. This way you even save the management.
herbst 3y ago
This so much. My hetzner (best choice for a media server within Europe) has 0 downtime in 1.5 years. And exactly as you said I am using bunny as well, which costs me a few $ per year.
raxxorraxor 3y ago
That is my setup after leaving AWS for some of my services (low user amount b2b). I put in far less resources and maintenance after I had the system running. Especially if you need to manage the software running anyway.

WyvernDrexx 3y ago

Imagine you lost your job. So you are here enjoying creating and hosting your hobby projects in theses services. Now, suddenly one fine morning you get slapped with $104K bill because someone decided to randomly ddos your one page dog lover website. Now, who in the would would be thinking of having ddos protection for their hobby project? This is just absurd thinking.

signaru 3y ago
Can't hosts just make a site unavailable once it reaches its plan's bandwidth limit, DDoS or not? I think being offline is a lesser headache than a large bill, especially for those who are inclined to a free tier to begin with.
- cxr 3y ago
  Folks regularly show up in HN comments during these discussions stating the opposite—that it's categorically better for all sites/projects, now matter how inconsequential, to stay online. It's weird. This includes some of the TPTB, too. Occasionally, though, someone'll say the quiet part out loud. E.g. re fly.io: > putting work into features specifically to minimize how much people spend seems like a good way to fail a company <https://news.ycombinator.com/item?id=24699292 https://news.ycombinator.com/item?id=24699292>
b112 3y ago
This may seem weird, but I believe ToS ae the real problem here. I call it the "car rental" problem. When I rent a car in person, I am often given a contract. And this contract is filled with tiny print, and pages of it. There are often people behind you, waiting, and bored/annoyed people behind the counter, waiting. This is beyond unreasonable. A point of sale contract should be short, in readable text, and understandable. For example, renting a car? Under a page, easily parseable, and if the person behind the counter cannot explain it, it is null and void. From a legal side, you can do this. And you can explain legal terms. Of course this means you are describing intent, which limits one in court, oh boo hoo Mr Lawyer. Cry me a river. Well the same should be true of any retail contract. Sign up for a service? One page with costs listed. At least then, there is hope of an end-user sort of understanding. And as one could claim that a DoS was actually targetting the provider, and not the website, that should be described too. So back to the topic at hand. I would write a demand letter, insistong Netify explain the charges, and ask them if they and their IP ranges were DoS, and if so that the charges be reversed. Because you shpuld not be paying, if someone attacks Netify. This letter should also be sent by mail, sig required, to the corporate address too.

niceice 3y ago

How do these hosting providers sleep?

chgs 3y ago
In enormous mansions atop large piles of money.
MikeDelta 3y ago
I guess on eiderdown pillows.

rozenmd 3y ago

This might be a good time to point out Cloudflare Pages: https://pages.cloudflare.com/ https://pages.cloudflare.com/ Under the free tier: > Unlimited bandwidth

itake 3y ago
I'm moving everything to Cloudflare.
- cod1r 3y ago
  Just looking at pages.cloudflare.com now and I think I'm going to be using cloudflare from now on.
- rubymamis 3y ago
  I'm trying to sign up but it keeps saying "Verification is taking longer than expected. Check your Internet connection and refresh the page if the issue persists." Does anyone else experience this as well?
  - Arnavion 3y ago
    That's the Cloudflare user experience in a nutshell. Your users will see the same thing when they visit your Cloudflare-hosted site.
    - piperswe 3y ago
      As a heads up, you can disable challenges almost entirely if you don't want any visitors challenged. Security -> Settings -> Security Level -> Essentially off
  - xrisk 3y ago
    Might be iCloud private relay if you use that
- ks2048 3y ago
  Ditto. "Migrating from Netlify to Pages" : https://developers.cloudflare.com/pages/migrations/migrating-from-netlify/ https://developers.cloudflare.com/pages/migrations/migrating...

jart 3y ago

Use a token bucket on your web server to catch abusive IPs and then blackhole them using `iptables -t raw -I PREROUTING -s ip -j DROP`. I know. I run https://ipv4.games/ https://ipv4.games/ which invites hackers to unleash their botnets, and the service runs on a small VM with only a few cores. It's been attacked by botnets with 49,131,669 IP addresses. There's no Cloudflare frontend or anything like that, because back when I used Cloudflare, the people who attacked the service would actually bring down the Cloudflare nodes before they brought down my web server. I doubt I've ever paid more than $100/month to operate the service. Please note that your service provider needs to have free ingress in order for this strategy to be effective.

wielebny 3y ago
This strategy may work for a (D)DoS that is targeted to an application layer, but won't work if the attack is designed to exhaust your bandwidth. Once you're receiving more traffic than you network cards can handle, it does not matter if you'll drop the packets with iptables or not. I was the target of attacks that caused Hetzner to terminate my contract. I was leasing physical servers there, so I assume the attacks were overwhelming their infrastructure.
- jart 3y ago
  I rent a GCE VM and there's not many if any people out there who can exhaust Google's network infrastructure. The only thing I have to worry about is making sure my server doesn't respond to abusive traffic.
- zettabomb 3y ago
  These days it seems that DDoS attacks are often not targeted at bandwidth either, but rather packets per second. It is (apparently) much easier to exhaust routing capacity with an inordinate number of tiny packets than with a still large number of large packets. Cloudflare has some fun ways to deal with this [0]. [0] https://blog.cloudflare.com/mitigating-a-754-million-pps-ddos-attack-automatically https://blog.cloudflare.com/mitigating-a-754-million-pps-ddo...

codewithcheese 3y ago

Vervel charges $400/TB for excess bandwidth, it's not even DDoS you should worry about, just moderate success.

chgs 3y ago
That’s a crazy high bandwidth. Bandwidth isn’t free, but $400 will get you a month of 10gig in my local peering point, that’s 1TB in 15 minutes.

Mandatum 3y ago

Wait until you learn that Vercel only supports blocking IP CIDR ranges on the Enterprise plan.

immibis 3y ago

Those services exist, and you have the option to use them. Netlify is not one. Apparently, you chose that the un-insured solution was best for you.

CalRobert 3y ago

Vercel seems to exist only to promote lock-in.

dstroot 3y ago

Can anyone share an example of edge middleware that might protect you on Vercel?

leerob 3y ago
You can also turn on soft and hard spend limits on Vercel (including SMS alerts) https://vercel.com/blog/introducing-spend-management-realtime-usage-alerts-sms-notifications https://vercel.com/blog/introducing-spend-management-realtim...
- Jaaneek 3y ago
  This is not a hard limit, this is just a webhook and you need to handle disabling everything on your own. 1. You can make mistakes in your code. 2. Some junior developer can by mistake change the code and make it ineffective. 3. Webhook is not instant so you can get billed more than the limit. 4. There is no information which project hit the limit, so you need to Fetch all of your project ID's and then disable all of them. You need to basically disable all the projects assigned to your team/organization. 5. Vercel doesn't guarantee in any way that you won't get billed more, they are just sending an information to you (with a delay). Hard limit should be a deal between user and company that the user won't ever get billed more than X$

redbell 3y ago

> Someone that doesn't want me to do well can simply ddos me into bankruptcy out of nowhere. An interesting story that expands on the above concept but a different vector entitled, "Illegal Life Pro Tip: Want to ruin your competitors business?" : https://news.ycombinator.com/item?id=36566634 https://news.ycombinator.com/item?id=36566634

op00to 3y ago

“We leave your safe deposit box unlocked. You might want to forge your own lock and key. If we happen to notice someone stealing out of your box, we will let them grab as much as they can for one minute, then maybe install our own lock if our revenue is close to target.”