7 ms·

While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that w

While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that way.

kauegimenes 2y ago
Its not only about protection, most web developers would use Cloudflare since its a free CDN and would increase the app load time considerably.
- chrisweekly 2y ago
  increase -> decrease
  - rkagerer 2y ago
    Except for those encountering that dreaded captcha.
- solardev 2y ago
  You can separately configure (to a large degree) the caching vs protection features, though.
warkdarrior 2y ago
Because if such hosting and developer tools are not protected against bots, the tools end up used for phishing, spamming, etc.
rad_gruchalski 2y ago
They are not targeting people like you. Bots are the target. If you look like a bot, how are they going to distinguish?
- KronisLV 2y ago
  > If you look like a bot, how are they going to distinguish? Some non-existant system of attesting that I'm person X (possibly through an e-ID card) who has issued a client certificate Y (cert chain, using my e-ID cert to sign) to be used with my device Z (presumably with a device fingerprint or IP range attached to the cert). Of course, this would mean no privacy, but that's not that different from being signed in through Google as an identity provider, we'd just shift the mechanism to be universal (like client certs already are). One of the options that would take more coordination than will probably happen (though very similar to some e-signature solutions in EU, which we already use) but I could see using something like that for a variety of professional/service sites, since signing in with the e-ID card directly is already a thing on some sites here (government sites, banking sites, utilities sites).
  - rad_gruchalski 2y ago
    Okay. Do that globally. And solve the ddos problem as you’re on it. If you add transparent tls termination, edge, caching, dns… maybe I’ll have a look! I had a guy like that working with me. Blocked every possible tracker, disabled javascript, used some niche browser, proton mail, and then complains that google doesn’t allow him to sign in. I get it, privacy and what not. But the guy was an outlier. Some random blogs, product pages aren’t gov, most likely have no way to opt-in for gov eID (maybe they aren’t based in the EU), and they only care that their service is available fast globally and that they get ddos protection for free (plus some other convenience features).
    - KronisLV 2y ago
      > Do that globally. We already do a simpler version of that with TLS and HTTPS, there are globally trusted root certs that ship with most OSes and browsers. It's just that we haven't extended the same approach to client certs and identity verification, instead having a bunch of walled gardens and governments running legacy methods of figuring out who someone is, as opposed to various eID mechanisms. If I trust news.ycombinator.com because I trust ISRG Root X1, I might similarly trust John Doe's iPhone because I trust the government of France's CA, as a hypothetical, as long as the certification chain is valid there. It's a problem that's technically solvable (say, in 20-50 years), but won't get done because good luck getting a bunch of governments to collaborate on that across the world. It's actually a surprise that we have TLS in the first place.
- Hizonner 2y ago
  Their problem. They are not entitled to make it other people's problem.
  - scarface_74 2y ago
    If I have a process that works for 95% of the people, why should I care about outliers who use Linux behind a VPN on a heavily customized version of Firefox?
    - Hizonner 2y ago
      Because they are standards compliant and you aren't, and you are legally required to provide an unsubscribe service or whatever without undue barriers around it.
      - scarface_74 2y ago
        For unsubscribe - yes. Everything else - no. But if I am using standards and they have an ad blocker that blocks some of the functioning of my site, am I also required to test my site against that?
        luckylion 2y ago
        > Everything else - no. I'd include _everything_ important in the "yes" category. If I cannot access the customer panel to update settings or notify them of a bug that is affecting me because I'm using Firefox ("works for 95% of users"), they're just not keeping up their end of the contract. Remember, 95% excludes everything but chromium/webkit-engines.
        scarface_74 2y ago
        Every SaaS company I’ve worked for has had a compatibility matrix where we say what we support. If we lost customers who were running a highly customized Firefox on Linux, so be it. Every company decides which customers are worth going after.
luckylion 2y ago
I'm convinced that's mostly incompetence on the side of the companies that implement that protection. "We have a problem with bots" - "Just create a firewall rule, whatever"
- rustc 2y ago
  What other way would you suggest to protect a free service from bots? Cloudflare is often the easiest to implement and has a generous limit on their free plan.
  - luckylion 2y ago
    Oh, they absolutely are, I don't disagree -- I use them too. But the immediate response to bots shouldn't be "make everyone go through a captcha". There's lots of nuance that you can tune to deal with your particular situation, but the first thing I'd do is block known bots or ASNs, set up a limit to trigger (bots usually don't make 1 document request a minute), set up higher limits for users who (seem to) have a valid cookie indicating that they are logged in, set up different thresholds for certain countries that are more risky etc etc. What you need to protect your service depends on your situation, it's not a one-size-fits-all solution. E.g. I find that I have no automated contact form spam once I add a simple JS to add some data that isn't standard, but I'm sure that wouldn't hold up if there was enough incentive to try to get past it. But the OP mentioned not just free services, but e.g. webhosting logins. That's just sad, as is Cloudflare's community being behind an aggressive captcha. I'm a user, I'm logged in, I've posted before, I'm in good standing, yet when I go there, I need to solve a captcha. When I then go there again an hour later, guess what, another captcha. Either there's another reason I'm not seeing or it's just lazyness as in "we need to have a forum but we really don't want to spend any resources on it, just put up an aggressive captcha that'll filter out most bots and everyone but the determined users".
    - hombre_fatal 2y ago
      Fwiw, Cloudflare does do a multivariate confidence check which is why it has multiple tiers: no captcha, a one-click captcha, the annoying puzzle captcha once, the annoying puzzle captcha six times in a row. > I'm a user, I'm logged in, I've posted before, I'm in good standing, yet when I go there, I need to solve a captcha. Though consider the fact that taking over someone's account shouldn't give you (a spammer) unlimited access either. The spambots you see on Twitter are mostly cred-stuffed accounts. It's a hard problem. Existing accounts are more dangerous than fresh accounts. Imo, "write your own password" should be a thing of the past. Services should just auto-gen a password or there should be a way to require the OS (like a password manager) to generate one to avoid cred-stuffing. We're letting down the average person by making them come up with unique passwords for every service instead of just helping them. Though I'm way off topic.
solardev 2y ago
Most developers I've met were actually similarly lazy... we just use Chrome on Mac, and don't really want to deal with VPNs unless our employers force us to. The last few Firefox holdouts also switched after running into various WebGL/Canvas/etc issues. The same attitude that leads us to focus on "happy path" users and ignore edge cases often also causes us to sheeple into that same basic dev group. Long gone are the days where most devs custom build Linux boxen from scratch and compile custom kernels to our liking... Anyway, I know the "Cloudflare's monopoly gating is killing web openness!" meme is common online, especially on HN, but in real life I've never actually heard anyone else complain about it (either a fellow dev or a customer or a manager). Instead, it's been universal praise for the actual issues Cloudflare exists to solve (CDN, bot protection, serverless, etc)... they are a godsend for small businesses that otherwise get immediately flooded by spam requests, especially from China, Russia, and India. And if you think Cloudflare is bad, it was even worse before they became dominant, with terrible services like Incapsula/Imperva charging way more but providing both worse bot protection AND more false positives, or the really hard early reCAPTCHAs (that Cloudflare was largely able to replace, for users who DO fit within the "norm"). That, or you'd have to fight every random sysadmin with their own lazy rules, like firewall rules that blacklisted entire regional ISPs and took weeks or months to resolve, if they ever even checked their emails. As inconvenient as Cloudflare is for users who take privacy seriously and try to be less trackable, for the other 90% of us who don't care as much and easily fit into their "norm" model, it's much nicer than what came before. Site downtime and slowness are also much less common now, in no small part because of their easy CDN and caching. From the implementation side, I've set up a few Cloudflare accounts in my career, but do take the time to try to configure it to balance security vs accessibility for any given target audience. Sometimes we'd block entire countries, other times we'd minimize security to ensure maximum reach, but usually we'd customize rulesets in the middle for any given company & audience. I never got a complaint about it (our emails were still available and not blocked). This was always a direct response to some business need, usually spambots or DDoS attempts that fail2ban etc. couldn't catch well enough. For the business, it was usually a "shit, our website is down again, what is it this time", and the choice between "for free or $20 we can get it back up again and not have this issue anymore" or "we can spend thousands of dollars and weeks of labor building our own security solution" is pretty easy. "What about that one guy who is proxied behind TOR and three VPNs with a random user agent using a text-only browser he wrote himself?" never really factors into that process =/ There's just not enough users like that out in the wild vs the very real constant threat of bots and malware. It's a shitty situation that the web is like this today, and I wish it weren't the case, but it really is an arms race, and these imperfect weapons are just what most of us have access to...

While you hit the nail on the head, I am still surprised that so many tools targeted at people like me (web hosting, developer tools, etc.) are protected that way.

kauegimenes 2y ago

Its not only about protection, most web developers would use Cloudflare since its a free CDN and would increase the app load time considerably.

chrisweekly 2y ago
increase -> decrease
- rkagerer 2y ago
  Except for those encountering that dreaded captcha.
solardev 2y ago
You can separately configure (to a large degree) the caching vs protection features, though.

warkdarrior 2y ago

Because if such hosting and developer tools are not protected against bots, the tools end up used for phishing, spamming, etc.

rad_gruchalski 2y ago

They are not targeting people like you. Bots are the target. If you look like a bot, how are they going to distinguish?

KronisLV 2y ago
> If you look like a bot, how are they going to distinguish? Some non-existant system of attesting that I'm person X (possibly through an e-ID card) who has issued a client certificate Y (cert chain, using my e-ID cert to sign) to be used with my device Z (presumably with a device fingerprint or IP range attached to the cert). Of course, this would mean no privacy, but that's not that different from being signed in through Google as an identity provider, we'd just shift the mechanism to be universal (like client certs already are). One of the options that would take more coordination than will probably happen (though very similar to some e-signature solutions in EU, which we already use) but I could see using something like that for a variety of professional/service sites, since signing in with the e-ID card directly is already a thing on some sites here (government sites, banking sites, utilities sites).
- rad_gruchalski 2y ago
  Okay. Do that globally. And solve the ddos problem as you’re on it. If you add transparent tls termination, edge, caching, dns… maybe I’ll have a look! I had a guy like that working with me. Blocked every possible tracker, disabled javascript, used some niche browser, proton mail, and then complains that google doesn’t allow him to sign in. I get it, privacy and what not. But the guy was an outlier. Some random blogs, product pages aren’t gov, most likely have no way to opt-in for gov eID (maybe they aren’t based in the EU), and they only care that their service is available fast globally and that they get ddos protection for free (plus some other convenience features).
  - KronisLV 2y ago
    > Do that globally. We already do a simpler version of that with TLS and HTTPS, there are globally trusted root certs that ship with most OSes and browsers. It's just that we haven't extended the same approach to client certs and identity verification, instead having a bunch of walled gardens and governments running legacy methods of figuring out who someone is, as opposed to various eID mechanisms. If I trust news.ycombinator.com because I trust ISRG Root X1, I might similarly trust John Doe's iPhone because I trust the government of France's CA, as a hypothetical, as long as the certification chain is valid there. It's a problem that's technically solvable (say, in 20-50 years), but won't get done because good luck getting a bunch of governments to collaborate on that across the world. It's actually a surprise that we have TLS in the first place.
Hizonner 2y ago
Their problem. They are not entitled to make it other people's problem.
- scarface_74 2y ago
  If I have a process that works for 95% of the people, why should I care about outliers who use Linux behind a VPN on a heavily customized version of Firefox?
  - Hizonner 2y ago
    Because they are standards compliant and you aren't, and you are legally required to provide an unsubscribe service or whatever without undue barriers around it.
    - scarface_74 2y ago
      For unsubscribe - yes. Everything else - no. But if I am using standards and they have an ad blocker that blocks some of the functioning of my site, am I also required to test my site against that?
      - luckylion 2y ago
        > Everything else - no. I'd include _everything_ important in the "yes" category. If I cannot access the customer panel to update settings or notify them of a bug that is affecting me because I'm using Firefox ("works for 95% of users"), they're just not keeping up their end of the contract. Remember, 95% excludes everything but chromium/webkit-engines.
        scarface_74 2y ago
        Every SaaS company I’ve worked for has had a compatibility matrix where we say what we support. If we lost customers who were running a highly customized Firefox on Linux, so be it. Every company decides which customers are worth going after.

luckylion 2y ago

I'm convinced that's mostly incompetence on the side of the companies that implement that protection. "We have a problem with bots" - "Just create a firewall rule, whatever"

rustc 2y ago
What other way would you suggest to protect a free service from bots? Cloudflare is often the easiest to implement and has a generous limit on their free plan.
- luckylion 2y ago
  Oh, they absolutely are, I don't disagree -- I use them too. But the immediate response to bots shouldn't be "make everyone go through a captcha". There's lots of nuance that you can tune to deal with your particular situation, but the first thing I'd do is block known bots or ASNs, set up a limit to trigger (bots usually don't make 1 document request a minute), set up higher limits for users who (seem to) have a valid cookie indicating that they are logged in, set up different thresholds for certain countries that are more risky etc etc. What you need to protect your service depends on your situation, it's not a one-size-fits-all solution. E.g. I find that I have no automated contact form spam once I add a simple JS to add some data that isn't standard, but I'm sure that wouldn't hold up if there was enough incentive to try to get past it. But the OP mentioned not just free services, but e.g. webhosting logins. That's just sad, as is Cloudflare's community being behind an aggressive captcha. I'm a user, I'm logged in, I've posted before, I'm in good standing, yet when I go there, I need to solve a captcha. When I then go there again an hour later, guess what, another captcha. Either there's another reason I'm not seeing or it's just lazyness as in "we need to have a forum but we really don't want to spend any resources on it, just put up an aggressive captcha that'll filter out most bots and everyone but the determined users".
  - hombre_fatal 2y ago
    Fwiw, Cloudflare does do a multivariate confidence check which is why it has multiple tiers: no captcha, a one-click captcha, the annoying puzzle captcha once, the annoying puzzle captcha six times in a row. > I'm a user, I'm logged in, I've posted before, I'm in good standing, yet when I go there, I need to solve a captcha. Though consider the fact that taking over someone's account shouldn't give you (a spammer) unlimited access either. The spambots you see on Twitter are mostly cred-stuffed accounts. It's a hard problem. Existing accounts are more dangerous than fresh accounts. Imo, "write your own password" should be a thing of the past. Services should just auto-gen a password or there should be a way to require the OS (like a password manager) to generate one to avoid cred-stuffing. We're letting down the average person by making them come up with unique passwords for every service instead of just helping them. Though I'm way off topic.

solardev 2y ago

Most developers I've met were actually similarly lazy... we just use Chrome on Mac, and don't really want to deal with VPNs unless our employers force us to. The last few Firefox holdouts also switched after running into various WebGL/Canvas/etc issues. The same attitude that leads us to focus on "happy path" users and ignore edge cases often also causes us to sheeple into that same basic dev group. Long gone are the days where most devs custom build Linux boxen from scratch and compile custom kernels to our liking... Anyway, I know the "Cloudflare's monopoly gating is killing web openness!" meme is common online, especially on HN, but in real life I've never actually heard anyone else complain about it (either a fellow dev or a customer or a manager). Instead, it's been universal praise for the actual issues Cloudflare exists to solve (CDN, bot protection, serverless, etc)... they are a godsend for small businesses that otherwise get immediately flooded by spam requests, especially from China, Russia, and India. And if you think Cloudflare is bad, it was even worse before they became dominant, with terrible services like Incapsula/Imperva charging way more but providing both worse bot protection AND more false positives, or the really hard early reCAPTCHAs (that Cloudflare was largely able to replace, for users who DO fit within the "norm"). That, or you'd have to fight every random sysadmin with their own lazy rules, like firewall rules that blacklisted entire regional ISPs and took weeks or months to resolve, if they ever even checked their emails. As inconvenient as Cloudflare is for users who take privacy seriously and try to be less trackable, for the other 90% of us who don't care as much and easily fit into their "norm" model, it's much nicer than what came before. Site downtime and slowness are also much less common now, in no small part because of their easy CDN and caching. From the implementation side, I've set up a few Cloudflare accounts in my career, but do take the time to try to configure it to balance security vs accessibility for any given target audience. Sometimes we'd block entire countries, other times we'd minimize security to ensure maximum reach, but usually we'd customize rulesets in the middle for any given company & audience. I never got a complaint about it (our emails were still available and not blocked). This was always a direct response to some business need, usually spambots or DDoS attempts that fail2ban etc. couldn't catch well enough. For the business, it was usually a "shit, our website is down again, what is it this time", and the choice between "for free or $20 we can get it back up again and not have this issue anymore" or "we can spend thousands of dollars and weeks of labor building our own security solution" is pretty easy. "What about that one guy who is proxied behind TOR and three VPNs with a random user agent using a text-only browser he wrote himself?" never really factors into that process =/ There's just not enough users like that out in the wild vs the very real constant threat of bots and malware. It's a shitty situation that the web is like this today, and I wish it weren't the case, but it really is an arms race, and these imperfect weapons are just what most of us have access to...