32 ms·

Another option would be to use a service like Cloudflare, which offers protections against scraping and other malicious behavior. This can help prevent the prox

Another option would be to use a service like Cloudflare, which offers protections against scraping and other malicious behavior. This can help prevent the proxy-mirror site from being able to access your site's content. https://blog.cloudflare.com/introducing-scrapeshield-discover-defend-dete/ https://blog.cloudflare.com/introducing-scrapeshield-discove...

stanislavb 4y ago
I have Cloudflare at the front already. The issue is that they are not actively scraping the content but rather mirroring it on demand.
- maybe_pablo 4y ago
  Just an idea but maybe you could cause big loads on their servers by requesting in parallel a large amount of urls where you actively serve a gzipped massive html file that is full of links to your website. EDIT: or building up on what user zhouyisu says above you can generate your perfect match IP blacklist by calling urls via the abusing site that automatically puts any caller into the blacklist.
  - is_true 4y ago
    I'm not sure how easy would be to serve a "zip bomb" without getting into trouble, but it would be neat
- fragmede 4y ago
  Then where they're coming from should be exceedingly visible in logs (via splunk or whatever), so deny those requests.
- DoctorOW 4y ago
  If you're on Cloudflare and they're stripping JS wouldn't a JS challenge be appropriate? https://developers.cloudflare.com/fundamentals/get-started/concepts/cloudflare-challenges/#js-challenge https://developers.cloudflare.com/fundamentals/get-started/c...
  - pharmakom 4y ago
    Pretty anti-user though
    - DoctorOW 4y ago
      Because it requires JS? How about this then... a[href*= "sukuns.us.to"] { display:none; } Then use SRI to enforce that CSS.
- nixcraft 4y ago
  Enable bot protection in Cloudflare as per your plan https://developers.cloudflare.com/bots/get-started/ https://developers.cloudflare.com/bots/get-started/

stanislavb 4y ago

I have Cloudflare at the front already. The issue is that they are not actively scraping the content but rather mirroring it on demand.

maybe_pablo 4y ago
Just an idea but maybe you could cause big loads on their servers by requesting in parallel a large amount of urls where you actively serve a gzipped massive html file that is full of links to your website. EDIT: or building up on what user zhouyisu says above you can generate your perfect match IP blacklist by calling urls via the abusing site that automatically puts any caller into the blacklist.
- is_true 4y ago
  I'm not sure how easy would be to serve a "zip bomb" without getting into trouble, but it would be neat
fragmede 4y ago
Then where they're coming from should be exceedingly visible in logs (via splunk or whatever), so deny those requests.
DoctorOW 4y ago
If you're on Cloudflare and they're stripping JS wouldn't a JS challenge be appropriate? https://developers.cloudflare.com/fundamentals/get-started/concepts/cloudflare-challenges/#js-challenge https://developers.cloudflare.com/fundamentals/get-started/c...
- pharmakom 4y ago
  Pretty anti-user though
  - DoctorOW 4y ago
    Because it requires JS? How about this then... a[href*= "sukuns.us.to"] { display:none; } Then use SRI to enforce that CSS.
nixcraft 4y ago
Enable bot protection in Cloudflare as per your plan https://developers.cloudflare.com/bots/get-started/ https://developers.cloudflare.com/bots/get-started/