7 ms·
It's ironic Perplexity itself blocks crawlers: $ curl -sI https://www.perplexity.ai | head -1 HTTP/2 403 Edit: trying to fake a browser user agent wit
by rustc 1y ago
It's ironic Perplexity itself blocks crawlers:
$ curl -sI https://www.perplexity.ai | head -1
HTTP/2 403
Edit: trying to fake a browser user agent with curl also doesn't work, they're using a more sophisticated method to detect crawlers.
- thambidurai 1y agosomeone asked this already to the CEO: https://x.com/AravSrinivas/status/1819610286036488625 https://x.com/AravSrinivas/status/1819610286036488625
- fireflash38 1y agoThe bots are coming from inside the house
- czk 1y agoironically... they use cloudflare.
- Trung0246 1y agoTry this then: https://github.com/lwthiker/curl-impersonate https://github.com/lwthiker/curl-impersonate