8 ms·
https://localghost.dev/robots.txt https://localghost.dev/robots.txt User-Agent: * Allow: /
by Dotnaught 1y ago
https://localghost.dev/robots.txt https://localghost.dev/robots.txt
User-Agent: *
Allow: /
- thrance 1y agoNot like anyone respects that anyways.
- a3w 1y agoAlso, I wanted tldrbot to summarize this page. /s
- criddell 1y agoThat's a good point. It's not a black and white issue. I personally see a bot working on behalf of an end user differently than OpenAI hoovering up every bit of text they can find to build something they can sell. I'd guess the owner of localghost.dev doesn't have a problem with somebody using a screen reader because although it's a machine pulling the content, it's for a specific person and is being pulled because they requested it. If the people making LLM's were more ethical, they would respect a Creative Commons-type license that could specify these nuances.
- charles_f 1y agoI contacted the author, she said because no-one respects it, she hasn't even tried.