5 ms·

just for you, I redeployed the old robots.txt (with an additional log-honeypot). I even manually submitted it to the web archive just now so you have something

just for you, I redeployed the old robots.txt (with an additional log-honeypot). I even manually submitted it to the web archive just now so you have something to look at: https://web.archive.org/web/20241231041718/https://wiki.diasporafoundation.org/robots.txt https://web.archive.org/web/20241231041718/https://wiki.dias... they ingested it twice since I deployed it. they still crawl those URLs - and I'm sure they'll continue to do so - as others in that thread have confirmed exactly the same. I'll be traveling for the next couple of days, but I'll check the logs again when I'm back. of course, I'll still see accessed from them, as most others in this thread do, too, even if they block them via robots.txt. but of course, that won't stop you from continuing to claim that "I lied". which, fine. you do you. luckily for me, there are enough responses from other people running medium-sized web stuffs with exactly the same observations, so I don't really care.

alphan0n 2y ago
What about the CommonCrawl archives? That clearly show the same robots.txt that allows all, from September through December? You’re a phony.
- denschub 2y ago
  Here's something for the next time you want to "expose" a phony: before linking me to your investigative source, ask for exact date-stamps when I made changes to the robots.txt and what I did, as well as when I blocked IPs. I could have told you those exactly, because all those changes are tracked in a git repo. If you asked me first, I could have answered you with the precise dates, and you would have realized that your whole theory makes absolutely no sense. Of course, that entire approach is mood now, because I'm not an idiot and I know when commoncrawl crawls, so I could easily adjust my response to their crawling dates, and you would of course claim I did. So I'll just wear my "certified-phony-by-orangesite-user" badge with pride. Take care, anonymous internet user.
  - alphan0n 2y ago
    Gentleman’s bet. If you can accurately predict the day of four of the next six months of commoncrawls crawl, I’ll donate $500 to the charity of your choice. Fail to, donate $100 to the charity of my choice.
    - alphan0n 2y ago
      Or heck, $1000 to the charity of your choice if you can do 6 of 6, no expectation on your end. Just name the day from February to July, since you’re no idiot.
  - alphan0n 2y ago
    ◔_◔
  - alphan0n 2y ago
    >I'm not an idiot and I know when commoncrawl crawls When will commoncrawl crawl your site again?
    - deleted 2y ago
      [deleted]

alphan0n 2y ago

What about the CommonCrawl archives? That clearly show the same robots.txt that allows all, from September through December? You’re a phony.

denschub 2y ago
Here's something for the next time you want to "expose" a phony: before linking me to your investigative source, ask for exact date-stamps when I made changes to the robots.txt and what I did, as well as when I blocked IPs. I could have told you those exactly, because all those changes are tracked in a git repo. If you asked me first, I could have answered you with the precise dates, and you would have realized that your whole theory makes absolutely no sense. Of course, that entire approach is mood now, because I'm not an idiot and I know when commoncrawl crawls, so I could easily adjust my response to their crawling dates, and you would of course claim I did. So I'll just wear my "certified-phony-by-orangesite-user" badge with pride. Take care, anonymous internet user.
- alphan0n 2y ago
  Gentleman’s bet. If you can accurately predict the day of four of the next six months of commoncrawls crawl, I’ll donate $500 to the charity of your choice. Fail to, donate $100 to the charity of my choice.
  - alphan0n 2y ago
    Or heck, $1000 to the charity of your choice if you can do 6 of 6, no expectation on your end. Just name the day from February to July, since you’re no idiot.
- alphan0n 2y ago
  ◔_◔
- alphan0n 2y ago
  >I'm not an idiot and I know when commoncrawl crawls When will commoncrawl crawl your site again?
  - deleted 2y ago
    [deleted]