5 ms·

Code here: https://github.com/google/robotstxt https://github.com/google/robotstxt

• glenneroo 7y ago
  It's also the 5th link "open sourced" in the article.