5 ms·Free web-based robots.txt parser based on Google’s open source C++ parser3 points by will_critchlow 7y ago