5 ms·
for those interested, I did look at it a bit and before I touched it was 116k~ lines. After I removed all the "bot", "spider"(but not "google" or "mozilla" or a
by Stormcaller 13y ago
for those interested, I did look at it a bit and before I touched it was 116k~ lines. After I removed all the "bot", "spider"(but not "google" or "mozilla" or anything that might be used by real useragents)(and yes I am aware chrome doesnt use google and mozilla is used 99% of those strings, I just couldnt give any better examples) It went down to 1k~.
I scrolled thru and most of those requests came from "216.151.137.36" so I removed that too. I had 500~ that wasn't "obviously" coming from bots. After this I stopped as what I wondered was "did this site get significant amount of real visits?" and the answer at that point was clearly no, or if any, less than 500.
Btw, does anyone know who this ip(216.151.137.36) belongs to? All I get is spam reports when I google it and doesnt seem to belong to any (major) spider. http://stopforumspam.com/ipcheck/216.151.137.36 http://stopforumspam.com/ipcheck/216.151.137.36
Also on another question, how does google deal with web apps that have unlimited pages(dynamic urls based on get and so)? As in, how does it say "this is legit" and this site here is not. Surely backlinks are one thing, but these can be "faked", too?
- toast0 13y ago> Btw, does anyone know who this ip(216.151.137.36) belongs to FWIW, I see this ip hitting my spider trap too. 14 requests in 30 seconds on March 14th, and 21 requests in 45 seconds on Feb 20th.
- laumars 13y ago$ whois 216.151.137.36 http://pastebin.com/XJawtmnw http://pastebin.com/XJawtmnw