I’ve added code to the referer checking to check for a few common bots:
- “http://help.yahoo.com/help/us/ysearch/slurp”
- “http://www.googlebot.com/bot.html”
- “http://www.blogdigger.com/”
- “http://search.msn.com/msnbot.htm”
- “Feedster, LLC.”
- “http://www.breakingblogs.com/timbo_bot.html”
- “fastbuzz.com”
- “http://www.pubsub.com/”
- “http://www.bloglines.com”
- “http://www.drupal.org/”
- “Ask Jeeves/Teoma”
- “ia_archiver”
- “http://minutillo.com/steve/feedonfeeds/”
There are others but these showed up in the access logs of the last few minutes!
Sorry guys, your referer lists will be a lot shorter now!
