• We get alot of spiders around the web logs. In particular, we saw new versions of hl_ftien_spider/1.1, Bigsearch.ca/Nutch-1.0-dev, RwB's spider for atUKplc., EntireWeb's Speedy Spider, and Opendium spider this week. related...

• If spiders weren't enough, we also had a Python-urllib/1.15 slithering around the traffic logs. Maybe pythons eat spiders? related...

• As far as mythical creatures in the traffic logs, we saw a gnome-vfs/2.16.2 neon/0.25.4 digging around and a Yeti/0.01 crawling through. The Yeti's user agent told us that it "check robots.txt daily and follows it", and is a variation of Naver. related...

• We added 2 variations of the Larbin Web Crawler to our database this week - here's one and here's the other. related...

• A couple retro user agents drove through, Microsoft Internet Explorer/3.11 (Windows 3.11) and Mozilla/3.0 (OS/2; U). We aren't sure if they were authentic or not, but it's still nice to see someone remembers the good old days. related...

• This one was unique - the facebookscraper/1.0 visited our sites, even though we are not facebook. I guess it got lost and scraped up on some other sites? related...

• Someone is actually keeping a meta tags directory and is harvesting their content using metatagsdir/0.7. Seems like an odd hobby, but then again, running a blog for web robots isn't exactly normal. related...

• As usual, we had a couple script injection hack attempts from syncrisis - <SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727)
  and <SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727).
  
  If you don't URL Encode your web traffic logs or your log viewers, these guys are a shining example of why you should. related...

Blog Archives: