Bots vs Browsers - Public Bot / User Agent Database & Commentary

Bots vs Browsers - database of 1,537,533 user agents and growing


DEC 12

2006

50,000 User Agents!



This month we spent putting a new backend on the site, as well as migrating to a new server.
In the meantime, we were visited by 39 new bots, so here's some quick catchup.
Oh, by the way, we also passed the 50,000 user agent milestone this month - hurray!
  • GurujiBot/1.0 stopped by, and in following their URL in the user agent, I found out that "GurujiBot is the user-agent for Guruji's web crawler. Guruji is crawling the web to build a Next Generation Search Engine.". According to their site, they obey the robots.txt exclusion standard. related...
  • Blog Conversation Project has been reading up on our sites working on a project for Harvard. According to their site, "The Blog Conversation Project is a research study seeking to develop methods of automatically reading and coding a large sample of blogs so that we might be able to measure the views expressed in this national or international conversation." The URL they provide is an interesting read and has more information about their project, so you should follow it and read more about them. related...
  • lanshanbot/1.0 passed by and included the MSN bot URL for their own. Upon following their URL, there is no mention of lanshanbot, so we are not sure if they are posing or legit. related...
  • Sunrise/0.42i hit our logs - this agent is used by an open source project called "Sunrise" that "converts websites and newsfeeds for offline reading on your handheld". related...
  • Straight from Hungaria, the LapozzBot/1.4 has been poking around our sites this month. They also have a website, but none of us read hungarian, so we'll leave it up to you to read up on their intentions. related...
This month leaves us at 50,350 user agents and 641 bots

Blog Archives:


FEB 29

2016

Leap Day Launch!











APR 18

2009

Where did Palm go?







JUL 13

2008

2,000 bots!





JUN 19

2008

And we're back...


JAN 21

2008

Happy 2nd Birthday!


OCT 25

2007

The Creature Post



JUL 22

2007

Hot Summer Bots




JUN 13

2007

Twiceler Strikes!


JUN 3

2007

Land of 1,000 bots




MAR 24

2007

March Bot Madness


MAR 10

2007

Got Spiders?




FEB 3

2007

February Stars


JAN 12

2007

Happy 1st Birthday!




DEC 12

2006

50,000 User Agents!



OCT 17

2006

2 weeks, 30 bots


OCT 2

2006

4 weeks, 23 bots



AUG 26

2006

500 bots!


AUG 20

2006

5 days, 8 bots











MAY 2

2006

A few new bots









JAN 12

2006

Day One


USER AGENT CATEGORIES

BOTS
BROWSERS
MOBILE BROWSERS
OLD SCHOOL
OTHER
SPAM / HACKS
  • Script Injections (298)
  • SQL Injection (7,697)
  • PHP Injection (1,435)
  • XSS Attacks (58)
  • Link Injection (250,841)