We have introduced some new categories for some of the various bots and user agents we are seeing for the first time.
and iTunes have browsers built into their systems, so their user agents will be tracked in these two new categories.
- We've seen several
RSS related user agents - mainly bots - and we'll be tracking these under this category going forward.
- Finally, we have started tracking Sitemap user agents - some of these are from us evaluating sitemap tools, and some are visitors that we did not invite.
Usually, these tools are used to build sitemaps for Google or Yahoo!, which both offer useful sitemap services to help webmasters publish all of their URLs directly to the search engines to help the web crawlers do a thorough job.
We hope that the new categories help with all of your bot research - enjoy!