Browsing Category "Nutch Bots" - Bots vs Browsers

Bots vs Browsers - Public Bot / User Agent Database & Commentary
Home \| Archives \| About User Agents \| User Agent Test Track \| Recent Additions \| Contributors \| Donate \| Top Searches \| User Agent IP Directory

Bots vs Browsers - database of 872,060 user agents and growing

Browsing Category "Nutch Bots"
Description: Variations of the Nutch bot User Agents
362 User Agents Found.
/Nutch-0.9 /Nutch-0.9-dev abc/Nutch-0.9-dev (abc; http://abc#11.us; abc at abc dot com) Abortion.sg/Nutch-1.1 (www.Abortion.sg; crawler@Abortion.sg) ACME Corporation/Nutch-1.0 (ACME Spider; http://www.acme.com; test123@spam.la) Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org) AdultsVisit.us/Nutch-1.0 (www.AdultsVisit.us; crawler@AdultsVisit.us) AdultsVisitUs/Nutch-1.1 (www.AdultsVisitUs.com; crawler@AdultsVisitUs.com) africa/Nutch-0.9 (africa; localhost; localhost) agentname/Nutch-1.0-dev Aghaven/Nutch-1.2 amd-source-bot/Nutch-1.0 Amit Singh/Nutch-0.9 (Amit Singh; www.cse.iitb.ac.in/~amitsingh; amitsingh@gmail.com) Ant/Ant-Nutch-1.1 (Ant Nutch Crawler; http://www.ant.com; crawler@ant.com) AntBot/Ant-Nutch-1.1 (Ant Nutch Crawler; http://www.ant.com; crawler@ant.com) AOL_Daniel_Clark_Spider/Nutch-0.9 (AOL Search; danielaclark1@aol.com) apache.org/Nutch-0.9 (apache; http://www.apache.org; users@apache.org) asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com) Attentio/Nutch-0.9-dev (Attentio's beta blog crawler; www.attentio.com; info@attentio.com) Attributor/Nutch-1.0-dev (Test crawler; http://www.attributor.com; info at attributor com) Ayna/Nutch-0.9 (Ayna Search Engine Crawler; http://www.ayna.com/; search at aynacorp dot com) baidu/Nutch-1.0 Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.co... beast/Nutch-0.9 (agentspider; beast@mail.com) becomex /Nutch-0.9 becomex /Nutch-1.0 bender/Nutch-0.8.1 (myd@cs.stanford.edu) Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com) Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com) blackcrawl/Nutch-0.9 (crawl for fun; www.zipfelchappe.com; zipfelchappe@localhost) Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.as... bob/Nutch-0.9 (bob; http://www.google.com; x@y) BobCrawl/Nutch-0.9 (Test/Development crawler; http://notavalable.com; notavailable@notavailable.com) boo/Nutch-1.0 boo/Nutch-1.0 (boo) Cabot/Nutch-1.0-dev (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; agent@amfibi.com) Cabot/Nutch-1.0-dev (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; help at amfibi dot com) Cabot/Nutch-1.2 (Amfibi's webcrawler robot; http://www.amfibi.com/cabot; cabot@amfibi.com) caizhi/infomation/Nutch-0.8.1 cancho/Nutch-1.0 (crawl test; http://asdf.net/; asdf@asdf.net) Cazoodle/Nutch-0.9-dev Cazoodle/Nutch-0.9-dev (Cazoodle Nutch Crawler; http://www.cazoodle.com; mqbot@cazoodle.com) CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com) CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.com) Chen Li/Nutch-1.0 (Nutch spiderman; http://chenli.com.cn; chenlibiti@163.com) Chickety China (the Chinese Chicken) / Nutch-0.9 cierzo-development/Nutch-1.1-dev COMODOspider/Nutch-1.0 complex_network_group/Nutch-0.9-dev (discovering the structure of the world-wide-web; http://cantor.ee.ucla.edu/~network... Covario Amazon Nutch Crawler/Nutch-1.2 Covario IDS/Nutch-1.2 crawl test/Nutch-1.0-dev DataPatrol/Nutch-1.0 (DataPatrol indexer from Garlik; http://www.garlik.com/products.php; crawler at garlik dot com) DERIbot/Nutch-1.0-dev (DERIbot; http://deri.org ; info@deri.ie) disco/Nutch-0.9 (experimental crawler ... please email imagine@gmail.com if problems observed; imagine@gmail.com) disco/Nutch-0.9 (experimental crawler ... please email imagine@gmail.com if problems observed; nedrocks@gmail.com) disco/Nutch-0.9 (experimental crawler; nedrocks@gmail.com) disco/Nutch-0.9 (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com) disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com/robot.html; disco-crawl@discoveryengine.com) disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; disco-crawl@discoveryengine.com) Domnutch-Bot/Nutch-1.0 (Domnutch; http://www.Nutch.de/) dpdev/Nutch-1.0 (datapatrol from garlik.com; http://www.garlik.com/crawler; crawler at garlik dot com) dpdev/Nutch-1.0 (datapatrol from garlik.com; http://www.garlik.com/products.php; crawler at garlik dot com) dpdev/Nutch-1.0-dev (datapatrol from garlik.com; http://www.garlik.com/crawler; crawler at garlik dot com) ealbum/Nutch-1.0 ecxi/Nutch-1.0 (esCERT-UPC-ecxi; http://escert.upc.edu/; admin escert edu) ecxi/Nutch-1.0-dev (esCERT-UPC-ecxi; http://escert.upc.edu/; admin escert edu) education portal/Nutch-0.9 (Please do not forbid, its for your benefit) Eric Osgood/Nutch-1.0 (Nutch spiderman; http://www.calpoly.edu/~eosgood ; MyEmail) Eurobot/Nutch-1.0-dev (1.0) Facet Engine Spider/Nutch-1.2 (Internet Crawler; spider@facetengine.com) fetch/Nutch-1.0 (TCGfetch; http://fetch.thecyberguardian.com; TCGEmail) foobar/Nutch-1.0-dev (foobar; foobar.com; foo@bar.com) GentilBot/Nutch-1.0 GeoHasher/Nutch-1.0 (GeoHasher Web Search Engine; geohasher.gotdns.org; geo_hasher at yahoo * com) gh-index-bot/Nutch-1.0 (GH Web Search.; lucene.apache.org; gh_email at someplace dot com) Googlebot/Nutch-1.0 googlepages/Nutch-0.9 (googlepages; http://www.googlepages.com; info@googlepages.com) healia/Nutch-0.9 (the personalized health search engine.; http://www.healia.com; mikes@healia.com) Heeii/Nutch-0.8.1 (Heeii; www.heeii.com; info@heeii.com) Heeii/Nutch-0.9 (Heeii; www.heeii.com; info@heeii.com) Horny Sex Search/Nutch-0.9 (HornySexSearch.com Crawler; http://www.hornysexsearch.com; Contact HornySexSearch.com) HouxouCrawler/Nutch-0.8.2-dev (houxou.com's nutch-based crawler which serves special interest on-line communities; http:... HouxouCrawler/Nutch-0.9 (houxou.com's nutch-based crawler which serves special interest on-line communities; http://www.... HPL/Nutch-0.9 hunter/Nutch-0.8.1 Iceweasel2.0.0.16/Nutch-1.0 (Webbrowser; http://iceweasel.com; info@iceweasel.com) Iceweasel2.0.0.16/Nutch-1.1 (Webbrowser; http://iceweasel.com; info@iceweasel.com) IIT Bombay CFILT NLP Bot/Nutch-1.1 (IITB CFILT Crawler) IITB-CFILT-Bot/Nutch-1.1 (This is the crawler of IIT Bombay, India. The data will be used for research purposes.; http:/... ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://w... ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; crawl@ilial... ilial/Nutch-0.9-dev Infoaxe./Nutch-0.9 Infoaxe./Nutch-1.0 InternetArchive/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) intumit/Nutch-1.1 IRP_edu_bot/Nutch-0.9 IS Alpha/Nutch-1.0 jupiter/Nutch-1.2 kindsight/Nutch-1.0 (kscrawler; www.projectrialto.com; crawler@projectrialto.com) KnowItAll/Nutch-0.9 (Nutch-UW-Crawler; http://cs.washington.edu/homes/mjc/crawler.html; uwcrawler08@gmail.com) Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://corp.krugle.com/crawler/info.html; webcrawler@krugle.com) Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://www.krugle.com/crawler/info.html; webcrawler@krugle.com) KS Crawler/Nutch-1.0 (http://www.kindsight.net/kscrawler; crawler@kindsight.net) KS Spider/Nutch-0.9 KSCrawler/Nutch-1.0 (http://www.kindsight.net/en/kscrawler; crawler@kindsight.net) LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/; info(a)lijit(d)com) LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/robot/crawler; info(a)lijit(d)com) linkdexbot/Nutch-1.0-dev (http://www.linkdex.com/; crawl at linkdex dot com) Lisboa/Nutch-1.2 lmspider/Nutch-0.9-dev (For research purposes.; www.nuance.com; lmspider@nuance.com) LPbot/Nutch-1.1 LTI/LemurProject Nutch Spider/Nutch-1.0-dev (lti crawler for CMU; http://www.lti.cs.cmu.edu; changkuk at cmu dot edu) LTI/LemurProject Nutch Spider/Nutch-1.0-dev (Research spider using Nutch; http://lucene.apache.org/nutch/bot.html; admin... male.com.sg/Nutch-1.1 (http://www.male.com.sg; crawler@male.com.sg) Manav/Nutch-0.9 (1.0; manavraman at yahoo dot com) Manav/Nutch-1.0-dev (1.0; manavraman at yahoo dot com) Martin/Nutch-1.0 (Nutch spiderman; MyEmail) mercury/Nutch-1.2 mmcrawler/Nutch-1.0 (MM Robots; http://; lindaoi1@hotmail.com) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.8.1 (http://lucene.apache.org/nutch/about.html... Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.9 (http://lucene.apache.org/nutch/about.html; ... Mozilla/4.0/Nutch-0.9 Mozilla/4.0/Nutch-1.0-dev (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) Mozilla/5.0 (compatible; ADIR Research Project; http://bradipo.net/mark) /Nutch-0.9 Mozilla/5.0 (compatible; Advisorbot/1.0)/Nutch-1.2 Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobi... Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:2.0b2) Gecko/20100720 AdultsVisitUs/Nutch-1.1 (www.AdultsVisitUs.com; cr... Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.1.9) Gecko/20100315 Firefox/3.5.9/Nutch-1.0 Mozilla/5.0 (Windows; N; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)/Nutch-1.0 (Crawler; lucene.apache.org/nutch/... Mozilla/5.0 (Windows; N; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)/Nutch-1.1 (Crawler; lucene.apache.org/nutch/... Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4/Nutch-0.9 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.9/Nutch-0.9 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10/... Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.12) Gecko/20080207 Ubuntu/7.10 (gutsy) Firefox/2.0.0.12/Nutch-0.9 Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.19) Gecko/20081202 Firefox (Debian-2.0.0.19-0etch1)/Nutch-1.0 Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.9.2.8) Gecko/20101230 Firefox/3.6.8/Nutch-1.0 Mozilla/Nutch-1.0 Mozilla/Nutch-1.1 MQBot/Nutch-0.8-dev (mqbot@cazoodle.com) MQBOT/Nutch-0.9-dev (MQBOT Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu) MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu) MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; mqbot@cs.uiuc.edu) MULTIPRISE/Nutch-1.0 (Robot d'indexation; http://www.multiprise.biz; admin@multiprise.fr) My Spider/Nutch-1.0 (My Bot; http://www.intersect.org.au; sridhar.reddapani@intersect.org.au) mybot/Nutch-1.0 (mybot; http://mybot.com; mybot@mybot.com) myfirsttest/Nutch-0.8.1 (myfirsttest; http://www.science.uva.nl/; xzhang1@science.uva.nl) mynutchcrawler/0.8.1 (nutch 0.8.1; http://localhost:8080; info at mysite dot com) Mysearch/Nutch-0.9 Neo Lee/Nutch-0.9 (Nutch spiderman; http://lucene.apache.org/nutch/; MyEmail) Netluchs/Nutch-1.0 ( ; http://www.netluchs.de/; _do_not_spam_me___humans_please_use_info_at_netluchs.de_without_the_dash... Netluchs/Nutch-1.0-dev ( ; http://www.netluchs.de/; _do_not_spam_me___humans_please_use_info_at_netluchs.de_without_the_... NetSeer/Nutch-0.9 (NetSeer Crawler; http://www.netseer.com; crawler@netseer.com) noopsis Spider/Nutch-1.1 (noopsis crawler) nsyght.com/Nutch-0.9 (nsyght.com; Nsyght.com) nsyght.com/Nutch-0.9 (nsyght.com; search.nsyght.com) nsyght.com/Nutch-1.0-dev (nsyght.com; Nsyght.com) Nutch 1.2/Nutch-1.2 (Facet Engine Nutch Crawler; spider@facetengine.com) Nutch agent name/Nutch-1.0 (Nutch agent description; http:// MyAgent.googlepages.com ; MyEmail) nutch test/Nutch-1.0 (nutch test) nutch-crawl/Nutch-1.0-dev (imcs; http://imcs.ro; admin@imcs.ro) nutch-crawler/Nutch-1.2 nutch-solr-integration-test/Nutch-1.2 (MoonValley Web Crawler using Nutch 1.2; http://www.moonvalley.com/; cwoolum@moonv... nutch-solr-integration/1.1 nutch-solr-integration/Nutch-1.0 nutch-solr/Nutch-1.0-dev nutch.biz/Nutch-1.0 (nutch.biz; crawler@nutch.biz) nutch.us/Nutch-1.0 (nutch.us; crawler@nutch.us) nutch.us/Nutch-1.0 (www.nutch.us; crawler@nutch.us) nutch.us/Nutch-1.0-dev (www.nutch.us; crawler@nutch.us) Nutch/1.2/Nutch-1.2 Nutch/Nutch-0.8.1 Nutch/Nutch-0.8.1 (Nutch; Nutch; Nutch) Nutch/Nutch-0.9 Nutch/Nutch-0.9 (Eurobot; http://www.ayell.eu ) nutch/Nutch-0.9 (nutch) Nutch/Nutch-0.9 (Nutch; http://lucene.apache.org/nutch/) Nutch/Nutch-0.9 (Nutch; http://nutch; nutch) Nutch/Nutch-1.0 (academic purpose; cats.kaist.ac.kr; smiler82@naver.com) nutch/Nutch-1.0 (nutch) Nutch/Nutch-1.0-dev (A Nutch-based crawler.; http://lucene.apache.org/nutch/bot.html; nutch-agent AT lucene.apache.org) nutch/Nutch-1.0-dev (nutch) nutch_princeton/Nutch-1.0-dev (princeton crawler for cass project; http://www.cs.princeton.edu/cass/; zhewang a_t cs ddo... nutch_test/Nutch-0.9 (nutch_test; http://www.8dorm.com; webmaster@8dorm.com) Nutch0.8/Nutch-0.9-dev nutch0.9/Nutch-0.9-dev NUTCHCRAWLER/Nutch-0.9 (anouar@yatinoo.com) NutchCVS/0.06-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu) NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) NutchCVS/0.7.1 (Nutch running at UW; http://crawlers.cs.washington.edu/; sycrawl@cs.washington.edu) NutchCVS/0.7.1 (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu) NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; raphael@unterreuth.de) NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; west@cis.poly.edu) NutchCVS/0.8-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu) NutchCVS/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) NutchCVS/0.8.1 (http://cis.poly.edu/westlab/; west@poly.edu) nutchCVS/Nutch-0.8.1 (nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) NutchCVS/Nutch-1.0 (http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2.; http://lucene.apache.org/nutch/bot.html; ec2test at lucene.com... nutchsearch/Nutch-0.9 (Nutch Search 1.0; herceg_novi at yahoo dot com) NutchTest/Nutch-1.0 nutchwax.com/Nutch-1.0 (nutchwax.com; crawler@nutchwax.com) NWBSpider/Nutch-2.0-dev OpenPlaces/Nutch-1.0-dev (OpenPlaces Content Crawler; http://www.openplaces.com; dnadeau 64th-ascii-char openplaces c o ... Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-0.9 (crawl for fun; www.zipfelchappe.com; zipfelchappe@localhost)/19.916; U; en)... Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-0.9 (crawl for fun; www.zipfelchappe.com; zipfelchappe@localhost)/20.2463; U; en... Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0 (Test crawl; lucene.apache.org/20.2477; U; en) Presto/2.5.25 Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0- dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots me... Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http:/18.684; U; en) Presto/2.4.15 PenTest.sg/Nutch-1.1 (www.PenTest.sg; crawler@PenTest.sg) Peter Wang/Nutch-0.9 (Nutch spiderman; http://peterpuwang.googlepages.com ; MyEmail) Pluggd/Nutch-0.9 (Pluggd automated crawler; http://www.pluggd.com; support at pluggd dot com) PR Crawler/Nutch-1.0 (data mining develpment project; crawler@projectrialto.com) PRCrawler/Nutch-0.9 (data mining development project) PRCrawler/Nutch-0.9 (data mining development project; crawler@projectrialto.com) QEAVis agent/Nutch-0.9 (http://nlp.uned.es/qeavis/) Raymond Balm�s/Nutch-1.0 (spiderman; http://www.balmes.com ; raymond.balmes@gmail.com) rdfbot/Nutch-1.0-dev REAP-crawler Nutch/Nutch-1.0-dev (Reap Project; http://reap.cs.cmu.edu/REAP-crawler/; Reap Project) REAP-crawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu) research-scan-bot/Nutch-1.0 RoadRunner/Nutch-1.0 (webmaster@fieldtech.org) roboo/Nutch-1.0 (roboo; http://wap.roboo.com; winter.pi@roboo.com) roboobot/Nutch-1.0 (roboobot; http://wap.roboo.com; winter.pi@roboo.com) Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net) robotgenius/Nutch-1.0-dev sait/Nutch-0.9 (SAIT Research; http://www.samsung.com) SapphireWebCrawler/1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.edu) SapphireWebCrawler/Nutch-1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; lezhao+crawl@cs.c... SapphireWebCrawler/Nutch-1.0 (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; philgooh@cs.cmu.e... SapphireWebCrawler/Nutch-1.0-dev (Sapphire Web Crawler using Nutch; http://boston.lti.cs.cmu.edu/crawler/; mhoy@cs.cmu.e... SBIder/Nutch-1.0-dev (http://www.sitesell.com/sbider.html) Schloerbot/Nutch-1.0 (Schloer consulting bot; http://schloerconsulting.com/schloerbot) searchdnabot/Nutch-1.0 (SearchDNA bot; http://searchenginedna.com; crawl at searchenginedna dot com) SearchEngineVerificationCrawler/Nutch-1.0 (The purpose of this crawling is to collect web pages for verifying search eng... SeekGen/Nutch-0.9 (SeekGenBot; http://www.seekgen.com; Email) Server/Nutch-1.2 SexShop.Sg/Nutch-1.1 (www.SexShop.Sg; crawler@SexShop.Sg) SexShop.sg/Nutch-1.1 (www.SexShop.sg; crawler@www.sexshop.sg) SHC/Nutch-1.0 (SemanticHacker Crawler; http://www.semantichacker.com/crawler-info; abuse@semantichacker.com) Sigram/Nutch-1.0-dev (Test agent for Nutch development; http://www.sigram.com/bot.html; bot at sigram dot com) SimilarPages/Nutch-1.0-dev (SimilarPages Nutch Crawler; http://www.similarpages.com; info at similarpages dot com) SimilarPages/Nutch-1.0-dev (SimilarPages Nutch Crawler; http://www.similarpages.com; info@similarpages.com) SindiceBot/Nutch-1.0-dev (http://sindice.com/dev?section=bot) SindiceBot/Nutch-1.0-dev (http://sindice.com/developers/bot) SonyEricssonK750i/R1N Browser/SEMC-Browser/4.2 Profile/MIDP-2.0 Configuration/CLDC-1.1/Nutch-1.0-dev sphsearch.org/Nutch-1.0 (sphsearch.org; crawler@sphsearch.org) sphsearch.org/Nutch-1.0-dev (www.sphsearch.org; crawler@sphsearch.org) SpiderMan/Nutch-0.9 (Nutch spiderman; http://spiderman.nutch.com ; MyEmail) ssearch_bot/Nutch-1.0 (sSearch Crawler; http://www.semantissimo.de) TANNER Spider/Nutch-1.1 temaseek.com/Nutch-1.0 (temaseek.com; crawler@temaseek.com) Teoma/Nutch-1.0 (Mozilla/5.0 (compatible; Ask Jeeves/Teoma); http://about.ask.com/en/docs/about/webmasters.shtml) Teoma/Nutch-1.2 ( Question and Answer Search; Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/t... Test crawler Nutch/Nutch-1.0-dev (Nutch Test Project; changkuk@cmu.edu) Test-Fetcher-0.1/Nutch-0.9 (Awesomeness) Test.Buzzz/Nutch-0.8.1 (Test.Buzz; http://test.com; test@test.com) test/Nutch-0.8.1 (Test robot; http://test.com; info at test.com>) test/unique/Nutch-0.8.1 TestBot/Nutch-1.1 TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://balihoo.com/index.aspx; tgautier at balihoo dot com) TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://chitchit.org/TestCrawler.html; amitjain at spro dot net) TestSpider/Nutch-1.0-dev Tipiweb/Nutch-1.0 (http://www.tipiweb.net) toofaan/Nutch-1.0 (http://www.toofaan.com) Trailfire/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) Tycoon Agent/Nutch-1.0-dev university/Nutch-1.0 (research) usyd.schwa.lab.nlp.research/Nutch-1.1 (http://it.usyd.edu.au/~smerity/schwa/crawler/; smerity %AT% it.usyd.edu.au) uw_cse_xwci uw.crawler@gmail.com http://myst.cs.washington.edu/index.html/Nutch-0.9 (University of Washington Computer S... vik-robot/Nutch-1.0 (vikspider; http://vik.com; chenlibiti@163.com) volverine/Nutch-0.9 (agentspider; beast@mail.com) VWBOT/Nutch-0.9-dev (VWBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; vwbot@cs.uiuc.edu) web2express.org/Nutch-0.9-dev (leveled playing field; http://web2express.org/; info at web2express.org) Webcrawler/Nutch-1.0 (Test crawl; lucene.apache.org/nutch/; a@b.net) Webcrawler/Nutch-1.0-dev (Test crawl; lucene.apache.org/nutch/; a@b.net) webmoney.advisor.bot/Nutch-1.0 webmoney.advisor/Nutch-1.0 webmoney.advisor/Nutch-1.2 WebMoney/1.0 (AdvisorBot 0.1)/Nutch-1.2 Webscope/Nutch-0.9-dev (http://www.cs.washington.edu/homes/mjc/agent.html) weivel/Nutch-0.9 (weive.com - web spider; http://www.weive.com/page/view/weivel; weivel at weive dot com) whiteiexpres/Nutch-0.9 (whiteiWebBot; whiteiexpress.com; whiteye@kon-x.com) Woovi/Nutch-1.0 (http://www.woovi.com/bot) workload-generator/Nutch-0.9 (web20-setup; https://twiki.hpl.hp.com/bin/view/Main/WebsearchNotes) yggdrasil/Nutch-0.9 (yggdrasil biorelated search engine; www dot biotec dot tu minus dresden do de slash schroeder; heik... ZipppBot/Nutch-0.9 (ZipppBot .02; http://www.zippp.net; crawlteam@zippp.net) ZipppBot/Nutch-1.0-dev (ZipppBot .02; http://www.zippp.net; crawlteam@zippp.net) Zscho.de Crawler/Nutch-1.0-Zscho.de-semantic_patch (Zscho.de Crawler, collecting for machine learning; http://zscho.de/ ... Zscho.de Crawler/Nutch-1.0-Zscho.de-semantic_patch (Zscho.de Crawler, collecting for machine learning; http://zscho.de/) zschobot/Nutch-0.9-semantic_patch (zschobot indexing; Zscho.de/de/bot.html) Aghaven/Nutch-1.2 (www.aghaven.com) AlexionResearchBot/Nutch-1.3 AskAboutOil/0.06-rcp (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@askaboutoil.com) Bigsearch.ca/Nutch-x.x-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; info@enhancededge.com) BilgiBetaBot/0.8-dev (bilgi.com (Beta) ; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org) blender.cs.qc.cuny.edu Spider (research purposes only)/Nutch-1.2 (This spider intends to collect individual webpages (no... boosker/Nutch-1.2 CjxSearch/Nutch-1.4 COMODOSpider/Nutch-1.2 CreativeCommons/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net) Filangy/1.0x (Filangy; http://www.nutch.org/docs/en/bot.html; filangy-agent@filangy.com) FreeNutch/Nutch-1.2 http://www.163.com/Nutch-1.0 (http://www.163.com) intumit/Nutch-1.2 intumit/Nutch-1.2 (intumit) intumit/Nutch-1.3 intumit/Nutch-1.4 IS Alpha/Nutch-1.1 KeywordSearchTool.co/Nutch-1.4 (http://KeywordSearchTool.co/robot) LiveYellow/1.0/Nutch-1.2 ly/1.0/Nutch-1.2 MaxPointCrawler/Nutch-1.1 MaxPointCrawler/Nutch-1.1 (maxpoint.crawler at maxpointinteractive dot com) MaxPointCrawler/Nutch-1.1 (MaxPoint.Crawler@maxpointinteractive.com) Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)/Nutch-1.0 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)/Nutch-1.0-dev Mozilla/5.0 (compatible; OpenindexDeepSpider/Nutch-1.5-dev; +http://openindex.io/spider.html; systemsATopenindexDOTio) Mozilla/5.0 (compatible; OpenindexShallowSpider/Nutch-1.5-dev; +http://openindex.io/spider.html; systemsATopenindexDOTio... Mozilla/5.0 (compatible; OpenindexSpider/Nutch-1.5-dev; +http://openindex.io/spider.html; systemsATopenindexDOTio) Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobi... Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 /Nutch-1.2 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1/Nutch-1.2 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0/Nutch-0.9 Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.19) Gecko/20081202 Firefox (Debian-2.0.0.19-0etch1)/Nutch-1.0 UNTRUSTED... Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.12) Gecko/2009070811 Ubuntu/9.04 (jaunty) Firefox/3.0.12/Nutch-1.0-dev ... Mozilla/5.0/Nutch-1.1 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Mozilla/Nutch-1.2 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.11) My Nutch Spider/Nutch-1.3 NSE/Nutch-1.2 nutch Nutch test crawler/Nutch-1.4-dev (Crawler testing; crawler/a/t/luminouslabs.com) nutch-1.3/Nutch-1.3 nutch-1.4/Nutch-1.4 nutch-solr-integration/Nutch-1.3 nutch-solr-integration/Nutch-1.4 nutch/1.2 (nutch) Nutch_Crawler/Nutch-1.3 Nutraspace/Nutch-1.2 (www.nutraspace.com) Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-0.9/838; U; en) Presto/2.4.15 Opera/9.80 (J2ME/MIDP; Opera Mini/Nutch-1.0-dev/838; U; en) Presto/2.4.15 OrangeCrawler/Nutch-1.0 (ldorange.crawler@orange-ftgroup.com) pic2u/Nutch-1.2 (http://www.pic2u.com) ScholarScope/Nutch-1.2 (Scholar Research Engine) Seengine/1.0/Nutch-2.0-dev Setooz/Nutch-1.0 (http://www.setooz.com) sgcrawler in-the-right-place/Nutch-1.3 sGroup crawler 1/Nutch-1.3 test/Nutch-1.2 TestNutch/Nutch-1.2 (testing Nutch; http://nutch.apache.org; info@nutch.apache.org) The Lemur Web Crawler/Nutch-1.3 (Lemur Web Crawler using Nutch 1.3; http://boston.lti.cs.cmu.edu/crawler_12/; admin@lemu... The Lemur Web Crawler/Nutch-1.3 (Lemur Web Crawler; http://boston.lti.cs.cmu.edu/crawler_12/; admin@lemurproject.org) WebCrawler/Nutch-1.2 (WebCrawler; WebCrawler) WK/Nutch-1.1 (Nutch spiderman; n/a ; MyEmail) WocBot/Nutch-1.4 (Wocodi Web Crawler 1.0; http://www.wocodi.com/; crawler@wocodi.com) nutch/Nutch-1.0-dev (nutch dev) tudelft-research/Nutch-1.0 viezae 0.1 beta/Nutch-0.9 (pre-beta search gathering; 127.0.0.1; viezae@sogetthis.com)