VoilaBot
Info: http://www.voila.com/
searchdnabot
Info: http://searchenginedna.com/
kalooga/KaloogaBot
Info: http://www.kalooga.com
iajaBot
Girafabot
Info: http://www.girafa.com/
Gigabot
Info: http://www.gigablast.com/spider.html
ExaBot
Info: http://beta.exalead.com/search/C=0/2p=Help.7
Exabot
Info: http://www.exabot.com/go/robot
EnaBot
Info: http://www.enaball.com/crawler.html
DNAbot/1.0
facebookexternalhit
Info: http://www.facebook.com/externalhit_uatext.php
ichiro/mobile
Info: http://help.goo.ne.jp/door/crawler.html
008/0.83
Info: http://www.80legs.com/spider.html
urlfan-bot
Info: http://www.urlfanx.com/site/bot/350.html
abot
Info: http://www.abot.com/
ABACHOBot
Info: http://www.abacho.com/
ABCdatos
Info: http://www.robotstxt.org/wc/active/html/abcdatos.html
AboutUsBot
Info: http://www.aboutus.org/AboutUs:Bot
Accelatech RSSCrawler
Accoona-AI-Agent
Info: http://accoona.com/about/about_accoona.jsp
aconon Index
Info: raubfische.de
Ahoy!
Info: http://www.robotstxt.org/wc/active/html/ahoythehomepagefinder.html
AideRSS
Info: http://www.postrank.com
ia_archiver
Info: http://www.alexa.com/site/help/webmasters
bitlybot
AlkalineBOT
Info: http://www.robotstxt.org/wc/active/html/Alkaline.html
scooter
JS-Kit URL Resolver
Info: http://aws.amazon.com/ec2/
crawler/3.0.0
Info: http://www.notconfigured.com/
EMC Spider
Anthill
Info: http://www.robotstxt.org/wc/active/html/anthill.html
Synapse
Info: http://synapse.apache.org/
Robot/v1.34
Info: http://smart.apnoti.com/index/aboutApnotiWebCrawler
Aport
Info: http://www.aport.ru/
AppleSyndication
Arachnophilia
Info: http://www.robotstxt.org/wc/active/html/arachnophilia.html
Araneo
Info: http://www.robotstxt.org/wc/active/html/araneo.html
ArchitextSpider
Info: http://www.atext.com
arks/1.0
Info: http://www.robotstxt.org/wc/active/html/arks.html
T312461
Info: http://www.artabus.com
Ask Jeeves
ASpider
Info: http://www.robotstxt.org/wc/active/html/aspider.html
ATN_Worldwide
Info: http://www.robotstxt.org/wc/active/html/atn.txt.html
Atomz
Info: atomz.com
atraxbot
Info: http://www.atraxsolutions.com/atraxbot
Attentio
Info: www.attentio.com
attributor/1.13.2
Info: http://www.attributor.com
AURESYS
Info: http://www.robotstxt.org/wc/active/html/auresys.html
bbot
Info: http://www.robotstxt.org/wc/active/html/bbot.html
BabalooSpider
Info: http://www.babaloo.si
BackRub
Info: http://www.robotstxt.org/wc/active/html/backrub.html
BaiduMobaider
Info: http://www.baidu.jp/spider/
BaiduImagespider
Info: http://www.baidu.jp/spider/
Baiduspider
Info: http://www.baidu.com/search/spider_jp.html
BecomeBot
Info: http://www.become.com/site_owners.html
Begun
Info: http://www.begun.ru
BejiBot
Big Brother
Info: http://www.robotstxt.org/wc/active/html/bigbrother.html
BigBoardsUpdater
BigmirSpider
Info: http://www.bigmir.net/
bingbot
Info: http://www.bing.com/bingbot.htm
Bitacle bot
Info: http://bitacle.org/
Biz360 Spider
Info: http://www.biz360.com
Bjaaland
Info: http://www.robotstxt.org/wc/active/html/bjaaland.html
BlackWidow
Info: http://www.robotstxt.org/wc/active/html/blackwidow.html
BlogCrawler by Xango
blogdb
Info: http://blogdb.jp
blog search engine by BlogFan.ORG
Info: http://www.blogfan.org
Bloglines
Info: http://www.bloglines.com
BlogPulse (ISSpider-3.0)
BlogSearch
Info: http://www.icerocket.com
BlogsNowBot
Info: http://www.blogsnow.com/
BlogStreetBot
Info: http://www.blogstreet.com/
BoardTracker
Info: http://www.boardtracker.com/spider.html
BoardPulse
Info: http://www.boardpulse.com/
BoardReader
Info: http://www.boardreader.com/aboutus.asp
BoardViewer
Info: http://www.boardviewer.com/
boitho.com-robot
Info: http://www.boitho.com/bot.html
boitho.com-dc
Info: http://www.boitho.com/dcbot.html
borg-bot
Info: http://www.robotstxt.org/wc/active/html/borg-bot.html
BotOnParade
Info: http://www.bots-on-para.de/bot.html
BOTW Spider
Info: http://botw.org
BDFetch
Info: http://www.brandprotect.com
BSpider
Info: http://www.robotstxt.org/wc/active/html/bspider.html
Bulkfeeds
Info: http://bulkfeeds.net
Butterfly
Info: http://labs.topsy.com/butterfly.html
CACTVS Chemistry Spider
Calif
CaRP/3.6Evolution
Info: http://www.biz360.com
CatchBot
Info: http://www.catchbot.com/
mxbot
Info: http://www.chainn.com/mxbot.html
Charlotte
Info: http://www.searchme.com/support/
Checkbot
ChitikaBot
Info: http://chitika.com/blog/bot/
ChristCrawler.com
Info: ChristCrawler.com
www.cienciaficcion.net
Info: www.cienciaficcion.net
CipinetBot
Info: http://www.cipinet.com/bot.html
CJNetworkQuality
Info: http://www.cj.com/networkquality/
YesupBot
Info: http://www.yesup.net/bot.html
CligooRobot
Info: http://www.cligoo.de/wk/technik.php
CLX.ru Bot
Info: http://www.clx.ru
CMC/0.01
ColdFusion
combine
Crawler (cometsearch@cometsystems.com)
Info: http://www.cometsystems.com
CCBot
Info: http://www.commoncrawl.org/
ComputingSite Robi
conceptbot
Cooby.de Crawler
Info: Cooby.de
CoolBot
Covario
Info: http://www.covario.com/ids
Twiceler-0.9
Info: http://www.cuil.com
twiceler.
Info: http://www.cuil.com/info/webmaster_info/
Cusco
CyberSpyder
daumoa
Info: http://ws.daum.net/aboutWebSearch_en.html
daypopbot
DeadLinkCheck
Info: http://dlc.sourceforge.net/
Deepnet Explorer
DesertRealm.com
Info: DesertRealm.com
Deweb
Die Blinde Kuh
Info: http://www.robotstxt.org/wc/active/html/blindekuh.html
dienstspider
Diffbot
Info: http://www.diffbot.com
Digger/1.0 JDK/1.3.0
Digimarc WebReader
Digimarc CGIReader
DIIbot
grabber
discobot
Info: http://www.discoveryengine.com/discobot.html
Dolphin
Info: http://telehouse.ru/crawler.html
DotBot
Info: http://www.dotnetdotcom.org/
DotBot/1.1
Info: http://www.dotnetdotcom.org
DotTK SiteCheck
Info: http://www.dot.tk
DragonBot/1.0 libwww/5.0
DWCP/2.0
e-SocietyRobot
Info: http://www.yama.info.waseda.ac.jp/~yamana/es/index_eng.htm
EbiNess/0.01a
dragonfly(ebingbong@playstarmusic.com)
Info: http://www.ebingbong.com/help/ourRobot.php
edgeio-retriever
Info: http://www.edgeio.com
EIT-Link-Verifier-Robot/0.2
elfinbot
abby
Info: http://www.ellerdale.com/crawler.html
Emacs-w3/v[0-9\.]+
Embedly
Info: http://support.embed.ly/
envolk
Info: http://www.envolk.com/envolkspiderinfo.html
ESISmartSpider
esther
eSyndiCat Bot
EuripBot
Info: http://www.eurip.com
Eurobot/1.1
Info: http://eurobot.ayell.eu/
Evliya Celebi
exactseek-pagereaper
Info: www.exactseek.com
ExactSeek_Spider
Info: http://www.askjeevs.com
NG/2.0
Info: http://botspotter.net/bs-389.html
Ezooms
factbot
Info: http://www.factbites.com/webmasters.php
fast-webcrawler
Info: http://help.yahoo.com/help/us/ysearch/slurp/index.html
FastCrawler
Feed24.com
Info: http://www.feed24.com
FeedBlitz
Feed Blitz
Info: http://www.feedblitz.com
UniversalFeedParser
Feed Parser
Info: http://www.feedparser.org
FeedBurner
Info: http://www.FeedBurner.com
FeedHub MetaDataFetcher
Info: http://www.feedhub.com
Feedster Crawler
Info: http://www.feedstermedia.com/
Feedtrace-bot
Info: http://feedtrace.com/
FeedValidator/
FEHLSTART Superspider
FelixIDE
ESIRover
fido
findlinks
Info: http://wortschatz.uni-leipzig.de/findlinks/
FindoryBot
Info: http://www.findory.com
Firebat
Info: http://lms.virtual-presence.org
Fish-Search-Robot
Mozilla/4.0 (compatible: FDSE robot)
FollowSite Bot
Info: http://www.followsite.com/bot.html
libwww-perl/5.810
Info: http://forexitalia.org/
fouineur.9bit.qc.ca
Freecrawl
FreeWebMonitoring
Info: http://www.freewebmonitoring.com
FunnelWeb
GaisBot
Info: http://gais.cs.ccu.edu.tw/robot.php
gamekitbot
Info: http://www.uchoose.de/crawler/gamekitbot/
gamma
gazz
gcreep
genieBot
Info: http://64.5.245.11/faq/faq.html
GeoHasher
Info: geohasher.gotdns.org
geourl
Info: http://geourl.org/bot.html
GetterroboPlus
GetURL.rexx
GingerCrawler
Info: http://www.gingersoftware.com/crawler_agent.htm
UnwindFetchor
Info: http://www.gnip.com
Goku
Info: http://goku.ru/bot.htm
Golem
gonzo
gooblogsearch
Info: http://help.goo.ne.jp/help/article/704/
Googlebot-Image
Info: http://www.google.com/webmasters/bot.html
Googlebot
Info: http://www.google.com/bot.html
Mediapartners-Google
Info: https://www.google.com/adsense/faq
Adsbot-Google
Info: http://www.google.com/bot.html
FeedFetcher-Google
Info: http://www.google.com/feedfetcher.html
kw-lp-suggest
Googlebot-Mobile
Info: http://www.google.com/support/webmasters/bin/answer.py?answer=34647&topic=9346
PageFetcher-Google-CoOp
Google-Sitemaps/1.0
Googlebot-Video
Info: http://www.google.com/bot.html
Gpostbot
Info: http://www.gpost.info/help.php?c=bot
griffon
Gromit
http://grub.org
Info: http://grub.org
Gulper Web Bot
GurujiBot
Info: http://www.guruji.com/en/WebmasterFAQ.html
H?m?h?kki
Info: http://www.robotstxt.org/wc/active/html/finnish.html
havIndex
HeinrichderMiragoRobot
HenryTheMiragoRobot
archive.org_bot
Info: http://crawler.archive.org/
HKU WWW Robot
HolyCowDude
Info: http://www.holycowdude.com/spider.htm
Hometown
HostTracker.com/1.0
Info: http://host-tracker.com
htdig
HTMLgobble
AITCSRobot
I Robot
http://www.almaden.ibm.com/cs/crawler
IBM Almaden
Info: http://www.almaden.ibm.com/cs/crawler
<email>crawler@us.ibm.com</email>
IBM_Planetwide
IBM Planetwide
+http://www.icerocket.com/
IceRocket
Info: http://www.icerocket.com/
ichiro
Ichiro
Info: http://help.goo.ne.jp/door/crawler.html
<email><![CDATA[ichiro [at] nttr [dot] co [dot] jp]]></email>
igde
igdeSpyder
Info: http://igde.ru
IlTrovatore-Setaccio
IlTrovatore Setaccio
Info: http://www.iltrovatore.it/aiuto/faq.html
<email>info@iltrovatore.it</email>
Mozilla 3.01 PBWF (Win95)
Imagelock
IncyWincy
IncyWincy
Indy Library
Indy Library
Informant
Informant
InfoSeek Robot
InfoSeek Robot 1.0
Infoseek Sidewinder
Infoseek Sidewinder
infoSpider
infoSpider
Info: http://www.infospider.com
<email>support@infospider.com</email>
INGRID
Ingrid
slurp@inktomi
Inktomi
Insitor
Insitor
Info: http://www.insitor.de/
inspectorwww
Inspector Web
IAGENT
IntelliAgent
Intelliseek
Intelliseek
Info: http://www.intelliseek.com/
Internet Cruiser Robot
Internet Cruiser
SCHOOLCARE; SV1; InfoPath.1
Internet for learning
Info: http://www.ifl.com/
InternetLinkAgent
Internet Link Agent
3GSE bot
Internet Research Institute UK
Info: http://iri-uk.com/
<email>coordinator@iri-uk.com</email>
internetseer
Internet Seer
sharp-info-agent
Internet Shinchakubin
Pogodak
Interseek
Info: http://www.interseek.com/
Iron33
Iron33
IsraeliSearch
Israeli Search
itchBot
Itch
Info: http://www.itch.com/infoforwebmasters.html
JavaBee
JavaBee
JBot
JBot
JCrawler
JCrawler
JetBot
JetEye
Info: http://www.jeteye.com/jetbot.html
JoBo
JoBo
Jobot
Jobot
JoeBot
JoeBot
JSpider
JSpider
Info: http://j-spider.sourceforge.net/
jumpstation
JumpStation
Jyxobot
Jyxo
Info: http://jyxo.cz/
image.kapsi.net
Kapsi Images
Info: image.kapsi.net
Katipo
Katipo
KDD-Explorer
KDD Explorer
KIT-Fireball
KIT Fireball
kmbot
knowmore
Info: http://knowmore.com/
<email>support@knowmore.com</email>
KO_Yappo_Robot
KO Yappo
LabelGrab
LabelGrabber
larbin
Larbin
LeapTag
LeapTag News Reader
Info: http://www.leaptag.com/privatebeta/faq.php
<email>support@leaptag.com</email>
LexxeBot
Lexxe
Info: http://lexxe.com/about/webmasters.cfm
<email>lexxebot@lexxe.com</email>
lwp-trivial
libwww-perl
Info: http://search.cpan.org/~gaas/libwww-perl-5.836/lib/LWP/Simple.pm
linkalarm
Link Alarm
Linkidator
Link Validator
LinkedInBot
LinkedIn
Info: http://www.linkedin.com
LinkScan Server
LinkScan
LinkWalker
LinkWalker
livedoorCheckers/
Livedoor Checkers
Lockon
Lockon
logo.gif crawler
logo.gif
Lycos
Lycos
Apple-PubSub/59
Mac OS X RSS
Magpie
Magpie
Mail.Ru
Mail.Ru
Info: http://www.mail.ru
<email>portal@corp.mail.ru</email>
MJ12bot
Majestics MJ12bot
Info: http://majestic12.co.uk/bot.php
Mammoth
Mammoth
Info: http://www.sli-systems.com
Marvin
Marvin
marvin/infoseek
marvin/infoseek
M/3.8
Mattie
MediaFox
MediaFox
mercator
Mercator
Info: http://research.compaq.com/SRC/mercator/
MerzScope
MerzScope
METASpider
Meta
Info: http://www.meta.com.ua/
Metaeuro
MetaEuro
Info: http://www.metaeuro.com/
<email>crawler@metaeuro.com</email>
MetaGer-LinkChecker
MetaGer
MetaURI
MetaURI
Info: metauri.com
MSR-ISRCCrawler
Microsoft Research
Info: http://research.microsoft.com/en-us/um/redmond/projects/msr-isrccrawler/msr-isrccrawler.htm
<email>isrc-bot@microsoft.com</email>
MindCrawler
MindCrawler
Miva
Miva
<email><![CDATA[AlgoFeedback [at] miva [dot] com]]></email>
MLBot
MLBot
Info: http://www.metadatalabs.com/mlbot
<email>support@metadatalabs.com</email>
UdmSearch
MNO GoSearch
mnoGoSearch
mnoGoSearch
Info: http://www.mnogosearch.org/
moget
Moget
MOMspider
MOM
Monster
Monster
Moreoverbot
Moreover
Info: http://www.moreover.com
Mp3Bot
Mp3Realm
Info: http://mp3realm.org/mp3bot/
<email>help@mp3realm.org</email>
msnbot-media
MSNBot (Media Search)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>
msnbot-mobile
MSNBot (Mobile)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>
msnbot-newsblogs
MSNBot (News Search)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>
msnbot-products
MSNBot (Product Search)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>
msnbot
MSNBot
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>
MSRBOT
MSRBot
Info: http://research.microsoft.com/research/sv/msrbot/
<email>msrbot@microsoft.com</email>
MuscatFerret
Muscat Ferret
MwdSearch
Mwd.Search
Najdi.si
Najdi.si
Info: www.najdi.si
NPBot
NameProtect
NaverBot
NaverBot
Info: http://help.naver.com/customer_webtxt_02.jsp
<email>nhnbot@naver.com</email>
NEC-MeshExplorer
NEC MeshExplorer
Nederland.zoek
Nederland.zoek
NetCarta CyberPilot Pro
NetCarta WebMap
Netcraft
Netcraft Web Server Survey
Info: http://news.netcraft.com/
NetMechanic
NetMechanic
NetNewsWire
NetNewsWire RSS
Info: http://www.newsgator.com/Individuals/NetNewsWire/
NetScoop
NetScoop
NIF
News is Free RSS
Info: http://www.newsisfree.com/robot.php
newscan-online
Newscan Online
(X11; compatible; crawler@newstin.com; HTTPClient 3.1
Newstin
Info: http://www.newstin.com/info/about_us
<email>crawler@newstin.com</email>
NextGenSearchBot 1
NextGen Search Bot
Info: http://about.zoominfo.com/About/NextGenSearchBot.aspx
NHSEWalker
NHSE Web Forager
NimbleCrawler
NimbleCrawler
Info: http://www.healthline.com/aboutus.jsp
<email>crawler@healthline.com</email>
NjuiceBot
NjuiceBot
Nomad
Nomad
Norbert the Spider
Norbert
Info: http://www.Burf.com
Gulliver
Northern Light
Nutch
Nutch
Info: http://lucene.apache.org/nutch/bot.html
explorersearch
NZ Explorer
Occam
Occam
Ocelli
Ocelli
Info: http://www.globalspec.com/Ocelli
omgilibot
Omgili
Info: http://www.omgili.com/Crawler.html
omgilibot/0.3 +http://www.omgili.com/Crawler.html
omgilibot
OneRiot
OneRiot
Info: http://www.oneriot.com/
Me.dium
OneRiot.com
Info: http://www.oneriot.com/company/about
LargeSmall
OneSpot
Info: http://www.onespot.com/
<email>info@onespot.com</email>
Online24-Bot
Online24-Bot
OOZBOT
OOZBOT
Info: http://www.setooz.com/oozbot.html
<email>agentname at setooz dot_com</email>
Openbot
Openfind
Info: http://www.openfind.com.tw/robot.html
<email>robot-response@openfind.com.tw</email>
Openfind
Openfind data gatherer
OpenISearch
OpenISearch
Info: http://www.openisearch.com/
Orbsearch
Orb Search
OWPBot
OWPBot
Info: http://www.openwhitepages.com
PackRat
Pack Rat
PageBoy
PageBoy
panscient.com
Panscient
Info: http://panscient.com/
ParaSite
ParaSite
ParchBot
ParchmentHill
Info: http://www.parchmenthill.com/search.htm
Patric
Patric
PEGASUS
Pegasus
PerlCrawler/1.0 Xavatoria/2.0
PerlCrawler 1.0
PGP-KA
PGP Key Agent
Duppies
Phantom
phpdig
PhpDig
psbot/0.1 (+http://www.picsearch.com/bot.html) (51dc65875976ac434c09274f7e46dec6)
Picsearch
Info: http://www.picsearch.com/bot.html
<email>info@picsearch.com</email>
PiltdownMan
Piltdown Man
Pimptrain’s robot
Pimptrain
Info: Pimptrain.com
pingalink
Ping A Link
pingdom.com_bot
Pingdom.com Bot
Info: http://www.pingdom.com/
Pioneer
Pioneer
PluckFeedCrawler
Pluck
Info: http://www.pluck.com
Plukkie
Plukkie
Info: http://www.botje.com/plukkie.htm
PlumtreeWebAccessor
Plumtree Web Accessor
PodNova
Pod Nova
Info: http://www.podnova.com
Pompos
Pompos
Info: http://dir.com/pompos.html
Poppi
Poppi
gestaltIconoclast
Popular Iconoclast
PortalBSpider
Portal B
PortalJuice.com
Portal Juice
Info: PortalJuice.com
PostRank
PostRank
Info: http://www.postrank.com/
<email>info@postrank.com</email>
psbot
PSBot
PycURL
PycURL
Info: http://pycurl.sourceforge.net
Qango.com Web Directory
Qango
Info: http://www.qango.com
R6_FeedFetcher
Radian6
Info: http://www.radian6.com/crawler/
<email>support@radian6.com</email>
R6_CommentReader
Radian6
Info: http://www.radian6.com/crawler/
<email>support@radian6.com</email>
R6_CommentReader
Radian6 Comment Reader
Info: http://www.radian6.com/crawler/
<email>support@radian6.com</email>
R6_FeedFetcher
Radian6 FeedFetcher
Info: http://www.radian6.com/crawler
<email>support@radian6.com</email>
R6_FeedFetcher
Radian6 FeedFetcher
Info: http://www.radian6.com/crawler
StackRambler
Rambler
Info: http://www.rambler.ru/
Raven
Raven Search
RixBot
REBOL IndeXer
Info: http://babelserver.org/rix/
rdfbot
Rediff
Info: http://www.rediff.com/
Resume Robot
Resume Robot
Road Runner: ImageScape Robot
Road Runner: ImageScape Robot
RHCS
RoadHouse Crawling System
Robbie
Robbie
RoboCrawl
RoboCrawl
Robofox
RoboFox
Robot du CRIM 1.0a
Robot Francoroute
Robozilla
Robozilla
Roverbot
Roverbot
RSS-SPIDER
RSS Feed Seeker
Info: http://www.rss-spider.com/fsb.php
RuLeS
RuLeS
SafetyNet Robot
SafetyNet
SBIder
SBIder RSS
Info: http://www.sitesell.com/sbider.html
Scarlett
Scarlett
Scharia
Scharia
Science-Index
Science Index
ScoutJet
ScoutJet
Info: http://www.scoutjet.com
Scrubby/3.0
Scrubby
Info: http://www.scrubtheweb.com/
SearchNZ
Search NZ
Info: http://www.searchnz.co.nz/
search17
Search17
Info: http://www.search17.com/bot.php
<email>crawler@search17.com</email>
searchprocess
SearchProcess
Seekbot
Seekbot
Info: http://www.seekbot.net/bot.html
SemtoBot
SemtoBot
Senrigan
Senrigan
Sensis Web Crawler
Sensis
Info: http://www.sensis.com.au/help.do
<email>search_comments\at\sensis\dot\com\dot\au</email>
spbot
SEOprofiler
Info: http://www.seoprofiler.com/bot/
<email>spbot _AT_ seoprofiler -DOT- com</email>
ServiceUptime.robot
ServiceUptime
SeznamBot
Seznam Fulltext Blog
Info: http://fulltext.sblog.cz/robot/
SG-Scout
SG Scout
Shagseeker
ShagSeeker
Shai’Hulud
<![CDATA[Shai’Hulud]]>
SheenBot
SheenBot
ShopWiki
<![CDATA[Shopwiki [Bot]]]>
Info: http://www.shopwiki.com/wiki/Help:Bot
<email>crawler@shopwiki.com</email>
SimilarPages/Nutch
<![CDATA[SimilarPages/Nutch [Crawler]]]>
Info: http://www.similarpages.com/
SimBot/1.0
Simmany Robot Ver 1.0
ssearcher100
Site Searcher
Site Valet
Site Valet
http://www.site-list.net
Site-List RSS
Info: http://www.site-list.net
SiteBot
SiteBot
Info: http://www.sitebot.org/robot/
SiteTech-Rover
SiteTech-Rover
SiteUptime.com
SiteUptime
Info: http://www.siteuptime.com
SiteVibeBot
SiteVibeBot
Info: http://www.sitevibe.nl
+SitiDi.net/SitiDiBot/
SitiDi
Info: SitiDi.net
aWapClient
Skymob
Info: Skymob.com
SLCrawler
SLCrawler
Sleek Spider
Sleek
Snapbot
Snap Shots
Info: http://www.snap.com
Snapbot/1.0
Snapbot
Info: http://www.snap.com
SnapPreviewBot
SnapPreviewBot
Info: http://www.snap.com/snapshots_faq.php
Snooper
Snooper
socbot
SocBot
Info: http://www.socbay.com
<email>search@socbay.com</email>
Sogou web spider
Sogou
Info: http://www.sogou.com
sohu-search
Sohu Search
Solbot
Solbot
Sosospider
Soso
Info: http://help.soso.com/webspider.htm
<email>sosospider@tencent.com</email>
Speedy
Speedy
Info: http://www.entireweb.com/about/search_tech/speedy_spider/
www.entireweb.com/speedy.html
Speedy
Info: http://www.entireweb.com/about/search_tech/speedy_spider/
Sphere Scout
Sphere
Info: www.scout.com
<email>scout at sphere dot com</email>
Sphider2
Sphider
mouse.house
Spider Monkey
SpiderBot
SpiderBot
spiderline
Spiderline Crawler
SpiderMan
SpiderMan
SpiderPig
SpiderPig
Info: http://www.icrossing.co.uk/bot/
SpiderView
SpiderView
Spinn3r
Spinn3r
Info: http://spinn3r.com/robot
suke
Suke
suntek
Suntek Search Engine
superbot.com
Super.info Search Bot
Info: http://www.super.info
Superfeedr
Superfeedr
Info: http://blog.superfeedr.com
<email>parser@superfeedr.com</email>
Synthesio
Synthesio
Info: http://www.synthesio.fr
<email>contact at synthesio dot fr</email>
Szukacz
Szukacz
Info: http://www.szukacz.pl/html/RobotEnglishVersion.html
<email>info@szukacz.pl</email>
Black Widow
TACH Black Widow
Tagoobot
Tagoo.ru
Info: http://www.tagoo.ru/
tailsweepblogcrawler
Tailsweep
Info: http://www.tailsweep.com/
<email><![CDATA[bot at [tailsweep] dot com]]></email>
Tarantula
Tarantula
tarspider
TarSpider
dlw3robot
Tcl W3 Robot
TechBOT
TechBOT
Technoratibot
Technorati
Info: http://technorati.com/about/
<email><![CDATA[press [at] technorati [dot] com]]></email>
Templeton
Templeton
teoma
Teoma/Ask Jeeves
Info: http://sp.teoma.com/docs/teoma/about/
JubiiRobot
The Jubii
NorthStar
The NorthStar Robot
w3index
The NWI Robot
Peregrinator-Mathematics
The Peregrinator
thumbshots-de-Bot
Thumbshots
Info: www.thumbshots.de
T-H-U-N-D-E-R-S-T-O-N-E
Thunderstone
TinEye
TinEye
Info: http://tineye.com/crawler.html
<email>crawler@tineye.com</email>
TITAN
Titan
TitIn
TitIn
TLSpider
TLSpider
turnitinbot
Turn it in
Info: http://www.turnitin.com/static/company.html
slysearch
Turn it in slysearch
Info: http://www.turnitin.com/static/products_services/search_engines.html
TurtleScanner
Turtle
Info: http://www.turtle.ru/
Tweetmeme
Tweetmeme.com
Info: http://tweetmeme.com/static.php?page=about
Twiceler
Twiceler (Cuill.com)
Info: http://www.cuill.com/twiceler/robot.html
Twingbot
Twingbot
Info: http://www.twingbot.com/
Twingly
Twingly
Info: http://www.twingly.com/
Twitterbot
Twitterbot
Twitturls
Twitturls.com
Info: http://twitturls.com
Python-urllib
Twitturls.com (Python-urllib)
Info: http://twitturls.com
UCSD-Crawler
UCSD Crawl
UMBC-memeta-Bot
UMBC RSS
Unpartisan
Unpartisan RSS
Info: http://www.unpartisan.com
urlck
URL Check
URL Spider Pro
URL Spider Pro
Valkyrie
Valkyrie
ClickSense
ValueClick LM
Info: http://www.lucidmedia.com/
<email>support@lucidmedia.com</email>
Mozilla/4.0 (vBSEO; http://www.vbseo.com)
vBSEO
Info: http://www.vbseo.com/
Verticrawl
Verticrawl
Victoria
Victoria
vision-search
Vision Search
Visions Search
Visions
Info: http://www.italicvisions.com/
<email>info@italicvisions.com</email>
voyager/1.0
Voyager
Info: http://www.kosmix.com/crawler.html
VWbot_K
VWbot
W3C-checklink
W3C
Info: http://validator.w3.org/checklink
W3C_CSS_Validator
W3C CSS Validator
Info: http://jigsaw.w3.org/css-validator/
W3C_Validator
W3C Validator
Info: http://validator.w3.org/
W3M2
W3M2
w3mir
W3mir
w@pspider
w@p
appie
Walhello Appie
Info: http://www.robotstxt.org/wc/active/html/appie.html
<email>aimo@walhello.com</email>
CrawlPaper
WallPaper
root
Web Core / Roots
WebMoose
Web Moose
WebAlta
WebAlta
Info: http://www.webalta.net/ru/about_webmaster.html
WebAlta Crawler
WebAlta
Info: http://www.webalta.net/ru/about_webmaster.html
WebBandit
WebBandit
WebCatcher
WebCatcher
Webclipping
Webclipping
WebCopy
WebCopy
WebFetcher
WebFetcher
weblayers
WebLayers
WebLinker
WebLinker
wlm
Weblog Monitor
WebQuest
WebQuest
WebReaper
WebReaper
webs@recruit.co.jp
Webs
websearchbench
WebSearchBench
Info: http://websearchbench.cs.uni-dortmund.de/
WOLP
WebStolperer
webvac
WebVac
webwalk
WebWalk
WebWalker
WebWalker
WebWatch
WebWatch
www.WebWombat.com.au
WebWombat
Info: http://www.webwombat.com.au/
Wget
Wget
whatUseek_winona
What U Seek Winona
Whitevector Crawler
Whitevector Crawler
Info: http://www.whitevector.com/
<email>support@whitevector.com</email>
www.whoisde.de
Whois DE
Info: http://www.whoisde.de
SurveyBot
Whois Source
Info: http://www.whois.sc/info/webmasters/surveybot.html
wikiwix
Wikiwix
Info: <![CDATA[http://www.wikiwix.com/?about=true&disp=robot]]>
Hazel’s Ferret Web hopper
Wild Ferret Web Hopper
Willow Internet Crawler by Twotrees
Willow
Windows-Live-Social-Object-Extractor-Engine
Windows Live SOEE
Windows-RSS-Platform/1.0
Windows RSS Platform 1.0
Info: http://blogs.msdn.com/rssteam/archive/2006/04/28/586220.aspx
Windows-RSS-Platform/2.0
Windows RSS Platform 2.0
Info: http://blogs.msdn.com/rssteam/archive/2008/02/28/the-rss-platform-user-agent-string.aspx
WinHTTP
WinHTTP
wired-digital-newsbot
Wired Digital
Vagabondo
Wise-Guys
Info: http://www.wise-guys.nl/webcrawler.php
Bilbo
Wise-Guys
Info: http://www.wise-guys.nl/webcrawler.php
zyborg
WiseNut
WordPress
WordPress
woriobot
Worio
Info: http://worio.com
OmniExplorer_Bot
WorldIndexer
Info: http://www.omni-explorer.com
Project Kolinka Forum Search
www.kolinka.com
Info: www.kolinka.com
<email><![CDATA[questions [at] kolinka [dot] com]]></email>
WWWC
WWWC
WWWeasel Robot
WWWeasel
wwwster
WWWSter
<email>gue@cis.uni-muenchen.de</email>
WWWWanderer
WWWWanderer
TECOMAC-Crawler
X-Crawler
Xenu
Xenu Link Sleuth
Info: http://home.snafu.de/tilman/xenulink.html
XGET
XGET
cosmos
XYLEME Robot
yacybot
YaCy
Info: http://yacy.net/bot.html
YahooYSMcm
Yahoo Publisher Network
Info: http://publisher.yahoo.com/
Yahoo-Blogs
Yahoo! Blogs
Info: http://help.yahoo.com/help/us/ysearch/crawling/crawling-02.html
YahooFeedSeeker
Yahoo! FeedSeeker
Info: http://publisher.yahoo.com/rssguide
Yahoo-MMCrawler
Yahoo! Image Search
Info: http://help.yahoo.com/l/us/yahoo/search/mediasearch/image-08.html
YahooSeeker/M1A1-R2D2
Yahoo! Mobile
Info: http://help.yahoo.com/l/us/yahoo/search/indexing/webcrawler-13.html
Yahoo! Slurp
Yahoo! Slurp
Info: http://help.yahoo.com/l/us/yahoo/search/webcrawler/
Yahoo-VerticalCrawler
Yahoo! Vertical Crawler
YandexAntivirus
Yandex Antivirus
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexBlog
Yandex Blog
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexBot
Yandex Bot
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexCatalog
Yandex Catalog
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexDirect
Yandex Direct
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexFavicon
Yandex Favicon
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexImageResizer
Yandex ImageResizer
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexImages
Yandex Images
Info: http://yandex.com/bots
YandexMedia
Yandex Media
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexMetrika
Yandex Metrika
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexNews
Yandex News
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexPagechecker
Yandex Pagechecker
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexVideo
Yandex Video
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexWebmaster
Yandex Webmaster
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
YandexZakladki
Yandex Zakladki
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>
Yanga WorldSearch Bot
Yanga
Info: http://www.yanga.co.uk/
<email>support@yanga.co.uk </email>
Yanga WorldSearch Bot
Yanga WorldSearch Bot
Info: http://www.yanga.co.uk
YebolBot
YebolBot
<email>yebolbot@gmail.com</email>
Yeti
Yeti
Info: http://help.naver.com/robots/
yeti
Yeti
Info: http://help.naver.com/robots/
<email>yetibot@naver.com</email>
Yeti/1.0
Yeti/1.0
Info: +http://help.naver.com/robots/
YodaoBot
Yodao
Info: http://www.yodao.com/help/webmaster/spider/
YoudaoBot
Youdao
Info: http://www.youdao.com/help/webmaster/spider/
YRSpider
YunRang
Info: http://www.yunrang.com/yrspider.html
zeus
Zeus Internet Marketing
Info: http://www.cyber-robotics.com/
http://www.zorkk.com
Zork RSS
Info: http://www.zorkk.com