List of User-Agents (Spiders, Robots, Crawlers)

searchdnabot
Info: http://searchenginedna.com/

kalooga/KaloogaBot
Info: http://www.kalooga.com

iajaBot

Girafabot
Info: http://www.girafa.com/

Gigabot
Info: http://www.gigablast.com/spider.html

ExaBot
Info: http://beta.exalead.com/search/C=0/2p=Help.7

Exabot
Info: http://www.exabot.com/go/robot

EnaBot
Info: http://www.enaball.com/crawler.html

DNAbot/1.0

facebookexternalhit
Info: http://www.facebook.com/externalhit_uatext.php

ichiro/mobile
Info: http://help.goo.ne.jp/door/crawler.html

008/0.83
Info: http://www.80legs.com/spider.html

urlfan-bot
Info: http://www.urlfanx.com/site/bot/350.html

abot
Info: http://www.abot.com/

ABACHOBot
Info: http://www.abacho.com/

ABCdatos
Info: http://www.robotstxt.org/wc/active/html/abcdatos.html

AboutUsBot
Info: http://www.aboutus.org/AboutUs:Bot

Accelatech RSSCrawler

Accoona-AI-Agent
Info: http://accoona.com/about/about_accoona.jsp

aconon Index
Info: raubfische.de

Ahoy!
Info: http://www.robotstxt.org/wc/active/html/ahoythehomepagefinder.html

AideRSS
Info: http://www.postrank.com

ia_archiver
Info: http://www.alexa.com/site/help/webmasters

bitlybot

AlkalineBOT
Info: http://www.robotstxt.org/wc/active/html/Alkaline.html

scooter

JS-Kit URL Resolver
Info: http://aws.amazon.com/ec2/

crawler/3.0.0
Info: http://www.notconfigured.com/

EMC Spider

Anthill
Info: http://www.robotstxt.org/wc/active/html/anthill.html

Synapse
Info: http://synapse.apache.org/

Robot/v1.34
Info: http://smart.apnoti.com/index/aboutApnotiWebCrawler

Aport
Info: http://www.aport.ru/

AppleSyndication

Arachnophilia
Info: http://www.robotstxt.org/wc/active/html/arachnophilia.html

Araneo
Info: http://www.robotstxt.org/wc/active/html/araneo.html

ArchitextSpider
Info: http://www.atext.com

arks/1.0
Info: http://www.robotstxt.org/wc/active/html/arks.html

T312461
Info: http://www.artabus.com

Ask Jeeves

ASpider
Info: http://www.robotstxt.org/wc/active/html/aspider.html

ATN_Worldwide
Info: http://www.robotstxt.org/wc/active/html/atn.txt.html

Atomz
Info: atomz.com

atraxbot
Info: http://www.atraxsolutions.com/atraxbot

Attentio
Info: www.attentio.com

attributor/1.13.2
Info: http://www.attributor.com

AURESYS
Info: http://www.robotstxt.org/wc/active/html/auresys.html

bbot
Info: http://www.robotstxt.org/wc/active/html/bbot.html

BabalooSpider
Info: http://www.babaloo.si

BackRub
Info: http://www.robotstxt.org/wc/active/html/backrub.html

BaiduMobaider
Info: http://www.baidu.jp/spider/

BaiduImagespider
Info: http://www.baidu.jp/spider/

Baiduspider
Info: http://www.baidu.com/search/spider_jp.html

BecomeBot
Info: http://www.become.com/site_owners.html

Begun
Info: http://www.begun.ru

BejiBot

Big Brother
Info: http://www.robotstxt.org/wc/active/html/bigbrother.html

BigBoardsUpdater

BigmirSpider
Info: http://www.bigmir.net/

bingbot
Info: http://www.bing.com/bingbot.htm

Bitacle bot
Info: http://bitacle.org/

Biz360 Spider
Info: http://www.biz360.com

Bjaaland
Info: http://www.robotstxt.org/wc/active/html/bjaaland.html

BlackWidow
Info: http://www.robotstxt.org/wc/active/html/blackwidow.html

BlogCrawler by Xango

blogdb
Info: http://blogdb.jp

blog search engine by BlogFan.ORG
Info: http://www.blogfan.org

Bloglines
Info: http://www.bloglines.com

BlogPulse (ISSpider-3.0)

BlogSearch
Info: http://www.icerocket.com

BlogsNowBot
Info: http://www.blogsnow.com/

BlogStreetBot
Info: http://www.blogstreet.com/

BoardTracker
Info: http://www.boardtracker.com/spider.html

BoardPulse
Info: http://www.boardpulse.com/

BoardReader
Info: http://www.boardreader.com/aboutus.asp

BoardViewer
Info: http://www.boardviewer.com/

boitho.com-robot
Info: http://www.boitho.com/bot.html

boitho.com-dc
Info: http://www.boitho.com/dcbot.html

borg-bot
Info: http://www.robotstxt.org/wc/active/html/borg-bot.html

BotOnParade
Info: http://www.bots-on-para.de/bot.html

BOTW Spider
Info: http://botw.org

BDFetch
Info: http://www.brandprotect.com

BSpider
Info: http://www.robotstxt.org/wc/active/html/bspider.html

Bulkfeeds
Info: http://bulkfeeds.net

Butterfly
Info: http://labs.topsy.com/butterfly.html

CACTVS Chemistry Spider

Calif

CaRP/3.6Evolution
Info: http://www.biz360.com

CatchBot
Info: http://www.catchbot.com/

mxbot
Info: http://www.chainn.com/mxbot.html

Charlotte
Info: http://www.searchme.com/support/

Checkbot

ChitikaBot
Info: http://chitika.com/blog/bot/

ChristCrawler.com
Info: ChristCrawler.com

www.cienciaficcion.net
Info: www.cienciaficcion.net

CipinetBot
Info: http://www.cipinet.com/bot.html

CJNetworkQuality
Info: http://www.cj.com/networkquality/

YesupBot
Info: http://www.yesup.net/bot.html

CligooRobot
Info: http://www.cligoo.de/wk/technik.php

CLX.ru Bot
Info: http://www.clx.ru

CMC/0.01

ColdFusion

combine

Crawler (cometsearch@cometsystems.com)
Info: http://www.cometsystems.com

CCBot
Info: http://www.commoncrawl.org/

ComputingSite Robi

conceptbot

Cooby.de Crawler
Info: Cooby.de

CoolBot

Covario
Info: http://www.covario.com/ids

Twiceler-0.9
Info: http://www.cuil.com

twiceler.
Info: http://www.cuil.com/info/webmaster_info/

Cusco

CyberSpyder

daumoa
Info: http://ws.daum.net/aboutWebSearch_en.html

daypopbot

DeadLinkCheck
Info: http://dlc.sourceforge.net/

Deepnet Explorer

DesertRealm.com
Info: DesertRealm.com

Deweb

Die Blinde Kuh
Info: http://www.robotstxt.org/wc/active/html/blindekuh.html

dienstspider

Diffbot
Info: http://www.diffbot.com

Digger/1.0 JDK/1.3.0

Digimarc WebReader

Digimarc CGIReader

DIIbot

grabber

discobot
Info: http://www.discoveryengine.com/discobot.html

Dolphin
Info: http://telehouse.ru/crawler.html

DotBot
Info: http://www.dotnetdotcom.org/

DotBot/1.1
Info: http://www.dotnetdotcom.org

DotTK SiteCheck
Info: http://www.dot.tk

DragonBot/1.0 libwww/5.0

DWCP/2.0

e-SocietyRobot
Info: http://www.yama.info.waseda.ac.jp/~yamana/es/index_eng.htm

EbiNess/0.01a

dragonfly(ebingbong@playstarmusic.com)
Info: http://www.ebingbong.com/help/ourRobot.php

edgeio-retriever
Info: http://www.edgeio.com

EIT-Link-Verifier-Robot/0.2

elfinbot

abby
Info: http://www.ellerdale.com/crawler.html

Emacs-w3/v[0-9\.]+

Embedly
Info: http://support.embed.ly/

envolk
Info: http://www.envolk.com/envolkspiderinfo.html

ESISmartSpider

esther

eSyndiCat Bot

EuripBot
Info: http://www.eurip.com

Eurobot/1.1
Info: http://eurobot.ayell.eu/

Evliya Celebi

exactseek-pagereaper
Info: www.exactseek.com

ExactSeek_Spider
Info: http://www.askjeevs.com

NG/2.0
Info: http://botspotter.net/bs-389.html

Ezooms

Facebook

factbot
Info: http://www.factbites.com/webmasters.php

fast-webcrawler
Info: http://help.yahoo.com/help/us/ysearch/slurp/index.html

FastCrawler

Feed24.com

Info: http://www.feed24.com

FeedBlitz
Feed Blitz
Info: http://www.feedblitz.com

UniversalFeedParser
Feed Parser
Info: http://www.feedparser.org

FeedBurner
Info: http://www.FeedBurner.com

FeedHub MetaDataFetcher
Info: http://www.feedhub.com

Feedster Crawler
Info: http://www.feedstermedia.com/

Feedtrace-bot
Info: http://feedtrace.com/

FeedValidator/

FEHLSTART Superspider

FelixIDE

ESIRover

fido

findlinks
Info: http://wortschatz.uni-leipzig.de/findlinks/

FindoryBot
Info: http://www.findory.com

Firebat
Info: http://lms.virtual-presence.org

Fish-Search-Robot

Mozilla/4.0 (compatible: FDSE robot)

FollowSite Bot
Info: http://www.followsite.com/bot.html

libwww-perl/5.810
Info: http://forexitalia.org/

fouineur.9bit.qc.ca

Freecrawl

FreeWebMonitoring
Info: http://www.freewebmonitoring.com

FunnelWeb

GaisBot
Info: http://gais.cs.ccu.edu.tw/robot.php

gamekitbot
Info: http://www.uchoose.de/crawler/gamekitbot/

gamma

gazz

gcreep

genieBot
Info: http://64.5.245.11/faq/faq.html

GeoHasher
Info: geohasher.gotdns.org

geourl
Info: http://geourl.org/bot.html

GetterroboPlus

GetURL.rexx

GingerCrawler
Info: http://www.gingersoftware.com/crawler_agent.htm

UnwindFetchor
Info: http://www.gnip.com

Goku
Info: http://goku.ru/bot.htm

Golem

gonzo

gooblogsearch
Info: http://help.goo.ne.jp/help/article/704/

Googlebot-Image
Info: http://www.google.com/webmasters/bot.html

Googlebot
Info: http://www.google.com/bot.html

Mediapartners-Google
Info: https://www.google.com/adsense/faq

Adsbot-Google
Info: http://www.google.com/bot.html

FeedFetcher-Google
Info: http://www.google.com/feedfetcher.html

kw-lp-suggest

Googlebot-Mobile
Info: http://www.google.com/support/webmasters/bin/answer.py?answer=34647&topic=9346

PageFetcher-Google-CoOp

Google-Sitemaps/1.0

Googlebot-Video
Info: http://www.google.com/bot.html

Gpostbot
Info: http://www.gpost.info/help.php?c=bot

griffon

Gromit

http://grub.org
Info: http://grub.org

Gulper Web Bot

GurujiBot
Info: http://www.guruji.com/en/WebmasterFAQ.html

H?m?h?kki
Info: http://www.robotstxt.org/wc/active/html/finnish.html

havIndex

HeinrichderMiragoRobot

HenryTheMiragoRobot

archive.org_bot
Info: http://crawler.archive.org/

HKU WWW Robot

HolyCowDude
Info: http://www.holycowdude.com/spider.htm

Hometown

HostTracker.com/1.0
Info: http://host-tracker.com

htdig

HTMLgobble

AITCSRobot

I Robot

http://www.almaden.ibm.com/cs/crawler
IBM Almaden
Info: http://www.almaden.ibm.com/cs/crawler
<email>crawler@us.ibm.com</email>

IBM_Planetwide
IBM Planetwide

+http://www.icerocket.com/
IceRocket
Info: http://www.icerocket.com/

ichiro
Ichiro
Info: http://help.goo.ne.jp/door/crawler.html
<email><![CDATA[ichiro [at] nttr [dot] co [dot] jp]]></email>

igde
igdeSpyder
Info: http://igde.ru

IlTrovatore-Setaccio
IlTrovatore Setaccio
Info: http://www.iltrovatore.it/aiuto/faq.html
<email>info@iltrovatore.it</email>

Mozilla 3.01 PBWF (Win95)
Imagelock

IncyWincy
IncyWincy

Indy Library
Indy Library

Informant
Informant

InfoSeek Robot
InfoSeek Robot 1.0

Infoseek Sidewinder
Infoseek Sidewinder

infoSpider
infoSpider
Info: http://www.infospider.com
<email>support@infospider.com</email>

INGRID
Ingrid

slurp@inktomi
Inktomi

Insitor
Insitor
Info: http://www.insitor.de/

inspectorwww
Inspector Web

IAGENT
IntelliAgent

Intelliseek
Intelliseek
Info: http://www.intelliseek.com/

Internet Cruiser Robot
Internet Cruiser

SCHOOLCARE; SV1; InfoPath.1
Internet for learning
Info: http://www.ifl.com/

InternetLinkAgent
Internet Link Agent

3GSE bot
Internet Research Institute UK
Info: http://iri-uk.com/
<email>coordinator@iri-uk.com</email>

internetseer
Internet Seer

sharp-info-agent
Internet Shinchakubin

Pogodak
Interseek
Info: http://www.interseek.com/

Iron33
Iron33

IsraeliSearch
Israeli Search

itchBot
Itch
Info: http://www.itch.com/infoforwebmasters.html

JavaBee
JavaBee

JBot
JBot

JCrawler
JCrawler

JetBot
JetEye
Info: http://www.jeteye.com/jetbot.html

JoBo
JoBo

Jobot
Jobot

JoeBot
JoeBot

JSpider
JSpider
Info: http://j-spider.sourceforge.net/

jumpstation
JumpStation

Jyxobot
Jyxo
Info: http://jyxo.cz/

image.kapsi.net
Kapsi Images
Info: image.kapsi.net

Katipo
Katipo

KDD-Explorer
KDD Explorer

KIT-Fireball
KIT Fireball

kmbot
knowmore
Info: http://knowmore.com/
<email>support@knowmore.com</email>

KO_Yappo_Robot
KO Yappo

LabelGrab
LabelGrabber

larbin
Larbin

LeapTag
LeapTag News Reader
Info: http://www.leaptag.com/privatebeta/faq.php
<email>support@leaptag.com</email>

LexxeBot
Lexxe
Info: http://lexxe.com/about/webmasters.cfm
<email>lexxebot@lexxe.com</email>

lwp-trivial
libwww-perl
Info: http://search.cpan.org/~gaas/libwww-perl-5.836/lib/LWP/Simple.pm

linkalarm
Link Alarm

Linkidator
Link Validator

LinkedInBot
LinkedIn
Info: http://www.linkedin.com

LinkScan Server
LinkScan

LinkWalker
LinkWalker

livedoorCheckers/
Livedoor Checkers

Lockon
Lockon

logo.gif crawler
logo.gif

Lycos
Lycos

Apple-PubSub/59
Mac OS X RSS

Magpie
Magpie

Mail.Ru
Mail.Ru
Info: http://www.mail.ru
<email>portal@corp.mail.ru</email>

MJ12bot
Majestics MJ12bot
Info: http://majestic12.co.uk/bot.php

Mammoth
Mammoth
Info: http://www.sli-systems.com

Marvin
Marvin

marvin/infoseek
marvin/infoseek

M/3.8
Mattie

MediaFox
MediaFox

mercator
Mercator
Info: http://research.compaq.com/SRC/mercator/

MerzScope
MerzScope

METASpider
Meta
Info: http://www.meta.com.ua/

Metaeuro
MetaEuro
Info: http://www.metaeuro.com/
<email>crawler@metaeuro.com</email>

MetaGer-LinkChecker
MetaGer

MetaURI
MetaURI
Info: metauri.com

MSR-ISRCCrawler
Microsoft Research
Info: http://research.microsoft.com/en-us/um/redmond/projects/msr-isrccrawler/msr-isrccrawler.htm
<email>isrc-bot@microsoft.com</email>

MindCrawler
MindCrawler

Miva
Miva
<email><![CDATA[AlgoFeedback [at] miva [dot] com]]></email>

MLBot
MLBot
Info: http://www.metadatalabs.com/mlbot
<email>support@metadatalabs.com</email>

UdmSearch
MNO GoSearch

mnoGoSearch
mnoGoSearch
Info: http://www.mnogosearch.org/

moget
Moget

MOMspider
MOM

Monster
Monster

Moreoverbot
Moreover
Info: http://www.moreover.com

Mp3Bot
Mp3Realm
Info: http://mp3realm.org/mp3bot/
<email>help@mp3realm.org</email>

msnbot-media
MSNBot (Media Search)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>

msnbot-mobile
MSNBot (Mobile)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>

msnbot-newsblogs
MSNBot (News Search)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>

msnbot-products
MSNBot (Product Search)
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>

msnbot
MSNBot
Info: <![CDATA[http://help.live.com/help.aspx?mkt=en-gb&project=wl_webmasters]]>

MSRBOT
MSRBot
Info: http://research.microsoft.com/research/sv/msrbot/
<email>msrbot@microsoft.com</email>

MuscatFerret
Muscat Ferret

MwdSearch
Mwd.Search

Najdi.si
Najdi.si
Info: www.najdi.si

NPBot
NameProtect

NaverBot
NaverBot
Info: http://help.naver.com/customer_webtxt_02.jsp
<email>nhnbot@naver.com</email>

NEC-MeshExplorer
NEC MeshExplorer

Nederland.zoek
Nederland.zoek

NetCarta CyberPilot Pro
NetCarta WebMap

Netcraft
Netcraft Web Server Survey
Info: http://news.netcraft.com/

NetMechanic
NetMechanic

NetNewsWire
NetNewsWire RSS
Info: http://www.newsgator.com/Individuals/NetNewsWire/

NetScoop
NetScoop

NIF
News is Free RSS
Info: http://www.newsisfree.com/robot.php

newscan-online
Newscan Online

(X11; compatible; crawler@newstin.com; HTTPClient 3.1
Newstin
Info: http://www.newstin.com/info/about_us
<email>crawler@newstin.com</email>

NextGenSearchBot 1
NextGen Search Bot
Info: http://about.zoominfo.com/About/NextGenSearchBot.aspx

NHSEWalker
NHSE Web Forager

NimbleCrawler
NimbleCrawler
Info: http://www.healthline.com/aboutus.jsp
<email>crawler@healthline.com</email>

NjuiceBot
NjuiceBot

Nomad
Nomad

Norbert the Spider
Norbert
Info: http://www.Burf.com

Gulliver
Northern Light

Nutch
Nutch
Info: http://lucene.apache.org/nutch/bot.html

explorersearch
NZ Explorer

Occam
Occam

Ocelli
Ocelli
Info: http://www.globalspec.com/Ocelli

omgilibot
Omgili
Info: http://www.omgili.com/Crawler.html

omgilibot/0.3 +http://www.omgili.com/Crawler.html
omgilibot

OneRiot
OneRiot
Info: http://www.oneriot.com/

Me.dium
OneRiot.com
Info: http://www.oneriot.com/company/about

LargeSmall
OneSpot
Info: http://www.onespot.com/
<email>info@onespot.com</email>

Online24-Bot
Online24-Bot

OOZBOT
OOZBOT
Info: http://www.setooz.com/oozbot.html
<email>agentname at setooz dot_com</email>

Openbot
Openfind
Info: http://www.openfind.com.tw/robot.html
<email>robot-response@openfind.com.tw</email>

Openfind
Openfind data gatherer

OpenISearch
OpenISearch
Info: http://www.openisearch.com/

Orbsearch
Orb Search

OWPBot
OWPBot
Info: http://www.openwhitepages.com

PackRat
Pack Rat

PageBoy
PageBoy

panscient.com
Panscient
Info: http://panscient.com/

ParaSite
ParaSite

ParchBot
ParchmentHill
Info: http://www.parchmenthill.com/search.htm

Patric
Patric

PEGASUS
Pegasus

PerlCrawler/1.0 Xavatoria/2.0
PerlCrawler 1.0

PGP-KA
PGP Key Agent

Duppies
Phantom

phpdig
PhpDig

psbot/0.1 (+http://www.picsearch.com/bot.html) (51dc65875976ac434c09274f7e46dec6)
Picsearch
Info: http://www.picsearch.com/bot.html
<email>info@picsearch.com</email>

PiltdownMan
Piltdown Man

Pimptrain’s robot
Pimptrain
Info: Pimptrain.com

pingalink
Ping A Link

pingdom.com_bot
Pingdom.com Bot
Info: http://www.pingdom.com/

Pioneer
Pioneer

PluckFeedCrawler
Pluck
Info: http://www.pluck.com

Plukkie
Plukkie
Info: http://www.botje.com/plukkie.htm

PlumtreeWebAccessor
Plumtree Web Accessor

PodNova
Pod Nova
Info: http://www.podnova.com

Pompos
Pompos
Info: http://dir.com/pompos.html

Poppi
Poppi

gestaltIconoclast
Popular Iconoclast

PortalBSpider
Portal B

PortalJuice.com
Portal Juice
Info: PortalJuice.com

PostRank
PostRank
Info: http://www.postrank.com/
<email>info@postrank.com</email>

psbot
PSBot

PycURL
PycURL
Info: http://pycurl.sourceforge.net

Qango.com Web Directory
Qango
Info: http://www.qango.com

R6_FeedFetcher
Radian6
Info: http://www.radian6.com/crawler/
<email>support@radian6.com</email>

R6_CommentReader
Radian6
Info: http://www.radian6.com/crawler/
<email>support@radian6.com</email>

R6_CommentReader
Radian6 Comment Reader
Info: http://www.radian6.com/crawler/
<email>support@radian6.com</email>

R6_FeedFetcher
Radian6 FeedFetcher
Info: http://www.radian6.com/crawler
<email>support@radian6.com</email>

R6_FeedFetcher
Radian6 FeedFetcher
Info: http://www.radian6.com/crawler

StackRambler
Rambler
Info: http://www.rambler.ru/

Raven
Raven Search

RixBot
REBOL IndeXer
Info: http://babelserver.org/rix/

rdfbot
Rediff
Info: http://www.rediff.com/

Resume Robot
Resume Robot

Road Runner: ImageScape Robot
Road Runner: ImageScape Robot

RHCS
RoadHouse Crawling System

Robbie
Robbie

RoboCrawl
RoboCrawl

Robofox
RoboFox

Robot du CRIM 1.0a
Robot Francoroute

Robozilla
Robozilla

Roverbot
Roverbot

RSS-SPIDER
RSS Feed Seeker
Info: http://www.rss-spider.com/fsb.php

RuLeS
RuLeS

SafetyNet Robot
SafetyNet

SBIder
SBIder RSS
Info: http://www.sitesell.com/sbider.html

Scarlett
Scarlett

Scharia
Scharia

Science-Index
Science Index

ScoutJet
ScoutJet
Info: http://www.scoutjet.com

Scrubby/3.0
Scrubby
Info: http://www.scrubtheweb.com/

SearchNZ
Search NZ
Info: http://www.searchnz.co.nz/

search17
Search17
Info: http://www.search17.com/bot.php
<email>crawler@search17.com</email>

searchprocess
SearchProcess

Seekbot
Seekbot
Info: http://www.seekbot.net/bot.html

SemtoBot
SemtoBot

Senrigan
Senrigan

Sensis Web Crawler
Sensis
Info: http://www.sensis.com.au/help.do
<email>search_comments\at\sensis\dot\com\dot\au</email>

spbot
SEOprofiler
Info: http://www.seoprofiler.com/bot/
<email>spbot _AT_ seoprofiler -DOT- com</email>

ServiceUptime.robot
ServiceUptime

SeznamBot
Seznam Fulltext Blog
Info: http://fulltext.sblog.cz/robot/

SG-Scout
SG Scout

Shagseeker
ShagSeeker

Shai’Hulud
<![CDATA[Shai’Hulud]]>

SheenBot
SheenBot

ShopWiki
<![CDATA[Shopwiki [Bot]]]>
Info: http://www.shopwiki.com/wiki/Help:Bot
<email>crawler@shopwiki.com</email>

SimilarPages/Nutch
<![CDATA[SimilarPages/Nutch [Crawler]]]>
Info: http://www.similarpages.com/

SimBot/1.0
Simmany Robot Ver 1.0

ssearcher100
Site Searcher

Site Valet
Site Valet

http://www.site-list.net
Site-List RSS
Info: http://www.site-list.net

SiteBot
SiteBot
Info: http://www.sitebot.org/robot/

SiteTech-Rover
SiteTech-Rover

SiteUptime.com
SiteUptime
Info: http://www.siteuptime.com

SiteVibeBot
SiteVibeBot
Info: http://www.sitevibe.nl

+SitiDi.net/SitiDiBot/
SitiDi
Info: SitiDi.net

aWapClient
Skymob
Info: Skymob.com

SLCrawler
SLCrawler

Sleek Spider
Sleek

Snapbot
Snap Shots
Info: http://www.snap.com

Snapbot/1.0
Snapbot
Info: http://www.snap.com

SnapPreviewBot
SnapPreviewBot
Info: http://www.snap.com/snapshots_faq.php

Snooper
Snooper

socbot
SocBot
Info: http://www.socbay.com
<email>search@socbay.com</email>

Sogou web spider
Sogou
Info: http://www.sogou.com

sohu-search
Sohu Search

Solbot
Solbot

Sosospider
Soso
Info: http://help.soso.com/webspider.htm
<email>sosospider@tencent.com</email>

Speedy
Speedy
Info: http://www.entireweb.com/about/search_tech/speedy_spider/

www.entireweb.com/speedy.html
Speedy
Info: http://www.entireweb.com/about/search_tech/speedy_spider/

Sphere Scout
Sphere
Info: www.scout.com
<email>scout at sphere dot com</email>

Sphider2
Sphider

mouse.house
Spider Monkey

SpiderBot
SpiderBot

spiderline
Spiderline Crawler

SpiderMan
SpiderMan

SpiderPig
SpiderPig
Info: http://www.icrossing.co.uk/bot/

SpiderView
SpiderView

Spinn3r
Spinn3r
Info: http://spinn3r.com/robot

suke
Suke

suntek
Suntek Search Engine

superbot.com
Super.info Search Bot
Info: http://www.super.info

Superfeedr
Superfeedr
Info: http://blog.superfeedr.com
<email>parser@superfeedr.com</email>

Synthesio
Synthesio
Info: http://www.synthesio.fr
<email>contact at synthesio dot fr</email>

Szukacz
Szukacz
Info: http://www.szukacz.pl/html/RobotEnglishVersion.html
<email>info@szukacz.pl</email>

Black Widow
TACH Black Widow

Tagoobot
Tagoo.ru
Info: http://www.tagoo.ru/

tailsweepblogcrawler
Tailsweep
Info: http://www.tailsweep.com/
<email><![CDATA[bot at [tailsweep] dot com]]></email>

Tarantula
Tarantula

tarspider
TarSpider

dlw3robot
Tcl W3 Robot

TechBOT
TechBOT

Technoratibot
Technorati
Info: http://technorati.com/about/
<email><![CDATA[press [at] technorati [dot] com]]></email>

Templeton
Templeton

teoma
Teoma/Ask Jeeves
Info: http://sp.teoma.com/docs/teoma/about/

JubiiRobot
The Jubii

NorthStar
The NorthStar Robot

w3index
The NWI Robot

Peregrinator-Mathematics
The Peregrinator

thumbshots-de-Bot
Thumbshots
Info: www.thumbshots.de

T-H-U-N-D-E-R-S-T-O-N-E
Thunderstone

TinEye
TinEye
Info: http://tineye.com/crawler.html
<email>crawler@tineye.com</email>

TITAN
Titan

TitIn
TitIn

TLSpider
TLSpider

turnitinbot
Turn it in
Info: http://www.turnitin.com/static/company.html

slysearch
Turn it in slysearch
Info: http://www.turnitin.com/static/products_services/search_engines.html

TurtleScanner
Turtle
Info: http://www.turtle.ru/

Tweetmeme
Tweetmeme.com
Info: http://tweetmeme.com/static.php?page=about

Twiceler
Twiceler (Cuill.com)
Info: http://www.cuill.com/twiceler/robot.html

Twingbot
Twingbot
Info: http://www.twingbot.com/

Twingly
Twingly
Info: http://www.twingly.com/

Twitterbot
Twitterbot

Twitturls
Twitturls.com
Info: http://twitturls.com

Python-urllib
Twitturls.com (Python-urllib)
Info: http://twitturls.com

UCSD-Crawler
UCSD Crawl

UMBC-memeta-Bot
UMBC RSS

Unpartisan
Unpartisan RSS
Info: http://www.unpartisan.com

urlck
URL Check

URL Spider Pro
URL Spider Pro

Valkyrie
Valkyrie

ClickSense
ValueClick LM
Info: http://www.lucidmedia.com/
<email>support@lucidmedia.com</email>

Mozilla/4.0 (vBSEO; http://www.vbseo.com)
vBSEO
Info: http://www.vbseo.com/

Verticrawl
Verticrawl

Victoria
Victoria

vision-search
Vision Search

Visions Search
Visions
Info: http://www.italicvisions.com/
<email>info@italicvisions.com</email>

voyager/1.0
Voyager
Info: http://www.kosmix.com/crawler.html

VWbot_K
VWbot

W3C-checklink
W3C
Info: http://validator.w3.org/checklink

W3C_CSS_Validator
W3C CSS Validator
Info: http://jigsaw.w3.org/css-validator/

W3C_Validator
W3C Validator
Info: http://validator.w3.org/

W3M2
W3M2

w3mir
W3mir

w@pspider
w@p

appie
Walhello Appie
Info: http://www.robotstxt.org/wc/active/html/appie.html
<email>aimo@walhello.com</email>

CrawlPaper
WallPaper

root
Web Core / Roots

WebMoose
Web Moose

WebAlta
WebAlta
Info: http://www.webalta.net/ru/about_webmaster.html

WebAlta Crawler
WebAlta
Info: http://www.webalta.net/ru/about_webmaster.html

WebBandit
WebBandit

WebCatcher
WebCatcher

Webclipping
Webclipping

WebCopy
WebCopy

WebFetcher
WebFetcher

weblayers
WebLayers

WebLinker
WebLinker

wlm
Weblog Monitor

WebQuest
WebQuest

WebReaper
WebReaper

webs@recruit.co.jp
Webs

websearchbench
WebSearchBench
Info: http://websearchbench.cs.uni-dortmund.de/

WOLP
WebStolperer

webvac
WebVac

webwalk
WebWalk

WebWalker
WebWalker

WebWatch
WebWatch

www.WebWombat.com.au
WebWombat
Info: http://www.webwombat.com.au/

Wget
Wget

whatUseek_winona
What U Seek Winona

Whitevector Crawler
Whitevector Crawler
Info: http://www.whitevector.com/
<email>support@whitevector.com</email>

www.whoisde.de
Whois DE
Info: http://www.whoisde.de

SurveyBot
Whois Source
Info: http://www.whois.sc/info/webmasters/surveybot.html

wikiwix
Wikiwix
Info: <![CDATA[http://www.wikiwix.com/?about=true&disp=robot]]>

Hazel’s Ferret Web hopper
Wild Ferret Web Hopper

Willow Internet Crawler by Twotrees
Willow

Windows-Live-Social-Object-Extractor-Engine
Windows Live SOEE

Windows-RSS-Platform/1.0
Windows RSS Platform 1.0
Info: http://blogs.msdn.com/rssteam/archive/2006/04/28/586220.aspx

Windows-RSS-Platform/2.0
Windows RSS Platform 2.0
Info: http://blogs.msdn.com/rssteam/archive/2008/02/28/the-rss-platform-user-agent-string.aspx

WinHTTP
WinHTTP

wired-digital-newsbot
Wired Digital

Vagabondo
Wise-Guys
Info: http://www.wise-guys.nl/webcrawler.php

Bilbo
Wise-Guys
Info: http://www.wise-guys.nl/webcrawler.php

zyborg
WiseNut

WordPress
WordPress

woriobot
Worio
Info: http://worio.com

OmniExplorer_Bot
WorldIndexer
Info: http://www.omni-explorer.com

Project Kolinka Forum Search
www.kolinka.com
Info: www.kolinka.com
<email><![CDATA[questions [at] kolinka [dot] com]]></email>

WWWC
WWWC

WWWeasel Robot
WWWeasel

wwwster
WWWSter
<email>gue@cis.uni-muenchen.de</email>

WWWWanderer
WWWWanderer

TECOMAC-Crawler
X-Crawler

Xenu
Xenu Link Sleuth
Info: http://home.snafu.de/tilman/xenulink.html

XGET
XGET

cosmos
XYLEME Robot

yacybot
YaCy
Info: http://yacy.net/bot.html

YahooYSMcm
Yahoo Publisher Network
Info: http://publisher.yahoo.com/

Yahoo-Blogs
Yahoo! Blogs
Info: http://help.yahoo.com/help/us/ysearch/crawling/crawling-02.html

YahooFeedSeeker
Yahoo! FeedSeeker
Info: http://publisher.yahoo.com/rssguide

Yahoo-MMCrawler
Yahoo! Image Search
Info: http://help.yahoo.com/l/us/yahoo/search/mediasearch/image-08.html

YahooSeeker/M1A1-R2D2
Yahoo! Mobile
Info: http://help.yahoo.com/l/us/yahoo/search/indexing/webcrawler-13.html

Yahoo! Slurp
Yahoo! Slurp
Info: http://help.yahoo.com/l/us/yahoo/search/webcrawler/

Yahoo-VerticalCrawler
Yahoo! Vertical Crawler

YandexAntivirus
Yandex Antivirus
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexBlog
Yandex Blog
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexBot
Yandex Bot
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexCatalog
Yandex Catalog
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexDirect
Yandex Direct
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexFavicon
Yandex Favicon
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexImageResizer
Yandex ImageResizer
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexImages
Yandex Images
Info: http://yandex.com/bots

YandexMedia
Yandex Media
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexMetrika
Yandex Metrika
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexNews
Yandex News
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexPagechecker
Yandex Pagechecker
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexVideo
Yandex Video
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexWebmaster
Yandex Webmaster
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

YandexZakladki
Yandex Zakladki
Info: http://yandex.com/bots
<email>support@search.yandex.com</email>

Yanga WorldSearch Bot
Yanga
Info: http://www.yanga.co.uk/
<email>support@yanga.co.uk </email>

Yanga WorldSearch Bot
Yanga WorldSearch Bot
Info: http://www.yanga.co.uk

YebolBot
YebolBot
<email>yebolbot@gmail.com</email>

Yeti
Yeti
Info: http://help.naver.com/robots/

yeti
Yeti
Info: http://help.naver.com/robots/
<email>yetibot@naver.com</email>

Yeti/1.0
Yeti/1.0
Info: +http://help.naver.com/robots/

YodaoBot
Yodao
Info: http://www.yodao.com/help/webmaster/spider/

YoudaoBot
Youdao
Info: http://www.youdao.com/help/webmaster/spider/

YRSpider
YunRang
Info: http://www.yunrang.com/yrspider.html

zeus
Zeus Internet Marketing
Info: http://www.cyber-robotics.com/