What crawls this site?
A live look at every bot, crawler, and scanner that has visited leedriggers.com — search engines, AI crawlers, SEO tools, and everything else that shows up.
67,409Total requests
60Distinct bots
20Paths hit
15Countries
Activity (last 30 days)
Chrome No User-Agent Human / unknown Other bot curl Scanner / probe AhrefsBot Google (by ASN)
Distribution Breakdown
Traffic by Bot Segment
<1% Search Engines
5% AI Crawlers
3% SEO Tools
<1% Social / Fetchers
6% Scanners & Probes
14% Other / Unclassified
44% Human / Browser
26% Dev Tools & Scripts
Search Engine Share
51% Googlebot
22% Bingbot
12% YandexBot
9% Applebot
3% SeznamBot
2% Baiduspider
<1% DuckDuckBot
AI Crawler Share
23% ClaudeBot
17% Perplexity (User)
14% PetalBot
13% ChatGPT (User)
7% Amazonbot
7% GPTBot
4% OpenAI SearchBot
4% Amzn SearchBot
Content Types Targeted
93% HTML / Pages
1% XML Data
2% Robots / TXT
4% Other
Most Crawled Paths
81% /
2% /blog/
2% /robots.txt
2% /contact/
2% /about/
2% /resume/
1% /wp-content/uploads/
1% /sitemap-main.xml
Top Origin Countries
38% US
11% FR
9% BR
5% DE
5% NO
4% NL
4% SE
4% ES
Infrastructure & Velocity
7-Day Rolling Velocity
Total crawl volume over the last week.
Crawler Controls
Hits checking permissions & site structures.
robots.txt249
*.xml (Sitemaps)178
/sitemap-main.xml178ASN Identifications
Known company IP ranges matched by network block — not user-agent. These crawlers deliberately omit or rotate their UA string.
Google 827 hits
Anthropic 2 hits
Bot Directory
Search Engines 596 hits
Googlebot 306
Bingbot 130
YandexBot 73
Applebot 51
SeznamBot 20
Baiduspider 11
DuckDuckBot 5
AI Crawlers 3,348 hits
ClaudeBot 762
Perplexity (User) 563
PetalBot 466
ChatGPT (User) 429
Amazonbot 234
GPTBot 223
OpenAI SearchBot 142
Amzn SearchBot 120
PerplexityBot 90
GoogleOther 71
Google Extended 70
Meta (Scraper) 68
CCBot 26
Bytespider 19
DeepSeekBot 12
Claude SearchBot 9
Diffbot 8
Cohere 7
Claude Code 6
Mistral (User) 6
Applebot Extended 6
xAI (Grok) 5
Claude (User) 4
Google VertexBot 2
Other Bot Segments
SEO tools, social fetchers, scanners, and unclassified traffic. "Other / Unclassified" includes UAs that don't match any of the 1,200+ known bot signatures — typically generic scrapers, headless browsers, or truly blank user-agents.
SEO Tools 1,931
AhrefsBot 1,213
MajesticBot 263
SemrushBot 235
MozBot 134
Screaming Frog 73
Barkrowler 11
DataForSEO 1
SeobilityBot 1
Social / Fetchers 54
Facebook 45
Twitterbot 9
Scanners & Probes 3,793
Scanner / probe 3,783
Security Scanner 10
Other / Unclassified 9,353
Other bot 9,353
Human / Browser 29,707
Chrome 17,861
Human / unknown 9,714
Firefox 804
Safari 748
Browser 445
Edge 128
Opera 5
Samsung Browser 1
Internet Explorer 1
Dev Tools & Scripts 17,794
No User-Agent 13,131
curl 4,038
Go HTTP 555
Python HTTP 70
Headless Browsers 4
Headless Browser 4
ASN-identified Bots 829
Google (by ASN) 827
Anthropic (by ASN) 2
Top paths
| Path | Hits |
|---|---|
/ | 11,454 |
/blog/ | 256 |
/robots.txt | 249 |
/contact/ | 232 |
/about/ | 229 |
/resume/ | 228 |
/wp-content/uploads/ | 186 |
/sitemap-main.xml | 178 |
/projects/ | 174 |
/.well-known/ | 111 |
/resources/ | 94 |
/cgi-bin/ | 91 |
/admin.php | 88 |
/about.php | 87 |
/file.php | 81 |
Countries
| Code | Hits |
|---|---|
| US | 23,874 |
| FR | 7,134 |
| BR | 6,011 |
| DE | 3,026 |
| NO | 2,872 |
| NL | 2,721 |
| SE | 2,284 |
| ES | 2,229 |
| CH | 2,208 |
| SG | 2,207 |
| CA | 2,163 |
| JP | 2,096 |
| PL | 2,009 |
| IT | 1,719 |
| HK | 1,046 |
Updated 1h ago. Aggregate data only — no visitor information shown.