What crawls this site?

A live look at every bot, crawler, and scanner that has visited leedriggers.com — search engines, AI crawlers, SEO tools, and everything else that shows up.

Crawl Surface Bot endpoints → Core Web Vitals Page Speed →
16,464Total requests
43Distinct bots
20Paths hit
15Countries

Activity (last 30 days)

Human / unknown Scanner / probe Other bot Chrome No User-Agent Google (by ASN) AhrefsBot PetalBot

Distribution Breakdown

Traffic by Bot Segment

<1% Search Engines
2% AI Crawlers
3% SEO Tools
<1% Social / Fetchers
10% Scanners & Probes
8% Other / Unclassified
67% Human / Browser
7% Dev Tools & Scripts

Search Engine Share

29% Bingbot
29% Applebot
21% YandexBot
13% Googlebot
3% Baiduspider
3% DuckDuckBot
1% SeznamBot

AI Crawler Share

54% PetalBot
14% Amazonbot
11% GPTBot
7% Meta (Scraper)
6% OpenAI SearchBot
3% CCBot
2% Claude Code
1% ClaudeBot

Content Types Targeted

68% HTML / Pages
5% XML Data
7% Robots / TXT
21% Other

Most Crawled Paths

54% /
7% /robots.txt
5% /sitemap-main.xml
3% /about/
3% /contact/
3% /blog/
2% /admin.php
2% /about.php

Top Origin Countries

18% US
14% FR
11% JP
11% BR
8% NL
7% CA
6% HK
6% SG

Infrastructure & Velocity

7-Day Rolling Velocity

Total crawl volume over the last week.

Crawler Controls

Hits checking permissions & site structures.

robots.txt249
*.xml (Sitemaps)178
/sitemap-main.xml178

ASN Identifications

Known company IP ranges matched by network block — not user-agent. These crawlers deliberately omit or rotate their UA string.

Google 304 hits
Anthropic 2 hits

Bot Directory

Search Engines 145 hits

Bingbot 42
first 2026-06-27 · last 2026-07-17
Applebot 42
first 2026-06-30 · last 2026-07-16
YandexBot 31
first 2026-06-30 · last 2026-07-16
Googlebot 19
first 2026-06-26 · last 2026-07-17
Baiduspider 5
first 2026-06-30 · last 2026-07-16
DuckDuckBot 4
first 2026-07-04 · last 2026-07-16
SeznamBot 2
first 2026-06-29 · last 2026-06-29

AI Crawlers 379 hits

PetalBot 206
first 2026-06-28 · last 2026-07-16
Amazonbot 52
first 2026-06-26 · last 2026-07-16
GPTBot 40
first 2026-06-27 · last 2026-07-16
Meta (Scraper) 25
first 2026-07-03 · last 2026-07-15
OpenAI SearchBot 22
first 2026-06-27 · last 2026-07-16
CCBot 10
first 2026-06-30 · last 2026-07-16
Claude Code 6
first 2026-06-27 · last 2026-06-27
ClaudeBot 5
first 2026-06-30 · last 2026-07-17
ChatGPT (User) 5
first 2026-07-04 · last 2026-07-16
DeepSeekBot 2
first 2026-06-30 · last 2026-06-30
PerplexityBot 2
first 2026-06-30 · last 2026-07-16
Bytespider 2
first 2026-07-02 · last 2026-07-16
GoogleOther 2
first 2026-07-16 · last 2026-07-16

Other Bot Segments

SEO tools, social fetchers, scanners, and unclassified traffic. "Other / Unclassified" includes UAs that don't match any of the 1,200+ known bot signatures — typically generic scrapers, headless browsers, or truly blank user-agents.

SEO Tools 503

AhrefsBot 282
MajesticBot 108
MozBot 64
SemrushBot 47
Barkrowler 2

Social / Fetchers 27

Facebook 18
Twitterbot 9

Scanners & Probes 1,695

Scanner / probe 1,692
Security Scanner 3

Other / Unclassified 1,236

Other bot 1,236

Human / Browser 11,079

Human / unknown 9,714
Chrome 1,157
Safari 88
Browser 87
Firefox 20
Edge 12
Samsung Browser 1

Dev Tools & Scripts 1,094

No User-Agent 922
Go HTTP 100
curl 56
Python HTTP 16

ASN-identified Bots 306

Google (by ASN) 304
Anthropic (by ASN) 2

Top paths

PathHits
/2,070
/robots.txt249
/sitemap-main.xml178
/about/104
/contact/102
/blog/99
/admin.php88
/about.php87
/resume/82
/file.php81
/goods.php79
/classwithtostring.php79
/adminfuns.php78
/as.php69
/wp-content/uploads/68

Countries

CodeHits
US2,778
FR2,145
JP1,752
BR1,738
NL1,262
CA1,167
HK948
SG875
KR769
CH515
DE462
NO401
IN374
ID226
PL216

Updated 23m ago. Aggregate data only — no visitor information shown.