What crawls this site?

A live look at every bot, crawler, and scanner that has visited leedriggers.com — search engines, AI crawlers, SEO tools, and everything else that shows up.

Crawl Surface Bot endpoints → Core Web Vitals Page Speed →
12,116Total requests
35Distinct bots
20Paths hit
15Countries

Activity (last 30 days)

Human / unknown Scanner / probe Google (by ASN) AhrefsBot MajesticBot PetalBot Amazonbot YandexBot

Distribution Breakdown

Traffic by Bot Segment

<1% Search Engines
1% AI Crawlers
2% SEO Tools
<1% Social / Fetchers
13% Scanners & Probes
<1% Other / Unclassified
80% Human / Browser
<1% Dev Tools & Scripts

Search Engine Share

40% YandexBot
21% Bingbot
17% Applebot
13% Googlebot
4% Baiduspider
3% SeznamBot
1% DuckDuckBot

AI Crawler Share

40% PetalBot
22% Amazonbot
14% GPTBot
8% OpenAI SearchBot
4% Meta (Scraper)
4% Claude Code
4% CCBot
2% ClaudeBot

Content Types Targeted

41% HTML / Pages
8% XML Data
11% Robots / TXT
40% Other

Most Crawled Paths

28% /
11% /robots.txt
8% /sitemap-main.xml
4% /admin.php
4% /about.php
4% /file.php
4% /about/
3% /blog/

Top Origin Countries

18% FR
13% JP
11% NL
9% US
8% HK
7% KR
6% BR
6% SG

Infrastructure & Velocity

7-Day Rolling Velocity

Total crawl volume over the last week.

Crawler Controls

Hits checking permissions & site structures.

robots.txt249
*.xml (Sitemaps)178
/sitemap-main.xml178

ASN Identifications

Known company IP ranges matched by network block — not user-agent. These crawlers deliberately omit or rotate their UA string.

Google 221 hits
Anthropic 2 hits

Bot Directory

Search Engines 75 hits

YandexBot 30
first 2026-06-30 · last 2026-07-03
Bingbot 16
first 2026-06-27 · last 2026-07-05
Applebot 13
first 2026-06-30 · last 2026-07-05
Googlebot 10
first 2026-06-26 · last 2026-07-04
Baiduspider 3
first 2026-06-30 · last 2026-06-30
SeznamBot 2
first 2026-06-29 · last 2026-06-29
DuckDuckBot 1
first 2026-07-04 · last 2026-07-04

AI Crawlers 169 hits

PetalBot 67
first 2026-06-28 · last 2026-07-05
Amazonbot 38
first 2026-06-26 · last 2026-07-05
GPTBot 23
first 2026-06-27 · last 2026-07-05
OpenAI SearchBot 14
first 2026-06-27 · last 2026-07-05
Meta (Scraper) 7
first 2026-07-03 · last 2026-07-04
Claude Code 6
first 2026-06-27 · last 2026-06-27
CCBot 6
first 2026-06-30 · last 2026-07-04
ClaudeBot 3
first 2026-06-30 · last 2026-06-30
DeepSeekBot 2
first 2026-06-30 · last 2026-06-30
PerplexityBot 1
first 2026-06-30 · last 2026-06-30
Bytespider 1
first 2026-07-02 · last 2026-07-02
ChatGPT (User) 1
first 2026-07-04 · last 2026-07-04

Other Bot Segments

SEO tools, social fetchers, scanners, and unclassified traffic. "Other / Unclassified" includes UAs that don't match any of the 1,200+ known bot signatures — typically generic scrapers, headless browsers, or truly blank user-agents.

SEO Tools 276

AhrefsBot 191
MajesticBot 79
SemrushBot 2
MozBot 2
Barkrowler 2

Social / Fetchers 20

Facebook 11
Twitterbot 9

Scanners & Probes 1,602

Scanner / probe 1,601
Security Scanner 1

Other / Unclassified 24

Other bot 24

Human / Browser 9,725

Human / unknown 9,714
Chrome 10
Safari 1

Dev Tools & Scripts 2

Python HTTP 2

ASN-identified Bots 223

Google (by ASN) 221
Anthropic (by ASN) 2

Top paths

PathHits
/644
/robots.txt249
/sitemap-main.xml178
/admin.php88
/about.php87
/file.php81
/about/81
/blog/80
/goods.php79
/classwithtostring.php79
/contact/79
/adminfuns.php78
/as.php69
/xmlrpc.php66
/chosen.php65

Countries

CodeHits
FR2,078
JP1,495
NL1,224
US1,054
HK943
KR765
BR693
SG655
CA615
CH515
NO400
IN370
DE347
ID226
PL216

Updated 2h ago. Aggregate data only — no visitor information shown.