How This Site Was Built
A topic-by-topic record of every major decision, system, and iteration — stack choices, crawl intelligence, performance monitoring, and AI readiness. First-person, specific, honest about tradeoffs.
10 entriesBuilt this changelog itself: a Keystatic-managed content collection with topic-based entries, a /changelog.md export for AI agents, a dedicated changelog.xml sitemap — with entry dates verified against the actual git history.
Implemented /entitymap.json (EntityMap v1.0) with 8 entities — Person, Organization, and 6 Concepts — with structured relations, hasChunks text extracts, and a human-readable /entitymap/ companion page.
Built the site's content discovery infrastructure: dual RSS feeds (blog + notes), Pagefind client-side search with post-build indexing, and a static blogroll with live descriptions fetched at build time.
Built /stack/ with a three-tier data pipeline diagram, /uses/ documenting hardware and software across machines, networking, finance, and fitness, and /blogroll/ with build-time-fetched meta descriptions.
Built the machine-readable surface that AI crawlers and agents need to navigate and understand the site: llms.txt, ai-catalog.json, entitymap.json, and a public /crawl-surface/ dashboard showing what's being accessed and by whom.
Built botClassify.js and botGroups.js — a UA-string-first classifier mapping 60+ known crawlers to named bots and groups, with ASN-based fallback for bots that spoof or omit user-agent strings.
Built the site's visual identity: LD monogram SVG favicon with light/dark inversion, no-flash dark mode via localStorage + inline script, system font stack with zero external requests, and a CrawlWidget in every page footer.
Integrated the web-vitals library to capture LCP, INP, CLS, FCP, and TTFB from real browser sessions, beaconing to /api/cwv → D1 → KV snapshot pipeline with p50/p75/p95 percentiles and indefinite retention.
Built a D1 + KV bot monitoring pipeline that logs every HTTP request, classifies crawlers by UA and ASN, and surfaces aggregated data on public dashboards — with zero D1 reads on page load.
Chose Astro 5 SSR with the Cloudflare Workers adapter as the runtime — no origin server, every request processed at the edge, static pages prerendered at deploy and dynamic routes served by Workers.