
Forty-plus distinct AI crawlers are hitting sites right now. Here's the robots.txt I actually run, how it changed twice after bad calls, and what the bot landscape looks like as of…
Read the field note ↗
Which anti-scraping techniques actually hold up in 2026. An honest look at tarpit patterns, Cloudflare and Fastly configurations, what survives adversarial AI scrapers, and what…
Read the field note ↗
Published 19 May 2026. Written from direct experiment data across 43 client and personal sites, Q4 2024 through Q1 2026.
Read the field note ↗
Published 19 May 2026. All numbers come from my own monitoring setup across 43 sites unless otherwise noted. I am not paid by Profound, AthenaHQ, or any vendor mentioned here.
Read the field note ↗
The email came in October 2025. A head of content at a mid-market SaaS company — project management tooling, boring in the best way — wanted to know whether OpenAI had trained on…
Read the field note ↗
In September 2025, I was doing a routine GSC audit for a SaaS client in the project management space when a number stopped me cold: 14.2 words . That was the average query length…
Read the field note ↗
In late November 2025, I was doing a routine log review for a mid-size SaaS documentation site I manage. We have about 340,000 sessions per month, a comfortable GA4 setup, and a…
Read the field note ↗
May 20, 2026. Google AI Mode has been out of Labs for the better part of a year now, and I am still watching SEOs treat it like a slightly smarter version of AI Overviews. That is…
Read the field note ↗
I want to tell you about a number that confused me for six weeks. In late October 2025, one of my clients (a mid-size recipe and food content publisher) started seeing a persistent…
Read the field note ↗
In the back half of 2025, I was brought in to help a mid-size travel brand retrofit alt text onto roughly 340,000 images — most of them AI-generated illustrations their content team…
Read the field note ↗
I am writing this on May 20, 2026, and I have 31 live Framer sites generating organic traffic right now. Eleven of those were built entirely or substantially using Framer AI or the…
Read the field note ↗