Tag
Technical SEO
5 posts
2026-09-08Cloudflare splits AI bots in three on 15 September
On 15 September Cloudflare replaces one AI bots switch with three categories and new defaults. Check what your site is set to before it is set for you.
2026-08-01A five-star hotel, a 62/100 site: the Shelbourne audit
We pointed Site Pulse at theshelbourne.com. It has an llms.txt and welcomes every AI crawler, then tells them its author is called originate_admin.
2026-05-21The web went majority-machine - can AI read your site?
Bots passed humans in June 2026 at 57.5% of web traffic - and 69% of AI crawlers can't run JavaScript. Client-side rendering leaves AI a blank page.
2026-04-02'Block all AI bots' is now actively costing you visibility
AI vendors split crawlers into training vs search bots. Block everything and you vanish from ChatGPT search. How to block training bots and stay visible.
2026-02-18llms.txt does nothing for Google Search - so what's it for?
Google's docs confirm Search ignores llms.txt and adoption studies show near-zero effect. So why do Stripe, Vercel and Anthropic still ship it?
