Tag
AI crawlers
4 posts
2026-09-08Cloudflare splits AI bots in three on 15 September
On 15 September Cloudflare replaces one AI bots switch with three categories and new defaults. Check what your site is set to before it is set for you.
2026-08-05Swedish is 0.61% of the training data
English is 45% of CulturaX and Swedish 0.61%. And 14 of 23 Swedish news sites block OpenAI's retrieval bot. How AI builds its picture of Sweden.
2026-05-21The web went majority-machine - can AI read your site?
Bots passed humans in June 2026 at 57.5% of web traffic - and 69% of AI crawlers can't run JavaScript. Client-side rendering leaves AI a blank page.
2026-04-02'Block all AI bots' is now actively costing you visibility
AI vendors split crawlers into training vs search bots. Block everything and you vanish from ChatGPT search. How to block training bots and stay visible.
