LINK

Cloudflare takes a stronger stance on AI crawlers

#seo (1)
Source
The Cloudflare Blog
Visit ↗
Your site, your rules: new AI traffic options for all customers

For our second Content Independence Day, we’re giving website owners finer options to manage AI traffic. Instead of a one-size-fits-all block, all customers can now easily distinguish and manage Search, Agent, and Training bots, alongside the new ability to protect ad-monetized pages.

As of September 15, 2026, the default settings on Cloudflare-managed AI bot tools will block crawlers that don’t distinguish intent between training and RAG.

Google is the biggest offender I think. They use the same User-Agent for training and for prompt-time RAG.