Cloudflare AI Bot Controls Can Affect Googlebot: What Website Owners Need to Check

Cloudflare has updated its AI traffic control framework, introducing separate settings for Search, Training, and Agent access. A critical takeaway for website owners is that blocking 'Training' can inadvertently restrict mixed-use crawlers like Googlebot, Applebot, and BingBot on ad-supported pages. Because these crawlers perform both search indexing and AI-related tasks, broad blocking policies may negatively impact a site's organic search visibility. Cloudflare now classifies major search engines as 'Accountable' crawlers, suggesting that users should specifically utilize the 'Disallow AI Training' setting to restrict model training while maintaining search indexing capabilities. The company also introduced 'Bot Preference Sync' to keep robots.txt files aligned with these dashboard settings. Web teams are advised to audit their current configurations, distinguish between search and training permissions, and monitor search console data to ensure that efforts to prevent AI scraping do not accidentally hinder essential search engine discovery.