AI Crawl Control - New Robots.txt tab for tracking crawler compliance
UpdateUnverifiedAdded Sep 24, 2026
AI Crawl Control now includes a Robots.txt tab that provides insights into how AI crawlers interact with your robots.txt files. What's new The Robots.txt tab allows you to: Monitor the health status of robots.txt files across all your hostnames, including HTTP status codes, and identify hostnames that need a robots.txt file.
Topics: Governance, Security, BI, Observability, Developer tools
More Cloudflare developer platform releases
Every Cloudflare developer platform release| Date | Release | Type |
|---|---|---|
| Oct 20 | Gateway - Schedule DNS policies from the UIunverified Update | Update |
| Oct 20 | WAF - WAF Release - 2025-10-20unverified Beta | Beta |
| Oct 23 | WAF - WAF Release - 2025-10-23 - Emergencyunverified Update | Update |
| Oct 23 | Workers AI - Workers AI Markdown Conversion: New endpoint to list supported formatsunverified Update | Update |
| Oct 23 | Workers - Workers Preview URL default behavior now matches your workers.dev settingunverified Preview | Preview |
| Oct 24 | WAF - WAF Release - 2025-10-24 - Emergencyunverified Update | Update |
| Oct 24 | Workers - Automatic resource provisioning for KV, R2, and D1unverified Beta | Beta |
| Oct 24 | Workers - Build TanStack Start apps with the Cloudflare Vite pluginunverified Update | Update |
Also shipped on Oct 21, 2025
Cloudflare in October 2025| Date | Company | Release | Product | Type |
|---|---|---|---|---|
| Oct 21 |  Sentry | AI-Powered User Feedback Summary and Categories in Open Betaunverified | Sentry | Beta |
| Oct 21 |  Netlify | Next.js 16 is ready to deploy on Netlifyunverified | Netlify | Update |
| Oct 21 | Netlify | Enforce Git-based workflows for production deploysunverified | Netlify | Update |
| Oct 21 |  Grafana Labs | Favorite data sourcesunverified | Grafana Cloud | Update |
| Oct 21 |  Shopify | Improved concurrency handling in the Cart AJAX API and Storefront Cart GraphQL APIunverified | Shopify developer platform | Update |
| Oct 21 | Shopify | New: Unlisted Product Statusunverified | Shopify developer platform | Update |
Sources: each vendor's own release notes, changelogs and GitHub releases, read daily to monthly by how often it posts. Logos via logo.dev; trademarks belong to their owners.