ChatGPT and DeepSeek: What Cloudflare's September Block Changes

Cloudflare plans to block AI crawlers from ChatGPT, DeepSeek and other neural networks: from September 2026, the company will begin blocking access by default on all pages with ads, unless the site owner explicitly opens it.

How the opt-in mechanism works

The new rule flips the familiar setup. Until now, pages were open to everyone, and publishers had to close access themselves through robots.txt or firewall settings, one crawler at a time. Now the logic is reversed: a page with ads is closed to AI by default, and a company that wants the data must first get explicit permission from the site owner.

Cloudflare sits between users' browsers and the servers of a huge number of sites worldwide. That is what makes the change large in scale: a single rule at the network infrastructure level covers all of the company's customers, from independent blogs to major media portals, with no extra effort on their part.

• Effective date: September 2026
• All pages with ad inventory fall under the block (unless there is an explicit opt-in)
• The site owner can open access manually, for a specific crawler or for all of them
• Cloudflare handles a significant share of global web traffic

Why publishers couldn't cope on their own

For years, AI companies have collected content to train models, ignoring or technically bypassing restrictions. Robots.txt is not legally binding: OpenAI, Perplexity and other companies have repeatedly been caught violating its directives. Lawsuits are expensive and drag on for years: NYT v. OpenAI and Getty Images v. Stability AI are still unresolved, and small publishers cannot afford to sue at all. Direct licensing deals went only to the largest media groups, which had bargaining power.

Small and mid-sized outlets, such as niche portals, news blogs and specialist media, were left effectively defenseless. Their content flowed into training datasets, ad revenue fell as AI answers displaced search traffic, and there was no compensation of any kind. Cloudflare offers a systemic answer: close access at the infrastructure level, centrally and for all its customers at once.

What this means for AI companies

If labs fail to strike deals with enough publishers before September 2026, their crawlers will lose access to a significant layer of current web content. Training on fresh data will get more expensive, and finding alternative sources (synthetic data, archives, licensing partnerships) will take time and resources.

The precedent matters strategically: if the model of a "CDN as intermediary between AI and content" takes hold at Cloudflare, other infrastructure providers may follow. For the AI industry, that means a gradual shift from effectively free access to data toward a licensing system, the one the publishing market has demanded since the start of the race for training corpora.

"Stop giving the web away for free" is how TNW frames

Cloudflare's position.

What this means

Cloudflare is moving the AI industry's dispute with publishers from the courtroom to the network infrastructure level. From September 2026, crawlers risk losing a significant layer of training data unless they negotiate and pay. For the AI market, this is not a one-off lawsuit but a systemic mechanism that could become a new standard across the CDN industry.

How does Cloudflare block ChatGPT and DeepSeek crawlers?

From September 2026, Cloudflare blocks AI crawlers by default on all sites with ads, unless the site owner opens access through a new opt-in system, replacing the old opt-out approach via robots.txt.

How does Cloudflare's protection against AI crawlers work?

Cloudflare flips the familiar setup: instead of publishers closing access to each crawler themselves, AI scrapers will now be blocked by default. Publishers must explicitly allow access if they agree to share content with companies that train neural networks.

How will the

Cloudflare block affect ChatGPT and DeepSeek training? From September 2026, Cloudflare will begin blocking by default the AI crawlers these services use to collect training data, on all pages with ads.

Can a publisher open access to AI companies?

Yes. The new rule flips the scheme to opt-in: if the site owner explicitly opens access, crawlers will be able to collect data for training neural networks.