TLDR;
Cloudflare launched Disallow AI Training to let brands refuse training use while retaining search access for designated accountable crawlers. That offers a more precise choice than blocking everything. Check each operator's commitments and the setting's effect on discovery before relying on it.
What happened
Cloudflare introduced Disallow AI Training and an Accountable designation for mixed-use crawler operators. The setting permits accountable crawlers to continue search access while publishing a no-training preference and blocking other training crawlers; some operator capabilities rely on time-bound commitments, and blocking settings still have different effects.
Why it matters
A blanket block can restrict valuable discovery alongside unwanted content reuse. Cloudflare's approach addresses that trade-off for designated mixed-use operators, but the designation and time-bound commitments deserve scrutiny. Enterprise brands need a documented access decision, rather than assuming one switch provides universal and permanent control over all AI use.
A more precise setting can help an enterprise preserve discovery while expressing a refusal of training use. Its value depends on the details: which operators are designated accountable, what commitments apply and what the chosen configuration actually allows or blocks. Mixed-use crawlers make this harder because the same access route can serve purposes with different commercial value. The enterprise should therefore treat the designation as part of a maintained decision, not a permanent assurance that needs no review. This development is distinct from charging for content or monitoring assistant recommendations; those mechanisms address other questions. The practical judgement is to align declared preferences, technical access and operator behaviour with the business objective, while keeping the limits of time-bound commitments clear to the people approving the policy.
How your brand can benefit / be affected
Review current bot settings and classify the access the business wants to preserve: search discovery, answer retrieval, user-triggered assistance and training. Check the accountable operator list and the commitments on which the setting relies.
Test actual request behaviour and monitor valuable discovery after any change. Keep evidence of the chosen configuration and review operator commitments as they evolve. Treat training preference, technical blocking and content compensation as separate controls with separate consequences.
Capture the current configuration and identify the discovery routes the organisation wants to preserve. Review the accountable operator list and the stated commitments, including any dates that require follow-up. Classify the main content types so public product information, expensive editorial work and restricted material are not automatically given the same treatment. Agree the intended training preference with the appropriate content and commercial owners. Then test representative requests to confirm the setting's supported effect. These checks should establish what changed in practice, rather than infer the result from the label on a switch or assume that every form of AI reuse is covered.
Monitor useful discovery and observed request behaviour after the change, keeping the implementation date and affected scope available for investigation. Revisit the policy when operator commitments or Cloudflare's supported behaviour change. Keep a route for content owners to raise exceptions where a broad configuration conflicts with a specific distribution or licensing need. Do not present a published preference, a technical block and compensation as interchangeable protections. Each requires its own evidence and review. The enterprise benefit is a more deliberate balance between unwanted training use and valuable access, with enough records to explain the decision and revise it when the underlying commitments no longer support the original commercial judgement.
News date: 15 September 2026. Editorial review: 16 September 2026. Analysis includes subsequent developments where stated.