Cloudflare Launches Markdown for AI Bots: Here’s What It Means for Your Website

Cloudflare has rolled out a new additional feature that can clearly enhance and change the way websites interact with AI crawlers.

It is called a Markdown for Agents and the update automatically converts HTML pages into markdown when AI systems request it.

What is the goal? To serve lighter and cleaner content to AI bots without creating separate pages.

What is the Markdown for Agents?

Markdown for agents works through a standard web method called HTTP content negotiation and it functions in the following way:

An AI crawler sends a request with the header: Accept: text/ markdown

Cloudflare intercepts the request at the network edge

• It fetches the original HTML page from the server
• It converts that page into markdown
• The markdown version is delivered to the AI bot

No extra coding is required in this and no duplicate pages are needed. The same URL simply serves a different format based on what the client requests.

Cloudflare says this treats AI agents as “first-class citizens” alongside human visitors.

Why this Markdown?

Markdown is much lighter than HTML. Cloudflare shared an example from its own blog:

• HTML version: 16,180 tokens
• Markdown version: 3,150 tokens

This is a major reduction.

As Cloudflare puts it, sending raw HTML to an AI is like “paying by the word to read packaging instead of the letter inside.”

For AI tools with limited context windows, fewer tokens mean faster processing and lower costs.

Available on Paid Plans

The feature is currently in beta and comes at no extra cost for:

• Pro
• Business
• Enterprise
• SSL for SaaS customers

Site owners can enable it per zone through the Cloudflare dashboard.

Some AI tools already request markdown using standard headers. Cloudflare specifically talks of the  Claude Code and OpenCode.

Each markdown response also includes an x-markdown-tokens header. This helps developers estimate token usage and plan chunking strategies.

The Google Debate: As we Know It

The launch comes shortly after criticism from John Mueller of Google.

Mueller recently called the idea of serving markdown to AI bots “a stupid idea.” He questioned whether bots can parse markdown links properly and warned about potential cloaking issues.

However, Cloudflare’s method is different from what Mueller criticized originally.

Some site owners create separate markdown pages and serve them based on user-agent detection. This practice can raise cloaking concerns.

Cloudflare instead uses content negotiation. When the client requests a particular format, the server responds accordingly henceforth. The information then remains the same with only the changes in the format.

Google defines cloaking as showing different content to manipulate rankings. In this case, the content is not different, just formatted differently.

Google has not yet clarified whether markdown served through content negotiation falls under cloaking rules.

New AI Traffic Insights

Cloudflare also added content-type tracking to Cloudflare Radar for AI bot traffic.

Website owners can now:

• See which content types AI bots receive
• Filter data by specific crawlers
• Track markdown responses by bot

Cloudflare demonstrated tracking for OpenAI’s OAI-SearchBot.

The data is available through public APIs and the Data Explorer.

Important Warning: Content Signals

When enabled, Markdown for Agents automatically sets a header:

Content-Signal: ai-train=yes, search=yes, ai-input=yes

This signals that your content can be used for AI training and search purposes.

Publishers who are cautious about AI usage need to review these defaults carefully before enabling the feature.

Why This Matters

If your website already runs on Cloudflare, you can activate markdown conversion with a single toggle.

There’s no need to build parallel pages. No extra infrastructure is also needed.

But it’s opt-in and limited to paid plans.