The short version
- 191 of 5,902 live top websites (3.2%) returned Markdown when asked for it; 71.2% of them are served through Cloudflare.
- The Markdown was small: a median 13,081 bytes against 401,073 bytes for the same homepage as HTML, a median reduction of 96.1%.
- Only 28 homepages (0.5% of HTML homepages) link to a Markdown version of themselves with a link tag, the other way to offer Markdown.
- 45.8% of homepages carry JSON-LD structured data, 36.1% describe the organization behind the site, and 28.2% list sameAs links to profiles elsewhere; 54.2% carry no JSON-LD at all.
- Of the sites that serve Markdown, 43.5% also publish a valid llms.txt: the same small group is adopting every agent-facing format at once.
- The Markdown is a lighter page, not a copy. Only 2 of 186 Markdown homepages contained everything their HTML did. The median one kept 83.8% of the HTML’s sentences and 29.9% of its links, mostly by leaving out menus and footers, and 23 of 178 kept less than 20% of the sentences.
What we measured
For each of the 5,902 domains in the Tranco top 10,000 whose homepage responded with a status below 400, we requested the homepage with the header an agent that prefers Markdown sends: “Accept: text/markdown, text/html;q=0.9, /;q=0.8”. A site that supports Markdown content negotiation answers with Content-Type text/markdown; every other site answers with its usual HTML.
For the sites that answered with Markdown we then requested the same homepage asking for HTML, to compare the two. For the 5,710 homepages returned as HTML we read the head for a link to a Markdown alternate and parsed every JSON-LD block, including nested @graph objects.
Findings
Markdown on request is rare, and mostly a platform feature
| Signal | Sites | Share |
|---|---|---|
| Returned Markdown for Accept: text/markdown | 191 | 3.2% of live sites |
| Of those, also sent “Vary: Accept” so caches keep the versions apart | 183 | 95.8% of Markdown sites |
| Of those, Markdown starts with a metadata block (front matter) | 152 | 79.6% of Markdown sites |
| HTML homepage links to a Markdown version | 28 | 0.5% of HTML homepages |
Server headers show where the Markdown comes from: 136 of the 191 sites (71.2%) are served through Cloudflare, which launched “Markdown for Agents” as an optional feature in February 2026, followed by Vercel (14 sites), nginx (7), Framer (6) and Netlify (5). Early adopters include cloudflare.com, wordpress.org, sentry.io, zendesk.com, gitlab.com, jotform.com, intercom.com, mixpanel.com, typeform.com, siemens.com, elsevier.com, laravel.com, asana.com and miro.com.
The Markdown version is a fraction of the size
For 185 sites we could fetch both versions. The median Markdown homepage was 13,081 bytes; the median HTML homepage for the same sites was 401,073 bytes. The median reduction was 96.1%, and for three sites in four it was at least 90.0%. For an agent that is billed by the token, that is a large difference in what each page costs to read.
The Markdown is rarely the same page
On 28 September we requested each of the 191 Markdown-serving homepages again, twice asking for HTML and twice asking for Markdown, and compared the two versions: numbers, dates, prices, headings, links, name-like phrases and sentences. Only items that appeared in both requests of the same version were compared, so content that changes on every load (a date, a rotating banner) does not count as a difference. Two requests of the same version agreed on a median of 100% of items, so the differences below are between the versions, not between requests.
| HTML versus Markdown, same homepage | Result |
|---|---|
| Markdown contains everything the HTML does (and the reverse) | 2 of 186 (1.1%; 95% interval 0.3% to 3.8%) |
| Median share of the HTML’s sentences found in the Markdown | 83.8% (178 pages with sentences) |
| Median share of the HTML’s links found in the Markdown | 29.9% (182 pages with links) |
| Markdown keeps less than 20% of the HTML’s sentences | 23 of 178 |
| Same comparison with menus, headers and footers removed from the HTML (exploratory) | 7 of 186 equivalent; median 98.3% of sentences kept |
Most of the gap is page furniture. Platform conversions drop navigation menus, footers and social links, which is much of what makes the Markdown small. For a minority the Markdown is a different document written for agents: sentry.io and cloudflare.com, for example, answer a Markdown request with a short page about their product for AI tools rather than a conversion of the homepage. An agent reading those sites sees different text from a person visiting them.
Structured data travels unevenly. Of the 137 pages that carry JSON-LD, the Markdown includes it for 76. On 127 of the 137, the JSON-LD holds facts that the visible text does not, such as addresses, founding dates and profile names, so an agent that receives Markdown without it loses those facts.
The feature is not fixed. Three of the 191 sites (circle.com, webflow.com and webflow.io) did not return Markdown on at least one request two days later. Among the rest, the Markdown was byte-for-byte identical across two requests for 176 of 186 pages.
Homepage structured data
| JSON-LD on the homepage | Share of 5,710 HTML homepages |
|---|---|
| Any JSON-LD | 45.8% |
| Organization (or a subtype such as NewsMediaOrganization or LocalBusiness) | 36.1% |
| WebSite | 31.0% |
| sameAs links to other profiles | 28.2% |
| A JSON-LD block that fails to parse | 0.6% |
The most common types were Organization (1,783 homepages), WebSite (1,770), SearchAction (1,226), ImageObject (1,222) and WebPage (921). Adoption barely changes with popularity: 43.8% of the top 1,000 homepages carry JSON-LD, against 45.3% of sites ranked 1,001 to 5,000 and 46.5% of sites ranked 5,001 to 10,000.
A handful of sites opt out in the page itself
18 homepages (0.3%) carry a “noai” value in their robots meta tag, a non-standard signal some platforms use to ask AI systems not to use the content.
How this compares with other studies
We found no earlier count of websites that answer a Markdown request. What has been published measures the other side, the agents: Checkly tested seven AI agents in February 2026 and found “only 3 out of 7 agents request markdown” (Claude Code, Cursor and OpenCode). Cloudflare’s launch post measured one page at 16,180 tokens as HTML against 3,150 as Markdown, an “80% reduction in token usage”; our median byte reduction across 185 homepages is larger, which may partly reflect that homepages carry more markup than article pages (we did not measure article pages). Vercel reported a 99.37% reduction in payload size for one page.
For structured data, the HTTP Archive Web Almanac 2025 (whole web, July 2025) found JSON-LD on 43% of desktop home pages and the Organization type on 26.74%. Our top-10,000 figures are similar for JSON-LD (45.8%) and higher for Organization (36.1%); the two samples differ (top sites against the whole web), so part of the gap may reflect larger sites investing more in describing who they are.
Sources: Checkly (opens in a new tab); Cloudflare, Markdown for Agents (opens in a new tab); Vercel (opens in a new tab); HTTP Archive Web Almanac 2025 (opens in a new tab).
What this means
What follows is our reading of the figures above; none of it was measured directly.
- Markdown on request is the cheapest agent upgrade available. For sites on a platform that supports it, it is a setting rather than a project, and it cuts what an agent must download by an order of magnitude. Few agents ask for Markdown today; in Checkly’s test, the ones that did were coding tools.
- Entity data is still missing on most homepages. More than half of top homepages give machines no structured statement of who is behind the site. For a brand that wants AI assistants to describe it correctly, an Organization record with sameAs links to its real profiles is the basic step.
- Check what your Markdown says before an agent does. A platform setting produces the Markdown, and it decides what to keep. For most sites that means losing menus, which is the point; for some it means losing most of the page, or the facts that sit only in the JSON-LD. Reading your own Markdown once takes minutes.
- The same sites adopt every format. The overlap between Markdown serving and llms.txt suggests these formats are being taken up by the same technical teams rather than spreading evenly. That is an opening for brands in categories where competitors have not started.
Methodology
- Sample: Tranco list L5PZ4, top 10,000 domains; 5,902 with a live homepage (status below 400, more than 200 bytes) on 26 September 2026.
- Markdown test: one HTTPS GET of the homepage with “Accept: text/markdown, text/html;q=0.9, /;q=0.8”; Markdown = response Content-Type text/markdown or text/x-markdown. Size comparison: a second request asking for HTML, same day.
- Structured data: every script of type application/ld+json parsed, types collected recursively. Homepages over 600 KB were re-fetched in full (790 of 807 succeeded); for the other 17 only the first 600 KB was searched.
- Serving platform: the Server response header, which a site can change or hide.
- Representation check (version 1.1): on 28 September 2026, four more requests to each of the 191 Markdown-serving homepages, two asking for HTML and two for Markdown, in a seeded random order. Two versions count as the same content only if every number, date, price, heading, link and name-like phrase of each appears in the other and at least 95% of the sentences do. The protocol was fixed before collection; 186 homepages were analyzed (three no longer served Markdown, one refused the HTML request, one timed out).
- Update schedule: quarterly.
Limitations
- Only homepages were tested; a site may serve Markdown or structured data on other pages.
- Sites that blocked our research user agent are outside the sample.
- A site can serve Markdown for some user agents and not others; we used one request profile.
- This study measures what sites send. It does not show whether AI crawlers ask for Markdown, whether AI search engines retrieve it, or whether it changes what an answer says; those need sites whose format can be switched on and off, which a survey of other people’s sites cannot do.
- Sentence and name matching in the representation check are text-matching proxies, not a judgment of meaning, and no person or model reviewed them. The comparison without menus and footers was added after seeing the results.
Data and downloads
- Every statistic on this page: stats.json
- The 191 sites that served Markdown, with sizes: md_vs_html.json
- The HTML and Markdown comparison for each of the 186 homepages: s3_md_html_pairs.csv
- The comparison protocol, fixed before collection: s3_representation_protocol.json
- Machine-readable methodology: methodology.json
Version 1.1 (28 September 2026) adds the representation check. No version 1.0 figure changed.
The data is free to reuse with attribution (CC BY 4.0).
To cite: Underneath. (2026). Do websites serve Markdown to AI agents? 2026 data. Underneath Research. https://underneath.agency/research/agent-readable-web-study
Frequently asked questions
What is Markdown content negotiation?
A website that supports it looks at the Accept header of each request. When an AI agent asks for text/markdown, the site returns a Markdown version of the page instead of HTML. Browsers keep getting the normal page.
How many websites serve Markdown to AI agents?
In September 2026, 191 of the 5,902 live websites in the Tranco top 10,000 (3.2%) returned Markdown when asked for it.
How much smaller is the Markdown version of a page?
Across 185 homepages we could compare, the Markdown version was a median 96.1% smaller than the HTML version of the same page.
Is the Markdown version the same as the web page?
Usually not in full. Only 2 of 186 Markdown homepages we checked contained everything their HTML did. The typical one kept 83.8% of the sentences and 29.9% of the links, mainly because menus and footers are left out.
What share of websites use Organization schema on the homepage?
36.1% of the top-site homepages we checked carry an Organization record (or a subtype) in JSON-LD, and 28.2% link to their other profiles with sameAs.