What is llms.txt? The spec, who reads it, and whether you need one

llms.txt is a proposed web standard: a plain-text Markdown file placed at the root of your domain (yoursite.com/llms.txt) that gives large language models a curated, human-readable map of your most important content. It was proposed in September 2024 by Jeremy Howard of Answer.AI, and its goal is narrow: help a model find and read the pages you actually want it to use, without wading through navigation, ads, and JavaScript.

That's the whole idea. It is not a ranking signal, not a magic switch that gets you cited, and not something every AI engine has agreed to read. The confusion around it comes almost entirely from people expecting it to do more than it claims to. Here's what the spec actually says, who reads the file, who ignores it, and whether publishing one is worth your time.

What does the llms.txt spec actually require?

The spec is deliberately minimal, and following it exactly is what makes the file machine-readable. A conforming file uses Markdown in a fixed order:

• An H1 with the name of the site or project. This is the only required element.
• A blockquote with a short summary — the key context a model needs to understand everything below it.
• Zero or more sections of prose or lists (no headings) giving more detail.
• Zero or more H2 sections, each a list of Markdown links to your key pages, optionally with a one-line description after each link.

There's also a convention of publishing llms-full.txt or .md versions of individual pages, so a model can fetch clean Markdown instead of rendered HTML. The structure matters: because the format is predictable, a parser can extract your links and summaries without guessing. You can generate and validate a file with our llms.txt tool, and see how it fits the wider toolkit in best llms.txt tools.

Who actually reads llms.txt?

This is where honesty matters more than hype. Publishing an llms.txt file is common — Anthropic, Stripe, Vercel, Cloudflare, and hundreds of documentation-heavy sites now serve one. But publishing a file is not the same as an engine consuming it to answer a question. As of mid-2026, no major AI provider has publicly confirmed that its answer engine fetches your llms.txt at retrieval time to decide what to cite.

What the file reliably helps with is narrower: developer-facing documentation that AI coding assistants and agents read on demand, where a curated index of clean Markdown genuinely saves context and improves answers. For a docs site, that's a real benefit. For a local business or a marketing site hoping to appear in ChatGPT recommendations, there is little evidence the file moves anything on its own. Treat it as a low-cost courtesy to the crawlers that might use it, not as a growth lever.

Does Google use llms.txt?

No. Google has said so directly and more than once. On June 17, 2025, Google's John Mueller stated that no AI system currently uses llms.txt, comparing it to the old keywords meta tag — a self-declared signal that was easy to game and eventually abandoned. At Search Central events in July 2025, Google's Gary Illyes said Google does not support llms.txt and is not planning to, and that ranking in AI Overviews just needs normal SEO (Search Engine Land).

Mueller also clarified a common point of confusion: when people noticed some Google-owned properties serving an llms.txt file, he explained the Search team neither uses nor endorses it — an internal content system had added support and some teams simply hadn't removed it. So if your goal is Google AI Overviews specifically, llms.txt does nothing. The signals that earn those citations are the ordinary ones: helpful content, clean structure, and corroboration across trusted sources — see how AI Overviews pick sources.

Is llms.txt the same as robots.txt?

No, and conflating them causes real mistakes. robots.txt tells crawlers what they are allowed to access; llms.txt tells models what content is worth reading and where the clean version lives. One is a permission boundary, the other is a curated index. Blocking a crawler in robots.txt while listing the same pages in llms.txt is a contradiction that helps no one.

They also serve different audiences over time. robots.txt is honored by essentially every well-behaved crawler because it is decades old and near-universally implemented. llms.txt is a young proposal with real publishing adoption but unproven consumption. If you care about which AI crawlers can reach your public pages at all, that's a robots.txt and firewall question — not an llms.txt one.

Do you actually need an llms.txt file?

Here's the honest take. Publish one if you run a documentation-heavy or developer-facing site, because the assistants and agents that read docs on demand can genuinely use a clean, curated index — and the cost of maintaining it is low. Skip it, or treat it as optional housekeeping, if you're a local business or a standard marketing site: there's no evidence it earns you AI citations, and Google has said outright it won't use it.

Either way, don't let llms.txt distract from the work that actually moves AI visibility: earning mentions on the third-party sources engines trust, structuring pages as clean answers, and keeping your facts consistent and machine-extractable. An llms.txt file is a tidy front door; it is not the house. If you want to know whether any of this is landing, measure it — check whether the engines name you today with the free AI-visibility check.

Frequently asked questions

Where does the llms.txt file go?

At the root of your domain, served as plain text at yoursite.com/llms.txt, exactly like robots.txt sits at yoursite.com/robots.txt. Individual clean-Markdown page versions can live alongside your pages as .md files or be listed in the index.

Will llms.txt get my site cited by ChatGPT?

There's no public evidence that it does on its own. Citations in ChatGPT come from retrieval over Bing's index plus OpenAI's own crawl and third-party corroboration — see how ChatGPT recommends brands. An llms.txt file may help a model read your docs, but it doesn't manufacture citations.

Does Google read llms.txt?

No. Google's John Mueller and Gary Illyes have both said publicly that Google does not use llms.txt and has no plans to. Ranking in AI Overviews relies on ordinary SEO signals, not this file.

Is llms.txt an official standard?

Not yet. It's a proposal from Answer.AI with growing adoption among publishers, but it is not ratified by a standards body and no major engine has committed to consuming it. Follow the spec if you publish one, but don't treat it as guaranteed to be read.

What's the difference between llms.txt and robots.txt?

robots.txt controls what crawlers are allowed to access; llms.txt suggests which content is worth reading and points to clean versions of it. One is a permission file, the other is a curated content map.