llms.txt is a small markdown file that tells AI systems which parts of your site are worth reading.It sits at your domain root like robots.txt, but it does the opposite job: instead of restricting access, it curates attention. Here is what goes in one, how it differs from the files you already have — and an honest read on how much it currently matters.
“llms txt” — 5,400 searches/mo (Google, US)Average monthly US search volume. Source: Google autocomplete demand via AnswerThePublic, 2026.
How it differs from the files you already have
robots.txt
- Access control — what crawlers may not fetch
- Honoured by every major crawler for decades
- Plain directives, machine-parsed
- Getting it wrong can hide your whole site
llms.txt
- Curation — what is worth reading, and why
- Proposed 2024; adoption still emerging
- Markdown, written for models to read directly
- Getting it wrong costs you nothing
What one looks like
The format is deliberately simple: a title, a summary blockquote, then grouped links with short descriptions.
# ProRank SEO
> All-in-one WordPress SEO and performance plugin: audits,
> schema, caching, image optimization, and local SEO.
## Docs
- [Getting started](https://prorank.io/docs/getting-started/installation): Install and activate
- [Schema markup](https://prorank.io/docs/on-page-seo/advanced-schema): Structured data setup
- [Core Web Vitals](https://prorank.io/docs/analytics/core-web-vitals): Field data tracking
## Guides
- [How to perform an SEO audit](https://prorank.io/blog/how-to-perform-seo-audit): The 10-step method
- [What is local SEO](https://prorank.io/blog/what-is-local-seo): Complete guide
## Optional
- [Changelog](https://prorank.io/changelog): Release historyThe discipline is in the curation. A file listing all 400 of your pages is a sitemap with extra steps; the point is to name the twenty pages that best explain what you do. The ## Optionalsection is part of the spec — it marks content a model can skip when its context window is tight.
The honest assessment
No major AI provider has publicly committed to using llms.txt for retrieval or citation, and anyone telling you it guarantees mentions in ChatGPT is selling something. What is true:
- It costs almost nothing to publish and maintain if generated automatically.
- It cannot hurt — it is additive, and unlike robots.txt a mistake has no downside.
- It forces useful clarity. Deciding which twenty pages represent your site is an exercise worth doing regardless of who reads the output.
- Early conventions sometimes stick. Sitemaps looked speculative once too.
Put it in the same category as structured data in 2011: cheap, plausible, and positioned for an outcome that may or may not arrive. That is a good bet at this price, and a bad thing to build a strategy on.
Where it fits in AI visibility
llms.txt is one small tactic inside a larger discipline. The things that demonstrably affect whether assistants can use your content are more mundane: being crawlable by retrieval bots, answering questions directly near the top of the page, and shipping clean structured data. The full picture is in what SEO for AI is called, and the crawler decisions are in the robots.txt guide.
Before optimising for assistants, make sure ordinary crawlers can read you: run the free audit.
Frequently asked questions
- What is an llms.txt file?
- llms.txt is a proposed convention: a markdown file at the root of your domain that gives AI systems a curated, plain-text map of your most useful content. Where robots.txt says what crawlers may not fetch, llms.txt says what is worth reading — a short description of the site plus categorised links to your best pages.
- Is llms.txt the same as robots.txt?
- No, and they are not alternatives. robots.txt is an access-control file that has been honoured for decades; llms.txt is a content-curation file proposed in 2024. One restricts, the other recommends. A site can and generally should have both, and neither replaces the other.
- Do AI systems actually use llms.txt?
- Honestly: adoption is still emerging and no major AI provider has committed to reading it as a ranking or retrieval input. The case for adding one is that it costs very little, it cannot hurt, and it standardises something models otherwise infer badly. Treat anyone promising llms.txt will get you cited as overselling it.
- What should go in llms.txt?
- An H1 with your site name, a blockquote summarising what the site is, then H2 sections grouping your genuinely useful pages as markdown links with a short description each. Keep it curated — the value is in what you leave out. An "Optional" section marks content that can be skipped when context is tight.
- How do I create llms.txt for WordPress?
- You can write it by hand and serve it at /llms.txt, but it goes stale the moment you publish. Generating it from your actual content — so new pages appear and removed ones disappear — is the maintainable route, which is what plugin support is for.
