The short answer: llms.txt is a plain-text file you publish at your site's root (yoursite.com/llms.txt) that gives AI language models a curated orientation: who you are, what your key pages contain, and where your canonical answers live. It's an emerging convention proposed in 2024 — not a standard, and no engine promises to honor it — but it costs minutes, can't hurt, and gives systems that do read it a clean map of your site instead of a guess. This site ships one.
Where it came from, and what it is
Robots.txt tells crawlers what they may read. Sitemaps tell them what exists. Neither tells a language model what anything means — and models working with limited context benefit from a human-curated summary more than a raw URL dump. The llms.txt proposal (September 2024, from the fast.ai community) fills that gap: a Markdown-formatted file listing your site's purpose and its most important pages with one-line descriptions, so a model landing on your domain can orient in one read.
What goes in it
The format is deliberately simple Markdown: an H1 with your name, a summary line, then sections of links with one-line descriptions. A trimmed example of the real thing:
# Houtos Labs
> Evaluation-led web, app & web3 studio. We don't guess. We grade.
## Key pages
- [The Grader](https://houtos.io/grader.html): free website audit across six lenses
- [What is SEO?](https://houtos.io/what-is-seo.html): the complete plain-English guide
- [Methodology](https://houtos.io/methodology.html): how the grading rubric works
Curate rather than dump: your identity, your commercial pages, your canonical answers — the pages you'd want quoted — each with a description a model can use verbatim.
What it does — and honestly doesn't
- It does give any system that reads it a clean, curated map — and some AI crawlers and tools do fetch it today.
- It does force a useful editorial exercise: deciding which pages are your canonical answers sharpens the site itself.
- It doesn't control anything. It's advisory — unlike robots.txt, nothing is obligated to respect it, and no engine has committed to using it for ranking or citation.
- It doesn't substitute for the fundamentals. A model that can't read your actual pages (script-dependent content, blocked crawlers) isn't rescued by a nice index of them.
Creating yours in four steps
- List your ten most important pages — identity, offers, and the answers you most want engines to quote.
- Write one honest line per page — what question it answers, in the customer's words.
- Format as Markdown — H1 name, blockquote summary, sectioned link lists — and save as plain text.
- Publish at the root (yoursite.com/llms.txt), keep it current when key pages change, and treat it like the sitemap's editorial sibling.
It's a five-minute layer of the larger discipline — being retrievable, quotable, and trustable to AI systems — covered end-to-end in our GEO guide and AEO guide.
Common questions
Do AI companies actually use llms.txt?
Some tools and crawlers fetch it today; no major engine has committed to honoring it, and adoption is early. The honest case is asymmetric: minutes of cost, zero downside, a cleaner map for whatever reads it — plus the editorial clarity of choosing your canonical pages.
Is llms.txt the same as robots.txt?
No — they're complements. Robots.txt is enforceable permission (what may be read); llms.txt is advisory meaning (what matters and why). A crawler can obey your robots.txt and never look at llms.txt, or vice versa.
Will llms.txt improve my rankings?
There's no evidence it affects search rankings, and it isn't designed to. Its aim is orientation for language models. Treat it as a cheap AEO/GEO layer, not an SEO tactic.
What should I put in mine?
Who you are in one line, then your curated key pages — commercial pages and canonical answers — each with a one-line description a model could quote. Ten to thirty links beats three hundred; curation is the point.
Start with a reading, not guesswork
The Grader's Findability lens measures how search and answer engines actually see your site — in plain English, every result labeled Measured or Estimated.
Grade my site The SEO & AEO audit