llms.txt
A Markdown file that gives AI tools a curated map of your site.
llms.txt is a proposed standard for a Markdown file, usually at the root of a site (/llms.txt), that gives language models a short, curated overview: what the site is, key background, and links to the most useful pages. It was proposed by Jeremy Howard on 3 September 2024.
The idea is that a model can read one small file instead of crawling a whole site full of navigation and scripts. The file starts with a heading for the project name, followed by a summary and sections of links. It is a proposal, not a rule every AI system follows.
Key characteristics
- A Markdown file, usually served at /llms.txt.
- Starts with a heading naming the site or project, then a short summary.
- Lists the pages an AI assistant should read first, with one-line descriptions.
- A proposal from September 2024; support varies between AI tools.
Example
A design studio's /llms.txt states its services, prices and contact details in a few lines, then links its pricing page, case studies and FAQ, so an assistant asked about the studio quotes current facts.
Frequently asked questions
Do AI assistants read llms.txt?
Some tools and agents use it, but it is a proposal, not a standard every AI crawler follows. It costs little to publish and keeps the facts you want quoted in one place.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may access. llms.txt tells AI tools what is worth reading and summarises the site for them.