llms.txt is a plain text file written in Markdown and published at the root of a website, at /llms.txt. It gives large language models and AI agents a short summary of the site and a curated list of links to its most useful pages. The idea was proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI, on llmstxt.org.
Why llms.txt exists
Modern websites are hard for AI systems to read: navigation menus, scripts, cookie banners and ads take up space, and a model has a limited context window. llms.txt solves this by offering a clean, human-written entry point: what the site is, and where to find the information that matters.
It is often compared to robots.txt and sitemaps, but it does a different job. robots.txt says what bots may crawl. A sitemap lists every URL. llms.txt explains and curates.
The llms.txt format
- H1 title with the name of the site or project. This is the only required part.
- Blockquote with a short summary of what the site is about.
- Optional paragraphs with key details an assistant should know.
- H2 sections, each with a Markdown list of links in the form
- [Title](URL): description. - An Optional section for secondary links that can be skipped when space is short.
A complete example
# SEOOn.page
> Free SEO tools and original research for the AI search era.
SEOOn.page offers free tools to check how search engines and AI assistants see a website.
## Tools
- [AI Bot Checker](https://seoon.page/tools/ai-bot-checker/): Check which AI crawlers a site allows
- [llms.txt Generator](https://seoon.page/tools/llms-txt-generator/): Create and validate llms.txt files
## Research
- [AI Bots and Spain's Top Websites](https://seoon.page/research/ai-bots-spain-top-websites-2026/): Which sites block AI crawlers
## Optional
- [Blog](https://seoon.page/blog/)Who uses llms.txt?
Adoption started with developer documentation and AI companies, and is spreading to other sectors. In our study of 344 leading websites, 15% of the websites already published an llms.txt file, with tech, hosting and telecom companies leading the way. News media, on the other hand, rarely publish one.
Does llms.txt help SEO or AI visibility?
Be realistic about it. Google has said it does not use llms.txt for Search, and no major AI company has confirmed that its crawlers read it systematically. Where it clearly helps is with AI agents and tools that read a site on demand, such as coding assistants and research agents, which can use it to find the right pages quickly.
Because it takes a few minutes to create and has no downside, many sites treat it as a low-cost bet on how AI agents will navigate the web.
llms.txt vs llms-full.txt
Some sites also publish llms-full.txt, a single file with the full text of their key pages in Markdown. It is useful for documentation sites, where an assistant may want everything in one request. For most websites, a short llms.txt is enough.
How to create your llms.txt
- Write a one-sentence summary of your website.
- Pick 5 to 20 pages that best explain your business: products, pricing, documentation, about, contact.
- Use our llms.txt generator to turn them into a valid file.
- Upload it to the root of your domain and check that it loads as plain text.
- Validate it with the same tool and update it when your key pages change.
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal, not a W3C or IETF standard, but its format is simple and widely copied.
Should llms.txt include every page?
No. That is what sitemaps are for. Keep llms.txt short and curated.
Can llms.txt block AI bots?
No. To control access, use robots.txt. llms.txt only describes your site.