What llms.txt is
llms.txt is a proposed file that sits at the root of your site, like robots.txt. It's written in Markdown and gives AI tools a short summary of your site, with links to the pages that matter most. The idea is that a language model can read one clean file instead of crawling menus, cookie banners and scripts. The format was proposed in 2024, and since then plenty of software companies have added one, mostly to their documentation sites.
Who reads it today
As of this autumn, none of the big AI search products have said they use it to decide what to cite. We checked our own server logs across 30 client sites for three months. The file was requested a few hundred times, almost all by small crawlers and developer tools. Requests from the crawlers behind ChatGPT, Perplexity and Google's AI features were close to zero.
So adding it won't get you mentioned in AI answers. Anyone selling it as an AI search fix is ahead of the evidence.
The one case where we'd add it now
If you publish documentation for developers, add it. Coding assistants and some editor tools do read llms.txt, and a clean Markdown version of your docs helps them give correct answers about your product. It takes an afternoon to set up and there's no downside. Keep it short: a paragraph on what the product does, then links to the getting-started guide, the API reference and pricing.
For a clinic, a law firm or a shop, we'd spend that afternoon on something else, like making sure there's a page that answers your customers' most common question in its first two lines.
What we're watching
Three things would change our advice: an AI search company saying it uses the file, a clear rise in requests from the major AI crawlers in our logs, or a test where adding the file changes what gets cited. We check the logs every month and will update this post if any of those happens.
- AI search
- Technical SEO
- llms.txt
Rania Haddad
Rania leads AI search at AdAscend. Before that she spent six years running SEO for a regional property portal. She reads Arabic search results for fun, which we've stopped questioning.
