llms.txt is a proposed plain-text file, written in Markdown and placed at the root of a website, that gives AI tools a short, curated map of the site's most important pages. It was proposed in September 2024 by Jeremy Howard of Answer.AI as a way to help language models find clean, useful content at the moment they answer a question. It is not an official web standard, and Google has said publicly that its search and AI features do not use it. For most business websites it is a cheap, harmless addition, not a ranking lever and not a substitute for crawlable, well-structured content.
At a glance
- It is a proposal, not a standard. llms.txt is a community convention published at llmstxt.org, not a protocol any search engine has committed to honoring.
- Google does not use it. Google's own documentation says you do not need new machine-readable files or AI text files to appear in AI Overviews or AI Mode.
- Its real home is documentation. Developer docs sites, where AI coding assistants pull reference material, are where llms.txt earns its keep today.
- It costs almost nothing. A small, accurate file does no harm, as long as nobody mistakes it for the thing that gets you cited.
What Is llms.txt?
llms.txt is a Markdown file served at yoursite.com/llms.txt that summarizes what your site is and links to the pages that matter most, each with a short note. The idea, set out in Jeremy Howard's original proposal on September 3, 2024, is that website pages are built for humans, full of navigation, scripts, and layout, and an AI system with a limited context window struggles to pull the useful parts out. A site owner knows which pages matter, so the file lets them say so directly in a format both people and language models can read.
The proposal is aimed at inference time, meaning the moment an AI tool is fetching information to answer a user, rather than at model training. That distinction matters. llms.txt was designed for an assistant that is actively reading your site on someone's behalf, like a coding tool pulling up library documentation, not as a signal for how a search engine ranks pages.
What Goes in an llms.txt File?
The format is deliberately simple. Per the specification, the file contains, in this order:
- An H1 title: the name of the project or site. This is the only required section.
- A blockquote summary: a short description with the key information needed to understand the rest of the file.
- Optional detail: zero or more paragraphs or lists giving extra context about the site.
- Link sections: zero or more H2-headed sections, each a Markdown list of links with an optional note after each one.
- An "Optional" section: by convention, secondary links an AI tool can skip when it needs a shorter context.
The proposal also suggests offering clean Markdown versions of key pages at the same URL with .md appended. A related convention, llms-full.txt, bundles the full text of a site's documentation into a single Markdown file; it was popularized by the documentation platform Mintlify in work with Anthropic, and Anthropic's own developer docs publish both files. Here is what a small business version of llms.txt might look like:
# Desert Ridge Plumbing
> Licensed residential plumbing company serving Henderson and Las Vegas, NV. Offers repairs, water heater installs, and 24/7 emergency service.
## Services
- [Water Heater Installation](https://example.com/water-heaters): tank and tankless installs and replacements
- [Emergency Plumbing](https://example.com/emergency): 24/7 service area and response details
## Company
- [About Us](https://example.com/about): license numbers, team, and service area
- [Contact](https://example.com/contact): phone, hours, and booking form
## Optional
- [Blog](https://example.com/blog): maintenance tips and seasonal guidesllms.txt vs robots.txt vs sitemap.xml: What Is the Difference?
The three files sit side by side at the root of a site, which is why they get confused. They do different jobs.
- robots.txt: tells crawlers what they may and may not fetch. It is about access, and it is a long-established standard that major crawlers, including the AI crawlers run by OpenAI, Anthropic, and Google, document how to respect.
- sitemap.xml: lists the URLs you want search engines to discover and index. It is about completeness, and search engines actively read it.
- llms.txt: offers a curated, human-readable summary of your most important content for AI tools. It is about context, and support for it is voluntary and limited.
In short, robots.txt controls access, sitemap.xml aids discovery, and llms.txt offers a guided summary. Only the first two are part of any search engine's documented crawling process. Getting robots.txt and your sitemaps right is still the higher-value work, and both are covered in our technical SEO checklist.
Do AI Engines Actually Use llms.txt?
This is where most coverage of llms.txt overpromises, so here is the plain answer: no major search engine or AI answer engine has publicly documented that it uses llms.txt to decide which sites to cite or rank.
Google has been the most explicit. Its guidance on AI features in Search states that there are no additional requirements to appear in AI Overviews or AI Mode, and that you do not need to create new machine-readable files or AI text files to appear in them. Google's John Mueller has said publicly that, as far as he knows, no AI service has said it uses llms.txt, and that server logs show AI bots do not even request it. Google's Gary Illyes has also said Google does not support it. Mueller has compared it to the old keywords meta tag: self-reported information that a system cannot trust to tell one website from another.
The picture outside Google search is more mixed. Some AI companies publish llms.txt files for their own developer documentation, and a few AI tools will read one when a user or agent points them at it. Google's own Lighthouse tool has even added an experimental llms.txt check aimed at browser-based agents. But publishing a file for your own docs, or reading it on request, is not the same as a crawler using it to choose citations. Until a provider says otherwise in its official documentation, treat any claim that llms.txt drives AI visibility as unproven.
Should You Add llms.txt to Your Website?
For most business websites, adding one is fine, but it belongs near the bottom of the priority list. The decision breaks down like this:
- Add it if you run developer documentation, an API, or a software product. This is the use case the format was built for, and AI coding assistants are the tools most likely to read it.
- Add it if it takes you ten minutes. A small, accurate file is low cost and low risk. We publish one on this site for exactly that reason.
- Do not add it instead of the fundamentals. If your pages are not crawlable, indexed, and clearly structured, no summary file will fix that.
- Do not pay for it as a premium service. Anyone selling llms.txt as a way to guarantee AI citations is selling something no provider has confirmed.
The work that actually earns AI citations is the work covered in what GEO means in SEO and our answer engine optimization guide: direct answers near the top of the page, question-shaped headings, structured data, genuine authority, and pages that engines can reach and index. llms.txt is, at best, a tidy footnote to that work.
How Do You Create an llms.txt File?
If you decide to add one, keep it short and keep it accurate.
- Write it in plain Markdown. Start with an H1 naming your business, then a one or two sentence blockquote describing what you do and where.
- List only the pages that matter. Your core service pages, about page, and contact page. A curated list of ten good links beats a dump of every URL, which is what your sitemap is for.
- Add a short note to each link. One line saying what the page covers.
- Upload it to your site root so it resolves at yoursite.com/llms.txt, and confirm it loads as plain text.
- Keep it current. A file pointing to deleted or redirected pages is worse than no file, so update it whenever you add, remove, or rename key pages.
- Consider a noindex header. Google's John Mueller has said noindexing llms.txt could make sense, since other sites may link to it and a raw text file showing up in search results would be odd for users. For a .txt file, that means an X-Robots-Tag noindex HTTP header.
The Bottom Line
llms.txt is a sensible idea with a narrow real-world job: helping AI tools read documentation and curated content cleanly when someone points them at it. It is not a standard, it is not used by Google's search or AI features, and no major AI answer engine has documented using it to pick sources. Add one if it is quick and you keep it accurate, then put your real effort where citations actually come from. For the definition of the term in one place, see our llms.txt glossary entry.