llms.txt

llms.txt is a plain-text Markdown file placed at the root of a website. It uses a short, structured list to tell large language models what the site is and which key pages are worth reading.

The format was proposed by Jeremy Howard in 2024. It deliberately echoes two older files: robots.txt and the XML sitemap. Like them, it lives at a fixed path, /llms.txt, and is served as text/plain. The difference is purpose:

File	What it tells a machine
robots.txt	Which paths not to crawl
sitemap.xml	Every URL you want indexed
llms.txt	“Read these few pages first; they explain who we are”

Typical structure

A standard llms.txt file has four parts:

• An H1 heading with the name of the site or organization.
• A blockquote with a one-sentence summary.
• Several H2 sections, each a list of links, with a short description after every link.
• An optional ## Optional section that flags secondary material a model can skip if it is short on space.

A minimal example:

# Example Co.

> Example Co. is a Shanghai-based firm that helps companies measure how AI assistants describe them.

## Services
- [GEO audit](https://example.com/audit): what the audit checks and what you receive

## Glossary
- [Mention rate](https://example.com/wiki/mention-rate): definition and formula

## Optional
- [Company news](https://example.com/news): announcements

Why it can help

When an AI client fetches a web page, it receives HTML mixed with navigation, scripts and boilerplate. The model’s context window fills up quickly with material that has nothing to do with the question. A few kilobytes of plain text gives it the skeleton of the whole site in one request: what the organization is, how the content is organized and where the authoritative pages are.

The benefit is clearest for sites that are already well structured, such as glossaries, documentation and knowledge bases, where a list of links with one-line descriptions maps cleanly onto the content.

Current status and limits

It is important to be honest about what llms.txt is not:

• It is a community convention, not a standard. Nothing obliges any AI company to read it, and none of the major providers publish whether or how they use it.
• It does not make content more credible. It lowers the cost of finding your pages; it does not change whether a model trusts or cites them.
• It does not change what a model already believes. If a model has your company confused with another one, a link list will not fix that. That is an entity problem (see brand entity).
• It does not replace structured data. Structured data labels factual fields such as legal name, address and founding date. llms.txt only provides navigation. The two work side by side.

Treat it as a low-cost addition after the fundamentals are in place: crawlable pages, consistent facts and clean markup.

Common variants

Some sites go further and publish:

• /llms-full.txt, a single file that concatenates the full text of the key pages.
• Markdown versions of individual pages, reachable by adding .md to the page URL, so a model can read the body text without navigation or scripts.

Both follow the same logic: give machines clean text that is cheap to read. Whichever you publish, keep the content in sync with the live pages. An llms.txt that points to retired URLs or describes old services adds to the inconsistency problem it was meant to reduce.

Where it fits in GEO work

In a generative engine optimization program, llms.txt belongs to the technical layer, alongside robots.txt rules for AI crawlers and server-side rendering. It is quick to add and easy to maintain, but on its own it will not measurably move mention rate. The content and entity work does that.