The honest guide to llms.txt

llms.txt is a proposed plain-text file, placed at the root of your domain, that gives AI systems a short, structured summary of your site and points them to your most important pages. It's worth adding because it's cheap and harmless, but be clear-eyed: as of now no major AI crawler has confirmed it reads or prioritises llms.txt, so treat it as a small hygiene item, not a fix for weak AI visibility.

Here's what the file actually is, what adoption really looks like today, what to put in it, and the mistakes that make people think it did something when it didn't.

What exactly is llms.txt supposed to do?

Answer first: llms.txt is a markdown file at yoursite.com/llms.txt that lists, in a short human-and-machine-readable format, what your site is, what it does, and links to the pages you consider most important — the idea being that an AI system fetching your site could read this one file instead of crawling everything.

It was proposed in 2024 as an equivalent to robots.txt or sitemap.xml, but for language models instead of search crawlers. robots.txt tells crawlers what they can't access; sitemap.xml lists every URL for indexing; llms.txt is meant to be curated and opinionated — a short brief, not a full index.

The typical structure looks like this (shown as plain text, not code, since the real file has no special syntax beyond markdown headings and links):

Title: your company name A one-line summary of what you do.

Section: Docs Link to your documentation homepage - short description Link to your API reference - short description

Section: About Link to your about/company page - short description

It's meant to be short — a few dozen lines, not a full sitemap dump.

Is llms.txt actually supported by ChatGPT, Perplexity, Gemini and Grok?

Answer first: no, not confirmed. There's no public statement from OpenAI, Anthropic, Google or Perplexity committing to fetch or prioritise llms.txt, and independent testing since it launched has found inconsistent or no evidence that it changes what gets retrieved or cited. Adoption on the publishing side has grown, but adoption on the reading side is the part that matters, and that's the unresolved half.

Being straight about where things stand:

• Plenty of sites publish one now. Frameworks, developer tools and SaaS companies have added llms.txt files, partly as a hedge and partly because tooling makes it nearly free to generate.
• No major AI lab has confirmed it's used in retrieval. These systems have not documented llms.txt as an input to how they crawl, index, or decide what to cite.
• It doesn't replace normal crawling or indexing. Assistants that browse the web still rely primarily on standard web content, search indexes, and whatever their retrieval pipeline already does — llms.txt is not known to override or supplement that in a documented way.
• It costs almost nothing to add, so the calculus is "small potential upside, near-zero downside" rather than "essential and skipping it hurts you."

So the honest framing for a client or reader is: add it because it's low effort and future-proofs you a little if support does arrive, not because it will visibly move your AI mentions this quarter. If you're choosing where to spend limited time on AI visibility, structured content, schema markup and third-party listings currently matter more — see the AI visibility checklist for the fuller priority order.

What should actually go in your llms.txt file?

Answer first: a one-paragraph description of what your company does, then a small number of curated sections linking to your highest-value pages — docs, pricing, key product pages, and an about/company page — each with a one-line description, kept short enough that the whole file could be read in under a minute.

A sensible structure:

1. 1Title and one-line summary. What you do, for whom, stated plainly. No slogans.
2. 2A short paragraph of context. Two or three sentences: category, main product, who it's for.
3. 3A "Docs" or "Product" section linking to your most important functional pages — documentation, key features, pricing.
4. 4An "About" section linking to your company/about page, so an entity-resolution step has somewhere authoritative to check facts like founding date or location.
5. 5Optionally, a "Blog" or "Resources" section with two or three of your most canonical, evergreen articles — not a dump of every post.

What to leave out:

• Marketing copy. This isn't a landing page; keep sentences factual and short.
• Every page on your site. That's what sitemap.xml is for. llms.txt should be a curated shortlist, ideally under 50-100 lines.
• Duplicate content from the file itself. Link to pages rather than pasting their content into the file — an llms-full.txt variant exists for that in some implementations, but it's a different, heavier file and not what most sites need.
• Anything that changes weekly. Pricing numbers, promotions, and dates go stale in a file nobody automatically re-crawls on a schedule.