llms.txt
What llms.txt means
llms.txt is a proposed markdown file offering AI agents a short, curated index of a site's most useful pages. It was proposed by Jeremy Howard of Answer.AI on 3 September 2024 and revised to version 2 in August 2026. It is a proposal from one author at one company, not an IETF, W3C or WHATWG standard, and the document calls itself an "informal overview" that is "open for community input."
More precisely
The only required element is an H1 naming the project. Optional parts are a blockquote summary, sections containing no headings, and H2 file lists of markdown links with optional notes. Version 2 permits the file at any subpath, where the most specific one applies, and adds discovery through `rel="alternate" type="text/markdown"` and `rel="describedby"`, in HTML link elements or in an HTTP Link header. `llms-full.txt` appears nowhere in either version of the specification; it is a convention from documentation platforms, and version 2 removed the context-expansion tooling that produced the older llms-ctx files.
Why it matters
Google lists these files under what you can ignore for Google Search, saying maintaining one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." No platform documents its crawler consuming a third-party llms.txt. Server-log measurement across 137,210 domains found that 97% of the files served received zero traffic in May 2026, with Claude-Code and GPTBot the top individual requesters, and the study's authors caution that fetched does not mean read.
Example
A minimal valid file is one H1 line such as `# Acme Docs`, a blockquote summary, and an H2 section listing links to the pages worth reading first.
Where this came from
Definitions here follow the specification or documentation that defines the term, not a paraphrase of another glossary.
-
The /llms.txt file, v2 — Answer.AI
Authorship, the v2 revision of August 2026, the file structure and required H1, subpath scoping, the link relation discovery mechanism, and the specification's self-description as a proposal open for community input.
Primary source · read 2026-09-03
-
Changes in v2 of the llms.txt proposal — Answer.AI
v2 removed the context-expansion tooling and the special meaning of the Optional section, and llms-full.txt is absent from the specification.
Primary source · read 2026-09-03
-
Optimizing your website for generative AI features on Google Search — Google
Google Search does not use llms.txt files and maintaining one neither harms nor helps visibility or rankings in Google Search.
Primary source · read 2026-09-03
developers.google.com/search/docs/fundamentals/ai-optimization-guide
-
We studied 137,210 domains. Almost nothing fetches llms.txt. — Ahrefs
Server-log finding that 97% of llms.txt files received zero traffic in May 2026, that Claude-Code and GPTBot were the top individual requesters, and the authors' caveat that fetched does not mean read. Published 2026-06-15.
Primary source · read 2026-09-03