An llms.txt file is a markdown file at /llms.txt that gives AI agents a short, linked map of your site. Only the H1 title is required; a summary and H2 lists of links do the work. Google says Search ignores the file, and Ahrefs found 97% of llms.txt files received zero requests in May 2026.
So write one if you want AI agents and coding tools to find your services, prices and proof quickly, keep it accurate, and do not expect it to move rankings or citations. This guide covers the current format (v2, August 2026), what the file does not do, a template you can copy, our own live file as a worked example, and how we keep the full-text version current.
- Format: one required H1, then an optional blockquote summary, optional notes and H2 sections of
[name](url): notelinks (llmstxt.org) - Location: the site root, or any path within it, covering the pages under that path
- Google Search, AI Overviews and AI Mode: the file neither helps nor harms, because Google Search ignores it (Google Search Central, updated 10 July 2026)
- How often it is read: 97% of llms.txt files in a sample of 137,210 domains received no requests in May 2026 (Ahrefs)
- Our worked example: orrjo.com/llms.txt is built by a script from our page files, and when we checked it on 14 October 2026, every URL in it returned HTTP 200
Disclosure: ORRJO publishes this guide and sells SEO and AEO work inside its Brand & Content service. Every third-party figure in the body links to the page it came from, and all of them were checked on 13 October 2026.
What is an llms.txt file?
An llms.txt file is a markdown file that gives large language models and AI agents a curated, linked overview of a website. The proposal, by Jeremy Howard, sets out its aim in one line: "We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content." It was first published in September 2024, and v2 followed in August 2026 (llmstxt.org changes).
The idea is that an agent reads the short file first, then follows only the links it needs. In the spec's words, agents "view or search llms.txt to find the information they need, then follow the relevant links." It is still a proposal, and the spec says it "is open for community input".
Use is heaviest on software documentation sites. The spec says llms.txt files "are used most heavily for software documentation, where coding agents follow them to find API references and tutorials", and it names "a business outlining its structure and policies" as another use. That second use is the one this guide is about.
It is easy to confuse llms.txt with the two files that already sit at the root of most sites. The spec separates them like this:
| File | What it is for, in the llms.txt spec's words |
|---|---|
| robots.txt | "lets automated tools know what access to a site is considered acceptable, such as for search indexing bots" |
| sitemap.xml | "a list of all the indexable human-readable information available on a site" |
| llms.txt | "a curated overview for LLMs", "used on demand, when an agent needs information about a topic while assisting a user" |
If your question is which AI crawlers to let in, that is a robots.txt job, and our guide to which AI crawlers to allow in robots.txt covers it.
What does an llms.txt file not do?
It does not improve your Google rankings, it does not control or block crawlers, and none of the studies we found shows that it gets you cited more often by AI engines. Here is the evidence, claim by claim.
| What people expect | What the evidence says |
|---|---|
| Better Google rankings, or more visibility in AI Overviews and AI Mode | Google: "Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." Its guide lists "creating unnecessary AI text files (like llms.txt)" among tactics you can ignore (Google Search Central, updated 10 July 2026). Google's AI features page adds: "You don't need to create new machine readable files, AI text files, or markup to appear in these features." |
| Control over which AI crawlers read your site | "robots.txt and llms.txt have different purposes" (llmstxt.org). Ahrefs: "it is not a robots.txt-style directive: it controls nothing and blocks nothing" (Ahrefs). |
| More citations in AI answers | SE Ranking, in a dataset of nearly 300,000 domains: "Both statistical analysis and machine learning showed no effect of LLMs.txt on how often a domain is cited by LLMs." It added: "At least not yet." (SE Ranking, 7 November 2025). Trakkr ran two citation tests on AI-cited domains and says they "give different results. Neither establishes what adding the file would cause." (Trakkr). |
| AI bots reading the file | Ahrefs looked at the 137,210 domains in Ahrefs Web Analytics that had traffic in May 2026: "97% of those files received zero traffic in May 2026. Nothing fetched them at all." AI search crawlers barely touched the rest: "OAI-SearchBot, PerplexityBot, and Claude's search crawler combined made only a couple of hundred fetches across thousands of sites." (Ahrefs). On one site, OtterlyAI counted 84 requests for /llms.txt out of 62,100+ AI bot visits over 90 days (OtterlyAI). |
| Bots finding the file on their own | "Zero requests came from AI bots for llms.txt files that don't exist. They never go looking." and "AI tools fetch llms.txt when a link, an index, or a user instruction tells them it exists." (Ahrefs). |
Ahrefs' verdict: "If your goal is showing up in ChatGPT, Perplexity, or AI Overviews, an llms.txt file is largely decoration." Two caveats on that study: Ahrefs ran it with its own Bot Analytics product, and it counted requests, which it calls the generous measure, not whether a bot used what it fetched.
We found no statement from OpenAI, Anthropic or Perplexity, as of 13 October 2026, that their search or answer products read other sites' llms.txt files. OpenAI and Anthropic do publish llms.txt files for their own developer documentation (OpenAI's crawler page says "For the complete documentation index, see llms.txt."), but publishing a file is not the same as reading everyone else's.
We cannot add evidence of our own here either. We do not yet have log data on requests to our own llms.txt, so we make no claim that any AI system reads it.
Do you need an llms.txt file?
No, it is optional, but it is cheap to publish and some AI tools do fetch it when they are pointed at it. Chrome's Lighthouse treats it as optional: "If the file is not provided by the server (resulting in a 404), the audit is marked as Not Applicable (N/A), as providing the file is optional at the moment." (Chrome for Developers, updated 5 May 2026).
The case for publishing one comes from the agent side, not the search side. Chrome describes the file as "an emerging convention used to provide a machine-readable summary of a website's content, specifically designed for LLMs and AI agents", and says: "Without this file, agents may spend more time crawling the site to understand its high-level structure and primary content." In the Ahrefs data, 19.5% of the fetches that did happen came from named AI tools, and "GPTBot is top and Claude-Code is second, ahead of every AI search and assistant bot."
Adoption depends on which sites you count. In a study published in November 2025, SE Ranking found an llms.txt file on 10.13% of nearly 300,000 domains. Ahrefs found one on 28% of its 137,210 domains in May 2026, and says its customers "skew more technical and SEO-aware than the web at large, so treat the 28% adoption figure as an upper bound."
- Publish one if you want AI agents and coding tools to reach your services, prices and proof in one step, and someone will keep it in step with your pages.
- Leave it for later if nobody will maintain it. A file nobody updates can end up stating old prices or linking to retired pages, and an agent that reads it gets the old version.
- Do not publish one to fix rankings, AI Overviews or AI citations. Google says Search ignores the file, and no study we found shows it lifting AI citations.
If citations are the goal, our guide to getting cited by ChatGPT and other AI answers covers what each engine says it reads.
What goes in an llms.txt file?
In this order: an H1 with your company name, a blockquote summary, optional notes without headings, then H2 sections that list links as [name](url), each with an optional note after a colon. This is the v2 format from llmstxt.org:
| Part | Required? | What the spec says |
|---|---|---|
| H1 | Yes | "An H1 with the name of the project or site. This is the only required section" |
| Blockquote | No | "A blockquote with a short summary of the project, containing key information necessary for understanding the rest of the file" |
| Notes | No | "Zero or more markdown sections (e.g. paragraphs, lists, etc) of any type except headings" |
| H2 link lists | No | Each list item contains "a required markdown hyperlink [name](url), then optionally a : and notes about the file" |
| An "Optional" H2 | No, a convention | "used, by convention, for secondary information: links an agent can skip when a shorter context is needed" |
The spec sets no size limit. It says only that "The file itself stays small enough to fit in context. The detail lives behind the links, and is fetched only when needed." Figures you may see elsewhere, such as Mintlify's "As a rough guideline, aim for under 50KB", are rules of thumb, not part of the spec.
v2 also proposes markdown versions of your pages: "pages with information that agents might need provide a clean markdown version of those pages at the same URL as the original page", with .md appended or replacing the extension. That is a separate, optional step. orrjo.com does not publish markdown versions of its pages today.
Is there an llms.txt template for a B2B website?
Yes. Copy the template below, replace every bracket with facts from your own live pages, and delete any line you cannot fill truthfully. It follows the v2 format and uses the sections a B2B buyer's questions tend to need: services, pricing, proof, research and company.
# [Company name] > [Company name] is a [what you are] that [what you sell] to [who buys it]. [Where you are based.] [The one fact a buyer most needs, such as who you are not for.] [Two to four plain sentences an agent needs to answer buyer questions accurately: each service in one line, and prices and contract terms worded exactly as your pricing page words them. Leave out anything your site does not publish.] ## Services - [Service name](https://[your-domain]/[service-page]): [What it is, who it is for, the price as published] - [Service name](https://[your-domain]/[service-page]): [What it is, who it is for, the price as published] ## Pricing - [Pricing](https://[your-domain]/[pricing-page]): [What the page covers, and the currency] ## Proof - [Case study: client name](https://[your-domain]/[case-study-page]): [The headline result, worded exactly as on the page] - [All case studies](https://[your-domain]/[case-studies-page]): [How many, and which sectors] ## Guides and research - [Guide title](https://[your-domain]/[guide-page]): [The buyer question it answers] - [Research title](https://[your-domain]/[research-page]): [Sample, date and method in one line] ## Company - [About](https://[your-domain]/[about-page]): [Founded, headquarters, team] - [Contact](https://[your-domain]/[contact-page]): [How buyers get in touch] ## Optional - [Blog](https://[your-domain]/[blog-page]): [Older articles an agent can skip] - [Careers](https://[your-domain]/[careers-page])
Before you publish: every price, figure and claim in the file should say exactly what the linked page says. Keep it to plain links and descriptions. Ahrefs advises "nothing instruction-shaped", and to "only link to resources you control".
How do you write and publish an llms.txt file, step by step?
Draft it from your live pages, save it as plain text at /llms.txt, check every link, point agents to it, then test it with an agent. In order:
- List the pages an agent needs first. Services, pricing, case studies, research and company pages, and only pages that are live.
- Write the H1 and the blockquote. Your company name, then one short summary. Copy prices and facts word for word from your own pages, so the file never says something your site does not.
- Add H2 sections of links. One
[name](url): noteline per page. Do not use bare URLs: the spec requires a markdown hyperlink in every list item. - Put secondary links under "## Optional". Older posts, careers and anything an agent can skip.
- Publish it as a plain text file named llms.txt at the root, or at a subpath such as /docs/llms.txt if it covers only that section.
- Check every link. Mintlify's checklist: "Every URL in your llms.txt should return a 200 status." (Mintlify).
- Tell agents where it is. v2 recommends "rel="describedby"" pointing to the llms.txt file, as an HTML
<link>element or an HTTPLink:header. Ahrefs found that agents "fetch llms.txt when directed, not speculatively." The spec's header example is below. - Decide whether to keep it out of search results. Google's John Mueller, quoted by Search Engine Journal on 21 July 2025: "using noindex for it could make sense, as sites might link to it and it could otherwise become indexed, which would be weird for users." A noindex HTTP header does that without touching the file.
- Test it the spec's way. "Test your file by asking an agent questions about your content, giving it only your llms.txt as a starting point."
- Run Lighthouse if you can. "Lighthouse flags the pages if a server error occurs when attempting to retrieve the llms.txt file." (Chrome for Developers). The Agentic Browsing category is experimental and needs Chrome 150 or later (Chrome for Developers).
The link header example from the spec, which points to a markdown version of a page and to the llms.txt file that covers it:
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"
What does a real B2B llms.txt look like?
Ours is at orrjo.com/llms.txt. A script builds it from our page files, so the file changes each time we rerun the script. When we checked it on 14 October 2026, every URL in it returned HTTP 200. This is how it is built, top to bottom:
| Part | What ours holds |
|---|---|
| H1 | The company name, "# ORRJO" |
| First blockquote | A pointer to llms-full.txt, which it describes as the "Full text of every key page in one file" |
| Second blockquote | What ORRJO does, who it works with and where it is based |
| Two plain paragraphs | Headline figures and timelines, then prices by service |
| H2 sections | Services; Pricing guides; Best-of lists (published by ORRJO; ORRJO entry marked as the publisher's); Comparisons and alternatives; Guides, templates and tools; Case studies (the only canon for client figures); Original research and methodology; Company; Authors |
| Each list item | A [name](url) link, some with a short note after a colon |
Its companion, llms-full.txt, holds the text of more than 150 pages, each under its own heading with its URL. Both files are served as plain text.
The top of the file, with the parts we have not reproduced marked:
# ORRJO > Full text of every key page in one file: https://orrjo.com/llms-full.txt [...a second blockquote with the company summary, then two paragraphs of figures and prices...] ## Services - [Lead Generation](https://orrjo.com/lead-generation): Outbound SDR-as-a-service across phone, email and LinkedIn, run by people, with senior strategy on every account. From £2,500 a month. - [Demand Generation](https://orrjo.com/demand-generation): Content, LinkedIn, webinars and events that warm named accounts before outbound. From £2,500 a month. [...] - [Brand & Content](https://orrjo.com/brand-and-content): Branding, websites, SEO and AEO, sales collateral and video. From £1,250 a month. [...] ## Case studies (the only canon for client figures) - [Nezasa: 369 meetings in 12 months (Lead Generation)](https://orrjo.com/case-study-nezasa) [...]
The choices in it you might copy:
- A pointer to the full text first. The opening blockquote sends an agent to llms-full.txt if it wants the text of every key page in one file.
- Facts before links. Headline figures, such as 12,500+ meetings booked and £300M+ pipeline generated, and prices by service sit in plain paragraphs above the first H2, so an agent can answer a price question without following a link.
- Prices on the service lines. Each of the four retainer services carries its from price on its link line, the same as our pricing page.
- Headings that say how to read the links. Two H2s carry a note in brackets: "Best-of lists (published by ORRJO; ORRJO entry marked as the publisher's)" and "Case studies (the only canon for client figures)". An agent reading the list knows who wrote it before it follows a link.
- Link names that carry the answer. Each case-study link names the client, a headline result and the services behind it, such as "Nezasa: 369 meetings in 12 months (Lead Generation)".
- Fixed lines where it matters, checks everywhere else. The Services list, its child-page line and the Discovery link are fixed lines; content pages are listed only if they exist and are not noindexed (more on that below).
- A pointer in robots.txt. Our robots.txt ends with "# Content index for language models" and the two file URLs. Those lines are comments, not robots.txt rules.
Where our file does not follow the v2 format, on 14 October 2026:
- The first blockquote points to llms-full.txt, and the company summary sits in a second blockquote. The spec puts the summary straight after the H1.
- The Services section ends with a "Child pages:" line of inline links. The spec's H2 sections hold markdown lists.
- Our links point to the HTML pages. The spec says links should point to agent-friendly content, such as markdown versions of pages, and orrjo.com does not publish those today.
- Our page templates carry no describedby link to the file, the server sends no Link header for it, and it is served without a noindex header.
- There is no "## Optional" section. That is a convention, not a rule.
If you copy our structure, use the template above for those parts.
What is llms-full.txt, and is it part of the standard?
llms-full.txt is a single file that holds the full text of a site's key pages, and the current v2 text of the llms.txt proposal does not mention it. Mintlify says it "originally developed llms-full.txt in a collaboration with Anthropic, who needed a cleaner way to feed their entire documentation into LLMs without parsing HTML." It adds that the file "was officially adopted into the llmstxt.org standard", but neither the v2 spec nor its changes page names llms-full.txt. Its docs describe the file as one that "combines your entire documentation site into a single file as context for AI tools and LLM indexing."
We publish one as an extra, at orrjo.com/llms-full.txt, with the text of more than 150 pages. It opens like this, then repeats the same block for every page:
# ORRJO, full content for language models Source: https://orrjo.com · Generated [date] Short index: https://orrjo.com/llms.txt [...a one-paragraph company summary...] --- ## [page title] URL: [page URL] [the page text, with its headings kept as markdown]
Whether you need one depends on how much text you want an agent to have in one fetch. Nothing in the proposal requires it, and it is the file most likely to drift out of date, because it copies your page text. That is why ours is generated, not written by hand.
How do you keep an llms.txt file current?
Rebuild it from your live pages, check every link, and treat every change like a code change. Ahrefs' advice: "Version-control it, restrict who can edit it, set an alert for unauthorized changes, keep the content to plain links and descriptions (nothing instruction-shaped), only link to resources you control, and review anything a platform auto-generates on your behalf." (Ahrefs).
How we keep both files current at ORRJO:
- A script writes llms.txt. It holds the summary, the facts paragraphs and the pages in each section. The Services list, the child-page line and the "Discovery without a call" link are fixed lines, written into the script on purpose. Every other section works from a set list of pages and includes one only if its page file exists in our site files and is not marked noindex, so a retired or hidden page drops out on the next run.
- It keeps the full-text page list clean. Each run adds any new page it lists to the page list that llms-full.txt is built from, and that list carries no duplicates: the script drops any repeated URL.
- A second script builds llms-full.txt.
scripts/gen-llms-full.pyfetches each page listed inscripts/llms-pages.txt("one URL per line, in output order") from the live site and writes the file. - It keeps the text and drops the chrome. It strips scripts, styles, navigation, the footer, the mobile menu and the cookie banner, keeps headings as markdown, and drops consecutive duplicate blocks.
- It will not write half a file. If any page fails to fetch, it stops with "refusing to write a partial file. Fix the failures and rerun."
- It can check without writing. A
--checkoption builds the file and reports, and writes nothing. - The page list follows llms.txt. The script's own instructions: "Keep that file in step with llms.txt when pages are added or retired."
Checks worth running whenever your pages change:
- Every URL in the file returns HTTP 200, none points to a retired page, and any link to a page you keep out of search, such as a form or questionnaire, is there on purpose.
- Every price and figure matches the page it links to, word for word.
- The file still opens with your H1 and passes the spec's test: ask an agent about your company, giving it only your llms.txt.
- If your platform generates the file, read what it generated before you rely on it.
Which tools generate an llms.txt file automatically?
Several website platforms and plugins do; the spec notes that "Many documentation platforms and CMSs can generate an llms.txt file automatically". The ones below each say so on their own pages:
| Tool | What its own page says | Price, as published |
|---|---|---|
| Yoast SEO (WordPress) | Generates an llms.txt file in one click, with a preview before publishing | "Built into Yoast, free for everyone" |
| All in One SEO (WordPress) | "Lite users can generate llms.txt files automatically"; llms-full.txt files and markdown settings need a Pro account | llms.txt free on Lite; Pro price not stated on that page |
| Wix | "Wix automatically generates and maintains your llms.txt file, but you can edit it at any time in your site's dashboard." Once you edit it, it stops updating automatically. | Not stated on the help page |
| Mintlify (documentation sites) | Hosts llms.txt and llms-full.txt for docs it hosts. "Automatically generated llms.txt index files cannot exceed 100,000 characters." | Not checked |
| Firecrawl llms.txt generator | Crawls a site and drafts the files. Its page says: "This API endpoint will still remain active but we will no longer be maintaining it after June 30, 2025." | Not checked |
A generated file lists what the platform finds, in the platform's words. Read it, cut what an agent does not need, and check the prices against your pricing page before you leave it live.
Questions buyers ask
Do I need an llms.txt file?
No. It is optional. Chrome's Lighthouse marks a missing file as not applicable because providing one is optional at the moment, and Google says Google Search ignores the file. Publish one if you want AI agents and coding tools to find your key pages quickly, and only if someone will keep it accurate. In the Ahrefs study of 137,210 domains, 97% of the llms.txt files it found received no requests at all in May 2026.
Does llms.txt help you rank in Google or get cited by ChatGPT?
Not on the evidence we could find on 13 October 2026. Google says the file will neither harm nor help your visibility or rankings in Google Search, including its generative AI features. SE Ranking's study of nearly 300,000 domains found no effect of the file on how often a domain is cited by LLMs. We found no statement from OpenAI, Anthropic or Perplexity that their answer products read other sites' llms.txt files.
Is llms-full.txt part of the llms.txt standard?
Not in the spec's own text. The llms.txt proposal, now at v2 from August 2026, does not mention llms-full.txt. Mintlify says it developed llms-full.txt with Anthropic, as one file that holds a whole documentation site, and that it was adopted into the llmstxt.org standard, but the v2 text does not define it. Some sites, ORRJO included, publish one alongside llms.txt. Treat it as an optional extra file, not part of the format, and keep it in step with the short index.
Does llms.txt replace robots.txt?
No. The spec says robots.txt and llms.txt have different purposes: robots.txt tells automated tools what access to a site is acceptable, while llms.txt is used on demand, when an agent needs information to help a user. Ahrefs puts it plainly: the file controls nothing and blocks nothing. To allow or block AI crawlers, use robots.txt.
Where should the llms.txt file go?
At the root of your site, at /llms.txt, or at any path within it, such as /docs/llms.txt, where it covers the pages under that path. That is the v2 rule on llmstxt.org, and where more than one file applies, agents should use the most specific one. To help agents find the file, the spec recommends a describedby link relation, as an HTML link element or an HTTP Link header.
How big should an llms.txt file be?
The spec sets no byte or link limit. It says only that the file itself stays small enough to fit in context, with the detail behind the links. Mintlify caps the index files it generates automatically at 100,000 characters. Ours stays small because each page gets one link, only a few carry a short note, and the full text lives in llms-full.txt.
How can I tell whether AI tools read my llms.txt?
Check your server or CDN logs for requests to /llms.txt and note which user agents made them. Ahrefs measured this across 137,210 domains and found that, among the fetches that did happen, GPTBot was the top named AI tool and Claude-Code second. A request shows that a tool fetched the file, not that it used what it read, so do not count it as a citation.
Sources
- llmstxt.org, The /llms.txt file, v2, Jeremy Howard. Purpose, placement, format, the Optional convention, size wording, link relations and the header example, markdown page versions, the test method, uses, the comparison with robots.txt and sitemaps, and the open status, checked 13 October 2026.
- llmstxt.org, Changes. First published September 2024, v2 dated August 2026, checked 13 October 2026.
- Google Search Central, Optimizing your website for generative AI features on Google Search, last updated 10 July 2026. Google Search ignores llms.txt files, checked 13 October 2026.
- Google Search Central, AI features and your website, last updated 10 December 2025, checked 13 October 2026.
- Chrome for Developers, llms.txt (Lighthouse agentic browsing audit), last updated 5 May 2026. How Chrome describes the file, the N/A result for a missing file and the server error flag, checked 13 October 2026.
- Chrome for Developers, Lighthouse agentic browsing scoring, last updated 5 May 2026. Experimental status and Chrome 150 or later, checked 13 October 2026.
- Ahrefs, We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read, Louise Linehan, published 15 June 2026, updated 27 August 2026. Sample, adoption, fetch rates, named AI fetchers, routing and security advice, checked 13 October 2026.
- SE Ranking, LLMs.txt: Why Brands Rely On It and Why It Doesn't Work, Yulia Deda, 7 November 2025. Adoption and the citation analysis, checked 13 October 2026.
- Trakkr, The llms.txt Effect, published 18 March 2026, modified 4 September 2026. The conclusion of its two citation tests, checked 13 October 2026.
- OtterlyAI, Llms.txt Experiment: What Marketers Get Wrong about llms.txt, Thomas Peham, 5 February 2026. One site's 90-day bot log, checked 13 October 2026.
- Search Engine Journal, Google Says It Could Make Sense To Use Noindex Header With LLMS.txt, Roger Montti, 21 July 2025. John Mueller's comment, checked 13 October 2026.
- OpenAI, Overview of OpenAI Crawlers. Link to OpenAI's own documentation llms.txt, checked 13 October 2026.
- Mintlify, llms.txt (documentation). Automatic hosting, the 100,000 character cap and its llms-full.txt definition, checked 13 October 2026.
- Mintlify, The value of llms.txt: hype or real?, 9 May 2025. Where llms-full.txt came from, as Mintlify tells it, checked 13 October 2026.
- Mintlify, How to generate llms.txt, Tiffany Chen, updated 15 March 2026. The link check and the 50KB rule of thumb, checked 13 October 2026.
- Yoast, llms.txt feature page, checked 13 October 2026.
- All in One SEO, WordPress LLMs.txt Generator, checked 13 October 2026.
- Wix Help Center, Understanding your site's llms.txt file, checked 13 October 2026.
- Firecrawl, llms.txt generator. Maintenance notice, checked 13 October 2026.
- ORRJO, llms.txt, llms-full.txt and robots.txt, fetched from the live site after the llms.txt rebuild, and every URL in llms.txt checked for HTTP 200, checked 14 October 2026.