What Is llms-full.txt and How Do You Implement It in 2026?

Q: Is llms-full.txt part of the llms.txt specification?

No. The proposal at llmstxt.org , revised to v2 in August 2026, defines only /llms.txt and per-page markdown versions of pages. The full-content file was introduced by the documentation platform Mintlify and spread from there, and it is now served by Cloudflare, OpenAI, Anthropic, Perplexity and the Svelte docs among others. Treat it as a de facto convention with wide adoption rather than a specified file, and do not expect a validator to enforce a format.

Q: Do I still need llms-full.txt if I publish per-page .md files?

They serve different access patterns. Per-page markdown at page.md or page.html.md , advertised with rel="alternate" type="text/markdown" , is what an agent uses when it already knows which page it wants, and it is the pattern the v2 proposal actually specifies. The full file is for the cases where the agent does not know yet: bulk ingestion, building a vector index, or a developer pasting one URL to give a model the whole product. Publish both if you can, since generating one is usually a by-product of generating the other.

One file at /llms-full.txt that holds the full text of every page you want an agent to read, so a model can load your whole corpus in a single fetch. This page shows three ways to generate it, and how to size, cache and serve it so agents actually keep it.

Check Your Site Now Browse the Glossary

CategoryCrawler access, licensing & indexing StatusDe facto standard Maintained byCommunity Glippy checkMachine Readability (category 6)

llms-full.txt (/llms-full.txt) is a single plain-text markdown file that concatenates the full body text of every page a site wants language models to read, rather than linking to them. It exists so an agent can pull an entire documentation set in one request instead of crawling dozens of URLs, and it is generated by the build, never written by hand. It is a community convention rather than a specified file: the llms.txt proposal at llmstxt.org defines the index file and per-page markdown versions, but has never defined a full-content variant.

Why llms-full.txt matters for AI visibility

An agent answering a question about your product has two ways to get your content. It can crawl your HTML, strip navigation, ads and script tags, and hope the extraction survived. Or it can fetch one markdown file that already contains exactly what you meant to say. The second path is cheaper, faster and lossless, and it is the path a coding agent or a research tool will take if you offer it. Sites that publish one are handing the model a clean corpus instead of leaving it to reconstruct one.

The trade-off is size. /llms.txt is deliberately small because the detail lives behind its links. /llms-full.txt is the opposite: it is the detail. Measured on 27 August 2026, the root file at developers.cloudflare.com was about 57 MB, docs.anthropic.com about 41 MB, developers.openai.com about 6.4 MB and svelte.dev about 1.2 MB. Only the last two fit comfortably in a current context window, which is why the interesting engineering here is not the concatenation, it is deciding what goes in, keeping it in sync with the site, and splitting it once it outgrows a single fetch.

Where the spec lives

There is no specification document for this file, so the useful sources are the llms.txt proposal it grew out of, the media type registration, and the generators that ship it in production.

The /llms.txt file, v2 - the primary proposal by Jeremy Howard. Read it for the index file format, the per-page .md convention, and the rel="alternate" and rel="describedby" link relations. Note that it does not mention llms-full.txt.
llms.txt changes: v1 to v2 - what the August 2026 revision dropped, including the llms_txt2ctx context-expansion tool and the mechanical meaning of the Optional section. Useful if you built tooling against v1.
AnswerDotAI/llms-txt - the reference repository and the place to open an issue if you want a full-content variant blessed by the proposal.
RFC 7763: The text/markdown Media Type - registers text/markdown and makes the charset parameter required. Reach for it when you are arguing about the Content-Type header your CDN should send.
Mintlify: llms.txt and llms-full.txt - the platform that popularised the file. Documents the exclusion rules it applies (hidden pages, noindex: true, non-default languages and versions) and the 100,000 character cap it puts on llms.txt but not on llms-full.txt.
Cloudflare: docs for agents - a production example of per-product scoping, with a root llms-full.txt plus one per product path, and per-page markdown via /index.md or an Accept: text/markdown header.

Three ways to implement llms-full.txt

Pick by where your content already lives. A build script over a markdown directory suits any static site and any framework, including ones with no plugin ecosystem. A documentation framework plugin suits teams already on Starlight, VitePress or Docusaurus, where the renderer knows more about your pages than a file walk ever will. A server-rendered endpoint suits sites whose content sits in a CMS or database, where there is no directory to walk and the file has to be assembled per request with proper caching.

Build script over a markdown content directory

The framework-agnostic version. Walk your content tree, drop the pages you already exclude from your sitemap, and write one file into your static output directory. Wire it into the build so it can never go stale, and fail the build when it grows past the size you are willing to serve.

javascriptscripts/build-llms-full.mjs

// Run this in CI before the static output is uploaded. Node 20.12 or newer.
import { readdir, readFile, writeFile, mkdir } from 'node:fs/promises';
import { join, relative, sep } from 'node:path';

const SITE = 'https://example.com';
const CONTENT = 'content';
const MAX_BYTES = 5 * 1024 * 1024;

const dirents = await readdir(CONTENT, { recursive: true, withFileTypes: true });
const files = dirents
  .filter((d) => d.isFile() && d.name.endsWith('.md'))
  .map((d) => join(d.parentPath, d.name))
  .sort();

const parts = [
  '# Example Docs',
  '',
  '> Full text of every published page, concatenated for single-fetch ingestion.',
  '',
  `Generated ${new Date().toISOString()}. Curated index: ${SITE}/llms.txt`,
  '',
];

for (const file of files) {
  const raw = await readFile(file, 'utf8');
  const fm = /^---\r?\n([\s\S]*?)\r?\n---\r?\n/.exec(raw);
  const head = fm ? fm[1] : '';
  // Same exclusions as the sitemap, so the two never disagree.
  if (/^(draft|noindex):\s*true\s*$/m.test(head)) continue;
  const title = (/^title:\s*["']?(.+?)["']?\s*$/m.exec(head) || [, file])[1];
  const slug = relative(CONTENT, file).split(sep).join('/').replace(/(index)?\.md$/, '');
  const body = raw.slice(fm ? fm[0].length : 0).trim();
  parts.push('---', '', `# ${title}`, '', `Source: ${SITE}/${slug}`, '', body, '');
}

const out = parts.join('\n');
await mkdir('public', { recursive: true });
await writeFile('public/llms-full.txt', out, 'utf8');

const bytes = Buffer.byteLength(out);
console.log(`llms-full.txt: ${files.length} pages, ${(bytes / 1048576).toFixed(2)} MB`);
if (bytes > MAX_BYTES) {
  console.error('Over 5 MB. Split it per section before shipping.');
  process.exit(1);
}

What this does: produces a file an agent can read top to bottom, with an H1 and a one-line summary so it knows what it is holding, and a Source: URL above each page so the model can cite the human page rather than the text dump. The size assertion is the part that matters in production: without it the file quietly grows past what any model will accept.

Documentation framework plugin, Astro Starlight

If your docs already run on a framework, use its plugin rather than a file walk. The plugin renders each page the way the site does, so components, code fences and includes resolve correctly, and it can emit scoped subsets in the same pass. This example uses starlight-llms-txt 0.11.x, which needs Astro 7 and @astrojs/starlight 0.41 or newer. VitePress has vitepress-plugin-llms and Docusaurus has docusaurus-plugin-llms, both shaped the same way.

javascriptastro.config.mjs

// npm i starlight-llms-txt
import starlight from '@astrojs/starlight';
import { defineConfig } from 'astro/config';
import starlightLlmsTxt from 'starlight-llms-txt';

export default defineConfig({
  // Required: the generated files use absolute URLs.
  site: 'https://example.com/',
  integrations: [
    starlight({
      title: 'Example Docs',
      plugins: [
        starlightLlmsTxt({
          projectName: 'Example',
          description: 'Server library for building agent-facing APIs.',
          // promote and demote reorder llms-full.txt and llms-small.txt, so the
          // pages worth reading survive if an agent truncates the fetch.
          promote: ['index*', 'getting-started*'],
          demote: ['reference/changelog*'],
          // Strip rendered chrome. The `full` bucket applies to llms-full.txt;
          // an array here would only affect llms-small.txt.
          customSelectors: { all: ['.sponsors-banner', 'interactive-demo'] },
          // Scoped subsets, linked from llms.txt, for models that cannot take
          // the whole corpus in one context.
          customSets: [
            { label: 'Reference', paths: ['reference/**'], description: 'API reference only.' },
            { label: 'Tutorial', paths: ['tutorial/**'], description: 'Step by step build.' },
          ],
          pageSeparator: '\n\n---\n\n',
        }),
      ],
    }),
  ],
});

What this does: emits /llms.txt, /llms-full.txt, /llms-small.txt and one file per custom set on every build, and writes a Documentation Sets section into llms.txt that links to all of them. That link is the discovery path: there is no registered link relation for a full-content file, so an agent finds it because your index names it.

On-demand endpoint with conditional requests

When content lives in a CMS or a database there is no directory to concatenate, so the file has to be assembled by the server. The risk is cost: agents refetch, and rebuilding several megabytes of markdown on every request is wasteful. Handle it with a revalidation window, a strong ETag and a 304 for repeat callers. This is a Next.js App Router route handler, verified against the 16.x route.js reference.

javascriptapp/llms-full.txt/route.js

import { createHash } from 'node:crypto';
import { getPublishedDocs } from '@/lib/docs';

export const runtime = 'nodejs';
export const revalidate = 3600; // rebuild the corpus at most once an hour

const SITE = 'https://example.com';

function render(pages) {
  const head = [
    '# Example Docs',
    '',
    `> Full text of every published page. Curated index: ${SITE}/llms.txt`,
    '',
  ];
  const bodies = pages.map((p) =>
    ['---', '', `# ${p.title}`, '', `Source: ${SITE}${p.path}`, '', p.markdown.trim()].join('\n')
  );
  return head.concat(bodies).join('\n') + '\n';
}

export async function GET(request) {
  // getPublishedDocs applies the same filter as the sitemap query:
  // no drafts, no noindex, no gated pages.
  const body = render(await getPublishedDocs());
  const etag = '"' + createHash('sha256').update(body).digest('base64url').slice(0, 27) + '"';

  if (request.headers.get('if-none-match') === etag) {
    return new Response(null, { status: 304, headers: { ETag: etag } });
  }

  return new Response(body, {
    headers: {
      'Content-Type': 'text/markdown; charset=utf-8',
      'Cache-Control': 'public, max-age=3600, stale-while-revalidate=86400',
      'Content-Disposition': 'inline',
      ETag: etag,
    },
  });
}

What this does: serves the corpus as text rather than as an HTML page, which is the single check most implementations fail, and gives repeat fetchers a cheap 304 instead of another multi-megabyte transfer. Content-Disposition: inline stops browsers offering the file as a download when the type is text/markdown.

Implementation guidelines

These are the failures that show up once the file is live rather than in local testing.

Generate it in the build, never by hand. A committed file drifts from the site within a release or two, and a stale corpus is worse than no corpus because the model has no way to tell it is reading last quarter's API. Regenerate in the same CI step that builds the site.
Reuse your sitemap's exclusion rules. Drafts, noindex pages, gated content, thank-you pages, internal runbooks and duplicate translations should be filtered by the same predicate that builds your sitemap, so the two can never disagree about what is public.
Order for truncation, not for navigation. Agents that hit a context limit keep the beginning and lose the end. Put the overview, quickstart and core concepts first and push changelogs, migration notes and deprecated pages to the bottom.
Cap the size, then split by path prefix. Once the root file passes a few megabytes it stops fitting in one context, so publish scoped files alongside it: /docs/llms-full.txt, /api/llms-full.txt. Cloudflare does exactly this, with a 57 MB root file and a 4.9 MB one under /workers/.
Serve it as text and verify the header. Use text/markdown; charset=utf-8 or text/plain; charset=utf-8. A single-page-app catch-all route that returns the HTML shell is the most common failure, and it looks like a 200 to your monitoring. Check with curl -sI https://example.com/llms-full.txt after every deploy.
Cache it like a build artefact. Set a real Cache-Control max-age, emit an ETag or Last-Modified, honour If-None-Match, and purge the CDN on deploy. Anthropic's docs serve theirs with max-age=3600; Perplexity's uses 86400.
Link it from llms.txt. The v2 proposal registers rel="describedby" for the index file and rel="alternate" type="text/markdown" for per-page markdown, but nothing for a full-content file. The only reliable discovery path is a named link inside /llms.txt, so put one there.

Do this, not that

Do

Open the file with an H1 and a one-line blockquote summary, then a Source: URL above each page body so the model can cite the human page.
Assert the byte size in CI and fail the build when it crosses your threshold, rather than discovering the growth from a support ticket.
Publish scoped files per product or section once the root file passes a few megabytes, and name them all in /llms.txt.
Keep code fences intact and unminified: a truncated or reflowed code block is worse than an omitted one.

Do not

Do not let a client-side router answer /llms-full.txt with the HTML shell. Agents and Glippy both read that as the file not existing.
Do not paste rendered HTML in, or the navigation menu, cookie banner, sponsor block and per-page table of contents that come with it.
Do not include pages you keep out of your sitemap, and do not include a second copy of every page in another language in the same file.
Do not serve it as application/octet-stream or with no charset: some fetchers refuse the first and mis-decode the second.

How Glippy checks this

Glippy probes /llms-full.txt as part of its site-level agent readiness fetch and scores the result under Machine Readability, category 6. It is a bonus check worth 4 points: a file that exists and is served as text scores the full 4; a file larger than 5 MB scores 2 with a warning that it may exceed agent context windows; a file that comes back as HTML, which is what a soft 404 or a single-page-app fallback produces, scores 0 and is flagged as needing a text/markdown or text/plain content type. If there is no file at all, Glippy records it as informational with no maximum score attached, so its absence cannot drag your score down. Full detail on the category is on the machine readability checker page.

Check your llms-full.txt setup

Glippy runs 240+ checks across 16 categories on any page, including Machine Readability (category 6). No sign-up required.

Try the Glippy Chrome extension

Frequently asked questions

Is llms-full.txt part of the llms.txt specification?

No. The proposal at llmstxt.org, revised to v2 in August 2026, defines only /llms.txt and per-page markdown versions of pages. The full-content file was introduced by the documentation platform Mintlify and spread from there, and it is now served by Cloudflare, OpenAI, Anthropic, Perplexity and the Svelte docs among others. Treat it as a de facto convention with wide adoption rather than a specified file, and do not expect a validator to enforce a format.

How large can llms-full.txt be before agents stop using it?

There is no defined limit, but a few megabytes of markdown is already on the order of a million tokens or more, which is beyond what most models will accept in one context. Real files measured in August 2026 range from roughly 1.2 MB for the Svelte docs to about 57 MB for the root file at developers.cloudflare.com. Anything past a few megabytes is realistically for bulk indexing and vectorisation rather than for pasting into a chat, so publish scoped per-section files alongside the root one. Glippy warns above 5 MB.

Should llms-full.txt be served as text/markdown or text/plain?

Either is fine, and both are used in production: Cloudflare sends text/markdown; charset=utf-8 while OpenAI, Anthropic and Svelte send text/plain; charset=utf-8. RFC 7763 registers text/markdown and makes the charset parameter required, so always include it. What matters far more than the choice between the two is that the response is not text/html, which is what a soft 404 or a client-side router fallback returns.

Do I still need llms-full.txt if I publish per-page .md files?

They serve different access patterns. Per-page markdown at page.md or page.html.md, advertised with rel="alternate" type="text/markdown", is what an agent uses when it already knows which page it wants, and it is the pattern the v2 proposal actually specifies. The full file is for the cases where the agent does not know yet: bulk ingestion, building a vector index, or a developer pasting one URL to give a model the whole product. Publish both if you can, since generating one is usually a by-product of generating the other.

Reviewed against the primary sources on 27 August 2026. These standards move quickly, so check the linked specs before you ship.