Skip to content
Bold PilotBold Pilot
Get started

llms.txt Generator

Your sitemap, turned into the file language models read first. Free, no sign-up.

Try , or

AI search guide

llms.txt: what the file is, who reads it, and how to write one that is worth reading

llms.txt is a Markdown file at the root of a site — next to robots.txt and sitemap.xml — that gives a language model a curated map: what the site is, in a sentence, and which pages are worth reading first, each with a line saying what it is for. The generator above writes one from your sitemap and the title and description of each page, and leaves it editable, because the editing is the part that matters.

The format

# Acme

> Invoicing for freelancers, with the tax side handled.

This file lists the pages worth reading first.

## Pages

- [Pricing](https://acme.io/pricing): Plans from $9, no per-invoice fees.
- [Security](https://acme.io/security): Where data lives and who can see it.

## Docs

- [Getting started](https://acme.io/docs/start): First invoice in five minutes.

## Optional

- [Changelog](https://acme.io/changelog): Every release, newest first.
  • One H1 with the site’s name. The only required line.
  • A blockquote with a one- or two-sentence summary — usually the homepage’s meta description, tightened.
  • H2 sections of link lists: - [Title](url): description. The description after the colon is what tells a model what the page is for without fetching it.
  • An Optional section for pages a model may skip when it is short on room.

A companion file, llms-full.txt, carries the full text of the important pages in one document. The generator writes the map; the full file is a separate, larger decision.

Who reads it — honestly

The proposal is from September 2024 and adoption is real but uneven. Documentation sites were first — Anthropic, Cloudflare, Stripe, Vercel and thousands of developer docs serve one — and developer tools such as coding assistants read them directly. The large answer engines have not committed to fetching it on every visit; what they do when it exists is not published. So the honest case for the file is this: it costs a few minutes, it cannot hurt, it is exactly what a model wants when it does look, and the sites that end up cited most are the ones that made themselves easy to read.

How the generator chooses pages

  1. It finds the sitemap through robots.txt, then /sitemap.xml, then /sitemap_index.xml, following one level of a sitemap index.
  2. It drops what a model has no use for: images, PDFs, feeds, tag and author archives, paginated lists, cart and account pages, anything with a query string.
  3. It keeps the shallowest pages first — the homepage, then top-level pages, then sections — up to sixty, and reads each one’s title and meta description.
  4. It groups by the first path segment: /blog/… becomes a Blog section, /docs/… a Docs section, top-level pages sit under Pages.

Then edit. Cut pages that do not earn a line, rewrite descriptions that only repeat the title, and move the long tail into an Optional section. Sixty links with sharp descriptions is a map; six hundred is a sitemap with extra steps.

Where to put it

  • Static hosting, Vercel, Netlify: put llms.txt in the public folder so it answers at /llms.txt with a text/plain or text/markdown content type. Next.js can also serve it from a route, the way this site does.
  • WordPress: upload it to the site root over SFTP, or use a plugin that serves it; several now do.
  • Docs platforms: Mintlify, GitBook, Docusaurus and Fumadocs generate one automatically; check /llms.txt before adding your own.

One trap: a single-page app that answers every unknown path with its HTML shell returns a 200 for /llms.txt that is not an llms.txt. The AI visibility checker catches that case and confirms the file parses.

Frequently asked

Does llms.txt replace robots.txt?

No. robots.txt says who may crawl; llms.txt says what is worth reading. A crawler that is refused in robots.txt never gets far enough to read the map. Set both — the robots.txt generator names the AI crawlers explicitly.

How often should it be regenerated?

When the site’s structure changes or an important page is added. Running the generator again and diffing against the edited file is the fastest way to see what is new.

Should every page be in it?

No. The file is a curated list, not an index; the sitemap already does the index. Put in what you would hand a new colleague on their first day.

Is there an official standard?

A proposal with a published spec at llmstxt.org, adopted widely enough to be a convention. There is no governing body and no certification; the checks the generator runs are the spec’s own structure.