llms.txt Guide — How to Write One for Your Website

llms.txt is a Markdown file served at your site root (e.g., https://example.com/llms.txt) that provides a curated summary of your website for Large Language Models. It was proposed by Jeremy Howard at Answer.AI and has been adopted by Stripe, Cloudflare, Anthropic, Vercel, and hundreds of other companies.

Unlike a sitemap (which lists URLs for crawlers) or robots.txt (which sets permissions), llms.txt provides context. It tells an LLM: "Here is who we are, what we offer, and where to find the important stuff." An agent that reads your llms.txt before interacting with your site will make fewer mistakes, hallucinate less, and complete tasks more reliably.

The file should be concise — ideally under 2,000 tokens. For sites with extensive documentation, a companion llms-full.txt file can provide the complete reference.

The format uses standard Markdown with a specific structure. Here are the required and recommended sections:

# Your Company Name

> Brief one-line description of what your company does.

## Docs

- [Getting Started](https://example.com/docs/getting-started): Quick start guide
- [API Reference](https://example.com/docs/api): Complete API documentation
- [Authentication](https://example.com/docs/auth): Auth flows and API keys

## API

- [OpenAPI Spec](https://example.com/openapi.json): Machine-readable API spec
- [MCP Server](https://example.com/.well-known/mcp.json): MCP discovery

## Products

- [Product A](https://example.com/products/a): Description of Product A
- [Pricing](https://example.com/pricing): Plans and pricing

## Optional

- [Blog](https://example.com/blog): Latest updates
- [Changelog](https://example.com/changelog): Version history

Key rules: The H1 heading must be your company or product name. The blockquote is a one-line summary. Each section uses H2 headings with bulleted lists of links. Every link should include a brief description after the colon.

The format is intentionally simple. LLMs parse Markdown natively — no special tooling required. The goal is not to replicate your entire site but to give an agent a map of the most important resources.

llms.txt has been adopted by major platforms including Stripe (links to all API docs and SDKs), Cloudflare (developer platform documentation), Anthropic (Claude API reference and guides), Vercel (Next.js docs and deployment guides), and Supabase (database and auth documentation).

The pattern is clear: developer-facing companies with APIs lead adoption because their users are already building with AI tools. But llms.txt is not limited to dev tools. E-commerce sites, SaaS platforms, and content publishers all benefit from telling agents what they offer.

The llms.txt directory at llmstxt.site tracks over 1,200 sites that publish the file. If your competitors are on that list and you are not, agents are learning about them and ignoring you.

In our AX Audit, llms.txt is one of seven files checked under Agent Discoverability, worth up to 12 of the 100 total points — shared across llms.txt, AGENTS.md, agent.json, agents.txt, agent-manifest.txt, ai-plugin.json, and agent-card.json, so no single file earns the full 12 on its own. Quality matters too: an llms.txt with broken links or outdated information is weaker evidence than one with accurate, well-structured content.

llms.txt also feeds the separate 4-point Token Efficiency check — specifically through a companion llms-full.txt, which gives agents a condensed, complete-context entry point instead of forcing them to crawl and parse your entire site.

You can write llms.txt by hand — it is just Markdown. But if you want a starting point, our AX Audit tool will analyze your site and suggest an llms.txt based on what it finds: your key pages, API docs, product listings, and documentation structure.

The generated file is a starting point, not a final product. Review it, remove irrelevant links, add missing resources, and make sure every link works. Then deploy it at your site root and re-run the audit to see the score improvement.