llms.txt: The Single File 95% of Businesses Are Missing
Of the 500 businesses scanned by AgentHermes, fewer than 25 serve an llms.txt file at their root. That one file is the fastest, cheapest way to make your site readable to AI agents — and it takes ten minutes to write. Here is the standard, a template you can copy, and the exact score impact we measure across every dimension.
The 95% Problem
AgentHermes has scanned 500 businesses across 27 verticals. The average Agent Readiness Score is 43 out of 100. Only one business scored Gold — Resend at 75. Most sites lose points because they are invisible to agents, not because their product is bad.
The easiest fix, the one we recommend before anyone touches OpenAPI or MCP, is to publish an llms.txt file. It is a plain markdown document served at your root that tells AI systems what your site is about and where the important parts live. Fewer than 5% of the businesses we scan have one. The ones that do score, on average, 8 to 12 points higher across the Discovery and Agent Experience dimensions.
The file was proposed by Jeremy Howard of Answer.AI in late 2024 as an AI-readable counterpart to robots.txt. Unlike robots.txt, which gates access, llms.txt guides attention. It tells a model: when someone asks about us, here are the pages you should prefer, the docs you should cite, and the APIs you should call.
Anatomy of an llms.txt File
The standard defines six canonical sections. You do not have to use all of them, but each one exists for a specific reason: a different question the agent might be trying to answer when it fetches your file.
# Project
H1 line with the name of your business or product. Agents use this as the canonical identity for search results and citations.
Example: # AgentHermes
> One-line summary
A blockquote with a single sentence that describes what you do. This is what agents quote when they mention you in an answer — treat it like your meta description.
Example: > The Agent Readiness Score for every business on the internet.
## Overview
A paragraph or two that gives agents enough context to answer the most common questions about you without crawling the full site.
Example: Context paragraphs explaining what the product does, who it is for, and key differentiators.
## Docs
Markdown links to your primary documentation pages. Each link gets a short description so the agent knows when to fetch it. Keep to 5 to 15 links.
Example: - [Quickstart](https://agenthermes.ai/docs/quickstart): Score your site in 60 seconds.
## API
Direct links to OpenAPI specs, MCP endpoints, agent cards, and SDKs. This is where agents go when they need to call something, not read about something.
Example: - [OpenAPI](https://agenthermes.ai/openapi.json): Machine-readable spec for 55+ endpoints.
## Examples
Short task recipes so agents can copy a working pattern instead of guessing. Each example is a heading plus a code block or direct URL.
Example: ### Score a domain: GET /api/score?domain=example.com
The order matters. Agents that fetch the file often read top-down and truncate at context limits. Put your identity, summary, and highest-value links first. Long prose and deep cross-references go in a companion /llms-full.txt that some agents fetch after the summary.
Copy This llms.txt Template
This is the exact structure we use at agenthermes.ai/llms.txt. Replace the AgentHermes details with your own. Ship it to the root of your domain as plain text and you are done.
# AgentHermes > The Agent Readiness Platform. Score, fix, and connect any business > to the agent economy in under 60 seconds. ## Overview AgentHermes is the FICO of the agent economy. We score every business across 9 dimensions — discovery, API quality, onboarding, pricing, payment, data, security, reliability, and agent experience — and help them close the gaps with auto-generated MCP servers, agent cards, and structured adapters. Tiers: Platinum 90+, Gold 75+, Silver 60+, Bronze 40+. ## Docs - [Quickstart](https://agenthermes.ai/audit): Score any domain in 60 seconds. - [Standard](https://agenthermes.ai/standard): The agent-hermes.json spec. - [ARL Levels](https://agenthermes.ai/blog/arl-levels-explained): 0 Dark to 5 Interoperable. - [Dimensions](https://agenthermes.ai/blog/what-is-agent-readiness): The 9 weighted dimensions. - [For Verticals](https://agenthermes.ai/for): Agent readiness guides per industry. ## API - [OpenAPI](https://agenthermes.ai/openapi.json): Machine-readable spec for 55+ endpoints. - [MCP Server](https://agenthermes.ai/api/mcp): Hosted Model Context Protocol endpoint. - [Agent Card](https://agenthermes.ai/.well-known/agent-card.json): A2A v0.3 agent card. - [NLWeb](https://agenthermes.ai/api/nlweb?q=): Natural-language query endpoint. ## Examples ### Score a domain GET https://agenthermes.ai/api/score?domain=example.com ### Fetch the leaderboard GET https://agenthermes.ai/api/leaderboard?limit=50 ### Generate an agent-card.json for your business POST https://agenthermes.ai/api/generate/agent-card
Keep every link absolute. Agents that fetch your llms.txt do not always parse the current page context — they expect fully qualified URLs so they can follow up with additional fetches. Use markdown links with a colon-separated description so the agent knows when the link is relevant without opening it.
llms.txt vs robots.txt: They Are Not the Same File
A common mistake is treating llms.txt as a second robots.txt. It is not. They serve different audiences, use different formats, and affect different parts of your readiness profile.
You should serve both — plus an agent-card.json and, ideally, an AGENTS.md at the root of your repository. AgentHermes detects all four during a scan and awards separate points for each. A site with robots.txt plus llms.txt plus agent-card.json signals that you understand both the old web and the new one.
If you also run AI crawlers like GPTBot, anthropic-ai, or Google-Extended against your own content, make sure your robots.txt allows them. Some teams accidentally block the same crawlers they are trying to reach with llms.txt. Align the two files.
How llms.txt Affects Your Agent Readiness Score
AgentHermes checks for llms.txt on every scan. When the file is present, well-formed, and returns 200, we award points against two of the nine dimensions:
Agents can find what matters
llms.txt explicitly lists the URLs you want cited — docs, APIs, agent cards. That removes guesswork for crawlers and raises Discovery from partial to full credit.
Agents get context without 40 fetches
A single llms.txt fetch replaces a dozen HTML crawls. That latency reduction is a direct signal of a site built for agent workflows, not just search engines.
Compounding effect: every link you add to your llms.txt improves the score of those destinations too — because agents follow the links and discover your agent-card.json, your OpenAPI spec, and your MCP server in the same session. One file, multiple dimensions lifted at once.
The Family of AI Discovery Files
llms.txt
Markdown content map served at root. Guides which pages models should prioritize. This article.
agent-card.json
A2A-compatible agent descriptor at /.well-known/agent-card.json. Declares skills, input modes, and capabilities.
AGENTS.md
Markdown file in repo root documenting coding agent conventions, tools, and workflows. Read by Claude, Cursor, and others.
robots.txt for AI
Existing robots.txt with explicit rules for GPTBot, anthropic-ai, Google-Extended, CCBot, and ClaudeBot user agents.
openapi.json
Machine-readable REST API spec. D2 weight 0.15 — the highest-scoring signal in the whole system.
MCP endpoint
Model Context Protocol server — the interactive counterpart to static files. Tools, resources, prompts.
Each file answers a different question. llms.txt answers “what is this site about and where should I go?” agent-card.json answers “what can this agent do?” AGENTS.md answers “how do I work in this codebase?” Together they form a complete agent-readable surface area.
If you only ship one, start with llms.txt. It is the lowest-friction file, the easiest to maintain, and the one that compounds the fastest because every agent that reads it follows the links to your richer assets — including your MCP server.
Frequently Asked Questions
What is llms.txt?
llms.txt is a markdown file served at the root of your domain that tells large language models and AI agents what your site contains, where your documentation lives, and which examples they should prefer when answering questions about you. It was proposed by Jeremy Howard of Answer.AI in 2024 as an AI-readable counterpart to robots.txt. The file is plain markdown, easy to author, and already supported by major AI systems that fetch it when a user asks about your domain.
Where does the llms.txt file go?
Serve llms.txt at the root of your primary domain — for AgentHermes that is https://agenthermes.ai/llms.txt. The file must be publicly accessible, return a 200 status, and be served with content type text/plain or text/markdown. Some sites also publish a /llms-full.txt with the complete prose of key docs embedded so agents can answer deep questions without additional fetches.
How is llms.txt different from robots.txt?
robots.txt tells search crawlers what not to index. llms.txt tells AI models what to prioritize when answering questions about your business. robots.txt is plain text with allow and disallow rules; llms.txt is markdown with sections like Overview, Docs, API, and Examples. They are complementary, not competitive. You should have both, plus an agent-card.json for richer agent metadata.
Does llms.txt improve my Agent Readiness Score?
Yes — directly. AgentHermes detects llms.txt during every scan and awards points against D1 Discovery (0.12 weight) and D9 Agent Experience (0.10 weight) combined. In our data from 500 scanned businesses, sites with a well-structured llms.txt score on average 8 to 12 points higher than sites without one, holding every other factor constant. It is one of the highest-leverage discovery files you can publish in under an hour.
What is the minimum viable llms.txt file?
The smallest useful llms.txt has four elements: an H1 with your product name, a blockquote with a one-line summary, an Overview paragraph, and a Docs section with three to five markdown links. Everything else is optional but helpful. You can ship a minimum viable file in 10 minutes, then iterate as you learn which questions agents ask most about your business.
Ship llms.txt, then fix the rest
Run a free Agent Readiness scan. See exactly which discovery files you are missing, what your score would be with them, and get an auto-generated llms.txt in the report.