Skip to content

Publish an llms.txt

What the scan checks: it looks for a file at https://yourdomain.com/llms.txt. Absent is a warning, not a hard fail, because adoption is still hedged, but publishing one is a cheap, professional signal.

llms.txt is a 2024 proposed standard for handing AI tools a curated, Markdown summary of your site and its most useful URLs. Adoption is real but uneven: Anthropic is the only major lab with a public commitment (ClaudeBot consults it to prioritize indexing), while GPTBot and PerplexityBot have been observed fetching it without a formal promise, and developer tools like Cursor and Continue actively consume it. The honest framing is a low-effort hedge: it costs almost nothing, signals professionalism to the labs, and the mechanism is likely to mature over 2026 and 2027.

Create a Markdown file and serve it at /llms.txt. The shape that works is an H1, a one-line blockquote pitch, then curated links with a description on each:

# Example Inc
> Example Inc builds AEO monitoring tools for marketing teams.
## Core pages
- [Pricing](https://example.com/pricing): Current per-seat plans and limits.
- [Docs](https://docs.example.com): Setup, API reference, integrations.
- [Case studies](https://example.com/customers): Named results by industry.
## About
- [Company](https://example.com/about): Team, funding, founding story.
  • Serve it as text/plain or text/markdown at the domain root, the same way robots.txt is served.
  • Write real descriptions. “Pricing, current per-seat plans and limits” tells the model whether the link is worth following. Bare labels do not.
  • Add /llms-full.txt only if your content is small (under ~200K tokens). For large sites, a full dump is performance theater.

The next step is authoring it well rather than dumping every URL. See author the llms.txt (L2).