Leadtype
Reference

LLM files

The leadtype/llm entry point produces five flavors of agent-facing output — four derived from a docs source tree, plus one that takes pages directly:

  • generateLlmsTxt — for hosted websites. Emits the /llms.txt convention with root-relative markdown mirror links.
  • generateLLMFullContextFiles — root /llms-full.txt fallback containing all generated markdown docs. Pairs with generateLlmsTxt.
  • generateAgentReadabilityArtifacts — root sitemap.xml, sitemap.md, robots.txt, and docs-scoped JSON manifest data that a host app can merge into site-level files.
  • generateAgentsMd — for npm-bundled docs. Emits an AGENTS.md index with relative ./docs/<topic>.md paths so the file works inside node_modules/<pkg>/.
  • generateAgentArtifacts — for sites without a docs tree. Takes an in-memory page list (CMS, database, route manifest) and emits the full artifact set in one call.

The CLI's default leadtype generate calls the website APIs; leadtype generate --bundle calls generateAgentsMd. Use these APIs directly when you need finer control.

What gets generated

FilePurpose
<out>/llms.txtTop-level routing index. Product summary, best starting points, and root-relative markdown mirror links.
<out>/docs/llms.txtDocs-scoped routing index. Lists every group with descriptions and markdown entry links.
<out>/llms-full.txtAll generated markdown docs flattened into one fallback file. Use when page-level links are not enough.
<out>/sitemap.xml / <out>/sitemap.mdOrigin-root crawler discovery files with canonical page URLs and docs navigation.
<out>/robots.txtOrigin-root crawl policy that points crawlers at <out>/sitemap.xml.
<out>/docs/agent-readability.jsonStructured manifest for host apps that need to merge docs pages with marketing, blog, changelog, or product pages.

Config navigation drives llms.txt sections, the sidebar, sitemap markdown, Agent Readability navigation, and AGENTS.md when present. Legacy group metadata remains the fallback for projects that have not adopted navigation, and still works as search/filter taxonomy. Site mode no longer emits per-group full-context files by default. See Evals for the benchmark behind that default.

Optional sections

Mark a navigation node optional: true to flag it "safe to drop for shorter context". Its pages don't get their own heading in docs/llms.txt — they collapse into a single trailing ## Optional section (the llms.txt convention for low-priority links), so an agent under a tight context budget knows what it can skip first.

Use strings at the top level for pages that should not live under a group. Use index for docs/index.mdx or a nested index.mdx page in navigation. Leadtype strips the index segment from the public URL, so top-level "index" still links to /docs, and base: "guides", pages: ["index"] links to /docs/guides.

Framework sections

Use defineFrameworkNavigation when several framework docs share the same sections. It expands templates into a normal navigation node, so sidebar generation, llms.txt, sitemap output, remote sources, and validation all keep using the existing navigation contract.

Framework base values are still relative to the parent base, so the React quickstart above resolves to /docs/frameworks/react/quickstart. A page that starts with / still points at the collection root, which keeps shared pages such as /ai-agents outside the framework tree.

The flag applies to the whole subtree, so nested children of an optional node are flattened into ## Optional too. It affects docs/llms.txt only — the website sidebar, sitemap, and search still list every page normally.

product (identity)

The product block in docs.config.ts is the identity of the documented thing — authored once and reused across llms.txt, the JSON-LD graph, and the agent card.

NameTypeDefaultDescription
namerequiredstring-Product display name. Rendered as the H1 of llms.txt and AGENTS.md.
taglinerequiredstring-One-line description rendered as the blockquote under the H1, and reused as the agent-card / JSON-LD description.
homepagestring-Canonical product homepage. Agent-card `url` fallback when no MCP endpoint is set.
docsstring-Docs entry point. Becomes the agent-card `documentationUrl`. Defaults to `${baseUrl}/docs`.
repositorystring-Source repository URL. Emitted as JSON-LD `codeRepository` for libraries.
kind"library" | "app"-`library` emits JSON-LD `SoftwareApplication` plus `SoftwareSourceCode`; anything else emits `SoftwareApplication`. Defaults to `app`.
categorystring-JSON-LD `applicationCategory`, e.g. `DeveloperApplication`.

Two sibling top-level blocks complete the identity:

  • organization{ name, url?, email?, logo?, sameAs?, contactPoint?, address? }. Who publishes the product. Feeds the JSON-LD Organization node and the agent-card provider. email publishes a top-level organization contact field, sameAs links disambiguate the brand, contactPoint emits typed Schema.org ContactPoint entries (with contactType plus email or telephone), and address emits a PostalAddress when you have a real business address to publish. Distinct from product: the product is the documented thing, the organization is who stands behind it.
  • llms.sections — the authored llms.txt body (see below).

The low-level generators (generateLlmsTxt, generateLLMFullContextFiles, generateAgentReadabilityArtifacts, generateAgentsMd) consume an internal { name, summary, blocks } product shape — not the config product directly. resolveAgentInputs(config) bridges the two: it maps product.taglinesummary, llms.sectionsblocks, and organization + product (kind/category/repository) → JSON-LD and agent-card inputs. leadtype generate uses it internally; call it yourself when composing the generators by hand (see "Typical sequence").

llms.sections

llms.sections describes the body of llms.txt. It is one ordered array, so a section's position is just its index — there are no placement flags.

Each LlmsBlock is one of two kinds:

  • markdown{ type: "markdown", heading?: string, body: string }. The body is emitted verbatim under an optional ## heading. Use it for prose, bullet lists, popularity (GitHub stars, npm downloads), a "Hosted by …" mention, community links, or badges. leadtype never fetches or computes values — supply them yourself (you can fetch at build time in your .ts config module, since it's just code).
  • links{ type: "links", heading: string, links: CuratedLink[] }. A curated link list under a ## heading, resolved against your source docs — titles and descriptions auto-fill from each page's frontmatter and URLs are rewritten for the website (/docs/<topic>.md) or the offline bundle (./docs/<topic>.md).

In AGENTS.md the same sections render as the product-content body, with generateAgentsMd adding the offline preamble and navigation around them.

Example llms.txt

This is the output of the llms.sections config above — each section becomes an ## heading, in array order, with links sections resolved to markdown mirror links:

Typical sequence

generateLLMFullContextFiles and generateAgentReadabilityArtifacts read from <outDir>/docs/, so always run conversion first.

Mounted URL prefixes

By default, generated markdown under <outDir>/docs/ maps to public URLs under /docs. Use mounts when part of that generated tree should have a different canonical URL, such as a changelog that lives beside docs in the source repo:

docs/docs.config.ts

With that mount, public/docs/quickstart.md is still /docs/quickstart.md, while public/docs/changelog/v1.md is described as /changelog/v1.md in llms.txt, llms-full.txt, navigation, sitemap data, Agent Readability metadata, and search metadata.

Custom build scripts should pass docsConfig.mounts to every artifact generator that reads generated markdown:

resolveDocsNavigation and resolveDocsTableOfContents also accept mounts so sidebars and page metadata use the same URL paths. If you use the CLI with docs/docs.config.ts, the config-level mount is loaded automatically and the CLI writes static markdown mirrors at public/changelog/*.md. For multiple sibling source folders, use --docs-dir changelog=/changelog; it creates the equivalent mount from the command line.

generateAgentReadabilityArtifacts

For hosted docs sites that need the Vercel Agent Readability discovery layer without letting leadtype own the whole website:

Writes crawler files at the output root (<outDir>/robots.txt, <outDir>/sitemap.xml, <outDir>/sitemap.md). The returned manifest contains the docs pages, markdown mirror URLs, group navigation, and freshness dates. Host apps can still merge result.manifest.pages with non-docs routes before serving custom root-level /sitemap.xml, /sitemap.md, and /robots.txt.

generateAgentArtifacts

For sites without a .mdx docs tree — CMS-backed blogs, data-driven pages, marketing sites. Takes an in-memory page list instead of a source directory and emits the full artifact set: llms.txt (+ the /.well-known/llms.txt copy), one markdown mirror per page at ${urlPath}.md with canonical_url/last_updated frontmatter, sitemap.xml, sitemap.md, robots.txt, and a root-level agent-readability.json:

Set emitRootCrawlerFiles: false when the site is one fragment of a larger origin (microfrontends) and the host app owns the root crawler files. See Generate artifacts without a docs tree for page-input fallback rules and the merge story.

Agent readability helpers

generateAgentReadabilityArtifacts gives you the manifest. The runtime helpers turn it into Web Response objects your framework can return directly:

The whole module is fs-free and edge-runtime safe: it works in Node, Bun, Vercel Edge, Cloudflare Workers, and any framework with a Web Request/Response API.

createAgentMarkdownResponse

Markdown content negotiation for docs pages. Returns Response | null (null means the path is not an agent-oriented markdown request — fall through to your HTML route). readMarkdownFile may be sync or async.

Headers it sets:

  • Content-Type: text/markdown; charset=utf-8
  • Vary: Accept (and , User-Agent when an AI UA is detected)
  • Link: <canonical>; rel="canonical"
  • Cache-Control: public, max-age=300, must-revalidate (override with cacheControl: "<directive>" or omit with cacheControl: null)

It also enriches frontmatter with canonical_url and last_updated, leaves discovery artifacts (llms.txt, sitemap.*, robots.txt, agent-readability.json, root llms-full.txt, search JSON) alone, and returns a 200 markdown "Page not found" body only for agent-oriented requests.

Optional userAgentPattern overrides the default AI user-agent regex (defaults cover GPTBot, ChatGPT-User, OAI-SearchBot, Claude-Web, ClaudeBot, anthropic-ai, CCBot, Google-Extended, AmazonBot, Bingbot, MetaExternalAgent, ByteSpider, PerplexityBot, MistralBot, AppleBot, YouBot).

createSitemapXmlResponse / createSitemapMarkdownResponse / createRobotsTxtResponse

Build sitemap and robots responses at request time so the <loc> and Sitemap: directives reflect the live origin (preview, staging, prod), no string-replace hacks needed.

requestOrigin is optional — when omitted, the helpers fall back to manifest.baseUrl.

createDocsJsonLd / stringifyJsonLd / createDocsHead

Build JSON-LD, canonical links, markdown alternate links, and OpenGraph-style metadata for a docs page from the manifest.

Reach for createDocsJsonLd when your framework wants a structured object, then pass it through stringifyJsonLd for safe <script type="application/ld+json"> contents. createDocsHead remains the generic { meta, links } helper for head APIs that accept descriptor arrays.

If the page is not in the manifest, both arrays are empty so the caller can fall back to its own metadata.

The default JSON-LD uses TechArticle and includes title, description, canonical URL, last modified time, product website, and breadcrumbs. Override fields only when your site has better product-specific data such as authors, publisher, OpenGraph image, publish date, keywords, or a different Schema.org type.

Lower-level helpers

For a step-by-step integration guide, see Optimize docs for agents.

HelperUse it for
createDocsJsonLdReturn page JSON-LD by urlPath, with per-site overrides.
stringifyJsonLdEscape JSON-LD for safe script contents.
renderJsonLdReturn the default Schema.org object for a manifest page.
renderJsonLdScriptReturn an escaped <script type="application/ld+json"> string.
createAgentMarkdownResponseBuild a complete markdown Response for agent Accept headers, AI user agents, and direct .md URLs.
createSitemapXmlResponse / createSitemapMarkdownResponse / createRobotsTxtResponseBuild sitemap/robots Responses rebased to the live origin.
createDocsHeadBuild canonical/alternate/og/JSON-LD head entries from the manifest.
resolveMarkdownMirrorTargetMap /docs/foo or /docs/foo.md to the generated docs/foo.md file path.
createMarkdownResponseHeadersProduce canonical markdown headers with Content-Type, Vary, Link, Cache-Control.
enrichMarkdownFrontmatterAdd canonical_url and last_updated aliases to generated markdown (CRLF-tolerant).
renderMissingMarkdownRender the 200 markdown body used for missing docs pages requested by agents.
acceptsMarkdownHeaderDetect whether an Accept header is asking for markdown (q-value aware).
isAgentUserAgentDetect known AI crawler and agent user agents. Accepts an optional regex override.
isAgentReadabilityArtifactPathAvoid rewriting llms.txt, llms-full.txt, sitemap, robots, search JSON, and manifest artifact paths.

Cache-Control and CDN

Every helper sets Cache-Control: public, max-age=300, must-revalidate by default and pairs it with Vary: Accept (and , User-Agent for AI UAs). When you cache responses on a CDN, make sure it shards by Accept and User-Agent so agents and browsers don't cross-pollinate cached HTML and markdown variants. Pass cacheControl: "<directive>" to override or cacheControl: null to omit the header (when you set caching at the CDN layer).

Manifest version

Every helper asserts manifest.version === 1 at the entry point and throws a clear error otherwise — so a stale dev agent-readability.json from a different leadtype version fails loudly instead of silently producing wrong responses.

generateAgentsMd

For npm-bundled docs that ship inside node_modules/<pkg>/ and need relative filesystem links:

Writes <outDir>/AGENTS.md (singular, at the root) with the same product summary and group structure as generateLlmsTxt, but every link is a relative path like ./docs/<segment>/<slug>.md instead of an absolute URL.

OptionDescription
srcDirRepo root containing the docs/ source.
outDirPackage root. AGENTS.md is written at <outDir>/AGENTS.md.
productSame ProductInfo shape as generateLlmsTxt. The blocks array renders here too, so author offline-friendly content (relative links, no website-routing prose).
groupsGroup tree from docs.config.ts. Same shape as generateLlmsTxt.
docsSubdirSubdirectory under outDir that holds the .md files. Default: docs. Used as the relative-path prefix in every link.

Returns { outputPath: string } pointing at the written AGENTS.md.

Example output

Pair with convertAllMdx to produce the .md files the links resolve to. See Bundle docs into a package for the full flow.

resolveDocsNavigation

Same nav-resolution logic the LLM files use, but returns the navigation manifest as a plain object — useful for driving top nav tabs and sidebar UI:

Write the result to src/generated/docs-nav.json and import it from your docs shell:

Now your top nav and sidebar can import a static manifest with the same nav tree the LLM files use. A common pattern is to render root nav nodes as top-level tabs, then render the active root node's pages and children in the sidebar.

Tables of contents

For the heading slug contract and renderer wiring, see Use the source primitive. This section covers the build-time APIs only.

resolveDocsNavigation includes a toc array on every page by default. The default range is h2h3, which keeps page-level h1 titles out of sidebars.

If you only need TOC data and not the full navigation tree, call resolveDocsTableOfContents:

For custom pipelines, extractDocsTableOfContents accepts a markdown or MDX string plus page URL metadata and returns plain JSON. It ignores frontmatter and fenced code blocks, and it uses the same slugifyDocsHeading helper that rendered headings must use to keep id attributes in sync.

Group design

The groups you pass to these APIs come from docs.config.ts. Two principles:

  • Use groups for routing, not sharding. Groups organize llms.txt, navigation, search metadata, and AGENTS.md. The root llms-full.txt remains the broad fallback. Per-group full-context bundles and an llms-full.txt router are not recommended: the eval run shows agents bypass them (context match 26–28% vs. 83–100% for page links, the monolith, and section indexes), so leadtype doesn't emit them and you shouldn't hand-roll them.
  • Write group descriptions for routing, not flavor text. Agents read those descriptions to decide which pages to load. "How to install and run" beats "Welcome to our guides!"

Base URL precedence

Pass baseUrl explicitly, or use environment variables for layered fallback:

The package-specific LEADTYPE_AGENT_BASE_URL lets each package override an org-wide default. BASE_URL covers most CI/deployment platforms, and a final hardcoded fallback keeps local builds working without env setup.