Migrating to v0.17.0

v0.17.0 updates AFDocs from the Agent-Friendly Documentation Spec v0.3.0 to v0.5.0. This is a breaking release. The sections below cover what changed and what you need to update.

What changed for CLI users

If you run AFDocs from the command line and don't use the programmatic API, here's what you need to know.

Your scores may be different

You may see score changes if you're using the latest version of the CLI, even if you don't change anything about how you run AFDocs. In smoke testing across sites spanning the A through F grade range, we saw overall score changes between -10 and +8 points. Most sites changed by ±5 points or fewer, and grade-letter changes were uncommon. The scoring changes reflect refinements from community conversations and based on additional testing around how agents actually interact with documentation:

• Content discoverability scores will likely drop. The directive check split (see below) and coverage changes combine to lower this category for most sites. In testing, every site we checked saw a content-discoverability drop, ranging from 3 to 14 points.

• The directive check now scores HTML and markdown separately. The old llms-txt-directive check has been replaced by two checks: llms-txt-directive-html (checks HTML pages) and llms-txt-directive-md (checks markdown pages). Most sites have the directive in one format but not the other. A site that previously scored "warn" on the combined check may now get a pass on one and a fail on the other. For example, a site that embeds the directive only in its HTML pages will pass llms-txt-directive-html but fail llms-txt-directive-md. The HTML check also now ignores matches in navigation sidebars, which eliminates false positives that some sites were seeing.

• HTML page size and content start position scores may improve. AFDocs now strips <script> and <style> elements from HTML before measuring page size, matching a change to how Claude Code processes pages. Sites with heavy inline JavaScript or CSS will see smaller measured sizes, which could improve your scores.

• Coverage scores may change in either direction. Some paths that were previously excluded from llms.txt coverage calculations (/changelog, /releases, /security, /status) are no longer excluded by default. If your sitemap includes these paths but your llms.txt doesn't link to them, your coverage percentage will be lower. You can restore the old behavior with --coverage-exclusions "/changelog/**,/releases/**,/security/**,/status/**". On the other hand, sites that use nested llms.txt indexes (where a top-level llms.txt links to section-level llms.txt files) may see coverage improve, because the check now recognizes subtrees covered by nested indexes instead of counting every individual page against the denominator.

• Runs that discover few pages now show N/A for some categories. If AFDocs discovers fewer than 5 pages during automatic sampling, page-level categories (Page Size, Content Structure, URL Stability, etc.) show as N/A instead of producing scores based on too little data. Site-level checks (llms.txt, coverage, authentication) still score normally.

Check names that changed

If you use --check-ids or --skip-check-ids to run specific checks, or if you have a config file, two check names changed:

• llms-txt-freshness is now llms-txt-coverage
• llms-txt-directive is now llms-txt-directive-html and llms-txt-directive-md

If you use old names in --check-ids, --skip-check-ids, or your config file, you'll get an error telling you the check ID is unknown. Update to the new names.

New diagnostics in your output

The scorecard may now include these new diagnostics that weren't in previous versions:

• Markdown support is only partially discoverable: Your site supports content negotiation, but most agents (which fetch HTML by default) have no signal to try the markdown path. Add a directive to your HTML pages.
• Single-page sample: AFDocs couldn't discover enough pages to score reliably. Check that your llms.txt has working links, or provide pages with --urls.
• All llms.txt links are cross-origin: Common when testing staging/preview deployments. Use --canonical-origin pointing to your public documentation site to fix.
• Gzipped sitemap skipped: A .gz sitemap was found but couldn't be read. Provide an uncompressed sitemap alongside it.
• Severe rate limiting: The site is returning 429 errors. Increase --request-delay to slow down.

New CLI flags

You can now configure llms-txt-coverage and markdown-content-parity check thresholds from the command line:

• --coverage-pass-threshold, --coverage-warn-threshold, --coverage-exclusions
• --parity-pass-threshold, --parity-warn-threshold, --parity-exclusions

No existing flags were removed or renamed. For details on these options, see CLI Reference.

---

The rest of this page covers changes relevant to developers building on the AFDocs programmatic API or parsing JSON output.

Check ID changes

Two check IDs changed. If you reference check IDs anywhere (config files, --check-ids / --skip-check-ids flags, JSON output parsers, CI scripts), update them.

Old ID	New ID(s)	What happened
llms-txt-freshness	llms-txt-coverage	Renamed. Same check, new name.
llms-txt-directive	llms-txt-directive-html, llms-txt-directive-md	Split into two checks: one for HTML pages, one for markdown pages.

The split means the total check count went from 22 to 23, and the max raw score changed from 126 to 130. llms-txt-directive-html kept the original's High/7 weight. llms-txt-directive-md is Medium/4.

Config file changes

If your .afdocs.yml or .afdocs.yaml references old check IDs in checkIds, skipCheckIds, or URL-specific overrides, update them to the new IDs. Unknown check IDs now produce errors instead of being silently ignored.

New optional config fields are available:

# To configure 'llms-txt-coverage', you can define your own minimum coverage thresholds
coveragePassThreshold: 95 # 0-100, default 95 (higher = stricter)
coverageWarnThreshold: 80 # 0-100, default 80
coverageExclusions: # glob patterns excluded from sitemap denominator
  - '/api/internal/**'

# To configure 'markdown-content-parity', you can define your own maximum disparity thresholds
parityPassThreshold: 5 # 0-100, default 5 (lower = stricter)
parityWarnThreshold: 20 # 0-100, default 20
parityExclusions: # CSS selectors stripped from HTML before comparison
  - '[data-markdown-ignore]'

For more details about configuration, including turning these into informational-only checks, refer to the check documentation.

YAML quoting note: Selectors like [data-markdown-ignore] must be quoted in YAML. Without quotes, YAML parses brackets as a nested array. v0.17.0 validates this at config load time and produces a clear error.

CLI changes

New flags for the coverage and parity checks:

• --coverage-pass-threshold, --coverage-warn-threshold, --coverage-exclusions
• --parity-pass-threshold, --parity-warn-threshold, --parity-exclusions

No existing flags were removed or renamed.

Score changes you may notice

Even without changing your configuration, you may see different scores after upgrading. In smoke testing across sites at every grade level, overall scores changed by -10 to +8 points. Most sites moved by ±5 or fewer, and grade-letter changes were uncommon (1 downgrade and 1 upgrade out of 6 tested sites). Category-level changes can be larger than the overall change because categories offset each other.

Content discoverability will likely drop

The directive check split is the primary driver. The old llms-txt-directive (weight 7) has been replaced by llms-txt-directive-html (weight 7) and llms-txt-directive-md (weight 4). Most sites have the llms.txt directive in one format but not the other, so the common outcome is one pass and one fail where the old check was a single warn. For example, a site that embeds a <!-- For AI agents: ... --> comment in its HTML templates will pass the HTML check but fail the markdown check (since HTML comments don't appear in the markdown output). Conversely, a site that prepends a directive line to its markdown content will pass the markdown check but fail the HTML check if that line doesn't also appear in the rendered HTML.

In testing, content-discoverability dropped for every site we checked (between 3 and 14 points). To improve this category, add the directive in both HTML and markdown formats.

llms-txt-directive-html may produce different results than the old llms-txt-directive

Beyond the split, the HTML check now strips <nav>, <script>, and <style> elements from the HTML body before searching for directives. The old logic may have produced false positives for pages that passed due to mentioning llms.txt in navigation sidebars or in documentation about generating llms.txt files.

page-size-html and content-start-position scores may improve

v0.17.0 strips <script> and <style> elements from HTML before conversion, matching how agents (including Claude Code) actually process pages. Sites with heavy inline JavaScript or CSS will see smaller post-conversion sizes and earlier content start positions, which may improve scoring.

llms-txt-coverage scores may change in either direction

Coverage scores may drop because the built-in exclusion list was trimmed: /changelog, /releases, /security, and /status paths are no longer automatically excluded from the sitemap denominator. If your sitemap includes these paths but your llms.txt does not, coverage percentages will be lower. If you intentionally exclude URLs that include these paths from your llms.txt, you can restore the old behavior with --coverage-exclusions "/changelog/**,/releases/**,/security/**,/status/**".

Coverage scores may improve if your llms.txt uses nested indexes (where the top-level file links to section-level llms.txt files like /docs/guides/llms.txt). The coverage check now recognizes these subtrees and counts them as covered, rather than requiring every individual page to appear in the top-level file. One site in testing went from 30% coverage (fail) to 100% (pass) after this change.

Low page count runs now produce N/A scores

When automatic discovery (random or deterministic sampling) finds fewer than 5 pages, page-level checks get scoreDisplayMode: "notApplicable" and are excluded from the overall score. This prevents a handful of pages from producing unreliable category scores. Site-level checks (llms.txt checks, coverage, auth-alternative-access) continue to score normally.

Programmatic API changes

Input validation

createContext() and runChecks() now validate options and throw Error on invalid input. Previously, invalid values (NaN, negative concurrency, out-of-range thresholds) were silently accepted. If you pass hardcoded valid options, this won't affect you. If you pass user-controlled values, add try/catch or call the new validateRunnerOptions() for pre-flight validation:

import { validateRunnerOptions } from 'afdocs';

const result = validateRunnerOptions(options);
if (!result.valid) {
  // result.errors: Array<{ field: string; message: string }>
  // result.warnings: Array<{ field: string; message: string }>
}

Type changes

CheckScore has a new required field:

interface CheckScore {
  // ... existing fields ...
  scoreDisplayMode: 'numeric' | 'notApplicable';
}

CategoryScore fields are now nullable:

interface CategoryScore {
  score: number | null; // was: number
  grade: Grade | null; // was: Grade
}

A null score means all checks in that category were notApplicable (insufficient data).

ReportResult has two new optional fields:

interface ReportResult {
  // ... existing fields ...
  testedPages?: number;
  samplingStrategy?: SamplingStrategy;
}

evaluateDiagnostics() signature change

The function now requires the full ReportResult as a second argument:

// Before
evaluateDiagnostics(resultsMap);

// After
evaluateDiagnostics(resultsMap, report);

New exports

import {
  validateRunnerOptions, // pre-flight option validation
  VALID_SAMPLING_STRATEGIES, // ['random', 'deterministic', 'curated', 'none']
  SPEC_VERSION, // 'v0.5.0'
} from 'afdocs';

import type { ValidationResult, ValidationIssue, SamplingStrategy, ScoreDisplayMode } from 'afdocs';

CheckOptions new fields

All optional with defaults matching previous behavior:

interface CheckOptions {
  // ... existing fields ...
  coveragePassThreshold?: number; // default 95
  coverageWarnThreshold?: number; // default 80
  coverageExclusions?: string[];
  parityPassThreshold?: number; // default 5
  parityWarnThreshold?: number; // default 20
  parityExclusions?: string[];
}

URL normalization

canonicalOrigin and llmsTxtUrl options are now normalized by createContext() (scheme prepended if missing), matching existing baseUrl behavior. Bare domains like example.com now work where they previously may have caused issues.

JSON output changes

If you parse JSON output from the CLI or programmatic API:

• Check IDs llms-txt-freshness and llms-txt-directive no longer appear. Look for llms-txt-coverage, llms-txt-directive-html, and llms-txt-directive-md instead.
• llms-txt-coverage details no longer include thresholdWarnings. Threshold validation now happens before checks run.
• llms-txt-coverage details have new optional fields: coveragePassThreshold, coverageWarnThreshold, userExcludedPages, omittedSubtrees, omittedSubtreePages.
• llms-txt-links-resolve details crossOrigin object has a new optional field: dominantOrigin.
• markdown-content-parity details have new optional fields: segmentationElementsStripped, parityPassThreshold, parityWarnThreshold.
• markdown-content-parity no longer returns status: 'error' for inverted thresholds. Invalid thresholds are caught at startup.
• Resolution text changed for several checks. If you match on resolution strings, update your patterns.

Diagnostic changes

New diagnostic IDs that may appear in the diagnostics array:

• markdown-partially-discoverable (warning): fires when content negotiation works but there's no HTML directive pointing to llms.txt.
• single-page-sample (warning): fires when fewer than 5 pages were discovered.
• cross-origin-llms-txt (warning): fires when all llms.txt links point to a different origin.
• gzipped-sitemap-skipped (info): fires when a .gz sitemap was encountered and skipped.
• rate-limiting-severe (warning): fires when more than 20% of URLs returned 429.

Changed trigger: markdown-undiscoverable now fires when markdown-url-support passes and llms-txt-directive-html does not pass and content-negotiation does not pass. Previously it required all three of content-negotiation, the old combined directive check, and llms-txt-links-markdown to fail. Sites where content negotiation passes but the HTML directive is missing will now see markdown-partially-discoverable instead.