feat(cli): move the Vale engine to 3.18.0 and re-measure its format reach

[thecodedrift]

Stack (root → tip):

• feat(cli): state each local engine's reach in the routing recipe #155
  • fix(cli): require both fixture buckets for an ast-grep rule #156
  • fix(cli): stop one AsciiDoc file from disabling every Vale rule #157
    • ➡️ feat(cli): move the Vale engine to 3.18.0 and re-measure its format reach #161 (you are here)

Stacked on #157. Review only the last two commits; everything below is that PR.

Moves the Vale engine to 3.18.0-20260824195610 — the stamp release-vale.yml published when the manifest PR (#158) merged — and re-measures the format table against the new binary.

Two commits, deliberately: the pin, then the re-measure. They cannot be split into separate PRs. The pin alone changes what the binary does without changing what the table says it does, and for .typ that is the crashing direction (below).

Seven rows moved, in both directions

extension	3.17.1	3.18.0
.mdx	converter:mdx2vast	markup	now parsed natively
.typ	plaintext	converter:typst2vast	now needs an external program
.rmd	plaintext	markup
.qml	plaintext	comment
.scss	plaintext	comment
.qmd	—	markup	new
.qdoc	—	comment	new

.typ is the row that mattered. An extension missing from the table is read as prose, which is harmless. The moment Vale routes it to a converter, that same omission is an E100 that takes down every Vale rule in the run — the failure this table exists to prevent, arriving under an extension nobody was watching. That is what the table's doc comment warns about, and it is now a worked example rather than a hypothetical.

The recipes carried the inverse of both facts

They told an author MDX was unsupported — which now costs them a format Vale reads fine — and they used [*.{md,mdx}] as the worked example of a matcher that breaks check. That matcher is now perfectly good. The cautionary example is [*.{md,typ}], and the recipe says why it changed, because the dangerous extension is whichever one the rendered list names, not the one an agent remembers.

Measured, not read off the release notes

Every row was re-probed by its own discriminating property: markup by a construct only a parser skips, comment-aware by a bare non-comment line yielding zero, plaintext by that same line yielding a finding, converter-backed by the E100 exit and its checker tag.

That caught one change the release notes do not mention: PHP comment extraction now requires a real <?php tag, where 3.17.1 linted a bare // comment without one. The fixture is corrected rather than the row.

It also produced four false positives that were the probe's fault, not Vale's — .html, .htm and .xhtml need prose inside an element, and .php needs its open tag. Feeding a format the wrong construct is indistinguishable from absent support, which is exactly why each tier is asserted by its own discriminating property and the per-extension fixtures live in one file rather than two.

End-to-end

A project containing doc.mdx and paper.typ:

Notice: Vale did not check 1 file(s): paper.typ. These formats are not supported by this
build — Vale parses them only through an external program (typst2vast), which this build
does not ship and does not check for. …

  doc.mdx:1:4   warning[no-simply]
4 issues (1 error, 3 warnings) across 4 files

MDX linted under [*.{html,md,mdx}]; only the Typst file skipped.

Provenance

The six SHA256 digests in the manifest were verified against upstream's vale_3.18.0_checksums.txt before #158 was merged, and all six packages carry the identical stamp — no platform out of step. This was also the first exercise of the OIDC trusted-publishing path since it moved to the npm-autopublish environment; it succeeded.

Refs #151

Built on top of #155

An agent reading route.txt could not tell whether a language was buildable locally or had to escalate to runtime, which needs a login. The reporter's case: two GitHub Actions YAML rules were routed to runtime because nothing said whether ast-grep handles YAML.

It does. The pinned @ast-grep/cli@0.41.0 lists 26 languages including Yaml, and a real pr-workflow-no-branches-filter rule (language: Yaml) passes verify and test. Both Actions rules belong in create-sg-rule with no login.

Refs #151

Derived, not transcribed

There was no source of truth for either engine's reach. $defs.Language in the vendored schema is a bare string with no enum, verify never validates language, and detect --json reports the repository's languages in a different vocabulary. The binaries were the only authority, and route.txt said nothing.

Transcribing two lists into a .txt would drift on a version bump with nothing to catch it, and stale prose is worse than the current silence — silence at least makes the agent ask. So: pinned constants in src/rules/capabilities.ts, substituted into recipes as %(KEY)s, with vendor-contract tests asserting the constants still match the binaries. That is the pattern ast-grep-vendor-contract.test.ts already exists for — exact-pinned binary plus a test that fires on the bump.

capabilities.ts is deliberately import-free: prompts/recipes.ts imports it, and the assert-prompts-graph vite plugin fails the build if that chunk reaches a host capability.

Rejected alternative: extending detect --json with an engines block. cli-detect's spec frames detect as an offline repo scan, engine reach is knowable at build time, and it would cost a third subprocess turn in a recipe already spending two.

Vale's reach was measured, not read

Vale self-reports no format list, so every tier here comes from probing the shipped 3.17.1 binary — and the measurement corrected the starting assumptions in five places:

• .css is comment-tier, not plaintext. The first probe said otherwise because it fed CSS a // comment, which is not one.
• .sass is comment-tier; .scss is not.
• Extension matching is case-sensitive and not closed over aliases — .R is comment-aware, .PY/.JS/.TS are not; .cc/.cxx/.pyw/.pm are, .hh/.hxx are not.
• Markup tier is wider than assumed (.htm, .xhtml).
• --no-exit does not suppress the converter tier's E100 exit 2.

Each tier is pinned by its discriminating property, because on ordinary prose all three readable tiers look identical: markup by a construct only a parser skips, comment-tier by the negative (a bare non-comment line must yield zero), converter-tier by non-zero exit plus E100 plus the converter's name. A test also asserts the fixture map covers the markup constant exactly, so an unprobed entry cannot be added to a constant.

create-vale-rule.txt was teaching the crash

Its worked example globbed [*.{md,mdx}]. .mdx needs mdx2vast, which this CLI does not ship, and one such file fails the entire Vale pass — so any rule following that example silently disabled every other Vale rule. Now [*.{md,markdown}], with the point stated plainly: a matcher like that is not a wider [*.md], it is a broken one. (The guard that makes this non-fatal is in the PR above this one.)

A correction, included deliberately

The last commit walks back a claim this branch originally shipped. Both the recipe and the constants said a plausible C++ or yaml "passes verify and fails inside the binary." Measured at 0.41.0, C++ and cpp are accepted aliases that resolve to Cpp, so an off-list spelling is not reliably an error.

The two failures that are real, neither caught locally: a name ast-grep does not recognize (C# for CSharp) aborts config parsing, taking every other rule's report down with it; and a valid name for the wrong parser (TypeScript over a .tsx tree) reports nothing and reads as a clean codebase. The advice to copy from the list is unchanged; the reason for it is now the accurate one. The vendor-contract assertions were already right — they pin the canonical list, not what the binary rejects — but their comment said otherwise.

Also

route.txt now states that a language on neither list does not route to runtime by default: create-legacy-rule comes first, since the repo's own linter may already speak it. And a new note separates reach (a property of the pinned version) from availability (a property of the host), which the existing "availability you can assert" bullet was conflating while giving the agent no way to assert it.

OpenSpec delta on cli-rule-routing — engine reach becomes a fourth thing the recipe is specified to state, with the normative core being that it is derived from the pinned versions rather than transcribed. Not archived; this is not the tip of the stack.

Built on top of #157

Stacked on #155. Review only the last two commits; the base is that PR.

One .adoc, .rst, .mdx, .xml, .dita or .asc anywhere a Vale rule's matcher reaches disabled every Vale rule in the repo. Measured: a run reporting 5 Vale findings across 5 files dropped to 0 with one .adoc added, and check still exited on an unrelated ast-grep finding — so the disappearance read as a normal red check.

Vale genuinely supports these formats upstream. The failure is a missing external converter in our distribution, not absent support.

Refs #151

Mechanism, established before designing the fix

Vale aborts the whole run; our code discards nothing. On the first E100, stdout is 0 bytes and exit is 2 — the Markdown findings are destroyed inside Vale before serialization. That ruled out "tolerate the exit code and parse partial output," which was the obvious first guess.

Two further measured facts shaped it: the trigger is the config, not the file (with [*.{html,md}] an .adoc is inert; with [*] it crashes), and per-file invocation would cost N subprocesses. A Vale config setting ([formats] adoc = md) would lint AsciiDoc with the Markdown parser and emit nonsense. So: exclude the converter-dependent files from the set handed to Vale, and name the skip rather than letting it be silent.

Notice: Vale did not check 3 file(s): guide.adoc, legacy.asc, spec.rest. Vale supports these
formats, but parsing them needs an external converter (asciidoctor, rst2html) that this build
does not ship. … every other file was checked normally.
5 issues (1 error, 4 warnings) across 4 files

Two things that looked correct and were not

• Vale honours exactly one --glob, last-wins. Two flags silently drop the first.
• --glob matches the basename only when the pattern has no /. Combined with the existing .taskless/** exclusion the expression goes path-wise, at which point a bare *.adoc excluded root-level d.adoc but not docs/e.adoc — a fix that looks right in whichever directory you happened to test it in. Hence **/*.adoc. Both are now pinned by tests.

Assert known support, with one deliberate inversion

The operative list is the converter tier, not a native allowlist. Vale lints far more than markup — source comments, plain text, and extensionless files (README, LICENSE, Makefile) — so a positive allowlist would have to enumerate every language Vale knows and would still drop every extensionless file, trading a loud crash for silent disabling across a much larger set. Unknown-to-us is safe because it is unknown-to-Vale.

The assert-known-support property is carried instead by the table naming what was measured, the exclusion being derived from it so the two cannot drift, and a per-extension test that re-measures every row against the real binary.

The exclusion applies even to explicitly named paths, overriding the prior "an explicit path is a request" rule in run.ts. Justified in-comment: honouring the request does not check that file badly, it costs the user the rest of their check.

Vale's stderr JSON is now also decoded to a sentence (E201: 'level' must be one of [...] in .../bogus.yml), preserving the error code — following decode-sg-stderr.md.

Reconciliation with capabilities.ts

This branch and #155 independently measured Vale's tiers and produced two tables. The second commit merges them into one record in capabilities.ts, with the converter name riding in the tier ("converter:asciidoctor") so one row states both the tier and the program to install — adding an extension is one line in one file. formats.ts now holds no data at all; everything derives, and a test asserts the derivation.

Both tables were wrong about different things, which is a decent argument for having built them independently:

	verdict	evidence
.asc, .rest	converter-dependent — missing from capabilities.ts	exit 2, E100 [lintAdoc] / [lintRST]
.tex, .rmd	plaintext, not native — overclaimed here	lints its own % comment / an R chunk, where .md skips a fence
.mkd, .mkdn	plaintext, not native — overclaimed here	lint through a fence, an HTML comment and an indented block
.mdown	native — missing from capabilities.ts	fence/comment/indent all yield 0
.ditamap	plaintext, deliberately not excluded	exit 0; excluding it would drop a file Vale lints fine

The .mkd/.mkdn error was the more expensive direction: calling them markup promises a scope: that has nothing to act on.

A per-extension probe now covers every row (probes every row of VALE_FORMAT_TIERS asserts set-equality between the table's keys and what the suites measure), so a row cannot be added without being measured. The old bare-prose tier loop was removed rather than kept — bare prose cannot discriminate any of the three readable tiers, and leaving it would let a weaker fixture overrule a stronger one.

MDX and the next bump

MDX is described as not supported yet, with Vale 3.18.0 parsing it natively and a CLI update expected to carry it. No date promised — a test asserts the recipe contains no 20NN-NN.

⚠️ 3.18.0 also adds a Typst converter (typst2vast). .typ is plaintext today, so a bump without re-measuring walks straight back into this bug under a new extension. The table's doc comment carries that warning and names both known 3.18.0 moves (.typ → converter, .mdx → markup). Vale updates within the 0.11.x lineage, separately from this fix.

On textlint

There is no recorded comparison anywhere — zero mentions of textlint, proselint, LanguageTool or alex in the working tree, all history, all 40 archived changes. Vale appears already chosen, on gap-filling grounds plus one substantive criterion: its Tengo sandbox exposes only text/math/fmt, so a Vale rule is inert data, which is what lets it ship static-tier with no login gate.

The converter gap is a real dent in the "self-sufficient binary" reasoning that decision rested on — it anticipated engine-level unavailability, not a per-file input killing an available engine mid-run. But textlint's rules are JavaScript, which cannot be static-tier; they would need the runtime harness, signing, and a login. That is the load-bearing property and Vale wins on it. Not close to justifying a migration; worth recording in the spec.

OpenSpec: none

cli-vale-rule-engine already requires that engine trouble is reported and does not abort the run. The crash violated that; the fix restores it. One caveat flagged rather than papered over: the spec says a rule's scope is expressed through its own matchers, and this narrows that above the user's config, including for explicitly named paths. If that should be recorded, the natural home is next to "Vale check executes against an assembled run config over the target paths."

[thecodedrift]

@claude /review

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[claude]

Claude finished @thecodedrift's task in 7m 24s —— View job

---

Review complete

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json contains no review threads and no actionable prior findings — the only prior activity was the @claude /review trigger comment itself and a Copilot review that failed with a quota error and left no content. This is effectively the first substantive review of this PR, so everything below is [New].

• Read .prior-review.json
• Fetched the diff (origin/fix/vale-formats...origin/fix/vale-3.18, 9 files, matches the PR's stated +139/-68)
• Verified the version bump is consistent across package.json, pnpm-lock.yaml, and VALE_VERSION (all 3.18.0-20260824195610 / 3.18.0)
• Verified the format-tier table, recipe prose (route.txt, create-vale-rule.txt), and test fixtures for the 7 moved/added rows are internally consistent
• Ran parallel finder passes (line-by-line, removed-behavior, cross-file, cleanup, altitude, conventions) and verified candidates
• Posted inline comments; posting this summary

Overall assessment: the mechanical parts of this PR are solid — the version pin is consistent everywhere it needs to be, the single-source-of-truth table (VALE_FORMAT_TIERS) is the only place the tier data lives, the recipe .txt files correctly derive their rendered lists from that table via %(VALE_...)s placeholders (not hand-maintained duplicates), and the new/moved fixtures for .mdx/.qmd/.rmd/.qdoc/.php genuinely discriminate their claimed tier. I did not find any actual misrouting bug — no extension landed in the wrong tier, no fixture is testing the wrong thing.

What I did find is a cluster of stale doc comments in packages/cli/src/rules/capabilities.ts and one test file that the table edit passed by. This file's own header insists "MEASURED, NOT DOCUMENTED" is the whole point — and several of its own prose paragraphs are now the kind of transcribed-and-wrong text that principle warns against:

• capabilities.ts:161-176 — the JSDoc above VALE_FORMAT_TIERS still frames 3.18.0 as "the known incoming bump" and says none of the moved rows are "reachable from {@link VALE_VERSION}," but VALE_VERSION is "3.18.0" as of this diff. It also mispredicts .qdoc landing as markup (it measured as comment) and never mentions that no MyST row was added.
• capabilities.ts:142 — still lists .rmd as a plaintext-tier example ("look like markup and are not"), but this diff moves .rmd to markup at line 188.
• capabilities.ts:238-239 — "we pin 3.17.1" is now wrong.
• capabilities.ts:342-345 — the VALE_CONVERTER_DEPENDENT doc still uses [*.{md,mdx}] as the "takes down the entire Vale pass" example. .mdx left the converter tier in this same diff; .typ is the new dangerous one. Notably, route.txt and create-vale-rule.txt both got this exact example correctly swapped to .typ — this source-level comment is the one place it was missed.
• vale-vendor-contract.test.ts:391 and :567 — both still cite .rmd as a plaintext/non-markup example; .rmd moved to MARKUP_FIXTURES in this PR.

None of these affect runtime behavior or test correctness — the tables and fixtures themselves are right — but they're exactly the kind of comment that outlives the code it described and misleads the next person who trusts prose over the table it's supposed to explain, in a file whose entire design premise is "don't trust prose, measure it."

One process/convention note: .changeset/vale-3-18-0.md is a new changeset file, but .changeset/vale-converter-formats.md already sits on this stack's base branch (fix/vale-formats) and explicitly forecasts this exact follow-up ("the same release adds a Typst converter, which will move .typ out of the plaintext tier..."). Per CLAUDE.md's "Growing the changeset" section, a later PR in a stack should extend the existing changeset rather than add a second one — two changesets for one continuous piece of work means two separate patch notes if the stack merges forward.

One minor test-quality nit: recipe-cross-references.test.ts:388's new .typ assertion (toContain(".typ")) is a bare substring check that doesn't pin any real claim and is largely redundant with the valeConverterList() assertion two lines above.

All findings are posted as inline comments on the relevant lines.

[thecodedrift]

Re: @claude[bot] — "Claude finished @thecodedrift's task in 7m 24s — Review complete… What I did find is a cluster of stale doc comments…"
#161 (comment)

All nine findings addressed in 922e378, and every one of them held up — nothing here was a false positive or already fixed. Each inline thread has a reply with the specifics; the two things worth surfacing at this level:

The MyST observation was a missing row, not just a stale sentence. The doc block predicted MyST would arrive as markup and no row was added — so I probed rather than deleted the prediction. .myst yields 1 finding on a bare line and 0 on a fenced code block containing the token, which is the markup discriminator. It is now ".myst": "markup" with its own MARKUP_FIXTURES entry (814 tests, up from 813). That is the exact failure mode this table's header warns about: a format Vale learned between releases, sitting unmeasured.

Re-probed rather than reasoned, on the pinned 3.18.0 binary, using each language's own comment syntax — a wrong delimiter is indistinguishable from absent support:

Extension	Probe	Result	Tier
.rmd, .qmd, .mdx, .myst	fenced block containing the token	0 findings	markup
.qdoc	bare line / /*! … */ block	0 / 1	comment
.qml, .scss, .php	bare line / own comment syntax (//, /* */, <?php + //)	0 / 1	comment
.typ	any prose	exit 2, E100 [lintTypst] Runtime error / typst2vast not found	converter
.pyi, .tex, .mkd, .mkdn	bare line / own syntax	1 / 1	plaintext

So the table as committed was right; the prose around it was the thing describing 3.17.1.

On the changeset: agreed and folded — vale-3-18-0.md is gone and vale-converter-formats.md absorbed it. That also caught two sentences in the existing file the bump falsified ("MDX is not supported yet"; ".pyi, .qml and .scss … stay in the plaintext tier"), which would otherwise have shipped in the same release note as their own contradiction.

One thing I found and deliberately did not change: .Rmd (capital R) also measures as markup on 3.18.0. It is not in the table, so it falls back to plaintext in the rendered recipe lists — a documentation inaccuracy, never a crash, since the dangerous direction is only the converter tier. Raising it here rather than expanding scope on a review pass.

— AI Coding Agent