Skip to content

fix(cli): let ast-grep rules see inside hidden directories - #153

Open
thecodedrift wants to merge 2 commits into
mainfrom
fix/hidden-scan
Open

fix(cli): let ast-grep rules see inside hidden directories#153
thecodedrift wants to merge 2 commits into
mainfrom
fix/hidden-scan

Conversation

@thecodedrift

Copy link
Copy Markdown
Member

ast-grep's file walker skips dot-directories unless told otherwise, and runAstGrepScan never told it otherwise. No sg rule could match anything under .github/, .circleci/, .vscode/, or .husky/check reported nothing and exited 0 on a workflow file it flags correctly the moment the same bytes live in a non-hidden directory.

Found while investigating #151: it is what made ast-grep look like it does not support YAML. It does — the blind spot was the directory, not the language.

$ taskless check                       → No issues found.
$ cp .github/workflows/bad.yml notdot/ && taskless check
  notdot/bad.yml:3:5  error[no-checkout-v2] …

The flag

Measured all six --no-ignore values against 0.41.0 on a fixture with hidden, gitignored, and plain copies of a match:

value reaches .github/ pulls in gitignored dist/
(none) no no
hidden yes no
dot, exclude, global, parent no no
vcs no yes

hidden is the only value that does the job, and vcs is the one that must not be passed — .gitignore is still respected, so the wider walk does not start reporting findings in dist/.

Also fixed in rules/runtime/narrow.ts, which runs the same sg scan shape, so a runtime rule's capture could not reach .github/ either. The sg test path needs nothing: it has no --no-ignore option at all and does not walk the project — it reads the testDir paths the assembled config names, and already reads hidden .tests/ today.

.taskless/ had to be excluded

.taskless/ is itself hidden, so the wider walk reached the CLI's own config — and a rule's YAML definition flagged itself:

  src/thing.yml:2:1                            error[no-sev]
  .taskless/rules/sg/no-sev/no-sev.yml:3:1     error[no-sev]   ← its own definition

Every taskless sg rule file contains id:, language:, severity:, message:, rule:, so any YAML rule a user writes would fire on config they did not author. Excluded with --globs '!**/.taskless/**'; the **/ prefix is load-bearing, since --globs is gitignore-style and a pattern containing / is root-anchored, which misses a .taskless/ inside a monorepo package. Measured — !.taskless/** and !.taskless both leave the nested one exposed.

Vale had already solved this, at vale/run.ts:145, applying its exclusion only when the CLI chose . itself: "An explicit path is a request, and silently declining to check a file someone named would be worse than checking one they did not." The first cut here excluded .taskless/ unconditionally, which would have re-created the two-engines-disagree failure this PR is about. It now matches Vale's semantics, in one exported function both call sites share. Vale needs no change.

Tests

Two, both asserting both halves so the fix cannot regress in either direction: .github/ is scanned while gitignored build/ is not (the temp dir is git inited first — ast-grep honours .gitignore only inside a repository, so without that the negative assertion passes vacuously), and a language: Yaml rule does not flag .taskless/ while still flagging .github/.

Both were confirmed to fail with the fix reverted.

Refs #151

ast-grep's walker skips dot-directories by default, so no `sg` rule could
match anything under `.github/`, `.circleci/`, `.vscode/` or `.husky/` —
`check` reported nothing and exited 0 on files it flags correctly one
directory over. Pass `--no-ignore hidden` from `runAstGrepScan` and the
runtime narrow.

Measured against the pinned ast-grep 0.41.0, `hidden` is the only value
that reaches those directories, and `vcs` is deliberately not passed so
`.gitignore` is still respected.

`.taskless/` is hidden too, so the wider walk reached CLI-managed config
that was never meant to be scanned: a rule definition is structured YAML
full of `id:`/`language:`/`severity:` keys, and an ordinary user-written
Yaml rule fires on the CLI's own rule files. Exclude it via `--globs`,
only on a whole-project walk, matching the rule the Vale runner already
follows — an explicit path stays a request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cyga14bww8rmazH2XrF8ms
Copilot AI lite review requested due to automatic review settings August 24, 2026 04:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@thecodedrift

Copy link
Copy Markdown
Member Author

@claude /review

@claude

claude Bot commented Aug 24, 2026

Copy link
Copy Markdown

Claude finished @thecodedrift's task in 3m 13s —— View job


Review mode: incremental — read 0 prior review thread(s) before reviewing.

.prior-review.json held no threads, review summaries with findings, or top-level comments other than the @claude /review request itself and Copilot's quota-limit message — so this is the first substantive review of this PR. Everything below is [New].

Approach

Eight parallel finder passes covered the diff from different angles (line-by-line, removed-behavior, cross-file tracing, duplication, simplification, efficiency, altitude, and CLAUDE.md conventions), followed by manual verification of every claim against the actual source before posting. Two findings were dropped after verification failed to confirm them cleanly; one is flagged with explicit uncertainty because I could not verify it against the actual sg binary.

Findings (posted inline where the diff allows it)

1. taskless check . silently loses the new .taskless/ exclusionpackages/cli/src/rules/scan.ts:182 (inline comment posted)
sgWalkArgv's .taskless/ exclusion is gated on paths.length === 0. But filterExistingPaths (packages/cli/src/commands/check.ts:64, not touched by this PR) normalizes an explicit . into the literal string "." rather than an empty array, so check . reaches sgWalkArgv with a non-empty paths and skips the exclusion — reintroducing the exact self-flagging bug this PR fixes, for a very natural invocation. Neither new test covers check .; the existing "positional path arguments" tests cover a real file, missing paths, all-missing paths, zero paths, and a subdirectory, but not .. The same gate is shared with the runtime narrow (narrow.ts:46), so both engines are affected.

2. .git/ is not excluded from the widened walkpackages/cli/src/rules/scan.ts:180 (inline comment posted, flagged as unverified)
--no-ignore hidden lets the walker descend into every dot-directory; the docblock is explicit about which ones become newly reachable (.github/, .circleci/, .vscode/, .husky/) but never mentions .git/, which is also hidden and was previously excluded only by the same default. I could not confirm from this environment whether ast-grep's walker hard-codes a separate .git/ exclusion independent of the hidden-file filter (plausible, since it's common in ignore-crate-based tools) — so this may be a non-issue. Given the PR's own methodology of measuring flag interactions against the pinned 0.41.0 binary, this specific one seems worth measuring the same way before merging, since the failure mode (scanning .git/objects/hooks on every run) would be silent otherwise.

3. Vale's .taskless/ exclusion glob may have the same anchoring bug this PR just fixed for ast-greppackages/cli/src/rules/vale/run.ts:145 (not in this diff, so no inline comment possible)
This PR's scan.ts docblock states a root-anchored .taskless/** glob "was measured to miss a .taskless/ nested inside a monorepo package," which is why ast-grep's exclusion uses !**/${TASKLESS_DIRECTORY}/**. Vale's own exclusion (!${TASKLESS_DIRECTORY}/**, no **/ prefix) wasn't touched. The PR description's "Vale needs no change" claim is about when to exclude (whole-project only), not the glob's anchoring — those weren't compared. If Vale's --glob has the same anchoring semantics, a nested packages/foo/.taskless/ is now correctly skipped by sg but still walked and linted by Vale, which is the "two engines disagree" failure this PR is otherwise about. Worth a quick check of Vale's actual glob-matching behavior.

4. The ast-grep vendor-contract test wasn't updated for the new argvpackages/cli/test/ast-grep-vendor-contract.test.ts:134 (not in this diff, so no inline comment possible)
The scan() helper's doc comment says it invokes scan "exactly as runAstGrepScan invokes it," but runAstGrepScan now also passes sgWalkArgv(paths) and this helper doesn't. That file exists specifically to pin undocumented ast-grep behavior against version bumps (per its own header), and this PR's docblock makes several such claims (--globs survives --no-ignore hidden, rule discovery is untouched, .tests/ is still skipped) that are asserted in prose but only exercised indirectly through check.test.ts integration tests — not pinned here the way the rest of this file's assumptions are.

Not raised

  • The TASKLESS_DIRECTORY constant duplication between scan.ts and vale/run.ts is called out and justified in the code's own comment ("keeps its own copy for the same reason") — a real minor duplication, but a deliberate one, not worth blocking on.
  • Efficiency observations about sequential per-rule sg re-walks in narrow.ts are pre-existing behavior this PR doesn't change, not a regression it introduces.
  • CLAUDE.md/STYLEGUIDE-CODE.md conventions (import grouping, barrel exports, "verify build output in the build") were checked explicitly — no violations.

Comment thread packages/cli/src/rules/scan.ts
Comment thread packages/cli/src/rules/scan.ts Outdated
`sgWalkArgv` gated its `.taskless/` exclusion on `paths.length === 0`, but
`filterExistingPaths` normalizes a positional path resolving to cwd into
the literal string ".", so `taskless check .` arrived with `paths = ["."]`.
A length test reads that as a user-named path and skips the exclusion,
which put the self-flagging bug back for the most ordinary way of asking
for a whole-project check: `check` was clean while `check .` reported
findings inside `.taskless/`.

Vale was already wrong in the same way at `run.ts`, independently of the
hidden-directory change and reachable by the same invocation — and since
Vale reads hidden directories by default, it had been reporting prose
findings in `.taskless/` on any `check .`. The defect is two engines
inferring the same thing from the same wrong signal, so the predicate is
now shared in `walk-scope.ts` rather than fixed in one copy.

Also excludes `.git/` from the wider walk. Measured against 0.41.0:
ast-grep has no exclusion of its own for it and `.gitignore` does not list
it, so the default hidden-directory skip was the only thing holding it
back. A whole-project scan descended into `.git/objects` and `.git/logs`
on every run, and `.git/hooks/*` are real source files that match language
rules never meant to lint VCS internals. Repeated `--globs` flags were
measured to both apply, unlike Vale's `--glob` where the last silently
wins, so the exclusions are separate flags rather than one brace pattern.

An explicit path is still a request: `check .taskless` reports it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cyga14bww8rmazH2XrF8ms
@thecodedrift

Copy link
Copy Markdown
Member Author

Re: @claude[bot] — "Claude finished @thecodedrift's task in 3m 13s — Review mode: incremental…"
#153 (comment)

Both findings were real and both are fixed in 85c0a16. Replies are on the inline threads; summarizing here since they interacted.

Finding 1 (check . loses the exclusion) — reproduced exactly: check reported 1 issue, check . reported 2, the second being the rule's own definition file. Fixed at the root rather than at the reported line, because paths.length === 0 was the wrong test in two places — rules/vale/run.ts:131 had the identical gate. That one is pre-existing on main and independent of this PR, but reachable by the same invocation, and since Vale reads hidden directories by default it had already been reporting prose findings inside .taskless/ on any check .. The predicate now lives once in rules/walk-scope.ts; the duplication was the defect.

Finding 2 (.git/, flagged as unverified) — measured, and reachable. --no-ignore hidden without vcs yielded [".git/probe.js", ".git/hooks/pre-commit.js", "src/a.js"], so ast-grep has no exclusion of its own for .git/ and .gitignore does not list it either. Both consequences you named hold. Excluded on the same whole-project terms. Rating it low was the one call I'd push back on — .git/hooks/* matching language rules is a false positive on files the user cannot reasonably fix, which is the same shape as finding 1.

Also worth recording from the fix: ast-grep honors repeated --globs flags, both applying, unlike Vale's --glob where the last silently wins. That is why the two exclusions are separate flags rather than one brace pattern, and it is now in the docblock.

Three tests added, each confirmed to fail without its corresponding fix: it.each over . and ./, one for .git/, and one pinning that an explicit check .taskless still reports it — the other half of the "an explicit path is a request" rule, which had no coverage before.

Noted and not actioned: Copilot's comment is a quota-limit notice with no findings.

— AI Coding Agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants