ci(mutants): an unmatched --test-name-pattern is setup-failure, not survived (#650)

A name-filtered `node --test` that matches no test exits 0 and TAP reports
only the file itself. The clean run took that for a healthy witness and the
mutant run for `survived`, although no assertion ran either time — as in
Validate 36151140394 on #649, where a renamed test left a guard pattern empty.

- mutation-guard-outcome: nodeTestSelection/executedTestNames/
  emptyTestSelection; a green oracle with a name filter and zero executed
  named tests is `setup-failure` (clean run: «FAIL чистая подготовка»,
  diff mode: attributed like any setup failure, #568).
- mutation-gate --check: staticTestSelectionProblems — every pattern must
  match a literal test(/it(/describe( name in the guard's files; files with
  ${…} names only warn. POSIX quoting keeps \( inside "…".
- Registry today: 349 unique name-filtered commands run on dev, all execute
  ≥ 1 test; --check: 0 errors, 3 warnings (dynamic names).
- Tests: 6 new in test/mutation-guard-outcome.test.mjs (incl. the installed
  node's real TAP and a registry-wide static check); 3 mutants.
- Docs: TESTING.md pointer, testing-notes/infrastructure.md section.

Issue: #650
User-Visible: no
This commit is contained in:
Claude
2026-09-25 18:31:17 +03:00
parent a55f564498
commit 43459bcbe0
7 changed files with 321 additions and 2 deletions
+21
View File
@@ -4753,6 +4753,27 @@ const MUTANT_DEFINITIONS = [
replace: ' if (!hunks || !hunks.length) return false;',
}],
},
// #650: a --test-name-pattern that matches nothing is a vacuous witness.
{
id: 'empty-test-selection-reads-as-survived',
guard: "node --test --test-name-pattern=\"#650 (clean run|mutant run|the installed node)\" test/mutation-guard-outcome.test.mjs",
because: '#650: a green oracle that executed no named test proves nothing; calling it '
+ 'survived blames the code, and a clean run calls it a healthy witness',
patches: [{ file: 'scripts/mutation-guard-outcome.mjs', find: " const empty = phase === 'oracle' ? emptyTestSelection(result?.command, outputOf(result)) : null;", replace: " const empty = null;" }],
},
{
id: 'static-test-selection-accepts-unmatched',
guard: "node --test --test-name-pattern=\"#650 static check\" test/mutation-guard-outcome.test.mjs",
because: '#650: mutation-gate --check must name an unmatched name filter before CI spends a shard on it',
patches: [{ file: 'scripts/mutation-guard-outcome.mjs', find: " if (names.some((name) => regexps.some((regexp) => regexp.test(name)))) continue;", replace: " if (names.length || !names.length) continue;" }],
},
{
id: 'shell-words-drops-regex-escape',
guard: "node --test --test-name-pattern=\"#650 (selection|static check)\" test/mutation-guard-outcome.test.mjs",
because: '#650: inside double quotes the shell keeps a backslash before ( — dropping it turns an '
+ 'escaped pattern into a group and flags healthy guards as empty',
patches: [{ file: 'scripts/mutation-guard-outcome.mjs', find: " if (char === '\\\\' && quote === '\"' && /[$`\"\\\\\\n]/.test(text[index + 1] || '')) {", replace: " if (char === '\\\\' && quote === '\"' && index + 1 < text.length) {" }],
},
{
id: 'ledger-version-sensitive',
guard: 'node --test --test-name-pattern="#481 AC1" test/mutation-gate.test.mjs',