First real run of the gate failed before reaching a single mutant: the clean
run of the golden guard was red on untouched code. demo/golden/policy.mjs
refuses `verify --scenario=...` on purpose — a partial verify is the "make CI
green" loophole the policy exists to close. The gate built to catch dishonest
tests had reached for a dishonest shortcut, and the policy caught it.
capture keeps the whole check: a failed semantic assertion becomes status
error, and goldenRunFailed treats an error as failure in either mode. The scene
carries warmPixelRegion with minPixels 2500 over the receiving half, so a lamp
moved out of reach still fails it — which is exactly what this mutant asserts.
Issue: #85
User-Visible: no
Five times in this project a green test meant nothing was checked. The
continuity smoke stayed green after the entire mechanism it guards was cut out.
The golden scene created to protect doorway light was empty — 1,177 warm pixels
against 107,119, all of them icons. The shadow smoke passed while no shadow was
drawn. Each time the test had been written alongside the code, went green at
once, and nobody ever asked whether it could go red.
The gate makes that question routine. Each mutant is a few lines of patch that
reproduce a known breakage, plus the name of the test that must fail on it. A
worktree is patched, the bundle rebuilt, the guard run — and a guard that stays
green fails the gate. Six mutants cover the holes documented in #85; the anchors
are exact strings from today's source, so the registry cannot silently drift —
a unit test that runs with the ordinary suite refuses a stale anchor.
The full run rebuilds the bundle per mutant, so it lives in its own workflow,
before a stable release and on a weekly schedule, not in Validate. The rules for
new tests are written at the top of docs/TESTING.md, and the sixth of them is
the cheapest: an assertion that reads back the property the code just set is
not written at all.
Issue: #85
User-Visible: no
Sergey Matyunin