mirror of
https://github.com/Matysh/houseplan-card
synced 2026-09-28 19:01:34 +00:00
ci: proof различает продуктовое дерево и overlay эталонов (#573)
Приёмка эталонов на beta.3 (`ad4000f9`) стоила второго полного Validate —
22 минуты, из них 17–22 на шард мутантов. Причина одна: корпус отпечатка
(`source-fingerprint.mjs`) называет `demo/golden` строкой-каталогом, а
замыкание входов раскрывало каталог во все текстовые файлы под ним, включая
`baselines-index.json`. Индекс становился входом smoke, performance_smoke и
каждого гарда через `serve.mjs`: на реальной паре C→B ключи smoke/perf были
DIFFERENT, отпечатки 181 из 183 браузерных свидетелей менялись, журнал их не
пропускал.
- `check-inputs.mjs`: `BASELINE_OVERLAY` — раскрытие каталога не выдаёт
overlay; явный корень golden и явная ссылка на файл — как были. На паре
C→B: ключи smoke/perf/parity/backend same, golden DIFFERENT; отпечатки
743 из 744 равны; план мутантов B с журналом C — 0–1 на шард вместо 38–44
- `ci-proof.mjs`: составное evidence — product tree без overlay, overlay
(tree, sha256 индекса, run из `Baseline-Reviewed`), content-ключи всех
реюзных job (исполненных тоже); `evaluateCiProof({expected, reviewedRun})`
сверяет с локальным расчётом, fail-closed на ключ, tree, индекс, reviewed
run, маркер с чужим ключом; proof без evidence при ожиданиях — stale
- `release-gate.mjs` / `release-prerelease.mjs`: ожидания считаются на
checkout кандидата (`candidateExpectations`), чужой checkout — notice
- мутанты: `baseline-overlay-leaks-into-every-key`,
`proof-trusts-evidence-it-could-verify`,
`reused-marker-key-unchecked-against-candidate`,
`product-tree-identity-counts-baselines`; перенацелен
`ci-proof-ignores-run-attempt`
- docs: TESTING (правило overlay), DEVELOPMENT (evidence в release proof),
STATUS
Issue: #573
User-Visible: no
This commit is contained in:
Claude
@@ -1216,6 +1216,10 @@ jobs:
|
||||
- name: Зафиксировать tree кандидата
|
||||
id: candidate
|
||||
run: echo "tree=$(git rev-parse 'HEAD^{tree}')" >> "$GITHUB_OUTPUT"
|
||||
# #573: proof несёт составное evidence — product tree без overlay эталонов,
|
||||
# сам overlay (tree, sha256 индекса, run из `Baseline-Reviewed`) и
|
||||
# content-ключи реюзных job; ключи сверяются с outputs job `reuse`, а
|
||||
# потребитель на checkout кандидата пересчитывает всё заново.
|
||||
- name: Собрать единый CI proof
|
||||
env:
|
||||
CANDIDATE_SHA: ${{ github.sha }}
|
||||
|
||||
+14
-1
@@ -492,7 +492,20 @@ requires a complete Validate proof for its SHA and
|
||||
Git tree (#541). The proof is tied to the workflow run ID and attempt and lists
|
||||
both the requested checks and the jobs that actually executed. A skipped heavy job counts only
|
||||
when its content-addressed reuse marker names an independently verified
|
||||
successful source job. Review, merge and release use the
|
||||
successful source job. Since #573 the proof also carries composite evidence:
|
||||
the identity of the product tree (every tracked path except the accepted
|
||||
golden overlay `demo/golden/baselines/**`), the overlay itself (its Git tree,
|
||||
the SHA-256 of `baselines-index.json` and the run named by the commit's
|
||||
`Baseline-Reviewed:` trailer) and the content key of every reusable job,
|
||||
executed or reused. Release consumers standing on the candidate checkout
|
||||
(`release-gate.mjs`, `release-prerelease.mjs`) recompute all of it locally and
|
||||
fail closed on any mismatch, on a reused marker whose key is not the
|
||||
candidate's, and on a declared review run that does not exist, was cancelled
|
||||
or is not a Validate run; a proof without the block is stale for them. The
|
||||
practical consequence is the beta.3 path: a candidate red only in golden,
|
||||
then a baseline-only commit that reuses smoke, performance smoke, parity and
|
||||
backend from the candidate's green jobs, skips every caught witness in the
|
||||
mutation ledger and re-runs golden, preflight and frontend only. Review, merge and release use the
|
||||
same `missing` / `pending` / `cancelled` / `stale` / `failed` state machine. A
|
||||
cancelled or light run is not a release verdict and cannot hide an older full
|
||||
failure; a later complete full proof can refresh it (#511). The release also requires Full
|
||||
|
||||
+1
-1
@@ -26,7 +26,7 @@ metadata). Only an explicit owner-approved emergency hotfix may skip this gate.
|
||||
| Hidden Alpha Stage | #89 Stage 1 ships in v1.63.0-beta.1, #122 Stage 2 in v1.64.0 and #160 Stage 3 in v1.73.0-beta.1; #570 adds the Stage 4 designer handoff and #583 refines it on `dev`: only the building ambient shadow remains, door/gate volumes pivot on the selected host face without strokes, and devices/lock badges receive deterministic mutual collision correction while room labels remain passive below them. The same hidden `iso` view uses a fixed 0°/20° camera and scale-aware wall height 84, low screen-facing device/room/lock overlays without tethers or ground dots, neutral full-height window frames with blue glass, and bounded theme materials. Since #448 the experiment is enabled only through the single indefinite browser-local `hp_alpha` gate; it is not expiring and has no per-stage key. Flat remains default; editors, `houseplan-space-card`, floor effects, stored coordinates and HA actions remain unchanged. Stage 4 stays internal and is absent from public changelog/user documentation. |
|
||||
| Workflow | Superseded 2026-08-12: the pre-1.62 rule of "local edits without tests or commits" is **dead** — since release 1.62 every product change follows `PROCESS.md` (issue in `S5-ready`+, branch `issue/<NN>-slug`, trailers on every commit, review pipeline; `AGENTS.md` is the summary). Release mechanics below remain current. A requested pre-release gets a production build plus the smallest targeted unit/smoke set covering the changed surfaces, one tested `dev` commit/tag and a GitHub Release with `prerelease=true`; `main` stays untouched. The complete local frontend/backend/smoke gate runs only before a stable release, after which `main` is fast-forwarded to the exact tested `dev` SHA and the stable release is produced by `release.yml` (`workflow_dispatch` on `main` with the tag) — the only publisher of installable assets since #540: gates on the exact SHA (Validate, Full Performance, E2E on the candidate commit), one build, `houseplan.zip` archived from the committed tree, `SHA256SUMS`, draft → publish → read-back verification; a release published by hand in the GitHub form is turned back into a draft and walked through the same path, and a re-dispatch on a public tag is a repair that adds only missing assets. Release bodies are short and bilingual (Russian first); every bullet links its GitHub issue (#NN) so the #328 rules stay machine-checkable. A STABLE body aggregates the changelog since the PREVIOUS STABLE release (never since the last beta): features/fixes described across the line's beta changelogs must appear, while bugs that were introduced and fixed strictly inside the beta line (never shipped in any stable) are excluded — draft with `npm run release:notes -- <tag>`, curate by hand, then `npm run release:notes -- <tag> --verify` must pass. `Мелкие исправления и улучшения` / `Small fixes and improvements` is allowed only when the range really contains user-visible work not itemised in the body; a single-issue hotfix ships without it (the verifier enforces this). Every body ends with separate links to the Russian and English changelogs. Open or partially delivered issues are never presented as shipped. Telegram announcements are sent only for stable releases; beta and RC publication is silent. `docs/RELEASE-NOTES.md` is the current canonical body instance; `npm run release:prerelease -- <tag> --issues=… --yes` is the primary local publication path and the manual `Publish prerelease` workflow is its GitHub-only equivalent once present on `main`. Nothing is copied to the home instance by hand |
|
||||
| GitHub | https://github.com/Matysh/houseplan-card — [Issues](https://github.com/Matysh/houseplan-card/issues) are the canonical task records; their labels carry priority and workflow status (`PROCESS.md` §9). GitHub Projects is no longer used. `main` carries stable releases; pre-release tags may point directly at `dev`. Work lands on `dev` and is merged into `main` for a stable release, so `dev` is normally equal to or ahead of `main`, never behind. Push via SSH key `ha_jb` (remote git@github.com:…); API releases via the fine-grained PAT in `~/.git-credentials` (Contents R/W, issued 2026-07-23) |
|
||||
| CI | #541 replaces three incompatible meanings of “green” with one machine-verifiable Validate proof: candidate SHA/tree, run ID/attempt, requested checks, actually executed jobs and independently checked content-addressed reuse. Review, merge and release share the same closed state machine; a light green dispatch cannot hide a full red run, and a dispatch without six executed mutant jobs cannot authorize review or merge. Prerelease publication requires a green full exact-SHA proof covering frontend/backend, smoke (including the #73 rAF frame sampler), golden, HACS/Hassfest and the short absolute-ceiling performance smoke. Obsolete same-ref Validate runs are cancelled. Full seven-sample base/candidate performance remains in `performance.yml` (`main` push, weekly, manual); stable release assets fail closed unless Validate and Full Performance are green for the exact tagged SHA and the stable-only CDP compositor screencast finds no empty/black presented frame. |
|
||||
| CI | #541 replaces three incompatible meanings of “green” with one machine-verifiable Validate proof: candidate SHA/tree, run ID/attempt, requested checks, actually executed jobs and independently checked content-addressed reuse. Review, merge and release share the same closed state machine; a light green dispatch cannot hide a full red run, and a dispatch without six executed mutant jobs cannot authorize review or merge. #573 makes the proof composite — product-tree identity, accepted golden overlay (tree, index hash, `Baseline-Reviewed` run) and the content key of every reusable job — and release consumers on the candidate checkout recompute and compare all of it; the accepted overlay is an input of `golden` only, so a baseline-only commit after a golden-red candidate reuses smoke, performance smoke, parity and backend, skips caught witnesses and re-runs golden alone. Prerelease publication requires a green full exact-SHA proof covering frontend/backend, smoke (including the #73 rAF frame sampler), golden, HACS/Hassfest and the short absolute-ceiling performance smoke. Obsolete same-ref Validate runs are cancelled. Full seven-sample base/candidate performance remains in `performance.yml` (`main` push, weekly, manual); stable release assets fail closed unless Validate and Full Performance are green for the exact tagged SHA and the stable-only CDP compositor screencast finds no empty/black presented frame. |
|
||||
| Local toolchain | #557 removes ambient-PATH claims from the owner's workstation: `scripts/windows-toolchain.ps1` keeps verified portable repository-pinned Node and a dedicated repository-pinned Python `.venv-ci` without changing system defaults; `toolchain:check` reports the current versions and exact executable/package/browser paths. The WSL entrypoint uses its own nvm + `.venv-ci`, and `--verify` runs a real HA subset and one Linux golden capture from an ext4 clone. These are early-feedback paths only; exact-SHA Linux CI remains canonical. |
|
||||
| HACS | **In the default catalog since 2026-08-25** (hacs/default#9004 merged). Install = plain HACS search. `houseplan.zip` is attached to stable tags automatically (verified on v1.72.0); forum/4pda announcement still pending |
|
||||
| Home instance | ha.jbstudio.pro (SSH port **22222**, key `ha_jb`; HA config root is `/mnt/data/supervisor/homeassistant` — `/config` does NOT exist in this SSH environment), last direct copy was **v1.57.0**; from v1.58.0 on it updates itself through HACS by tag (no scp) |
|
||||
|
||||
+10
-1
@@ -155,6 +155,15 @@ manifest читают и `classify-changes.mjs` (job `changes`: job запуск
|
||||
`test/check-inputs.test.mjs`) требует, чтобы каждый такой файл был чьим-то
|
||||
входом либо стоял в `NOT_AN_INPUT` с причиной: новый скрипт без записи —
|
||||
красный юнит, не вечное расширение прогонов;
|
||||
- **overlay принятых эталонов** (`demo/golden/baselines/**`, #573) — вход
|
||||
только `golden`, у которой он стоит явным корнем. Раскрытие каталога по
|
||||
строке его не выдаёт: корпус отпечатка называет `demo/golden` каталогом,
|
||||
но берёт из него только `*.mjs`, а до #573 индекс эталонов через это
|
||||
раскрытие становился входом smoke, perf и 181 из 183 браузерных
|
||||
свидетелей — приёмка 13 кадров на beta.3 (`ad4000f9`) сменила их ключи и
|
||||
отпечатки, и второй полный Validate повторил 22 минуты уже сделанной
|
||||
работы. Явная ссылка на файл индекса (как в `test/check-inputs.test.mjs`)
|
||||
входом остаётся;
|
||||
- `--check=<job>` печатает входы, `--why=<файл>` — цепочку, по которой файл
|
||||
стал входом.
|
||||
|
||||
@@ -162,7 +171,7 @@ manifest читают и `classify-changes.mjs` (job `changes`: job запуск
|
||||
`test/check-inputs.test.mjs`) держат представителей каждой категории входов и
|
||||
обратную пробу для UI ↔ backend; мутанты `manifest-drops-workflow-input`,
|
||||
`classify-unknown-input-is-unaffected`, `reuse-backend-hashes-ui`,
|
||||
`backend-dynamic-inputs-dropped`,
|
||||
`backend-dynamic-inputs-dropped`, `baseline-overlay-leaks-into-every-key`,
|
||||
`guard-inputs-ignore-wrapper-defaults`, `registry-diff-not-selected`,
|
||||
`merge-pushes-unvalidated-candidate`, `merge-ignores-lease-rejection`,
|
||||
`nightly-does-not-wait` держат сам протокол.
|
||||
|
||||
@@ -35,6 +35,22 @@ export const BUILD_OUTPUT = [
|
||||
'dist/**', 'custom_components/houseplan/frontend/**', 'demo/srv/assets/**',
|
||||
];
|
||||
|
||||
/**
|
||||
* Принятые эталоны и их индекс — overlay поверх продуктового дерева (#573).
|
||||
*
|
||||
* С ними сравнивает ровно одна проверка — `golden`, и у неё они стоят явным
|
||||
* корнем. Никакая другая job их не читает: смок и перф-смок сверяют бандл с
|
||||
* отпечатком исходников, а отпечаток (`source-fingerprint.mjs`) берёт из
|
||||
* `demo/golden` только `*.mjs`. Но корпус там назван строкой-каталогом, и
|
||||
* раскрытие каталога отдавало всем этим проверкам ещё и
|
||||
* `baselines-index.json` — так приёмка 13 кадров на `v1.76.0-beta.3`
|
||||
* (`ad4000f9`) сменила ключи smoke и performance_smoke и отпечатки 181 из 183
|
||||
* свидетелей с браузерным гардом, и второй полный Validate повторил 22 минуты
|
||||
* работы, которую первый уже сделал. Раскрытие каталога overlay не выдаёт;
|
||||
* явный корень и явная ссылка на файл — как были.
|
||||
*/
|
||||
export const BASELINE_OVERLAY = ['demo/golden/baselines/**'];
|
||||
|
||||
/**
|
||||
* Не входы Validate — с причиной. Каждая запись отвечает на вопрос «кто это
|
||||
* исполняет и почему не Validate».
|
||||
@@ -83,6 +99,7 @@ export const globToRegExp = (glob) => {
|
||||
return new RegExp(`^${re}$`);
|
||||
};
|
||||
const matchesAny = (file, globs) => globs.some((glob) => globToRegExp(glob).test(file));
|
||||
export const isBaselineOverlay = (file) => matchesAny(file, BASELINE_OVERLAY);
|
||||
|
||||
/** Отслеживаемые файлы (git), либо обход дерева там, где git недоступен. */
|
||||
export function trackedFiles(root) {
|
||||
@@ -253,8 +270,13 @@ export function closure(root, entries, { tracked = trackedFiles(root), stopAt =
|
||||
for (const ref of data) {
|
||||
if (trackedSet.has(ref)) { note(ref, file); seen.add(ref); continue; }
|
||||
// каталог по строке — данные; бинарные файлы под ним код по строке не
|
||||
// читает (эталоны golden входят в свою проверку явным корнем)
|
||||
if (isDir(ref)) for (const f of tracked) if (f.startsWith(`${ref}/`) && !BINARY.test(f)) { note(f, file); seen.add(f); }
|
||||
// читает, а overlay эталонов принадлежит только своей проверке (#573):
|
||||
// и то и другое входит в golden явным корнем
|
||||
if (isDir(ref)) {
|
||||
for (const f of tracked) {
|
||||
if (f.startsWith(`${ref}/`) && !BINARY.test(f) && !isBaselineOverlay(f)) { note(f, file); seen.add(f); }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...seen].sort();
|
||||
|
||||
+155
-9
@@ -4,9 +4,13 @@
|
||||
// а skipped job без доказанного content-addressed reuse ничего не доказывает.
|
||||
|
||||
import { inflateRawSync } from 'node:zlib';
|
||||
import { appendFileSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { appendFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
||||
import { dirname, resolve } from 'node:path';
|
||||
import { isMainModule } from './spawn-portable.mjs';
|
||||
import { BASELINE_OVERLAY, REUSE_JOBS, globToRegExp } from './check-inputs.mjs';
|
||||
import { reuseKey } from './gate-reuse.mjs';
|
||||
|
||||
export const CI_PROOF_SCHEMA = 'houseplan-ci-proof/v1';
|
||||
export const CI_PROOF_ARTIFACT_PREFIX = 'ci-proof';
|
||||
@@ -52,6 +56,95 @@ export function ciProofArtifactName(runId, attempt) {
|
||||
return `${CI_PROOF_ARTIFACT_PREFIX}-${Number(runId)}-${Number(attempt)}`;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Составное evidence (#573): proof называет ОТДЕЛЬНО продуктовое дерево,
|
||||
// overlay принятых эталонов и content-ключи реюзных job. Один `tree`
|
||||
// кандидата отвечал только «то же ли это дерево»; после приёмки эталонов
|
||||
// ответ всегда «нет», хотя продукт не менялся, — и потребитель не мог ни
|
||||
// объяснить, ни проверить, почему smoke и perf законно переиспользованы, а
|
||||
// golden перегнан. Теперь он сверяет каждую часть с тем, что сам считает на
|
||||
// checkout кандидата (`expected` в evaluateCiProof).
|
||||
|
||||
const overlayMatchers = BASELINE_OVERLAY.map((glob) => globToRegExp(glob));
|
||||
export const isBaselineOverlayPath = (path) => overlayMatchers.some((re) => re.test(path));
|
||||
|
||||
/**
|
||||
* Identity продуктового дерева: строки `git ls-tree -r <sha>` без overlay
|
||||
* эталонов. Два коммита с одним значением отличаются только принятыми
|
||||
* кадрами и их индексом — ровно случай baseline-only коммита.
|
||||
*/
|
||||
export function productTreeId(lsTreeText) {
|
||||
const lines = String(lsTreeText).split(/\r?\n/).filter(Boolean)
|
||||
.filter((line) => !isBaselineOverlayPath(line.split('\t').slice(1).join('\t')))
|
||||
.sort();
|
||||
if (!lines.length) throw new Error('product tree is empty — ls-tree output has no entries');
|
||||
const hash = createHash('sha256');
|
||||
for (const line of lines) { hash.update(line); hash.update('\0'); }
|
||||
return hash.digest('hex');
|
||||
}
|
||||
|
||||
/** Run из трейлера `Baseline-Reviewed: …/actions/runs/<id>`; null, когда трейлера нет. */
|
||||
export function baselineReviewedRun(commitMessage) {
|
||||
const trailer = String(commitMessage).match(/^Baseline-Reviewed:\s*(\S+)\s*$/mi)?.[1];
|
||||
if (!trailer) return null;
|
||||
const id = Number(trailer.match(/\/actions\/runs\/(\d+)/)?.[1] || 0);
|
||||
if (!id) throw new Error(`Baseline-Reviewed trailer does not name an actions run: ${trailer}`);
|
||||
return id;
|
||||
}
|
||||
|
||||
const gitOut = (root, args) => execFileSync('git', ['-C', root, ...args], { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 });
|
||||
|
||||
/**
|
||||
* Evidence по checkout: то, что пишет job `proof`, и то, что независимо
|
||||
* считает потребитель на том же SHA. Ключи — те же `reuseKey`, что у job
|
||||
* `reuse`; `keys` в аргументе (её outputs) обязаны совпасть — иначе
|
||||
* дерево, по которому приняли решение о реюзе, не то, по которому написан proof.
|
||||
*/
|
||||
export function localEvidence(root = process.cwd(), { keys = null, rev = 'HEAD' } = {}) {
|
||||
const computed = Object.fromEntries(REUSE_JOBS.map((job) => [job, reuseKey(root, job)]));
|
||||
if (keys) {
|
||||
for (const job of REUSE_JOBS) {
|
||||
if (keys[job] && keys[job] !== computed[job]) {
|
||||
throw new Error(`${job}: reuse job key ${keys[job]} differs from the proof checkout key ${computed[job]}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
const overlayDir = BASELINE_OVERLAY[0].replace(/\/\*\*$/, '');
|
||||
let baselineTree = null;
|
||||
try { baselineTree = gitOut(root, ['rev-parse', `${rev}:${overlayDir}`]).trim() || null; } catch { baselineTree = null; }
|
||||
const manifestPath = resolve(root, overlayDir, 'baselines-index.json');
|
||||
const manifestSha256 = existsSync(manifestPath)
|
||||
? createHash('sha256').update(readFileSync(manifestPath)).digest('hex') : null;
|
||||
return {
|
||||
product: { tree: productTreeId(gitOut(root, ['ls-tree', '-r', '--full-tree', rev])) },
|
||||
baselines: {
|
||||
tree: baselineTree,
|
||||
manifestSha256,
|
||||
reviewedRun: baselineReviewedRun(gitOut(root, ['log', '-1', '--format=%B', rev])),
|
||||
},
|
||||
keys: computed,
|
||||
};
|
||||
}
|
||||
|
||||
const EVIDENCE_FIELDS = [
|
||||
['product.tree', (e) => e?.product?.tree],
|
||||
['baselines.tree', (e) => e?.baselines?.tree ?? null],
|
||||
['baselines.manifestSha256', (e) => e?.baselines?.manifestSha256 ?? null],
|
||||
['baselines.reviewedRun', (e) => e?.baselines?.reviewedRun ?? null],
|
||||
...REUSE_JOBS.map((job) => [`keys.${job}`, (e) => e?.keys?.[job]]),
|
||||
];
|
||||
|
||||
/** Первое расхождение evidence proof с ожиданием потребителя, либо null. */
|
||||
export function evidenceMismatch(actual, expected) {
|
||||
for (const [name, read] of EVIDENCE_FIELDS) {
|
||||
const have = read(actual);
|
||||
const want = read(expected);
|
||||
if (want === undefined) continue;
|
||||
if (have !== want) return `${name}: proof says ${have ?? 'null'}, candidate checkout says ${want ?? 'null'}`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function parseReuseMarker(text) {
|
||||
const sha = String(text).match(/^SHA:\s*([0-9a-f]{40})\s*$/mi)?.[1] || null;
|
||||
const runId = Number(String(text).match(/\/actions\/runs\/(\d+)/)?.[1] || 0) || null;
|
||||
@@ -82,7 +175,7 @@ const reuseClaim = (outputs, id) => ({
|
||||
/** Build the immutable JSON uploaded by the final Validate job. */
|
||||
export function buildCiProof({
|
||||
candidateSha, candidateTree, runId, attempt, event, needs,
|
||||
requestedFull = false, requestedMutants = false,
|
||||
requestedFull = false, requestedMutants = false, evidence = null,
|
||||
}) {
|
||||
const changes = needs?.changes?.outputs || {};
|
||||
const reuse = needs?.reuse?.outputs || {};
|
||||
@@ -130,6 +223,16 @@ export function buildCiProof({
|
||||
if (selection.geometry_parity) executedOrReused('geometry_parity');
|
||||
if (selection.backend) executedOrReused('backend');
|
||||
const requiredChecks = requiredCheckIds({ request, selection });
|
||||
// #573: content-ключ записывается и у ИСПОЛНЕННОЙ реюзной job — иначе
|
||||
// следующий прогон не докажет, что его reuse ссылается на те же входы.
|
||||
if (evidence?.keys) {
|
||||
for (const id of REUSE_JOBS) {
|
||||
if (checks[id]?.mode === 'executed') checks[id].key = evidence.keys[id] || null;
|
||||
if (checks[id]?.mode === 'reused' && evidence.keys[id] && checks[id].reuse?.key !== evidence.keys[id]) {
|
||||
throw new Error(`${id}: reuse marker key ${checks[id].reuse?.key} differs from the candidate key ${evidence.keys[id]}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
return {
|
||||
schema: CI_PROOF_SCHEMA,
|
||||
candidate: { sha: candidateSha, tree: candidateTree },
|
||||
@@ -140,6 +243,7 @@ export function buildCiProof({
|
||||
executedChecks: requiredChecks.filter((id) => checks[id]?.mode === 'executed'),
|
||||
reusedChecks: requiredChecks.filter((id) => checks[id]?.mode === 'reused'),
|
||||
checks,
|
||||
...(evidence ? { evidence } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -160,19 +264,21 @@ function executedCheckIsGreen(id, jobs) {
|
||||
* One state machine for all consumers. `reuseRuns` maps source run id to
|
||||
* `{run,jobs}` fetched independently from the marker claim.
|
||||
*/
|
||||
export function evaluateCiProof({ run, proof, jobs = [], reuseRuns = new Map(), candidate = {}, policy }) {
|
||||
export function evaluateCiProof({
|
||||
run, proof, jobs = [], reuseRuns = new Map(), candidate = {}, policy, expected = null, reviewedRun = undefined,
|
||||
}) {
|
||||
const result = (status, note) => ({ status, note, url: runUrlOf(run) });
|
||||
if (!run) return result('missing', 'Validate run is missing');
|
||||
if (run.status !== 'completed') return result('pending', `Validate run ${runIdOf(run)} is ${run.status || 'pending'}`);
|
||||
if (run.conclusion === 'cancelled') return result('cancelled', `Validate run ${runIdOf(run)} was cancelled`);
|
||||
if (!proof) return result('missing', `Validate run ${runIdOf(run)} has no proof artifact`);
|
||||
if (proof.schema !== CI_PROOF_SCHEMA) return result('stale', `unsupported proof schema ${proof.schema || 'missing'}`);
|
||||
const expected = {
|
||||
const identity = {
|
||||
runId: runIdOf(run), attempt: runAttemptOf(run), sha: candidate.sha || runShaOf(run), tree: candidate.tree,
|
||||
};
|
||||
if (proof.run?.id !== expected.runId || proof.run?.attempt !== expected.attempt
|
||||
|| proof.run?.workflow !== 'validate.yml' || proof.candidate?.sha !== expected.sha
|
||||
|| (expected.tree && proof.candidate?.tree !== expected.tree)) {
|
||||
if (proof.run?.id !== identity.runId || proof.run?.attempt !== identity.attempt
|
||||
|| proof.run?.workflow !== 'validate.yml' || proof.candidate?.sha !== identity.sha
|
||||
|| (identity.tree && proof.candidate?.tree !== identity.tree)) {
|
||||
return result('stale', 'proof does not belong to the candidate SHA/tree and run attempt');
|
||||
}
|
||||
if (runShaOf(run) && proof.candidate.sha !== runShaOf(run))
|
||||
@@ -181,6 +287,28 @@ export function evaluateCiProof({ run, proof, jobs = [], reuseRuns = new Map(),
|
||||
return result('stale', 'run event differs from proof event');
|
||||
if (policy?.full && !asBool(proof.request?.full)) return result('stale', 'proof is light; full gates were not requested');
|
||||
if (policy?.mutants && !asBool(proof.request?.mutants)) return result('stale', 'proof has no requested mutant jobs');
|
||||
// #573: потребитель, у которого есть checkout кандидата, сверяет составное
|
||||
// evidence, а не верит ему. Proof без блока при наличии ожиданий устарел.
|
||||
if (expected) {
|
||||
if (!proof.evidence) return result('stale', 'proof predates composite evidence (#573)');
|
||||
const mismatch = evidenceMismatch(proof.evidence, expected);
|
||||
if (mismatch) return result('failed', `evidence does not match the candidate checkout — ${mismatch}`);
|
||||
}
|
||||
if (proof.evidence) {
|
||||
for (const id of REUSE_JOBS) {
|
||||
const claim = proof.checks?.[id];
|
||||
if (claim?.mode === 'reused' && claim.reuse?.key !== proof.evidence.keys?.[id])
|
||||
return result('failed', `${id}: reused marker key differs from the candidate content key`);
|
||||
}
|
||||
const declared = proof.evidence.baselines?.reviewedRun ?? null;
|
||||
if (declared && reviewedRun !== undefined) {
|
||||
const source = reviewedRun?.run;
|
||||
if (!source || runIdOf(source) !== declared || !/validate\.yml$/.test(String(source.path || source.workflow || 'validate.yml'))
|
||||
|| source.status !== 'completed' || source.conclusion === 'cancelled') {
|
||||
return result('failed', `Baseline-Reviewed run ${declared} is missing, cancelled or not a Validate run`);
|
||||
}
|
||||
}
|
||||
}
|
||||
const derived = requiredCheckIds(proof);
|
||||
if (!sameSet(derived, proof.requiredChecks || []))
|
||||
return result('failed', 'proof required-check list is incomplete or inconsistent');
|
||||
@@ -310,7 +438,17 @@ export async function loadGithubProofContext({ repo, run, token, fetchImpl = fet
|
||||
);
|
||||
reuseRuns.set(sourceKey, { run: sourceRun, jobs: sourceJobs?.jobs || [] });
|
||||
}
|
||||
return { proof, jobs, reuseRuns };
|
||||
// #573: объявленный человеком run просмотра кадров обязан существовать.
|
||||
let reviewedRun;
|
||||
const declared = proof?.evidence?.baselines?.reviewedRun;
|
||||
if (declared) {
|
||||
try {
|
||||
reviewedRun = { run: await githubJson(`https://api.github.com/repos/${repo}/actions/runs/${declared}`, token, fetchImpl) };
|
||||
} catch {
|
||||
reviewedRun = null;
|
||||
}
|
||||
}
|
||||
return { proof, jobs, reuseRuns, reviewedRun };
|
||||
}
|
||||
|
||||
if (isMainModule(import.meta.url)) {
|
||||
@@ -323,20 +461,28 @@ if (isMainModule(import.meta.url)) {
|
||||
if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, output);
|
||||
process.stdout.write(output);
|
||||
} else if (emit) {
|
||||
const needs = JSON.parse(process.env.NEEDS_JSON || '{}');
|
||||
const reuseOutputs = needs?.reuse?.outputs || {};
|
||||
const evidence = localEvidence(process.cwd(), {
|
||||
keys: Object.fromEntries(REUSE_JOBS.map((job) => [job, reuseOutputs[`${job}_key`] || null])),
|
||||
});
|
||||
const proof = buildCiProof({
|
||||
candidateSha: process.env.CANDIDATE_SHA,
|
||||
candidateTree: process.env.CANDIDATE_TREE,
|
||||
runId: process.env.CI_RUN_ID,
|
||||
attempt: process.env.CI_RUN_ATTEMPT,
|
||||
event: process.env.CI_EVENT,
|
||||
needs: JSON.parse(process.env.NEEDS_JSON || '{}'),
|
||||
needs,
|
||||
requestedFull: process.env.REQUEST_FULL,
|
||||
requestedMutants: process.env.REQUEST_MUTANTS,
|
||||
evidence,
|
||||
});
|
||||
const target = resolve(emit);
|
||||
mkdirSync(dirname(target), { recursive: true });
|
||||
writeFileSync(target, `${JSON.stringify(proof, null, 2)}\n`);
|
||||
console.log(`CI proof: ${target} (${proof.requiredChecks.join(', ')})`);
|
||||
console.log(`product tree ${evidence.product.tree.slice(0, 12)} · baselines ${evidence.baselines.tree?.slice(0, 12) || 'none'}`
|
||||
+ ` · reviewed run ${evidence.baselines.reviewedRun || 'none'}`);
|
||||
} else {
|
||||
console.error('usage: ci-proof.mjs --emit=<proof.json> | --marker=<.reuse-marker>');
|
||||
process.exitCode = 2;
|
||||
|
||||
@@ -9418,8 +9418,8 @@ const MUTANT_DEFINITIONS = [
|
||||
+ 'attempt cannot vouch for the current result (#541)',
|
||||
patches: [{
|
||||
file: 'scripts/ci-proof.mjs',
|
||||
find: ' if (proof.run?.id !== expected.runId || proof.run?.attempt !== expected.attempt\n',
|
||||
replace: ' if (proof.run?.id !== expected.runId || false && proof.run?.attempt !== expected.attempt\n',
|
||||
find: ' if (proof.run?.id !== identity.runId || proof.run?.attempt !== identity.attempt\n',
|
||||
replace: ' if (proof.run?.id !== identity.runId || false && proof.run?.attempt !== identity.attempt\n',
|
||||
}],
|
||||
},
|
||||
{
|
||||
@@ -9904,6 +9904,54 @@ const MUTANT_DEFINITIONS = [
|
||||
replace: ' "zones_v1": ("zones", ()),',
|
||||
}],
|
||||
},
|
||||
{
|
||||
id: 'baseline-overlay-leaks-into-every-key',
|
||||
guard: 'node --test --test-name-pattern="#573" test/check-inputs.test.mjs test/gate-reuse.test.mjs test/mutation-gate.test.mjs',
|
||||
because: '#573: строка-каталог `demo/golden` в корпусе отпечатка раскрывалась во все текстовые '
|
||||
+ 'файлы под ним, и индекс эталонов становился входом smoke, perf и 181 из 183 браузерных '
|
||||
+ 'свидетелей. Без фильтра overlay приёмка кадров снова перегоняет 22 минуты чужой работы, '
|
||||
+ 'а ключи, которые proof предъявляет как «те же входы», меняются от PNG',
|
||||
patches: [{
|
||||
file: 'scripts/check-inputs.mjs',
|
||||
find: " if (f.startsWith(`${ref}/`) && !BINARY.test(f) && !isBaselineOverlay(f)) { note(f, file); seen.add(f); }",
|
||||
replace: " if (f.startsWith(`${ref}/`) && !BINARY.test(f)) { note(f, file); seen.add(f); }",
|
||||
}],
|
||||
},
|
||||
{
|
||||
id: 'proof-trusts-evidence-it-could-verify',
|
||||
guard: 'node --test --test-name-pattern="#573" test/ci-proof.test.mjs test/release-gate.test.mjs',
|
||||
because: '#573: потребитель с checkout кандидата обязан СВЕРЯТЬ evidence, а не верить. Без '
|
||||
+ 'сравнения подменённый content-ключ, product tree или индекс эталонов проходят как свои — '
|
||||
+ 'и proof снова говорит только «вот SHA»',
|
||||
patches: [{
|
||||
file: 'scripts/ci-proof.mjs',
|
||||
find: ' const mismatch = evidenceMismatch(proof.evidence, expected);',
|
||||
replace: ' const mismatch = evidenceMismatch(proof.evidence, proof.evidence);',
|
||||
}],
|
||||
},
|
||||
{
|
||||
id: 'reused-marker-key-unchecked-against-candidate',
|
||||
guard: 'node --test --test-name-pattern="#573" test/ci-proof.test.mjs',
|
||||
because: '#573: маркер реюза несёт ключ источника; если он не сверяется с ключом кандидата, '
|
||||
+ 'зелёная job с ДРУГИМИ входами засчитывается этому дереву',
|
||||
patches: [{
|
||||
file: 'scripts/ci-proof.mjs',
|
||||
find: " if (claim?.mode === 'reused' && claim.reuse?.key !== proof.evidence.keys?.[id])",
|
||||
replace: " if (claim?.mode === 'reused' && claim.reuse?.key !== (proof.evidence.keys?.[id] ?? claim.reuse?.key) && false)",
|
||||
}],
|
||||
},
|
||||
{
|
||||
id: 'product-tree-identity-counts-baselines',
|
||||
guard: 'node --test --test-name-pattern="#573" test/ci-proof.test.mjs',
|
||||
because: '#573: identity продуктового дерева существует ради одного сравнения — «C и B '
|
||||
+ 'отличаются только принятыми кадрами». Считая overlay, оно совпадает с `tree` кандидата и '
|
||||
+ 'перестаёт отличать приёмку эталонов от правки продукта',
|
||||
patches: [{
|
||||
file: 'scripts/ci-proof.mjs',
|
||||
find: " .filter((line) => !isBaselineOverlayPath(line.split('\\t').slice(1).join('\\t')))",
|
||||
replace: " .filter((line) => typeof line === 'string')",
|
||||
}],
|
||||
},
|
||||
{
|
||||
id: 'room-orphan-colour-wins-again',
|
||||
guard: 'node --test test/logic.test.mjs',
|
||||
|
||||
@@ -1,13 +1,30 @@
|
||||
// Exact-SHA GitHub Actions gate used by release workflows. Prereleases require
|
||||
// Validate; stable releases additionally require the dedicated full
|
||||
// performance workflow.
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
CI_PROOF_POLICIES, evaluateCiProof, githubCandidateTree,
|
||||
CI_PROOF_POLICIES, evaluateCiProof, githubCandidateTree, localEvidence,
|
||||
loadGithubProofContext, selectCiProofVerdict,
|
||||
} from './ci-proof.mjs';
|
||||
|
||||
/**
|
||||
* #573: ожидания потребителя. Гейты релиза стоят на checkout кандидата, и
|
||||
* тогда evidence proof (product tree, overlay эталонов, content-ключи,
|
||||
* reviewed run) не принимается на веру, а сверяется с тем, что посчитано
|
||||
* здесь. Checkout не на кандидате — считать нечего, и об этом говорится вслух.
|
||||
*/
|
||||
export function candidateExpectations({ sha, root = process.cwd(), log = console.log } = {}) {
|
||||
let head = null;
|
||||
try { head = execFileSync('git', ['-C', root, 'rev-parse', 'HEAD'], { encoding: 'utf8' }).trim(); } catch { head = null; }
|
||||
if (!sha || head !== sha) {
|
||||
log(`::notice::checkout ${head ? head.slice(0, 8) : 'absent'} is not the candidate ${String(sha).slice(0, 8)} — proof evidence is verified against GitHub only (#573)`);
|
||||
return null;
|
||||
}
|
||||
return localEvidence(root);
|
||||
}
|
||||
|
||||
/**
|
||||
* The verdict is the LATEST run that was not cancelled (#511). A cancelled run
|
||||
* proves nothing either way — concurrency or a hand superseded it — and the
|
||||
@@ -37,7 +54,7 @@ const newestFirst = (runs) => [...(Array.isArray(runs) ? runs : [])].sort((a, b)
|
||||
|
||||
/** #541: proof-aware verdict shared with review and merge. */
|
||||
export async function classifyValidateProofs({
|
||||
runs, repo, sha, tree, token, fetchImpl = fetch,
|
||||
runs, repo, sha, tree, token, fetchImpl = fetch, expected = null,
|
||||
loadContext = (run) => loadGithubProofContext({ repo, run, token, fetchImpl }),
|
||||
}) {
|
||||
const evaluations = [];
|
||||
@@ -48,7 +65,7 @@ export async function classifyValidateProofs({
|
||||
try {
|
||||
const context = await loadContext(run);
|
||||
evaluations.push(evaluateCiProof({
|
||||
run, ...context, candidate: { sha, tree }, policy: CI_PROOF_POLICIES.release,
|
||||
run, ...context, candidate: { sha, tree }, policy: CI_PROOF_POLICIES.release, expected,
|
||||
}));
|
||||
} catch (error) {
|
||||
evaluations.push({
|
||||
@@ -72,12 +89,18 @@ const sleep = (ms) => new Promise((done) => setTimeout(done, ms));
|
||||
|
||||
export async function waitForGreenWorkflow({
|
||||
repo, sha, token, workflow = 'validate.yml', label = 'Validate', timeoutMs = 60 * 60 * 1000,
|
||||
root = process.cwd(),
|
||||
}) {
|
||||
if (!repo || !sha || !token || !workflow) throw new Error('repo, sha, token and workflow are required');
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
const url = workflowRunsUrl({ repo, workflow, sha });
|
||||
const proofRequired = workflow === 'validate.yml';
|
||||
const tree = proofRequired ? await githubCandidateTree({ repo, sha, token }) : null;
|
||||
const expected = proofRequired ? candidateExpectations({ sha, root }) : null;
|
||||
if (expected) {
|
||||
console.log(`candidate evidence: product tree ${expected.product.tree.slice(0, 12)}, baselines `
|
||||
+ `${expected.baselines.tree?.slice(0, 12) || 'none'}, reviewed run ${expected.baselines.reviewedRun || 'none'} (#573)`);
|
||||
}
|
||||
while (true) {
|
||||
const response = await fetch(url, {
|
||||
headers: {
|
||||
@@ -92,7 +115,7 @@ export async function waitForGreenWorkflow({
|
||||
const runs = Array.isArray(body?.workflow_runs) ? body.workflow_runs : [];
|
||||
const latest = latestRelevantRun(runs);
|
||||
const verdict = proofRequired
|
||||
? await classifyValidateProofs({ runs, repo, sha, tree, token })
|
||||
? await classifyValidateProofs({ runs, repo, sha, tree, token, expected })
|
||||
: { status: classifyValidateRuns(runs) === 'success' ? 'green'
|
||||
: classifyValidateRuns(runs) === 'fail' ? 'failed' : 'pending', url: latest?.html_url, note: '' };
|
||||
if (verdict.status === 'failed') {
|
||||
|
||||
@@ -12,7 +12,7 @@ import { spawnSync } from 'node:child_process';
|
||||
import { createInterface } from 'node:readline/promises';
|
||||
import { stdin, stdout } from 'node:process';
|
||||
import { assertReleaseContract } from './release-contract.mjs';
|
||||
import { classifyValidateProofs } from './release-gate.mjs';
|
||||
import { candidateExpectations, classifyValidateProofs } from './release-gate.mjs';
|
||||
import { assertBundleManifest } from './bundle-tree.mjs';
|
||||
import { SUMS_FILE, compareSums, formatSums, parseSums, sumsOfDirectory } from './release-assets.mjs';
|
||||
import {
|
||||
@@ -416,7 +416,9 @@ if (invokedDirectly) {
|
||||
]);
|
||||
const tree = run('git', ['rev-parse', `${sha}^{tree}`]).stdout;
|
||||
const token = run('gh', ['auth', 'token']).stdout;
|
||||
const verdict = await classifyValidateProofs({ runs, repo, sha, tree, token });
|
||||
// #573: локальный публикатор стоит на checkout кандидата — сверяет evidence.
|
||||
const expected = candidateExpectations({ sha, root });
|
||||
const verdict = await classifyValidateProofs({ runs, repo, sha, tree, token, expected });
|
||||
if (verdict.status !== 'green') {
|
||||
throw new Error(`Exact-SHA Validate proof is ${verdict.status} for ${sha}: ${verdict.note}`);
|
||||
}
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
CHECKS, CHECK_NAMES, NOT_AN_INPUT, REUSE_JOBS, checksAffectedBy, closure, coverage, globToRegExp,
|
||||
inputsOf, isDeclaredNotAnInput, isExecutableInput, manifest, referencesOf, stripComments,
|
||||
BASELINE_OVERLAY, CHECKS, CHECK_NAMES, NOT_AN_INPUT, REUSE_JOBS, checksAffectedBy, closure, coverage,
|
||||
globToRegExp, inputsOf, isBaselineOverlay, isDeclaredNotAnInput, isExecutableInput, manifest, referencesOf,
|
||||
stripComments,
|
||||
} from '../scripts/check-inputs.mjs';
|
||||
|
||||
// Единый manifest входов (#492 §5). Две группы доказательств: чистая механика
|
||||
@@ -103,6 +103,48 @@ test('замыкание: код транзитивно, данные — лис
|
||||
assert.equal(parents.get('scripts/helper.mjs'), 'demo/compat.mjs');
|
||||
});
|
||||
|
||||
// #573: overlay принятых эталонов принадлежит только golden. Строка-каталог
|
||||
// `demo/golden` в корпусе отпечатка раскрывалась во ВСЕ текстовые файлы под
|
||||
// ним, и индекс эталонов становился входом smoke, perf и каждого гарда через
|
||||
// serve.mjs — приёмка 13 кадров на beta.3 перегнала 22 минуты чужой работы.
|
||||
test('#573: раскрытие каталога не выдаёт overlay эталонов; явный корень и явная ссылка — выдают', () => {
|
||||
const files = {
|
||||
'demo/smoke_a.mjs': "import './serve.mjs';\n",
|
||||
'demo/serve.mjs': "import '../scripts/source-fingerprint.mjs';\n",
|
||||
'scripts/source-fingerprint.mjs': "const corpus = ['demo/fixtures', 'demo/golden'];\n",
|
||||
'demo/golden/run.mjs': "const baselineRoot = 'demo/golden/baselines';\n",
|
||||
'demo/golden/matrix.mjs': 'export const GOLDEN_SCENARIOS = [];\n',
|
||||
'demo/golden/baselines/baselines-index.json': '{"scenarios":{}}',
|
||||
'demo/golden/baselines/scene.png': 'binary',
|
||||
'demo/golden/baselines/.gitkeep': '',
|
||||
'test/golden-index.test.mjs': "const index = 'demo/golden/baselines/baselines-index.json';\n",
|
||||
};
|
||||
const tracked = Object.keys(files).sort();
|
||||
const viaSmoke = closure('/virtual', ['demo/smoke_a.mjs'], { tracked, read: (f) => files[f] });
|
||||
assert.ok(viaSmoke.includes('demo/golden/matrix.mjs'), 'код под demo/golden — по-прежнему вход');
|
||||
assert.ok(!viaSmoke.some(isBaselineOverlay), `overlay не течёт через каталог: ${viaSmoke.join(', ')}`);
|
||||
// golden сама называет каталог эталонов строкой — и всё равно получает их не
|
||||
// раскрытием, а явным корнем manifest (CHECKS.golden.roots)
|
||||
const viaGolden = closure('/virtual', ['demo/golden/run.mjs'], { tracked, read: (f) => files[f] });
|
||||
assert.ok(!viaGolden.some(isBaselineOverlay), 'каталог overlay по строке — тоже не раскрывается');
|
||||
assert.ok(CHECKS.golden.roots.includes('demo/golden/**'), 'эталоны входят в golden корнем');
|
||||
// явная ссылка на файл overlay — честная зависимость, она остаётся
|
||||
const viaTest = closure('/virtual', ['test/golden-index.test.mjs'], { tracked, read: (f) => files[f] });
|
||||
assert.ok(viaTest.includes('demo/golden/baselines/baselines-index.json'));
|
||||
assert.deepEqual(BASELINE_OVERLAY, ['demo/golden/baselines/**']);
|
||||
assert.equal(isBaselineOverlay('demo/golden/baselines/baselines-index.json'), true);
|
||||
assert.equal(isBaselineOverlay('demo/golden/matrix.mjs'), false);
|
||||
});
|
||||
|
||||
test('#573: на живом дереве индекс эталонов — вход golden и ничьей другой реюзной job', () => {
|
||||
// путь собран из кусков: литерал сделал бы индекс входом frontend через этот тест
|
||||
const index = p('demo', 'golden', 'baselines', 'baselines-index.json');
|
||||
for (const check of ['smoke', 'performance_smoke', 'geometry_parity', 'backend']) {
|
||||
assert.ok(!MANIFEST[check].has(index), `${check} не читает эталоны`);
|
||||
}
|
||||
assert.ok(MANIFEST.golden.has(index));
|
||||
});
|
||||
|
||||
test('замыкание останавливается на копиях бандла (класс D)', () => {
|
||||
const files = {
|
||||
'demo/smoke_a.mjs': "import '../custom_components/houseplan/frontend/houseplan-card.js';\n",
|
||||
|
||||
+146
-2
@@ -2,10 +2,14 @@ import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { deflateRawSync } from 'node:zlib';
|
||||
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
import {
|
||||
CI_PROOF_POLICIES, buildCiProof, evaluateCiProof, parseReuseMarker, readCiProofArtifact,
|
||||
requiredCheckIds, selectCiProofVerdict,
|
||||
CI_PROOF_POLICIES, baselineReviewedRun, buildCiProof, evaluateCiProof, localEvidence, parseReuseMarker,
|
||||
productTreeId, readCiProofArtifact, requiredCheckIds, selectCiProofVerdict,
|
||||
} from '../scripts/ci-proof.mjs';
|
||||
import { REUSE_JOBS } from '../scripts/check-inputs.mjs';
|
||||
import { reuseKey } from '../scripts/gate-reuse.mjs';
|
||||
|
||||
export const SHA = 'a'.repeat(40);
|
||||
export const TREE = 'b'.repeat(40);
|
||||
@@ -210,3 +214,143 @@ test('#541: uploaded deflated artifact is read without an external ZIP dependenc
|
||||
const zip = Buffer.concat([local, name, compressed, central, name, eocd]);
|
||||
assert.deepEqual(readCiProofArtifact(zip), { schema: 'test', ok: true });
|
||||
});
|
||||
|
||||
// #573 — составное evidence. Proof называет отдельно продуктовое дерево,
|
||||
// overlay принятых эталонов и content-ключи реюзных job; потребитель с
|
||||
// checkout кандидата сверяет их, а не верит. Сценарий beta.3: кандидат C
|
||||
// красный только по golden, baseline-only коммит B переиспользует зелёные
|
||||
// job C и перегоняет golden.
|
||||
const KEYS = Object.fromEntries(REUSE_JOBS.map((job, index) => [job, String(index + 1).repeat(64)]));
|
||||
const evidenceOf = (over = {}) => ({
|
||||
product: { tree: 'p'.repeat(64) },
|
||||
baselines: { tree: 'c'.repeat(40), manifestSha256: 'd'.repeat(64), reviewedRun: 34853080375 },
|
||||
keys: { ...KEYS },
|
||||
...over,
|
||||
});
|
||||
|
||||
/** B: smoke и performance_smoke reused из красного-по-golden C, golden исполнена. */
|
||||
function baselineOnlyFixture() {
|
||||
const fixture = proofFixture({ id: 20, attempt: 1 });
|
||||
const evidence = evidenceOf();
|
||||
const reuseFrom = (id) => ({
|
||||
mode: 'reused', result: 'success',
|
||||
reuse: { key: evidence.keys[id], sourceRun: 10, sourceAttempt: 1, sourceSha: 'e'.repeat(40) },
|
||||
});
|
||||
fixture.proof.checks.smoke = reuseFrom('smoke');
|
||||
fixture.proof.checks.performance_smoke = reuseFrom('performance_smoke');
|
||||
fixture.proof.checks.golden.key = evidence.keys.golden;
|
||||
fixture.proof.checks.geometry_parity.key = evidence.keys.geometry_parity;
|
||||
fixture.proof.checks.backend.key = evidence.keys.backend;
|
||||
fixture.proof.executedChecks = fixture.proof.executedChecks.filter((id) => !['smoke', 'performance_smoke'].includes(id));
|
||||
fixture.proof.reusedChecks = ['performance_smoke', 'smoke'];
|
||||
fixture.proof.evidence = evidence;
|
||||
fixture.jobs = fixture.jobs.filter((job) => !/Смоки|Перф-смок/.test(job.name));
|
||||
// C: run красный (golden different), но smoke и perf — зелёные job
|
||||
fixture.reuseRuns.set('10:1', {
|
||||
run: { id: 10, run_attempt: 1, status: 'completed', conclusion: 'failure', head_sha: 'e'.repeat(40) },
|
||||
jobs: [...smokeJobs(), success(names.smokeDone), success(names.performance),
|
||||
{ name: names.golden, conclusion: 'failure' }, ...mutantJobs()],
|
||||
});
|
||||
fixture.reviewedRun = { run: { id: 34853080375, path: '.github/workflows/validate.yml', status: 'completed', conclusion: 'failure' } };
|
||||
fixture.expected = evidenceOf();
|
||||
return fixture;
|
||||
}
|
||||
|
||||
test('#573 AC1: baseline-only коммит — reused smoke/perf из красного-по-golden кандидата, golden исполнена, proof green', () => {
|
||||
const fixture = baselineOnlyFixture();
|
||||
const verdict = evaluateCiProof({ ...fixture, policy: CI_PROOF_POLICIES.release });
|
||||
assert.equal(verdict.status, 'green', verdict.note);
|
||||
assert.deepEqual(fixture.proof.reusedChecks, ['performance_smoke', 'smoke']);
|
||||
assert.ok(fixture.proof.executedChecks.includes('golden'), 'golden сравнивает с новыми эталонами и перегоняется всегда');
|
||||
// те же семантики без ожиданий — review/merge потребители не ломаются
|
||||
assert.equal(evaluateCiProof({ ...fixture, expected: null, reviewedRun: undefined, policy: CI_PROOF_POLICIES.merge }).status, 'green');
|
||||
});
|
||||
|
||||
test('#573 AC3: красный smoke кандидата не прячется за зелёной golden — источник reuse обязан быть зелёной job', () => {
|
||||
const fixture = baselineOnlyFixture();
|
||||
fixture.reuseRuns.get('10:1').jobs = fixture.reuseRuns.get('10:1').jobs
|
||||
.map((job) => (job.name.startsWith('Смоки в браузере (шард 2') ? { ...job, conclusion: 'failure' } : job));
|
||||
const verdict = evaluateCiProof({ ...fixture, policy: CI_PROOF_POLICIES.release });
|
||||
assert.equal(verdict.status, 'failed');
|
||||
assert.match(verdict.note, /smoke: source run does not verify/);
|
||||
});
|
||||
|
||||
test('#573 AC4: подмена content-ключа, product tree, overlay, индекса или reviewed run — fail-closed', () => {
|
||||
const fixture = baselineOnlyFixture();
|
||||
const withExpected = (over) => evaluateCiProof({
|
||||
...fixture, expected: evidenceOf(over), policy: CI_PROOF_POLICIES.release,
|
||||
});
|
||||
assert.equal(withExpected({}).status, 'green');
|
||||
assert.match(withExpected({ keys: { ...KEYS, smoke: 'f'.repeat(64) } }).note, /keys\.smoke/);
|
||||
assert.equal(withExpected({ keys: { ...KEYS, smoke: 'f'.repeat(64) } }).status, 'failed');
|
||||
assert.match(withExpected({ product: { tree: 'q'.repeat(64) } }).note, /product\.tree/);
|
||||
assert.match(withExpected({ baselines: { tree: 'x'.repeat(40), manifestSha256: 'd'.repeat(64), reviewedRun: 34853080375 } }).note, /baselines\.tree/);
|
||||
assert.match(withExpected({ baselines: { tree: 'c'.repeat(40), manifestSha256: 'y'.repeat(64), reviewedRun: 34853080375 } }).note, /manifestSha256/);
|
||||
assert.match(withExpected({ baselines: { tree: 'c'.repeat(40), manifestSha256: 'd'.repeat(64), reviewedRun: 1 } }).note, /reviewedRun/);
|
||||
// маркер реюза ссылается на ключ, отличный от ключа кандидата — внутренняя несогласованность
|
||||
const tampered = structuredClone(fixture.proof);
|
||||
tampered.checks.smoke.reuse.key = 'a'.repeat(64);
|
||||
assert.match(evaluateCiProof({ ...fixture, proof: tampered, policy: CI_PROOF_POLICIES.release }).note, /smoke: reused marker key differs/);
|
||||
// объявленный run просмотра кадров не существует / отменён / не Validate
|
||||
assert.match(evaluateCiProof({ ...fixture, reviewedRun: null, policy: CI_PROOF_POLICIES.release }).note, /Baseline-Reviewed run 34853080375 is missing/);
|
||||
assert.equal(evaluateCiProof({
|
||||
...fixture, reviewedRun: { run: { id: 34853080375, path: '.github/workflows/validate.yml', status: 'completed', conclusion: 'cancelled' } },
|
||||
policy: CI_PROOF_POLICIES.release,
|
||||
}).status, 'failed');
|
||||
assert.equal(evaluateCiProof({
|
||||
...fixture, reviewedRun: { run: { id: 34853080375, path: '.github/workflows/nightly.yml', status: 'completed', conclusion: 'success' } },
|
||||
policy: CI_PROOF_POLICIES.release,
|
||||
}).status, 'failed');
|
||||
// proof без evidence при наличии ожиданий — устарел, а не «сойдёт»
|
||||
const legacy = structuredClone(fixture.proof);
|
||||
delete legacy.evidence;
|
||||
assert.equal(evaluateCiProof({ ...fixture, proof: legacy, policy: CI_PROOF_POLICIES.release }).status, 'stale');
|
||||
});
|
||||
|
||||
test('#573: identity продуктового дерева не видит overlay эталонов, но видит всё остальное', () => {
|
||||
const lines = [
|
||||
'100644 blob 1111\tsrc/logic.ts',
|
||||
'100644 blob 2222\tdemo/golden/matrix.mjs',
|
||||
'100644 blob 3333\tdemo/golden/baselines/scene.png',
|
||||
'100644 blob 4444\tdemo/golden/baselines/baselines-index.json',
|
||||
];
|
||||
const before = productTreeId(lines.join('\n'));
|
||||
const accepted = productTreeId(lines.map((line) => line.replace('3333', '5555').replace('4444', '6666')).join('\n'));
|
||||
assert.equal(accepted, before, 'приёмка эталонов — то же продуктовое дерево');
|
||||
assert.notEqual(productTreeId(lines.map((line) => line.replace('1111', '9999')).join('\n')), before);
|
||||
assert.notEqual(productTreeId(lines.map((line) => line.replace('2222', '9999')).join('\n')), before, 'сцены — продукт');
|
||||
assert.throws(() => productTreeId(''), /empty/);
|
||||
assert.equal(baselineReviewedRun('Accept frames\n\nBaseline-Reviewed: https://github.com/x/y/actions/runs/777\n'), 777);
|
||||
assert.equal(baselineReviewedRun('no trailer'), null);
|
||||
assert.throws(() => baselineReviewedRun('Baseline-Reviewed: somewhere-else\n'), /does not name an actions run/);
|
||||
});
|
||||
|
||||
test('#573: buildCiProof пишет ключ исполненной реюзной job и отвергает маркер с чужим ключом', () => {
|
||||
const evidence = evidenceOf();
|
||||
const { proof } = proofFixture({ id: 30 });
|
||||
const needs = { preflight: { result: 'success' }, changes: { result: 'success', outputs: { heavy: 'true', mutants_requested: 'true', frontend: 'true', backend: 'true', geometry_parity: 'true', integration: 'true' } },
|
||||
reuse: { result: 'success', outputs: { smoke: 'true', smoke_key: evidence.keys.smoke, smoke_source_run: '10', smoke_source_attempt: '1', smoke_source_sha: 'e'.repeat(40) } },
|
||||
frontend: { result: 'success' }, hacs: { result: 'success' }, hassfest: { result: 'success' }, changed_mutants: { result: 'success' },
|
||||
smoke: { result: 'skipped' }, smoke_done: { result: 'skipped' }, golden: { result: 'success' }, performance_smoke: { result: 'success' },
|
||||
geometry_parity: { result: 'success' }, backend: { result: 'success' } };
|
||||
const built = buildCiProof({ candidateSha: SHA, candidateTree: TREE, runId: 30, attempt: 1, event: 'push', needs, evidence });
|
||||
assert.equal(built.checks.golden.key, evidence.keys.golden, 'исполненная job несёт свой content-ключ');
|
||||
assert.equal(built.checks.smoke.mode, 'reused');
|
||||
assert.deepEqual(built.evidence, evidence);
|
||||
assert.ok(!proof.evidence, 'без evidence блока нет — обратная совместимость записи');
|
||||
const foreign = { ...needs, reuse: { ...needs.reuse, outputs: { ...needs.reuse.outputs, smoke_key: 'f'.repeat(64) } } };
|
||||
assert.throws(() => buildCiProof({ candidateSha: SHA, candidateTree: TREE, runId: 30, attempt: 1, event: 'push', needs: foreign, evidence }),
|
||||
/smoke: reuse marker key .* differs from the candidate key/);
|
||||
});
|
||||
|
||||
test('#573: evidence живого дерева считается детерминированно и совпадает с ключами gate-reuse', () => {
|
||||
const root = fileURLToPath(new URL('..', import.meta.url));
|
||||
const first = localEvidence(root);
|
||||
const second = localEvidence(root, { keys: first.keys });
|
||||
assert.deepEqual(second, first);
|
||||
assert.match(first.product.tree, /^[0-9a-f]{64}$/);
|
||||
assert.match(first.baselines.tree, /^[0-9a-f]{40}$/);
|
||||
assert.match(first.baselines.manifestSha256, /^[0-9a-f]{64}$/);
|
||||
for (const job of REUSE_JOBS) assert.equal(first.keys[job], reuseKey(root, job));
|
||||
assert.throws(() => localEvidence(root, { keys: { ...first.keys, smoke: 'f'.repeat(64) } }), /smoke: reuse job key/);
|
||||
});
|
||||
|
||||
@@ -143,6 +143,34 @@ test('a behaviour input changes every browser key and a version bump changes all
|
||||
}
|
||||
});
|
||||
|
||||
// #573 — воспроизведение beta.3 на синтетическом дереве: кандидат C и
|
||||
// baseline-only коммит B отличаются только overlay эталонов (PNG + индекс).
|
||||
// Корпус отпечатка называет `demo/golden` каталогом ровно как настоящий
|
||||
// source-fingerprint.mjs; до #573 это делало индекс входом smoke и perf.
|
||||
test('#573: приёмка эталонов меняет только ключ golden — smoke, perf, parity и backend переиспользуются', () => {
|
||||
const { dir, put } = makeTree();
|
||||
try {
|
||||
put('scripts/source-fingerprint.mjs', "const corpus = ['demo/fixtures', 'demo/golden'];\nexport const fp = 1;\n");
|
||||
put('demo/bundle-freshness.mjs', "import '../scripts/source-fingerprint.mjs';\nexport const fresh = 1;\n");
|
||||
put('demo/golden/baselines/baselines-index.json', '{"scenarios":{"one":"a"}}\n');
|
||||
const candidate = keys(dir);
|
||||
// B: 13 кадров и индекс переписаны, продукт не тронут
|
||||
put('demo/golden/baselines/one.png', Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x00, 0x02]));
|
||||
put('demo/golden/baselines/baselines-index.json', '{"scenarios":{"one":"b"},"acceptedAt":"later"}\n');
|
||||
const accepted = keys(dir);
|
||||
for (const job of JOBS) {
|
||||
if (job === 'golden') assert.notEqual(accepted[job], candidate[job], 'golden сравнивает с эталонами и обязана перегоняться');
|
||||
else assert.equal(accepted[job], candidate[job], `${job}: входы побайтово те же, что у кандидата`);
|
||||
}
|
||||
// но правка продукта вместе с эталонами перегоняет всё, что собирает бандл
|
||||
put('src/card.ts', "export const CARD_VERSION = '1.0.0'; export const changed = true;\n");
|
||||
const both = keys(dir);
|
||||
for (const job of ['smoke', 'golden', 'performance_smoke']) assert.notEqual(both[job], accepted[job], `${job} видит правку исходника`);
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('harness edits are isolated to their own job (#208)', () => {
|
||||
const { dir, put } = makeTree();
|
||||
const only = (changed) => {
|
||||
|
||||
@@ -446,6 +446,35 @@ test('#481 AC1: отпечаток свидетеля не меняется от
|
||||
assert.notEqual(witnessFingerprint({ ...LEDGER_MUTANT, patches: [{ file: 'src/x.ts', find: 'a', replace: 'c' }] }, base), fp, 'объявление патча');
|
||||
});
|
||||
|
||||
// #573. Гард-смок доходит до `source-fingerprint.mjs`, а тот называет корпус
|
||||
// строкой-каталогом `demo/golden`. До #573 раскрытие каталога делало индекс
|
||||
// эталонов входом каждого такого гарда: приёмка кадров на beta.3 сменила
|
||||
// отпечатки 181 из 183 браузерных свидетелей, и журнал их не пропустил.
|
||||
test('#573: приёмка эталонов не меняет отпечаток свидетеля со смок-гардом', () => {
|
||||
const tree = (index) => ({
|
||||
'src/x.ts': 'let a = 1;',
|
||||
'demo/smoke_x.mjs': "import './serve.mjs';\n",
|
||||
'demo/serve.mjs': "import '../scripts/source-fingerprint.mjs';\n",
|
||||
'scripts/source-fingerprint.mjs': "const corpus = ['demo/fixtures', 'demo/golden'];\n",
|
||||
'demo/golden/matrix.mjs': 'export const GOLDEN_SCENARIOS = [];\n',
|
||||
'demo/golden/baselines/baselines-index.json': index,
|
||||
});
|
||||
const fsOf = (files) => ({
|
||||
root: '/repo',
|
||||
read: (file) => files[file] ?? '',
|
||||
exists: (file) => file in files,
|
||||
normalize: (text) => text,
|
||||
inputsOf: (guard) => guardInputs(guard, { exists: (f) => f in files, read: (f) => files[f] ?? '', files: Object.keys(files).sort() }),
|
||||
});
|
||||
const mutant = { id: 'x', guard: 'node demo/smoke_x.mjs', patches: [{ file: 'src/x.ts', find: 'a', replace: 'b' }] };
|
||||
const candidate = witnessFingerprint(mutant, fsOf(tree('{"scenarios":{"one":"a"}}')));
|
||||
const accepted = witnessFingerprint(mutant, fsOf(tree('{"scenarios":{"one":"b"},"acceptedAt":"later"}')));
|
||||
assert.equal(accepted, candidate, 'overlay эталонов — не вход смок-гарда');
|
||||
const harness = tree('{"scenarios":{"one":"a"}}');
|
||||
harness['demo/golden/matrix.mjs'] = 'export const GOLDEN_SCENARIOS = [1];\n';
|
||||
assert.notEqual(witnessFingerprint(mutant, fsOf(harness)), candidate, 'код под demo/golden — по-прежнему вход');
|
||||
});
|
||||
|
||||
// #518. Хост-файлы карты — тринадцать тысяч строк. Отпечаток и отбор по файлу
|
||||
// целиком означали, что правка в одном их конце перегоняет свидетелей из
|
||||
// другого: на #500 двенадцать изменённых строк тянули 53 мутанта из 75.
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import {
|
||||
classifyValidateProofs, classifyValidateRuns, latestRelevantRun, workflowRunsUrl,
|
||||
candidateExpectations, classifyValidateProofs, classifyValidateRuns, latestRelevantRun, workflowRunsUrl,
|
||||
} from '../scripts/release-gate.mjs';
|
||||
import { buildCiProof } from '../scripts/ci-proof.mjs';
|
||||
import { buildCiProof, localEvidence } from '../scripts/ci-proof.mjs';
|
||||
|
||||
const SHA = 'a'.repeat(40);
|
||||
const TREE = 'b'.repeat(40);
|
||||
@@ -124,3 +126,40 @@ test('#541: the release documents describe proof semantics', () => {
|
||||
const performance = readFileSync(new URL('../demo/performance/README.md', import.meta.url), 'utf8');
|
||||
assert.match(performance, /latest\nnon-cancelled run on the SHA/);
|
||||
});
|
||||
|
||||
// #573: гейт релиза стоит на checkout кандидата и сверяет составное evidence
|
||||
// proof с тем, что считает сам; чужой checkout — честное «проверяю только по
|
||||
// GitHub», а не молчаливый пропуск.
|
||||
test('#573: ожидания считаются только на checkout кандидата и уходят в classifyValidateProofs', async () => {
|
||||
const root = fileURLToPath(new URL('..', import.meta.url));
|
||||
const head = execFileSync('git', ['-C', root, 'rev-parse', 'HEAD'], { encoding: 'utf8' }).trim();
|
||||
const notes = [];
|
||||
const foreign = candidateExpectations({ sha: 'f'.repeat(40), root, log: (line) => notes.push(line) });
|
||||
assert.equal(foreign, null);
|
||||
assert.match(notes[0], /is not the candidate ffffffff — proof evidence is verified against GitHub only/);
|
||||
const own = candidateExpectations({ sha: head, root, log: (line) => notes.push(line) });
|
||||
assert.deepEqual(own, localEvidence(root));
|
||||
|
||||
// proof без evidence при наличии ожиданий — stale, а старее его нет → missing; с evidence и совпадением — green
|
||||
const legacy = proofContext({ id: 40 });
|
||||
const verdict = await classifyValidateProofs({
|
||||
runs: [legacy.run], repo: 'x/y', sha: SHA, tree: TREE, token: 'x', expected: own,
|
||||
loadContext: async () => legacy.context,
|
||||
});
|
||||
assert.equal(verdict.status, 'missing', verdict.note);
|
||||
const modern = proofContext({ id: 41 });
|
||||
modern.context.proof.evidence = structuredClone(own);
|
||||
const green = await classifyValidateProofs({
|
||||
runs: [modern.run], repo: 'x/y', sha: SHA, tree: TREE, token: 'x', expected: own,
|
||||
loadContext: async () => modern.context,
|
||||
});
|
||||
assert.equal(green.status, 'green', green.note);
|
||||
const substituted = structuredClone(own);
|
||||
substituted.keys.golden = '0'.repeat(64);
|
||||
const red = await classifyValidateProofs({
|
||||
runs: [modern.run], repo: 'x/y', sha: SHA, tree: TREE, token: 'x', expected: substituted,
|
||||
loadContext: async () => modern.context,
|
||||
});
|
||||
assert.equal(red.status, 'failed');
|
||||
assert.match(red.note, /keys\.golden/);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user