test(perf): pull the Long Task ceilings of the switchCycle profiles down to the warmed level (#770)

longTasks.maxCountP95 / maxTotalP95Ms sum every window of a sample,
switchCycle included. Since #735 that window is warm, and the 30 tasks /
12000 ms of the cold cycle sat 1.25-3.2 times above the family maxima;
performance_smoke judges only these absolute ceilings.

Series (CI only, as #747): every Full Performance run 2026-10-01..10-05,
14 runs, each with a runner containing #735 (5e318168), both sides,
7 samples, one runner type (linux x64, Chromium 151.0.7922.34, Node
22.23.3) - 27 reports / 189 samples per profile. Point = what the ceiling
judges, the nearest-rank p95 of a report (its largest sample). The window
is warm in all of them: largest single Long Task in switchCycle 190 ms
against 427-1008 ms for the smallest cold floor switch; the #735 guard
fired in none.

                      count p50/p95/max   total p50/p95/max (ms)
  large-house-v1         10 / 12 / 13     2826 / 3140 / 3301
  plan-snap              14 / 14 / 15     3387 / 3610 / 3759
  interaction            12 / 13 / 13     3056 / 3242 / 3289
  isometric              20 / 21 / 22     4761 / 5101 / 5104
  stage3-dense           23 / 24 / 24     5516 / 5935 / 5935
  isometric-backdrop     21 / 22 / 22     4931 / 5286 / 5331

Rule (#747, per metric, one number per family): first step (one task,
50 ms) at or above 1.15 x M, no higher than 1.2 x M:
  flat (lh, plan-snap, interaction, interaction smoke)
    30 -> 18 (1.15 x 15 = 17.25, +20 %), 12000 -> 4350 (1.15 x 3759, +15.7 %)
  2.5D (isometric, dense, backdrop, isometric smoke)
    30 -> 28 (1.15 x 24 = 27.6, +16.7 %), 12000 -> 6850 (1.15 x 5935, +15.4 %)

Out of M: the base of the v1.79.0 stable comparison 37149329461 is
v1.78.0 (7d4d75bd, 09-28, before #735 and #694/#725/#739). On the same
runner as its candidate it reads 1.2-2 x higher - code, not noise - and
serves as the known slower variant: its dense sum of 8754 ms fails 6850;
its other points (isometric 24/6783, backdrop 22/5159, flat at most
17/3959) pass and stay the relative comparison's job. The one 3-sample
interaction smoke of the period (36910217188, 12/3047) passes. The
relative half (ratios, 3/5 tasks, 150 ms) is unchanged.

The test pins one pair per family in every file, replays all 162 points
and the smoke point through the --absolute-only evaluation, recomputes
the rule, requires doubling to fail on each metric and the v1.78.0
verdicts above, and keeps the relative fields; #507's count pin follows
to 28. Red on the previous budgets, and red with 12000 left in the
backdrop file.

The README records the series and the rule ("CI contracts") and, in
"Changing budgets", the recalibration method; it also drops the stale
#585 text: the 450/150/120 gesture allowances returned to 150/60/75 with
6f226a5b (v1.77.0-beta.2), and the backdrop twin carries the iso count
allowance too.

Issue: #770
User-Visible: no
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZfe7YS4rqEMKoVeS3GKd
This commit is contained in:
Claude
2026-10-07 00:26:16 +00:00
committed by claude[bot]
parent 283200c789
commit cb7faab3e2
10 changed files with 232 additions and 33 deletions
+131 -1
View File
@@ -419,6 +419,134 @@ for (const [family, spec] of Object.entries(SWITCH_CYCLE_FAMILIES)) {
});
}
// #770: обоснование потолков `longTasks.maxCountP95` / `maxTotalP95Ms` и ряд —
// demo/performance/README.md, «CI contracts». Суммы Long Task — по всем окнам
// образца, и окно `switchCycle` после #735 тёплое, поэтому прежние 30 / 12000
// стояли в 1,25–3,2 раза над уровнем. Ряд — все прогоны Full Performance
// 01–05.10 (раннер кандидата содержит #735 во всех), обе стороны, 7 образцов,
// один тип раннера (linux x64, Chromium 151.0.7922.34, Node 22.23.3), в порядке
// [база, кандидат] прогонов 36883292495, 36904753178, 36904816677, 36905707715,
// 36905760140, 37055609391, 37062790080, 37065283633, 37134606046, 37136641943,
// 37149329461 (только кандидат), 37176616088, 37203867985, 37302579095. Точка —
// то, что судит потолок: p95 ближайшего ранга по 7 образцам, то есть максимум.
// База 37149329461 — v1.78.0 (7d4d75bd, 28.09, до #735 и тёплых работ
// #694/#725/#739): на том же раннере она в 1,2–2 раза медленнее кандидата, это
// код, а не шум, и в M она не входит — это известный более медленный вариант.
// Смоковая точка (3 образца) — 36910217188, interaction, adc2d7c5.
const LONG_TASK_FAMILIES = {
flat: {
count: 18, total: 4350,
smoke: 'budgets-interaction-smoke.json',
files: ['budgets.json', 'budgets-large-house-plan-snap.json', 'budgets-large-house-interaction.json',
'budgets-interaction-smoke.json'],
full: { maxCountRegressionRatio: 0.35, countNoiseAllowance: 3, maxTotalRegressionRatio: 0.3, noiseAllowanceMs: 150 },
series: {
'large-house-v1': {
count: [12, 11, 11, 10, 11, 12, 9, 7, 11, 12, 10, 9, 9, 9, 10, 11, 9, 9, 8, 8, 10, 10, 10, 13, 9, 10, 10],
total: [3301, 3101, 3068, 2825, 3140, 2994, 2269, 2108, 2948, 2927, 2923, 2811, 2803, 2781, 3047, 3140,
2823, 2748, 2183, 2128, 2836, 2852, 2826, 3133, 2759, 2823, 2818],
},
'large-house-plan-snap-v1': {
count: [8, 7, 14, 14, 14, 14, 14, 13, 14, 14, 14, 14, 12, 12, 8, 9, 14, 14, 14, 12, 6, 14, 13, 15, 12, 13, 13],
total: [1752, 1647, 3550, 3323, 3498, 3610, 3546, 3313, 3418, 3433, 3759, 3433, 3192, 3179, 2327, 2359,
3387, 3411, 3416, 3296, 1798, 3407, 3400, 3509, 3157, 3266, 3279],
},
'large-house-interaction-v1': {
count: [11, 12, 11, 10, 12, 12, 13, 12, 12, 13, 10, 10, 13, 11, 12, 13, 11, 11, 12, 12, 12, 5, 5, 13, 12, 11, 11],
total: [3104, 3240, 3116, 2851, 3142, 3105, 3159, 3038, 3112, 3208, 2450, 2270, 3056, 2919, 3242, 3289,
2895, 2864, 3203, 3046, 2957, 1480, 1558, 3111, 3057, 2671, 2676],
},
},
smokePoints: [{ count: 12, total: 3047 }],
slower: {
'large-house-v1': { count: 17, total: 3959 },
'large-house-plan-snap-v1': { count: 12, total: 2562 },
'large-house-interaction-v1': { count: 16, total: 3660 },
},
},
isometric: {
count: 28, total: 6850,
smoke: 'budgets-isometric-smoke.json',
files: ['budgets-large-house-isometric.json', 'budgets-isometric-stage3-dense.json',
'budgets-large-house-isometric-backdrop.json', 'budgets-isometric-smoke.json'],
full: { maxCountRegressionRatio: 0.2, countNoiseAllowance: 5, maxTotalRegressionRatio: 0.2, noiseAllowanceMs: 150 },
series: {
'large-house-isometric-v1': {
count: [15, 16, 21, 21, 20, 21, 20, 20, 21, 22, 19, 20, 20, 21, 21, 21, 14, 14, 12, 15, 19, 21, 20, 12, 12, 20, 21],
total: [3713, 3543, 4944, 5088, 5104, 5022, 4135, 3956, 5101, 5078, 4618, 4709, 4816, 4754, 4831, 4889,
3525, 3404, 2651, 2854, 4761, 4812, 5073, 2571, 2569, 4926, 4978],
},
'isometric-stage3-dense-v1': {
count: [23, 24, 23, 23, 23, 24, 23, 23, 23, 23, 24, 23, 23, 23, 24, 23, 23, 23, 23, 23, 23, 23, 20, 23, 22, 21, 21],
total: [5670, 5769, 5480, 5458, 5618, 5935, 5935, 5492, 5645, 5527, 5520, 5643, 5375, 5407, 5894, 5555,
5740, 5502, 5488, 5516, 5627, 4380, 4181, 5419, 5474, 4088, 4086],
},
'large-house-isometric-backdrop-v1': {
count: [21, 21, 20, 21, 21, 22, 12, 12, 21, 21, 20, 22, 20, 21, 21, 22, 13, 12, 22, 21, 11, 21, 20, 22, 20, 12, 14],
total: [4946, 4887, 4736, 4816, 5331, 5182, 2694, 2524, 5010, 5014, 5086, 4931, 4748, 4907, 5286, 4975,
2913, 2698, 5158, 5160, 2634, 5036, 5026, 5034, 4908, 2591, 2669],
},
},
smokePoints: [],
slower: {
'large-house-isometric-v1': { count: 24, total: 6783 },
'isometric-stage3-dense-v1': { count: 27, total: 8754 },
'large-house-isometric-backdrop-v1': { count: 22, total: 5159 },
},
},
};
for (const [family, spec] of Object.entries(LONG_TASK_FAMILIES)) {
test(`#770 Long Task count/total (${family}): потолки над тёплым уровнем после #735 с запасом на шум раннера`, () => {
// Одно число на семью, как у #747: смок = полный (#473 AC4), plan-snap и
// interaction — потолки large-house-v1, двойники — исторические (#160, #743).
for (const file of spec.files) {
const { longTasks } = readBudget(file);
assert.equal(longTasks.maxCountP95, spec.count, `${file}: общий потолок числа Long Task семьи ${family}`);
assert.equal(longTasks.maxTotalP95Ms, spec.total, `${file}: общий потолок суммы Long Task семьи ${family}`);
// Ужесточение, а не ослабление: прежние 30 / 12000 #770 только опускает.
assert.ok(longTasks.maxCountP95 < 30 && longTasks.maxTotalP95Ms < 12000, `${file}: потолок не ниже прежнего`);
}
const smoke = readBudget(spec.smoke);
const failures = (count, total) => evaluatePerformanceBudget({
candidate: { ...absoluteSmokeReport(smoke), longTasks: { maxSingleMs: 1, countP95: count, totalP95Ms: total } },
budgets: smoke, absoluteOnly: true,
}).failures.map((check) => check.id);
const counts = Object.values(spec.series).flatMap((series) => series.count);
const totals = Object.values(spec.series).flatMap((series) => series.total);
for (const series of Object.values(spec.series)) {
assert.equal(series.count.length, 27, 'ряд — 14 прогонов × 2 стороны без базы v1.78.0');
assert.equal(series.total.length, 27);
series.count.forEach((count, index) => assert.deepEqual(failures(count, series.total[index]), [],
`${count} / ${series.total[index]} из ряда краснеет на ${spec.count} / ${spec.total}`));
}
for (const { count, total } of spec.smokePoints) assert.deepEqual(failures(count, total), [], 'смоковая точка');
// Правило #747 для обеих метрик: первый шаг (1 задача, 50 мс) не ниже
// 1.15 × M и не выше 1.2 × M, M — максимум ряда семьи.
const maxCount = Math.max(...counts);
const maxTotal = Math.max(...totals);
assert.equal(spec.count, Math.ceil(maxCount * 1.15), `число — первое целое над 1.15 × ${maxCount}`);
assert.equal(spec.total, Math.ceil((maxTotal * 1.15) / 50) * 50, `сумма — первое кратное 50 над 1.15 × ${maxTotal}`);
assert.ok(spec.count >= maxCount * 1.15 && spec.count <= maxCount * 1.2, 'число: запас 15–20 %');
assert.ok(spec.total >= maxTotal * 1.15 && spec.total <= maxTotal * 1.2, 'сумма: запас 15–20 %');
assert.deepEqual(failures(2 * maxCount, 1), ['longTask.countP95'], 'удвоение числа краснеет');
assert.deepEqual(failures(1, 2 * maxTotal), ['longTask.totalP95Ms'], 'удвоение суммы краснеет');
// Известный более медленный вариант — база v1.78.0 того же раннера: в M не
// входит. Возврат плотного двойника к её сумме (8754 мс) потолок ловит;
// остальные её точки ниже потолков — их ловит относительное сравнение.
for (const [profile, { count, total }] of Object.entries(spec.slower)) {
const expected = profile === 'isometric-stage3-dense-v1' ? ['longTask.totalP95Ms'] : [];
assert.deepEqual(failures(count, total), expected, `${profile} v1.78.0: ${count} / ${total}`);
}
// Меньшее ловит относительное сравнение полного прогона: его коэффициенты и
// допуски не рычаг.
for (const file of spec.files.filter((name) => name !== spec.smoke)) {
const { longTasks } = readBudget(file);
for (const [key, value] of Object.entries(spec.full)) assert.equal(longTasks[key], value, `${file}: ${key} не рычаг`);
}
});
}
test('boundary collision search restores the ordinary isometric allowances (#585)', () => {
const isometric = readBudget('budgets-large-house-isometric.json');
const dense = readBudget('budgets-isometric-stage3-dense.json');
@@ -452,7 +580,9 @@ test('isometric long-task count allowance covers the lazy iso-chunk split, nothi
const isometric = readBudget('budgets-large-house-isometric.json');
assert.equal(isometric.longTasks.countNoiseAllowance, 5, 'owner-accepted +2 tasks of the lazy iso-scene-render split plus jitter');
assert.equal(isometric.longTasks.maxCountRegressionRatio, 0.2, 'the ratio is not the lever');
assert.equal(isometric.longTasks.maxCountP95, 30);
// #770: the family ceiling, above the 21 the allowance yields for that base,
// so the relative half still decides this comparison.
assert.equal(isometric.longTasks.maxCountP95, 28);
assert.equal(isometric.longTasks.maxTotalRegressionRatio, 0.2, 'total work is still gated as before');
assert.equal(isometric.longTasks.maxSingleRegressionRatio, 0.2);
assert.equal(readBudget('budgets-isometric-stage3-dense.json').longTasks.countNoiseAllowance, 5, 'the dense twin shares the lazy iso chunk and the #160 common-ceiling contract');