mirror of
https://github.com/Matysh/houseplan-card
synced 2026-10-07 23:19:14 +00:00
test(perf): pull the Long Task ceilings of the switchCycle profiles down to the warmed level (#770)
longTasks.maxCountP95 / maxTotalP95Ms sum every window of a sample, switchCycle included. Since #735 that window is warm, and the 30 tasks / 12000 ms of the cold cycle sat 1.25-3.2 times above the family maxima; performance_smoke judges only these absolute ceilings. Series (CI only, as #747): every Full Performance run 2026-10-01..10-05, 14 runs, each with a runner containing #735 (5e318168), both sides, 7 samples, one runner type (linux x64, Chromium 151.0.7922.34, Node 22.23.3) - 27 reports / 189 samples per profile. Point = what the ceiling judges, the nearest-rank p95 of a report (its largest sample). The window is warm in all of them: largest single Long Task in switchCycle 190 ms against 427-1008 ms for the smallest cold floor switch; the #735 guard fired in none. count p50/p95/max total p50/p95/max (ms) large-house-v1 10 / 12 / 13 2826 / 3140 / 3301 plan-snap 14 / 14 / 15 3387 / 3610 / 3759 interaction 12 / 13 / 13 3056 / 3242 / 3289 isometric 20 / 21 / 22 4761 / 5101 / 5104 stage3-dense 23 / 24 / 24 5516 / 5935 / 5935 isometric-backdrop 21 / 22 / 22 4931 / 5286 / 5331 Rule (#747, per metric, one number per family): first step (one task, 50 ms) at or above 1.15 x M, no higher than 1.2 x M: flat (lh, plan-snap, interaction, interaction smoke) 30 -> 18 (1.15 x 15 = 17.25, +20 %), 12000 -> 4350 (1.15 x 3759, +15.7 %) 2.5D (isometric, dense, backdrop, isometric smoke) 30 -> 28 (1.15 x 24 = 27.6, +16.7 %), 12000 -> 6850 (1.15 x 5935, +15.4 %) Out of M: the base of the v1.79.0 stable comparison 37149329461 is v1.78.0 (7d4d75bd, 09-28, before #735 and #694/#725/#739). On the same runner as its candidate it reads 1.2-2 x higher - code, not noise - and serves as the known slower variant: its dense sum of 8754 ms fails 6850; its other points (isometric 24/6783, backdrop 22/5159, flat at most 17/3959) pass and stay the relative comparison's job. The one 3-sample interaction smoke of the period (36910217188, 12/3047) passes. The relative half (ratios, 3/5 tasks, 150 ms) is unchanged. The test pins one pair per family in every file, replays all 162 points and the smoke point through the --absolute-only evaluation, recomputes the rule, requires doubling to fail on each metric and the v1.78.0 verdicts above, and keeps the relative fields; #507's count pin follows to 28. Red on the previous budgets, and red with 12000 left in the backdrop file. The README records the series and the rule ("CI contracts") and, in "Changing budgets", the recalibration method; it also drops the stale #585 text: the 450/150/120 gesture allowances returned to 150/60/75 with6f226a5b(v1.77.0-beta.2), and the backdrop twin carries the iso count allowance too. Issue: #770 User-Visible: no Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018qZfe7YS4rqEMKoVeS3GKd
This commit is contained in:
@@ -419,6 +419,134 @@ for (const [family, spec] of Object.entries(SWITCH_CYCLE_FAMILIES)) {
|
||||
});
|
||||
}
|
||||
|
||||
// #770: обоснование потолков `longTasks.maxCountP95` / `maxTotalP95Ms` и ряд —
|
||||
// demo/performance/README.md, «CI contracts». Суммы Long Task — по всем окнам
|
||||
// образца, и окно `switchCycle` после #735 тёплое, поэтому прежние 30 / 12000
|
||||
// стояли в 1,25–3,2 раза над уровнем. Ряд — все прогоны Full Performance
|
||||
// 01–05.10 (раннер кандидата содержит #735 во всех), обе стороны, 7 образцов,
|
||||
// один тип раннера (linux x64, Chromium 151.0.7922.34, Node 22.23.3), в порядке
|
||||
// [база, кандидат] прогонов 36883292495, 36904753178, 36904816677, 36905707715,
|
||||
// 36905760140, 37055609391, 37062790080, 37065283633, 37134606046, 37136641943,
|
||||
// 37149329461 (только кандидат), 37176616088, 37203867985, 37302579095. Точка —
|
||||
// то, что судит потолок: p95 ближайшего ранга по 7 образцам, то есть максимум.
|
||||
// База 37149329461 — v1.78.0 (7d4d75bd, 28.09, до #735 и тёплых работ
|
||||
// #694/#725/#739): на том же раннере она в 1,2–2 раза медленнее кандидата, это
|
||||
// код, а не шум, и в M она не входит — это известный более медленный вариант.
|
||||
// Смоковая точка (3 образца) — 36910217188, interaction, adc2d7c5.
|
||||
const LONG_TASK_FAMILIES = {
|
||||
flat: {
|
||||
count: 18, total: 4350,
|
||||
smoke: 'budgets-interaction-smoke.json',
|
||||
files: ['budgets.json', 'budgets-large-house-plan-snap.json', 'budgets-large-house-interaction.json',
|
||||
'budgets-interaction-smoke.json'],
|
||||
full: { maxCountRegressionRatio: 0.35, countNoiseAllowance: 3, maxTotalRegressionRatio: 0.3, noiseAllowanceMs: 150 },
|
||||
series: {
|
||||
'large-house-v1': {
|
||||
count: [12, 11, 11, 10, 11, 12, 9, 7, 11, 12, 10, 9, 9, 9, 10, 11, 9, 9, 8, 8, 10, 10, 10, 13, 9, 10, 10],
|
||||
total: [3301, 3101, 3068, 2825, 3140, 2994, 2269, 2108, 2948, 2927, 2923, 2811, 2803, 2781, 3047, 3140,
|
||||
2823, 2748, 2183, 2128, 2836, 2852, 2826, 3133, 2759, 2823, 2818],
|
||||
},
|
||||
'large-house-plan-snap-v1': {
|
||||
count: [8, 7, 14, 14, 14, 14, 14, 13, 14, 14, 14, 14, 12, 12, 8, 9, 14, 14, 14, 12, 6, 14, 13, 15, 12, 13, 13],
|
||||
total: [1752, 1647, 3550, 3323, 3498, 3610, 3546, 3313, 3418, 3433, 3759, 3433, 3192, 3179, 2327, 2359,
|
||||
3387, 3411, 3416, 3296, 1798, 3407, 3400, 3509, 3157, 3266, 3279],
|
||||
},
|
||||
'large-house-interaction-v1': {
|
||||
count: [11, 12, 11, 10, 12, 12, 13, 12, 12, 13, 10, 10, 13, 11, 12, 13, 11, 11, 12, 12, 12, 5, 5, 13, 12, 11, 11],
|
||||
total: [3104, 3240, 3116, 2851, 3142, 3105, 3159, 3038, 3112, 3208, 2450, 2270, 3056, 2919, 3242, 3289,
|
||||
2895, 2864, 3203, 3046, 2957, 1480, 1558, 3111, 3057, 2671, 2676],
|
||||
},
|
||||
},
|
||||
smokePoints: [{ count: 12, total: 3047 }],
|
||||
slower: {
|
||||
'large-house-v1': { count: 17, total: 3959 },
|
||||
'large-house-plan-snap-v1': { count: 12, total: 2562 },
|
||||
'large-house-interaction-v1': { count: 16, total: 3660 },
|
||||
},
|
||||
},
|
||||
isometric: {
|
||||
count: 28, total: 6850,
|
||||
smoke: 'budgets-isometric-smoke.json',
|
||||
files: ['budgets-large-house-isometric.json', 'budgets-isometric-stage3-dense.json',
|
||||
'budgets-large-house-isometric-backdrop.json', 'budgets-isometric-smoke.json'],
|
||||
full: { maxCountRegressionRatio: 0.2, countNoiseAllowance: 5, maxTotalRegressionRatio: 0.2, noiseAllowanceMs: 150 },
|
||||
series: {
|
||||
'large-house-isometric-v1': {
|
||||
count: [15, 16, 21, 21, 20, 21, 20, 20, 21, 22, 19, 20, 20, 21, 21, 21, 14, 14, 12, 15, 19, 21, 20, 12, 12, 20, 21],
|
||||
total: [3713, 3543, 4944, 5088, 5104, 5022, 4135, 3956, 5101, 5078, 4618, 4709, 4816, 4754, 4831, 4889,
|
||||
3525, 3404, 2651, 2854, 4761, 4812, 5073, 2571, 2569, 4926, 4978],
|
||||
},
|
||||
'isometric-stage3-dense-v1': {
|
||||
count: [23, 24, 23, 23, 23, 24, 23, 23, 23, 23, 24, 23, 23, 23, 24, 23, 23, 23, 23, 23, 23, 23, 20, 23, 22, 21, 21],
|
||||
total: [5670, 5769, 5480, 5458, 5618, 5935, 5935, 5492, 5645, 5527, 5520, 5643, 5375, 5407, 5894, 5555,
|
||||
5740, 5502, 5488, 5516, 5627, 4380, 4181, 5419, 5474, 4088, 4086],
|
||||
},
|
||||
'large-house-isometric-backdrop-v1': {
|
||||
count: [21, 21, 20, 21, 21, 22, 12, 12, 21, 21, 20, 22, 20, 21, 21, 22, 13, 12, 22, 21, 11, 21, 20, 22, 20, 12, 14],
|
||||
total: [4946, 4887, 4736, 4816, 5331, 5182, 2694, 2524, 5010, 5014, 5086, 4931, 4748, 4907, 5286, 4975,
|
||||
2913, 2698, 5158, 5160, 2634, 5036, 5026, 5034, 4908, 2591, 2669],
|
||||
},
|
||||
},
|
||||
smokePoints: [],
|
||||
slower: {
|
||||
'large-house-isometric-v1': { count: 24, total: 6783 },
|
||||
'isometric-stage3-dense-v1': { count: 27, total: 8754 },
|
||||
'large-house-isometric-backdrop-v1': { count: 22, total: 5159 },
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
for (const [family, spec] of Object.entries(LONG_TASK_FAMILIES)) {
|
||||
test(`#770 Long Task count/total (${family}): потолки над тёплым уровнем после #735 с запасом на шум раннера`, () => {
|
||||
// Одно число на семью, как у #747: смок = полный (#473 AC4), plan-snap и
|
||||
// interaction — потолки large-house-v1, двойники — исторические (#160, #743).
|
||||
for (const file of spec.files) {
|
||||
const { longTasks } = readBudget(file);
|
||||
assert.equal(longTasks.maxCountP95, spec.count, `${file}: общий потолок числа Long Task семьи ${family}`);
|
||||
assert.equal(longTasks.maxTotalP95Ms, spec.total, `${file}: общий потолок суммы Long Task семьи ${family}`);
|
||||
// Ужесточение, а не ослабление: прежние 30 / 12000 #770 только опускает.
|
||||
assert.ok(longTasks.maxCountP95 < 30 && longTasks.maxTotalP95Ms < 12000, `${file}: потолок не ниже прежнего`);
|
||||
}
|
||||
const smoke = readBudget(spec.smoke);
|
||||
const failures = (count, total) => evaluatePerformanceBudget({
|
||||
candidate: { ...absoluteSmokeReport(smoke), longTasks: { maxSingleMs: 1, countP95: count, totalP95Ms: total } },
|
||||
budgets: smoke, absoluteOnly: true,
|
||||
}).failures.map((check) => check.id);
|
||||
const counts = Object.values(spec.series).flatMap((series) => series.count);
|
||||
const totals = Object.values(spec.series).flatMap((series) => series.total);
|
||||
for (const series of Object.values(spec.series)) {
|
||||
assert.equal(series.count.length, 27, 'ряд — 14 прогонов × 2 стороны без базы v1.78.0');
|
||||
assert.equal(series.total.length, 27);
|
||||
series.count.forEach((count, index) => assert.deepEqual(failures(count, series.total[index]), [],
|
||||
`${count} / ${series.total[index]} из ряда краснеет на ${spec.count} / ${spec.total}`));
|
||||
}
|
||||
for (const { count, total } of spec.smokePoints) assert.deepEqual(failures(count, total), [], 'смоковая точка');
|
||||
// Правило #747 для обеих метрик: первый шаг (1 задача, 50 мс) не ниже
|
||||
// 1.15 × M и не выше 1.2 × M, M — максимум ряда семьи.
|
||||
const maxCount = Math.max(...counts);
|
||||
const maxTotal = Math.max(...totals);
|
||||
assert.equal(spec.count, Math.ceil(maxCount * 1.15), `число — первое целое над 1.15 × ${maxCount}`);
|
||||
assert.equal(spec.total, Math.ceil((maxTotal * 1.15) / 50) * 50, `сумма — первое кратное 50 над 1.15 × ${maxTotal}`);
|
||||
assert.ok(spec.count >= maxCount * 1.15 && spec.count <= maxCount * 1.2, 'число: запас 15–20 %');
|
||||
assert.ok(spec.total >= maxTotal * 1.15 && spec.total <= maxTotal * 1.2, 'сумма: запас 15–20 %');
|
||||
assert.deepEqual(failures(2 * maxCount, 1), ['longTask.countP95'], 'удвоение числа краснеет');
|
||||
assert.deepEqual(failures(1, 2 * maxTotal), ['longTask.totalP95Ms'], 'удвоение суммы краснеет');
|
||||
// Известный более медленный вариант — база v1.78.0 того же раннера: в M не
|
||||
// входит. Возврат плотного двойника к её сумме (8754 мс) потолок ловит;
|
||||
// остальные её точки ниже потолков — их ловит относительное сравнение.
|
||||
for (const [profile, { count, total }] of Object.entries(spec.slower)) {
|
||||
const expected = profile === 'isometric-stage3-dense-v1' ? ['longTask.totalP95Ms'] : [];
|
||||
assert.deepEqual(failures(count, total), expected, `${profile} v1.78.0: ${count} / ${total}`);
|
||||
}
|
||||
// Меньшее ловит относительное сравнение полного прогона: его коэффициенты и
|
||||
// допуски не рычаг.
|
||||
for (const file of spec.files.filter((name) => name !== spec.smoke)) {
|
||||
const { longTasks } = readBudget(file);
|
||||
for (const [key, value] of Object.entries(spec.full)) assert.equal(longTasks[key], value, `${file}: ${key} не рычаг`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
test('boundary collision search restores the ordinary isometric allowances (#585)', () => {
|
||||
const isometric = readBudget('budgets-large-house-isometric.json');
|
||||
const dense = readBudget('budgets-isometric-stage3-dense.json');
|
||||
@@ -452,7 +580,9 @@ test('isometric long-task count allowance covers the lazy iso-chunk split, nothi
|
||||
const isometric = readBudget('budgets-large-house-isometric.json');
|
||||
assert.equal(isometric.longTasks.countNoiseAllowance, 5, 'owner-accepted +2 tasks of the lazy iso-scene-render split plus jitter');
|
||||
assert.equal(isometric.longTasks.maxCountRegressionRatio, 0.2, 'the ratio is not the lever');
|
||||
assert.equal(isometric.longTasks.maxCountP95, 30);
|
||||
// #770: the family ceiling, above the 21 the allowance yields for that base,
|
||||
// so the relative half still decides this comparison.
|
||||
assert.equal(isometric.longTasks.maxCountP95, 28);
|
||||
assert.equal(isometric.longTasks.maxTotalRegressionRatio, 0.2, 'total work is still gated as before');
|
||||
assert.equal(isometric.longTasks.maxSingleRegressionRatio, 0.2);
|
||||
assert.equal(readBudget('budgets-isometric-stage3-dense.json').longTasks.countNoiseAllowance, 5, 'the dense twin shares the lazy iso chunk and the #160 common-ceiling contract');
|
||||
|
||||
Reference in New Issue
Block a user