Owner's decision after the incident: a detached plan is never deleted, at any
age. v1.46.4 gave it a month; this makes it permanent and, more importantly,
writes the reasoning where the next change will trip over it — docs/SCOPE.md now
carries the standing rule. The component may delete a file only when a user
action says so. 'Nothing points at this any more' is not such an action, because
the two errors are not symmetrical: wasted disk is visible, cheap and
reversible; a deleted file is none of those.
Went through every other automatic deletion with the same question. One more
was wrong: houseplan/files/cleanup rmtree'd whatever folder the card named. A
partial migration leaves urls pointing into it — files/migrate deliberately does
not rewrite the ones it could not confirm — so those were live links to files
being deleted; and a wrong or stale id from any client destroyed a live device's
manuals. The server now reads the stored config under its lock and removes only
what nothing references, keeping the rest and saying so.
Also: a plan of a DELETED space now waits thirty days rather than an hour.
Deleting a space is deliberate; an hour is a short window to notice a misclick.
The rest came out clean: layout/delete and marker/room/space removal are all
confirm-guarded user actions, upload temporaries are never user-visible, and
dropping legacy 'segments' is a documented migration.
Protecting every file of a live space also protected the ones a commit had just
superseded, and gave rejected uploads immortality. The distinction that matters
is narrower: a space with NO plan_url has had its image detached and may want it
back; a space that has one can only be holding its own rejects. Attachments:
staging folders keep the hour, marker folders get the month.
Also: the layout event test asserted the order of separately fired bus events,
which nothing promises — it came back [2,1,3] in CI.
Deployed v1.46.3 to my own instance, restarted, and the startup sweep deleted
both floor plans: config/houseplan/plans/ went from f1.svg + f2.png to empty.
The backup is a SecureTar, so they are gone.
The rule was wrong, not the code. v1.46.0 introduced collection that treats
'nothing references this right now' as abandoned and gives it an hour. But
detaching a plan — switching a space to 'draw' — is a normal, reversible action,
and the editor's own comment says the file stays on disk. Those two plans had
been detached for weeks; every pass since v1.46.0 was entitled to remove them,
and the one that finally ran did.
New rule, one for every path:
* superseded by a commit (was in the old revision, is not in the new) — goes
immediately; that is the one thing a commit knows for certain;
* belongs to a space or marker that still exists — never collected, at any
age, because unreferenced is not abandoned;
* a per-dialog staging folder (up_*) — one hour, unchanged: by construction it
only ever holds an upload from a dialog that was never saved;
* anything else — thirty days.
The flag I added an hour ago is gone with it: two rules for the same
question is how this happened. Tests updated to the new grace, plus two that pin
the distinction directly.
I am sorry about the files.
HP-1460-01: v1.46.0 stopped overwriting attachments, but picking a free name
and taking it were two steps. Two uploads racing between them agreed on the
same name, both answered 200, and one set of bytes replaced the other;
files/migrate had the same check-then-copy gap. reserve_filename now claims the
name with O_CREAT|O_EXCL as it picks it, and both paths use it. It also splits
the extension off the RAW name and budgets the stem against MAX_FILENAME
including the collision tag — a maximal name lost its '.pdf' and then grew past
the limit, so the view sanitised the request back to a different name and the
attachment 404'd for good.
HP-1460-02: cleanup lived in an 'except Exception', which CancelledError walks
past, only one tmp_path was tracked, promotion had no finally, and the
collector only walks marker folders — an aborted transfer stranded a .upload-*
that nothing would ever remove. An outer finally owns every temporary, a second
'file' part is refused, promotion failure cleans up, and sweep_upload_temps
runs at setup, daily, and inside the commit-scoped collector. Chunks are
batched to 1 MB per disk task instead of one per 64 KB.
HP-1460-03: the layout event reached the static card and not the full one, so
two full cards diverged until a reload. The full card subscribes now and
re-reads ONLY the layout, keyed on its revision. Two hazards handled: it
records revisions it produced itself, and the reaction is deferred ~200 ms
because the event can beat the reply to our own write over the same socket;
positions dragged but not yet sent are flushed and merged on top, so a fix for
a stale UI cannot become a lost drag.
Tests: smoke_layout_sync (fails on a v1.46.0 build), four pure tests for atomic
reservation incl. 20-thread concurrency and the length boundary, a backend test
walking every failing exit path of an upload, and — as the report asked — an
HA-harness test that a repair issue disappears with its space.
Docs: CHANGELOG.md + CHANGELOG.ru.md + ARCHITECTURE.md + TESTING.md + STATUS.md.
unique_filename produced 'manual (2).pdf'; HouseplanContentView sanitises the
name in the REQUEST too, turning ' (2)' into '_2_', so the file was written and
then 404'd. The same pattern was already in files/migrate, so a rebind that hit
a name collision has been producing dead links. Both use the shared helper now,
with '-2', which round-trips sanitize_filename — asserted.
HP-1454-01 (high, release blocker): an uploaded SVG plan opened directly is a
top-level document of Home Assistant's own origin, so a <script> inside it
reaches the session's localStorage and API. Uploading needs write access, which
by default every authenticated user has. SVG responses now carry a sandbox CSP;
only SVG, because a CSP on a PDF can break the browser's viewer and a raster
image has nothing to disable. Verified in Chromium both ways: the script runs
without the header and does not with it.
HP-1454-02: attachment uploads wrote straight to <marker>/<filename>, outside
the config transaction — a cancelled dialog or a rejected save left the stored
url serving new bytes, and every new icon shared one 'new' folder, so two of
them attaching manual.pdf pointed at one file. Uploads take a free name, a new
icon gets a per-dialog staging folder promoted on an accepted save, and
config/set collects superseded and aged-orphan attachments like it does plans.
HP-1454-03: the debounce spaced out the starts of a write, not the writes. A
save slower than 500 ms let the next edit go out with the same expected_rev;
the server accepted the first, rejected the second, and the conflict handler
reloaded over the local copy. Writes are chained now — one in flight, each with
the revision the previous returned.
HP-1454-04: _openPairsCache keyed on room ids and links only, so an aspect
change or a dragged vertex left open boundaries and their glow cuts at old
coordinates. It keys on the rendered model object now — the same invalidation
the model cache already has, not a second strategy. The fingerprint also gained
an O(1) geometry roll-up per room.
HP-1454-05: outer collections were capped, inner ones were not. Limits for
poly points, open_to, controls, pdfs, text and url lengths, plus a total
serialized size cap; legacy is dropped server-side.
HP-1454-06: upload streams to a temp file and downloads use FileResponse, so a
50 MB manual no longer costs ~100 MB of RSS per transfer.
HP-1454-07: spaceModels() dropped room.settings, so the static card ignored the
per-room fill override. HP-1454-08: layout had no revision on point-wise writes
and no event, leaving static cards stale forever; it now keeps a revision,
returns it and fires houseplan_layout_updated. HP-1454-09: repair cleanup only
walked existing spaces, so a deleted space kept its warning. HP-1454-10:
serialize-javascript pinned past two advisories.
Tests: smoke_svg_sandbox (proves both directions), smoke_config_writer and
smoke_render_parity (both verified failing against a v1.45.4 build), six pure
tests for attachment collection and inner limits, four HA-harness tests for the
CSP, non-overwriting uploads, the size cap and layout revisions.
Docs: CHANGELOG.md + CHANGELOG.ru.md + ARCHITECTURE.md + TESTING.md + STATUS.md.
R4-1: collecting superseded plan files runs after the configuration is already
durable, but an error listing the directory propagated out of config/set. The
client saw a failure for a revision the server had committed, and its retry
came back as a conflict. collect_plans now reports 0 instead of raising, and
config/set logs and proceeds — the event fires, the revision is returned.
R4-2: the pending set was cleared when a batch went out, not when it came back,
so every render during an in-flight content/sign queued another request: six
calls where one was needed, and unbounded on a socket that is slow rather than
busy. Queued and in-flight are separate states now; a failure backs off (2 s
doubling to 60 s) instead of retrying on the next frame; an in-flight entry
expires after 15 s so a promise that never settles cannot wedge retries; a late
answer after dispose() no longer renders.
Tests: test/signing.test.mjs — eight cases with hand-settled promises, verified
against a v1.45.1 checkout where four of them fail (2 sign calls instead of 1,
no backoff, a late answer rendering after teardown). Backend: a broken
collector still yields a successful save whose revision the next CAS accepts.
Pure collector: a disappearing directory returns 0.
Docs: CHANGELOG.md + CHANGELOG.ru.md + ARCHITECTURE.md + TESTING.md + STATUS.md.
R3-1 (high): v1.45.0 made the upload safe but left deletion to the client —
after a successful save the card asked the backend to remove everything but the
file it had just committed. Two open editors cannot be ordered: a delayed
request from one deleted the plan the other had just saved, leaving the
accepted configuration pointing at nothing, the exact damage copy-on-write was
introduced to prevent.
houseplan/plan/cleanup is removed. config/set collects inside its own write
lock from the two configurations that bracket the commit (plans.collect_plans):
a file the old revision referenced and the new one does not is superseded and
goes; any other unreferenced upload waits out PLAN_ORPHAN_TTL_S, because a
fresh one may belong to a transaction that has not committed yet. The collector
lives in a pure module so it can be reasoned about and unit-tested without the
HA harness.
R3-2: houseplan-space-card signed its plan url and threw the result away —
getCardSize() mutated a throwaway model while render() rebuilt its own from the
config, so the <image> requested the protected path and got 401 on every
render. Both cards now share ContentSigner (src/signing.ts), which also gives
the static card batching, expiry handling and periodic re-signing. is
released in finally: one failed request no longer wedges a url for the life of
the page.
Tests: five backend interleaving cases from the report, six unit tests for the
pure collector, smoke_space_card_bg (verified to fail against a v1.45.0 build:
the raw url reaches the DOM and no retry happens). 57 smokes, 124 unit, 22
backend-pure.
Docs: CHANGELOG.md + CHANGELOG.ru.md + ARCHITECTURE.md + TESTING.md + STATUS.md.
Matysh