v1.46.1: re-check of v1.46.0 — HP-1460-01, -02, -03

HP-1460-01: v1.46.0 stopped overwriting attachments, but picking a free name
and taking it were two steps. Two uploads racing between them agreed on the
same name, both answered 200, and one set of bytes replaced the other;
files/migrate had the same check-then-copy gap. reserve_filename now claims the
name with O_CREAT|O_EXCL as it picks it, and both paths use it. It also splits
the extension off the RAW name and budgets the stem against MAX_FILENAME
including the collision tag — a maximal name lost its '.pdf' and then grew past
the limit, so the view sanitised the request back to a different name and the
attachment 404'd for good.

HP-1460-02: cleanup lived in an 'except Exception', which CancelledError walks
past, only one tmp_path was tracked, promotion had no finally, and the
collector only walks marker folders — an aborted transfer stranded a .upload-*
that nothing would ever remove. An outer finally owns every temporary, a second
'file' part is refused, promotion failure cleans up, and sweep_upload_temps
runs at setup, daily, and inside the commit-scoped collector. Chunks are
batched to 1 MB per disk task instead of one per 64 KB.

HP-1460-03: the layout event reached the static card and not the full one, so
two full cards diverged until a reload. The full card subscribes now and
re-reads ONLY the layout, keyed on its revision. Two hazards handled: it
records revisions it produced itself, and the reaction is deferred ~200 ms
because the event can beat the reply to our own write over the same socket;
positions dragged but not yet sent are flushed and merged on top, so a fix for
a stale UI cannot become a lost drag.

Tests: smoke_layout_sync (fails on a v1.46.0 build), four pure tests for atomic
reservation incl. 20-thread concurrency and the length boundary, a backend test
walking every failing exit path of an upload, and — as the report asked — an
HA-harness test that a repair issue disappears with its space.
Docs: CHANGELOG.md + CHANGELOG.ru.md + ARCHITECTURE.md + TESTING.md + STATUS.md.

custom_components/houseplan/plans.py

@@ -9,40 +9,66 @@ and tested.
 from __future__ import annotations
 
 import logging
+import os
 import time
 from pathlib import Path
 from typing import Any
 
 from .const import PLAN_ORPHAN_TTL_S
-from .validation import PLAN_EXTENSIONS, sanitize_filename
+from .validation import MAX_FILENAME, PLAN_EXTENSIONS, sanitize_filename
 
 _LOGGER = logging.getLogger(__name__)
 
+# Streaming uploads land here first. The prefix is a dot so the name can never
+# collide with an attachment (sanitize_filename strips leading dots) and is easy
+# to sweep.
+TMP_PREFIX = ".upload-"
 
-def unique_filename(directory: Path, name: str) -> str:
-    """A name inside `directory` that is not taken, deriving from `name`.
 
-    Uploads never overwrite. The bytes already under a name may be referenced by
-    the stored configuration, and an upload is not part of that transaction: a
-    cancelled dialog or a rejected save would otherwise leave a live url serving
-    someone else's file (HP-1454-02).
+def reserve_filename(directory: Path, name: str) -> str:
+    """Atomically claim a free name inside `directory` and return it.
+
+    Creates the file, empty, with `O_CREAT | O_EXCL`, so the name is *taken* the
+    moment it is chosen. The previous version asked `exists()` and returned a
+    string; two uploads racing between the check and the write agreed on the
+    same name and one silently overwrote the other, both reporting success
+    (HP-1460-01). The caller writes the real bytes over the placeholder — it
+    owns the name by then — and must remove it if it never gets that far.
+
+    The result is guaranteed to satisfy `sanitize_filename(result) == result`:
+    the content view sanitises the name in the request too, so a name it would
+    shorten or rewrite is a file that is written and then never served.
     """
-    safe = sanitize_filename(name)
-    if not (directory / safe).exists():
-        return safe
-    stem, dot, suffix = safe.rpartition(".")
+    directory.mkdir(parents=True, exist_ok=True)
+    # Split the extension off the RAW name: sanitize_filename() truncates to
+    # MAX_FILENAME, so sanitising first would cut ".pdf" off a long name and the
+    # attachment would be stored — and served — without its type.
+    base = name.rsplit("/", 1)[-1].rsplit("\\", 1)[-1]
+    stem, dot, suffix = base.rpartition(".")
     if not dot:
-        stem, suffix = safe, ""
-    i = 2
+        stem, suffix = base, ""
+    stem = sanitize_filename(stem)
+    ext = f".{sanitize_filename(suffix)[:16]}" if suffix else ""
+    i = 1
     while True:
-        # "-2", not " (2)": the content view sanitizes the name in the REQUEST
-        # too, and a space or a bracket there turns into "_", so a file called
-        # "manual (2).pdf" was written and then never served. Only characters
-        # that survive sanitize_filename may be used to build a name.
-        candidate = f"{stem}-{i}{'.' + suffix if suffix else ''}"
-        if not (directory / candidate).exists():
-            return candidate
-        i += 1
+        tag = "" if i == 1 else f"-{i}"
+        # budget the stem so the WHOLE name fits, including the collision tag —
+        # appending "-2" to an already maximal name produced a url the view
+        # truncated back to something else, i.e. a permanent 404
+        room = MAX_FILENAME - len(ext) - len(tag)
+        candidate = (stem[:room] if room > 0 else "f") + tag + ext
+        candidate = sanitize_filename(candidate)
+        if candidate.startswith("."):  # a name that is only an extension
+            candidate = "file" + candidate
+        try:
+            fd = os.open(directory / candidate, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
+        except FileExistsError:
+            i += 1
+            if i > 10000:  # pathological directory; do not spin forever
+                raise
+            continue
+        os.close(fd)
+        return candidate
 
 
 def attachment_refs(cfg: dict[str, Any] | None) -> set[str]:
@@ -59,6 +85,34 @@ def attachment_refs(cfg: dict[str, Any] | None) -> set[str]:
     return out
 
 
+def sweep_upload_temps(files_dir: Path, now: float | None = None) -> int:
+    """Remove abandoned streaming temporaries (HP-1460-02).
+
+    The request itself deletes its own, but a hard kill — a restart mid-upload,
+    an OOM — leaves one behind, and the attachment collector only walks marker
+    folders, so it would never be seen. Age-gated for the same reason as the
+    rest: a fresh one belongs to a request still in flight.
+    """
+    cutoff = (time.time() if now is None else now) - PLAN_ORPHAN_TTL_S
+    removed = 0
+    try:
+        items = [p for p in files_dir.iterdir() if p.is_file()] if files_dir.is_dir() else []
+    except OSError as err:
+        _LOGGER.warning("House Plan: could not list %s: %s", files_dir, err)
+        return 0
+    for item in items:
+        if not item.name.startswith(TMP_PREFIX):
+            continue
+        try:
+            if item.stat().st_mtime >= cutoff:
+                continue
+            item.unlink()
+            removed += 1
+        except OSError:
+            continue
+    return removed
+
+
 def collect_attachments(
     files_dir: Path,
     old_cfg: dict[str, Any] | None,
@@ -82,6 +136,7 @@ def collect_attachments(
     except OSError as err:
         _LOGGER.warning("House Plan: could not list %s: %s", files_dir, err)
         return 0
+    removed += sweep_upload_temps(files_dir, now)
     for folder in folders:
         try:
             items = sorted(p for p in folder.iterdir() if p.is_file())