kayg commented

Owner

Owner report (2026-10-06): "updating tags takes too long (sequential? synchronous) — we are not writing performant code by default!"

Orchestrator reading of crates/calternal-tags/src/rewrite.rs: prepare runs index::reconcile_user (a full Home reconcile) before every rename/merge, then run_until processes targets one by one: take the namespace mutation lock, write the file, save the journal, refresh the Index, checkpoint (disk write) — per file.

Do (keep crash safety, resumability and exact Undo from #1110):

Measure first on the perf-test VM (root@10.69.69.63) with a Home of ~1,000 tagged items: wall time for rename and merge, p95 of other requests during it. Report before/after.
No full reconcile per operation: plan from the Index; verify each target's revision at write time instead (the checked writer already refuses stale bytes).
Batch: write targets in batches (e.g. 64) with bounded parallel file IO (the files are independent), one journal append and one checkpoint per batch, Index updates in one transaction per batch. Release the mutation lock between batches so ordinary edits are not starved.
UI: optimistic — the sidebar and views show the renamed tag immediately with a progress indicator for the background rewrite; Undo stays available.
Target: ≥10× faster on 1,000 items, no p95 regression for other requests.
Gates per crate + bun run check + bun run test quoted; numbers in the report and docs/perf/. A data-safety review follows.

## Owner report (2026-10-06): "updating tags takes too long (sequential? synchronous) — we are not writing performant code by default!" Orchestrator reading of crates/calternal-tags/src/rewrite.rs: `prepare` runs `index::reconcile_user` (a full Home reconcile) before every rename/merge, then `run_until` processes targets one by one: take the namespace mutation lock, write the file, save the journal, refresh the Index, checkpoint (disk write) — per file. Do (keep crash safety, resumability and exact Undo from #1110): 1. Measure first on the perf-test VM (root@10.69.69.63) with a Home of ~1,000 tagged items: wall time for rename and merge, p95 of other requests during it. Report before/after. 2. No full reconcile per operation: plan from the Index; verify each target's revision at write time instead (the checked writer already refuses stale bytes). 3. Batch: write targets in batches (e.g. 64) with bounded parallel file IO (the files are independent), one journal append and one checkpoint per batch, Index updates in one transaction per batch. Release the mutation lock between batches so ordinary edits are not starved. 4. UI: optimistic — the sidebar and views show the renamed tag immediately with a progress indicator for the background rewrite; Undo stays available. 5. Target: ≥10× faster on 1,000 items, no p95 regression for other requests. Gates per crate + `bun run check` + `bun run test` quoted; numbers in the report and docs/perf/. A data-safety review follows.

tagperf-1186-tags-mac.zip

3.4 MiB

tagperf-1186-tag-detail-mac.zip

3.6 MiB

tagperf-1186-undo-toast-mac.zip

3.4 MiB

tagperf-1186-sidebar-mac.zip

3 MiB

tagperf-1186-progress-mac.zip

3.4 MiB

sidebar-mouse-rename-phone-dark-mac.png

130 KiB

sidebar-mouse-rename-phone-light-mac.png

164 KiB

sidebar-mouse-rename-tablet-dark-mac.png

608 KiB

sidebar-mouse-rename-tablet-light-mac.png

610 KiB

sidebar-shortcut-rename-desktop-dark-mac.png

807 KiB

sidebar-shortcut-rename-desktop-light-mac.png

826 KiB

sidebar-shortcut-rename-phone-dark-mac.png

130 KiB

sidebar-shortcut-rename-tablet-dark-mac.png

608 KiB

sidebar-shortcut-rename-tablet-light-mac.png

610 KiB

sidebar-progress-desktop-dark-mac.png

803 KiB

sidebar-progress-desktop-light-mac.png

820 KiB

sidebar-progress-phone-dark-mac.png

129 KiB

sidebar-progress-phone-light-mac.png

165 KiB

sidebar-progress-tablet-dark-mac.png

603 KiB

sidebar-progress-tablet-light-mac.png

604 KiB

sidebar-done-phone-dark-mac.png

129 KiB

sidebar-done-phone-light-mac.png

164 KiB

sidebar-done-tablet-dark-mac.png

608 KiB

gates-verbatim.log.gz

418 KiB

sidebar-done-desktop-dark-mac.png

809 KiB

sidebar-mouse-rename-desktop-light-mac.png

826 KiB

sidebar-done-tablet-light-mac.png

609 KiB

sidebar-mouse-rename-desktop-dark-mac.png

807 KiB

sidebar-done-desktop-light-mac.png

826 KiB

sidebar-shortcut-rename-phone-light-mac.png

165 KiB

kayg commented

2026-10-06 09:31:59 +00:00

Author

Owner

Starting #1186 on job/tagperf-1186, based at f5fbced3c733370b5b37544d7816732628be2fd0 (worktree reports three commits behind origin/dev). I have read CLAUDE.md, CONTEXT.md, and the relevant Tag/index and instant interaction sections of docs/DESIGN.md. I am inspecting the existing benchmark and will record the ~1,000-item rename/merge baseline on the locked perf VM before changing the rewrite path.

Starting #1186 on `job/tagperf-1186`, based at `f5fbced3c733370b5b37544d7816732628be2fd0` (worktree reports three commits behind `origin/dev`). I have read `CLAUDE.md`, `CONTEXT.md`, and the relevant Tag/index and instant interaction sections of `docs/DESIGN.md`. I am inspecting the existing benchmark and will record the ~1,000-item rename/merge baseline on the locked perf VM before changing the rewrite path.

kayg referenced this issue from a commit

2026-10-06 10:14:55 +00:00

bench(tags): profile large rewrites under read load (#1186)

kayg commented

2026-10-06 10:31:41 +00:00

Author

Owner

Baseline setup is ready: the 1,000-item runner and concurrent request sampler are committed in 65a5a1f77 and 77629d50b, and a release server from base f5fbced3c is staged on the perf VM. Source inspection confirms rewrite::prepare calls index::reconcile_user before querying candidate Tag paths, then reads all indexed MIME values; run_until writes and checkpoints one source at a time, and refresh_index performs separate item lookups and writes. The required /root/perf.lock is held by another run (observed lock owners are flock and bash); this job is waiting for the exclusive measurement window. No baseline timings are available yet, and no implementation files have been changed.

Baseline setup is ready: the 1,000-item runner and concurrent request sampler are committed in `65a5a1f77` and `77629d50b`, and a release server from base `f5fbced3c` is staged on the perf VM. Source inspection confirms `rewrite::prepare` calls `index::reconcile_user` before querying candidate Tag paths, then reads all indexed MIME values; `run_until` writes and checkpoints one source at a time, and `refresh_index` performs separate item lookups and writes. The required `/root/perf.lock` is held by another run (observed lock owners are `flock` and `bash`); this job is waiting for the exclusive measurement window. No baseline timings are available yet, and no implementation files have been changed.

kayg referenced this issue from a commit

2026-10-06 10:35:04 +00:00

bench(tags): print rewrite sample progress (#1186)

kayg commented

2026-10-06 11:23:55 +00:00

Author

Owner

The first baseline attempt did not reach a rewrite sample. Seeding 1,000 Notes with 16 concurrent POSTs hit HTTP 500 at item 985 (Notes operation failed). The server log had repeated SQLite busy errors (sqlite_code=5) and a search-indexer TooManyOpenFiles error. The partial Home and profile were discarded. I am changing the fixture seeder to create Notes serially and report progress; this keeps setup outside the measured interval and avoids turning profile setup into a concurrency storm.

The first baseline attempt did not reach a rewrite sample. Seeding 1,000 Notes with 16 concurrent POSTs hit HTTP 500 at item 985 (`Notes operation failed`). The server log had repeated SQLite busy errors (`sqlite_code=5`) and a search-indexer `TooManyOpenFiles` error. The partial Home and profile were discarded. I am changing the fixture seeder to create Notes serially and report progress; this keeps setup outside the measured interval and avoids turning profile setup into a concurrency storm.

kayg commented

2026-10-06 13:28:02 +00:00

Author

Owner

The serial API seed reached 1,000 Notes, but the server then logged repeated Notes Index SQLite busy errors, and the final Tag-count read stalled beyond its 120-second timeout; /readyz also stopped responding (server RSS was about 811 MB). I discarded that Home and added crates/calternal-tags/examples/tag_rewrite_1186_seed.rs: it creates actual Markdown Notes through calternal-fs, runs one normal Tags reconciliation, and checks both Index counts. The profile now verifies that fixture before timing rewrites. cargo fmt --check, focused Clippy and all 44 calternal-tags tests pass; the baseline timing is still pending.

The serial API seed reached 1,000 Notes, but the server then logged repeated Notes Index SQLite busy errors, and the final Tag-count read stalled beyond its 120-second timeout; `/readyz` also stopped responding (server RSS was about 811 MB). I discarded that Home and added `crates/calternal-tags/examples/tag_rewrite_1186_seed.rs`: it creates actual Markdown Notes through `calternal-fs`, runs one normal Tags reconciliation, and checks both Index counts. The profile now verifies that fixture before timing rewrites. `cargo fmt --check`, focused Clippy and all 44 `calternal-tags` tests pass; the baseline timing is still pending.

kayg commented

2026-10-06 13:57:10 +00:00

Author

Owner

Performance baseline finding: the 1,000-item fixture is now seeded directly through calternal-fs and its Tag counts verify. Under the perf lock, the first baseline rename had not reached a terminal receipt after more than five minutes of observation, with no completed sample yet. The current path therefore includes substantial setup/serialized work; I am keeping the run active to capture end-to-end wall time and overlapping Tag-read p95 before implementation changes.

kayg commented

2026-10-06 14:14:23 +00:00

Author

Owner

Baseline finding: with the direct 1,000-Note fixture, the pre-change server accepted the first rename but did not reach a terminal receipt within the configured 1,800-second timeout. The locked VM reported load average 2.46875, 2.68115234375, 1.79248046875 at start and 5.14990234375, 7.09521484375, 6.75341796875 after stop. The initial profiler discarded concurrent-read samples on timeout, so it did not produce a p95; I am fixing that reporting path and will record the incomplete wall time as a lower bound rather than claim a completed sample.

kayg referenced this issue from a commit

2026-10-06 14:38:15 +00:00

perf(tags): batch durable rewrites and index updates (#1186)

kayg commented

2026-10-06 14:47:49 +00:00

Author

Owner

Finding for #1186: after the first batch implementation, receipt progress counted Sidecar files as source records. A folder Sidecar can represent many visible Files, so 1,000 tagged Files would show fewer than 1,000 completed items. Progress now counts the visible items in each source and validates journal cursors before indexing offsets. cargo clippy -p calternal-tags --all-targets -- -D warnings passed; cargo test -p calternal-tags passed (45 tests).

Finding for #1186: after the first batch implementation, receipt progress counted Sidecar files as source records. A folder Sidecar can represent many visible Files, so 1,000 tagged Files would show fewer than 1,000 completed items. Progress now counts the visible items in each source and validates journal cursors before indexing offsets. `cargo clippy -p calternal-tags --all-targets -- -D warnings` passed; `cargo test -p calternal-tags` passed (45 tests).

kayg commented

2026-10-06 16:36:35 +00:00

Author

Owner

Baseline evidence for #1186 (pre-change release, perf VM, each run under flock /root/perf.lock, 1,000 source Notes + one destination, 16 concurrent Tag readers): rename remained nonterminal after 61.12 s; reader p95 416.96 ms, max 902.24 ms, 5,215 reads, 0 errors; server CPU 33.83 s, average RSS 264,019,955 bytes, peak RSS 268,505,088 bytes. Merge remained nonterminal after 61.04 s; reader p95 430.16 ms, max 892.24 ms, 4,763 reads, 0 errors; server CPU 37.40 s, average RSS 257,196,516 bytes, peak RSS 264,237,056 bytes. The older rename run also remained nonterminal after 1,800 s. Lock-held load averages (1/5/15 min) were 2.39→3.63 for rename and 2.43→3.78 for merge. The two 60-second operations are censored lower bounds; their disposable Homes were not reused.

Baseline evidence for #1186 (pre-change release, perf VM, each run under `flock /root/perf.lock`, 1,000 source Notes + one destination, 16 concurrent Tag readers): rename remained nonterminal after 61.12 s; reader p95 416.96 ms, max 902.24 ms, 5,215 reads, 0 errors; server CPU 33.83 s, average RSS 264,019,955 bytes, peak RSS 268,505,088 bytes. Merge remained nonterminal after 61.04 s; reader p95 430.16 ms, max 892.24 ms, 4,763 reads, 0 errors; server CPU 37.40 s, average RSS 257,196,516 bytes, peak RSS 264,237,056 bytes. The older rename run also remained nonterminal after 1,800 s. Lock-held load averages (1/5/15 min) were 2.39→3.63 for rename and 2.43→3.78 for merge. The two 60-second operations are censored lower bounds; their disposable Homes were not reused.

kayg commented

2026-10-06 16:59:52 +00:00

Author

Owner

Finding for #1186: the first optimized 1,000-item rename reached 832/1,000, then the receipt failed with tag filesystem operation failed and can_undo=true. The admin job detail recorded last_error=tag filesystem operation failed. Sanitized server diagnostics at the same time show Too many open files (os error 24) from accept, search-index and file-index work, followed by Index pool closure during shutdown. The sample had 16 continuous /api/v1/tags readers. I stopped the run, preserved the separate untouched fixture, and am isolating whether the repeated-read profile or rewrite path exhausts descriptors before continuing the performance comparison.

Finding for #1186: the first optimized 1,000-item rename reached 832/1,000, then the receipt failed with `tag filesystem operation failed` and `can_undo=true`. The admin job detail recorded `last_error=tag filesystem operation failed`. Sanitized server diagnostics at the same time show `Too many open files (os error 24)` from accept, search-index and file-index work, followed by Index pool closure during shutdown. The sample had 16 continuous `/api/v1/tags` readers. I stopped the run, preserved the separate untouched fixture, and am isolating whether the repeated-read profile or rewrite path exhausts descriptors before continuing the performance comparison.

kayg commented

2026-10-06 17:31:03 +00:00

Author

Owner

Performance and data-safety finding: the first 1,000-item run reached item 832 with 484 of 499 server descriptors held by users/<user>/Notes directory proofs. The queued filesystem changes kept each optional proof fd alive until consumers drained them, then the rewrite hit EMFILE. The batch writer now uses the same checked BLAKE3 replacement and path event without retaining that optional proof; the Files projection keeps its existing conservative check. A focused filesystem test covers this event path.

The interrupted run's Undo reported 103 conflicts. A byte audit showed that a background calternal-id backfill changed Notes while the old fixture was being rewritten, so the journal no longer represented a stable byte state. The performance fixture now uses stable IDs and upgrades only the exact known legacy fixture bytes. I will rerun both versions from that clean fixture before reporting comparable timings.

cargo clippy -p calternal-fs --all-targets -- -D warnings and cargo clippy -p calternal-tags --all-targets -- -D warnings passed. All 45 calternal-tags tests passed. The full filesystem test run had one unrelated history_rejects_symlinked_segments_and_keeps_lock_after_fold EAGAIN from nonblocking flock; that test passes when run alone. Its assertion was not changed.

Performance and data-safety finding: the first 1,000-item run reached item 832 with 484 of 499 server descriptors held by `users/<user>/Notes` directory proofs. The queued filesystem changes kept each optional proof fd alive until consumers drained them, then the rewrite hit `EMFILE`. The batch writer now uses the same checked BLAKE3 replacement and path event without retaining that optional proof; the Files projection keeps its existing conservative check. A focused filesystem test covers this event path. The interrupted run's Undo reported 103 conflicts. A byte audit showed that a background `calternal-id` backfill changed Notes while the old fixture was being rewritten, so the journal no longer represented a stable byte state. The performance fixture now uses stable IDs and upgrades only the exact known legacy fixture bytes. I will rerun both versions from that clean fixture before reporting comparable timings. `cargo clippy -p calternal-fs --all-targets -- -D warnings` and `cargo clippy -p calternal-tags --all-targets -- -D warnings` passed. All 45 `calternal-tags` tests passed. The full filesystem test run had one unrelated `history_rejects_symlinked_segments_and_keeps_lock_after_fold` `EAGAIN` from nonblocking `flock`; that test passes when run alone. Its assertion was not changed.

kayg referenced this issue from a commit

2026-10-06 17:42:04 +00:00

test(web): cover Tag rewrite progress and Undo (#1186)

kayg commented

2026-10-06 19:46:42 +00:00

Author

Owner

The first lock-held post-change rename profile completed all 1,000 targets and exact Undo on the perf VM. One sample took 575.96 s (9m 35.96s); other-request p95 was 13.90 ms, with 0 read errors. Server CPU was 120.03 s, average RSS 257,457,544 bytes (peak 279,846,912), and open descriptors averaged 116.11 (peak 136). Host load under the lock was 3.14/4.56/4.36 before and 4.35/4.11/4.16 after.

The old implementation did not finish within 180 s: rename reached 70/1,000 and other-request p95 was 11.98 ms; merge reached 70/1,000 and p95 was 12.29 ms. The first optimized run reached 256/1,000 in 180 s. The latest run is about 4.5x the old observed per-target throughput, below the 10x goal, and p95 is higher. CPU time accounts for 120 s of the 576 s wall time, which points to synchronous durable filesystem work as the remaining cost. I am grouping the per-file install intents and directory durability work into the existing 64-target batch while keeping the batch journal and exact Undo.

The first lock-held post-change rename profile completed all 1,000 targets and exact Undo on the perf VM. One sample took 575.96 s (9m 35.96s); other-request p95 was 13.90 ms, with 0 read errors. Server CPU was 120.03 s, average RSS 257,457,544 bytes (peak 279,846,912), and open descriptors averaged 116.11 (peak 136). Host load under the lock was 3.14/4.56/4.36 before and 4.35/4.11/4.16 after. The old implementation did not finish within 180 s: rename reached 70/1,000 and other-request p95 was 11.98 ms; merge reached 70/1,000 and p95 was 12.29 ms. The first optimized run reached 256/1,000 in 180 s. The latest run is about 4.5x the old observed per-target throughput, below the 10x goal, and p95 is higher. CPU time accounts for 120 s of the 576 s wall time, which points to synchronous durable filesystem work as the remaining cost. I am grouping the per-file install intents and directory durability work into the existing 64-target batch while keeping the batch journal and exact Undo.

kayg commented

2026-10-06 20:47:53 +00:00

Author

Owner

Status

The #1186 implementation is committed on job/tagperf-1186, but this run is incomplete and is not ready for merge. I stopped at the owner’s four-hour job cap.

Head: 6a481b0a1200fd0076d22aba4339d8694171b17a.

Built

Tag rewrite planning reads Index candidates without reconcile_user. The checked Root writer verifies each source revision before publication.
Tag writes stage with a bound of 8 workers in batches of at most 64. Each Root batch validates every target, writes one durable journal intent, applies replayable Version/Blob/rename steps, and syncs affected data filesystems before and after publication. Index rows update in one transaction per batch. Locks release between batches.
The Tags UI keeps the optimistic Tag name, shows background progress, and retains Undo.
Added the #1186 benchmark/seeder and a real-server screenshot driver.

Files owned by this job: apps/web/e2e/tag-rewrite-1186.mjs, apps/web/src/lib/components/TagRow.svelte, apps/web/src/lib/components/TagRow.svelte.test.ts, apps/web/src/lib/components/app-sidebar.svelte, apps/web/src/lib/stores/tagRewrite.svelte.test.ts, apps/web/src/lib/stores/tagRewrite.svelte.ts, apps/web/src/routes/tag/[tag]/+page.svelte, apps/web/src/routes/tags/+page.svelte, bench/tag-rewrite-1186-controller.py, bench/tag-rewrite-1186.py, crates/calternal-fs/src/journal.rs, crates/calternal-fs/src/lib.rs, crates/calternal-fs/src/quota.rs, crates/calternal-fs/src/root.rs, crates/calternal-fs/src/versions.rs, crates/calternal-fs/src/write.rs, crates/calternal-fs/tests/storage.rs, crates/calternal-tags/examples/tag_rewrite_1186_seed.rs, crates/calternal-tags/src/index.rs, crates/calternal-tags/src/rewrite.rs, crates/calternal-tags/src/source.rs, plus exact generated perf contracts.

Gates

cargo fmt --check: exit 0, no output.

cargo clippy -p calternal-fs --all-targets -- -D warnings:

    Checking calternal-path v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-path)
    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.26s

cargo test -p calternal-fs:

test result: ok. 99 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 98.81s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.31s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.43s
test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 31.00s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.27s

cargo clippy -p calternal-tags --all-targets -- -D warnings:

    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-fs)
    Checking calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-db)
    Checking calternal-api v0.0.1 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-api)
    Checking calternal-notes-core v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-notes-core)
    Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-plugin)
    Checking calternal-tags v0.0.1 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-tags)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 48.27s

cargo test -p calternal-tags:

test result: ok. 45 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 160.22s

The first post-merge bun run check found a stale exact render.bound pin in app-sidebar.svelte; scripts/perf-lint --adopt-7c refreshed exact live hashes and reduced the ratchet from 22,615 to 22,605. Its output ended:

perf-lint: PASS; 0 violations; 22605 scoped exceptions

The second bun run check did not finish. It was stopped at the time cap while Svelte diagnostics were still running:

error: script "check" was terminated by signal SIGTERM (Polite quit request)

bun run test was not run.

Performance and UX gaps

Perf VM baseline (1000 source items; measurements held under /root/perf.lock): rename was incomplete after 180.151 s at 70/1000, other-request p95 11.98 ms; merge was incomplete after 180.193 s at 70/1000, p95 12.29 ms. A later interim rename profile, before the final per-batch journal/sync change, completed in 575.962 s with p95 13.90 ms and exact Undo. Host load was higher during that sample. There is no after measurement for the final batch journal code, so the ≥10× target and p95 target remain unverified. No docs/perf/ result was added.

When a Home quota is configured, reserve_batch_write_quota still walks Home once per 64-target batch. The Index reconcile is gone, but this remaining quota walk needs review against the no-full-Home-scan performance rule.

Only three phone/light macOS screenshots exist from the earlier real-server run: artifacts/tag-rewrite-1186/progress-phone-light-mac.png, artifacts/tag-rewrite-1186/tags-phone-light-mac.png, and artifacts/tag-rewrite-1186/tag-detail-phone-light-mac.png. The sidebar locator was corrected afterward, but the final real-server matrix was not rerun or attached. Tablet/desktop, dark theme, and sidebar evidence are missing. A one-time adversarial round and the full web tests also remain for the merge round.

Decisions

Batch durability uses one pre-intent syncfs per affected data filesystem so staged data and Version directories exist before the WAL can name them, followed by one post-publication syncfs per affected filesystem before deleting the intent. The existing per-file journal path is unchanged.
The Tags rewrite uses the existing TagRow, ItemActionMenu, status/Undo toast, and sidebar; no new visual primitive was added.

For the merge round: run cd apps/web && bun run check, cd apps/web && bun run test -- --maxWorkers=2, cd apps/web && bun e2e/tag-rewrite-1186.mjs, then run one adversarial pass with bash tests/adversarial/run.sh against the real local server. The UI run must capture macOS-emulated phone (390), tablet (820), and desktop (1440) in light and dark. Run final rename and merge profiles on the perf VM under separate flock /root/perf.lock phases, record load average inside each lock, and add the final numbers to docs/perf/.

## Status The #1186 implementation is committed on `job/tagperf-1186`, but this run is incomplete and is not ready for merge. I stopped at the owner’s four-hour job cap. Head: `6a481b0a1200fd0076d22aba4339d8694171b17a`. ## Built - Tag rewrite planning reads Index candidates without `reconcile_user`. The checked Root writer verifies each source revision before publication. - Tag writes stage with a bound of 8 workers in batches of at most 64. Each Root batch validates every target, writes one durable journal intent, applies replayable Version/Blob/rename steps, and syncs affected data filesystems before and after publication. Index rows update in one transaction per batch. Locks release between batches. - The Tags UI keeps the optimistic Tag name, shows background progress, and retains Undo. - Added the `#1186` benchmark/seeder and a real-server screenshot driver. Files owned by this job: `apps/web/e2e/tag-rewrite-1186.mjs`, `apps/web/src/lib/components/TagRow.svelte`, `apps/web/src/lib/components/TagRow.svelte.test.ts`, `apps/web/src/lib/components/app-sidebar.svelte`, `apps/web/src/lib/stores/tagRewrite.svelte.test.ts`, `apps/web/src/lib/stores/tagRewrite.svelte.ts`, `apps/web/src/routes/tag/[tag]/+page.svelte`, `apps/web/src/routes/tags/+page.svelte`, `bench/tag-rewrite-1186-controller.py`, `bench/tag-rewrite-1186.py`, `crates/calternal-fs/src/journal.rs`, `crates/calternal-fs/src/lib.rs`, `crates/calternal-fs/src/quota.rs`, `crates/calternal-fs/src/root.rs`, `crates/calternal-fs/src/versions.rs`, `crates/calternal-fs/src/write.rs`, `crates/calternal-fs/tests/storage.rs`, `crates/calternal-tags/examples/tag_rewrite_1186_seed.rs`, `crates/calternal-tags/src/index.rs`, `crates/calternal-tags/src/rewrite.rs`, `crates/calternal-tags/src/source.rs`, plus exact generated perf contracts. ## Gates `cargo fmt --check`: exit 0, no output. `cargo clippy -p calternal-fs --all-targets -- -D warnings`: ```text Checking calternal-path v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-path) Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.26s ``` `cargo test -p calternal-fs`: ```text test result: ok. 99 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 98.81s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.31s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.43s test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 31.00s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.27s ``` `cargo clippy -p calternal-tags --all-targets -- -D warnings`: ```text Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-fs) Checking calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-db) Checking calternal-api v0.0.1 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-api) Checking calternal-notes-core v0.1.0 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-notes-core) Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-plugin) Checking calternal-tags v0.0.1 (/home/kayg/Developer/calternal-wt/tagperf-1186/crates/calternal-tags) Finished `dev` profile [unoptimized + debuginfo] target(s) in 48.27s ``` `cargo test -p calternal-tags`: ```text test result: ok. 45 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 160.22s ``` The first post-merge `bun run check` found a stale exact `render.bound` pin in `app-sidebar.svelte`; `scripts/perf-lint --adopt-7c` refreshed exact live hashes and reduced the ratchet from 22,615 to 22,605. Its output ended: ```text perf-lint: PASS; 0 violations; 22605 scoped exceptions ``` The second `bun run check` did not finish. It was stopped at the time cap while Svelte diagnostics were still running: ```text error: script "check" was terminated by signal SIGTERM (Polite quit request) ``` `bun run test` was not run. ## Performance and UX gaps Perf VM baseline (1000 source items; measurements held under `/root/perf.lock`): rename was incomplete after 180.151 s at 70/1000, other-request p95 11.98 ms; merge was incomplete after 180.193 s at 70/1000, p95 12.29 ms. A later interim rename profile, before the final per-batch journal/sync change, completed in 575.962 s with p95 13.90 ms and exact Undo. Host load was higher during that sample. There is no after measurement for the final batch journal code, so the ≥10× target and p95 target remain unverified. No `docs/perf/` result was added. When a Home quota is configured, `reserve_batch_write_quota` still walks Home once per 64-target batch. The Index reconcile is gone, but this remaining quota walk needs review against the no-full-Home-scan performance rule. Only three phone/light macOS screenshots exist from the earlier real-server run: `artifacts/tag-rewrite-1186/progress-phone-light-mac.png`, `artifacts/tag-rewrite-1186/tags-phone-light-mac.png`, and `artifacts/tag-rewrite-1186/tag-detail-phone-light-mac.png`. The sidebar locator was corrected afterward, but the final real-server matrix was not rerun or attached. Tablet/desktop, dark theme, and sidebar evidence are missing. A one-time adversarial round and the full web tests also remain for the merge round. ## Decisions - Batch durability uses one pre-intent `syncfs` per affected data filesystem so staged data and Version directories exist before the WAL can name them, followed by one post-publication `syncfs` per affected filesystem before deleting the intent. The existing per-file journal path is unchanged. - The Tags rewrite uses the existing `TagRow`, `ItemActionMenu`, status/Undo toast, and sidebar; no new visual primitive was added. For the merge round: run `cd apps/web && bun run check`, `cd apps/web && bun run test -- --maxWorkers=2`, `cd apps/web && bun e2e/tag-rewrite-1186.mjs`, then run one adversarial pass with `bash tests/adversarial/run.sh` against the real local server. The UI run must capture macOS-emulated phone (390), tablet (820), and desktop (1440) in light and dark. Run final rename and merge profiles on the perf VM under separate `flock /root/perf.lock` phases, record load average inside each lock, and add the final numbers to `docs/perf/`.

kayg commented

2026-10-07 04:02:00 +00:00

Author

Owner

Continuing #1186 on job/tagperf-1186. Worktree head at resume: 6a481b0a1200fd0076d22aba4339d8694171b17a; current origin/dev base to merge before final gates: 3601719b80a39f712daf07b37879b22929dfffa0. I read the previous report and the required CLAUDE.md, CONTEXT.md, and relevant docs/DESIGN.md sections. The implementation is committed; this run will finish the requested full web gates, final 1,000-target interleaved rename/merge measurements, quota-walk assessment, production screenshot matrix, and e2e driver. No new code findings yet.

Continuing #1186 on `job/tagperf-1186`. Worktree head at resume: `6a481b0a1200fd0076d22aba4339d8694171b17a`; current `origin/dev` base to merge before final gates: `3601719b80a39f712daf07b37879b22929dfffa0`. I read the previous report and the required `CLAUDE.md`, `CONTEXT.md`, and relevant `docs/DESIGN.md` sections. The implementation is committed; this run will finish the requested full web gates, final 1,000-target interleaved rename/merge measurements, quota-walk assessment, production screenshot matrix, and e2e driver. No new code findings yet.

kayg commented

2026-10-07 04:43:47 +00:00

Author

Owner

Finding during the required full web suite after merging origin/dev: Vitest reported × a tag chip opens the tag menu and a pick rewrites only that token (5,736 ms) at apps/web/src/lib/composer/Composer.svelte.test.ts:262. This is in the merged Composer code, outside the #1186 Tag rewrite files. I have not changed the assertion. The full suite is still running; I will record its final diagnostic and aggregate result when it exits.

Finding during the required full web suite after merging `origin/dev`: Vitest reported `× a tag chip opens the tag menu and a pick rewrites only that token` (5,736 ms) at `apps/web/src/lib/composer/Composer.svelte.test.ts:262`. This is in the merged Composer code, outside the #1186 Tag rewrite files. I have not changed the assertion. The full suite is still running; I will record its final diagnostic and aggregate result when it exits.

kayg commented

2026-10-07 04:52:24 +00:00

Author

Owner

Additional finding from the still-running full bun run test: Vitest reports failures in apps/web/src/routes/settings/mail/MailSection.svelte.test.ts (links selected Mail options through shared identity shared-account, 5,895 ms), apps/web/src/lib/mail/MailSidebar.svelte.test.ts (keeps the sidebar empty of folder placeholders until real folders arrive (#1094), 5,638 ms), and apps/web/src/routes/settings/admin/LicenceGroup.svelte.test.ts (saves a pasted key, clears it from the page, and removes it without changing Users, 6,700 ms; requires confirmation to replace a saved key and lets Cancel keep it, 5,814 ms). These merged Mail and Admin tests are outside the #1186 files. No existing expectation has been changed. The suite is still running; final stacks and totals are pending.

Additional finding from the still-running full `bun run test`: Vitest reports failures in `apps/web/src/routes/settings/mail/MailSection.svelte.test.ts` (`links selected Mail options through shared identity shared-account`, 5,895 ms), `apps/web/src/lib/mail/MailSidebar.svelte.test.ts` (`keeps the sidebar empty of folder placeholders until real folders arrive (#1094)`, 5,638 ms), and `apps/web/src/routes/settings/admin/LicenceGroup.svelte.test.ts` (`saves a pasted key, clears it from the page, and removes it without changing Users`, 6,700 ms; `requires confirmation to replace a saved key and lets Cancel keep it`, 5,814 ms). These merged Mail and Admin tests are outside the #1186 files. No existing expectation has been changed. The suite is still running; final stacks and totals are pending.

kayg commented

2026-10-07 04:54:55 +00:00

Author

Owner

Further output from the same full bun run test: apps/web/src/routes/settings/apps/CalendarFeedsGroup.svelte.test.ts fails freezes edits and retries the complete submitted body after assertion (8,798 ms), and apps/web/src/lib/history/VersionHistory.svelte.test.ts fails preflights an author session and shows the items kept before undo (12,298 ms). Vitest has reported seven failures so far across Composer, Mail, Admin settings, Calendar settings and VersionHistory. They are outside the #1186 files; all assertions remain unchanged. The full suite is still running and its final totals are pending.

Further output from the same full `bun run test`: `apps/web/src/routes/settings/apps/CalendarFeedsGroup.svelte.test.ts` fails `freezes edits and retries the complete submitted body after assertion` (8,798 ms), and `apps/web/src/lib/history/VersionHistory.svelte.test.ts` fails `preflights an author session and shows the items kept before undo` (12,298 ms). Vitest has reported seven failures so far across Composer, Mail, Admin settings, Calendar settings and VersionHistory. They are outside the #1186 files; all assertions remain unchanged. The full suite is still running and its final totals are pending.

kayg commented

2026-10-07 04:56:48 +00:00

Author

Owner

Additional full-suite finding: apps/web/src/lib/backgroundGroup.svelte.test.ts fails offers theme colour and picture without a shared light/dark toggle (5,761 ms). Vitest has reported eight failures so far across unrelated merged Composer, Mail, Admin, Calendar settings, VersionHistory and background settings tests. No expectation was changed; final suite output is pending.

Additional full-suite finding: `apps/web/src/lib/backgroundGroup.svelte.test.ts` fails `offers theme colour and picture without a shared light/dark toggle` (5,761 ms). Vitest has reported eight failures so far across unrelated merged Composer, Mail, Admin, Calendar settings, VersionHistory and background settings tests. No expectation was changed; final suite output is pending.

kayg commented

2026-10-07 05:14:45 +00:00

Author

Owner

Full web gate finding after origin/dev merge:

Command: cd apps/web && bun run test

Vitest summary (verbatim):

 Test Files  9 failed | 278 passed (287)
      Tests  11 failed | 2026 passed (2037)
   Start at  06:37:54
   Duration  2051.53s (transform 34%, environment 22%, import 22%, tests 16%, setup 5%)

Each failed assertion reported Test timed out in 5000ms. All 11 are outside the Tag rewrite files. The suite ran concurrently with Rust compilation on the shared build host, which may have added contention; no test expectations were changed.

Full web gate finding after origin/dev merge: Command: `cd apps/web && bun run test` Vitest summary (verbatim): ``` Test Files 9 failed | 278 passed (287) Tests 11 failed | 2026 passed (2037) Start at 06:37:54 Duration 2051.53s (transform 34%, environment 22%, import 22%, tests 16%, setup 5%) ``` Each failed assertion reported `Test timed out in 5000ms`. All 11 are outside the Tag rewrite files. The suite ran concurrently with Rust compilation on the shared build host, which may have added contention; no test expectations were changed.

kayg commented

2026-10-07 07:02:34 +00:00

Author

Owner

The production E2E rerun reached the real 1,000-Note rewrite and saved four progress screenshots (phone and tablet, light and dark). It then hit Playwright's 30 s setTheme data-theme wait while moving to the desktop capture, before checking the completed receipt. I changed the screenshot matrix to seed the review theme in the test User's local settings without writing Appearance during the background rewrite, and added explicit receipt/Undo-journal checks. I am rerunning the driver.

The production E2E rerun reached the real 1,000-Note rewrite and saved four progress screenshots (phone and tablet, light and dark). It then hit Playwright's 30 s `setTheme` `data-theme` wait while moving to the desktop capture, before checking the completed receipt. I changed the screenshot matrix to seed the review theme in the test User's local settings without writing Appearance during the background rewrite, and added explicit receipt/Undo-journal checks. I am rerunning the driver.

kayg commented

2026-10-07 07:17:35 +00:00

Author

Owner

The next screenshot retry confirmed that local userStorage alone cannot select Dark: the rendered state remained Light while the test seam reported Dark and the server Appearance mode was Light. The driver now changes Appearance through the real API on /readyz before TagRow mounts. It also checks the completed receipt before mounting the Undo toast, so the toast timer does not run during theme setup.

The next screenshot retry confirmed that local `userStorage` alone cannot select Dark: the rendered state remained Light while the test seam reported Dark and the server Appearance mode was Light. The driver now changes Appearance through the real API on `/readyz` before TagRow mounts. It also checks the completed receipt before mounting the Undo toast, so the toast timer does not run during theme setup.

kayg commented

2026-10-07 07:39:43 +00:00

Author

Owner

The production run captured all six Undo-toast screenshots. The final assertion then waited for the original page's Undo action after the driver had spent about a minute capturing those six contexts, so its documented eight-second toast had expired. The captured fresh contexts prove the durable receipt still exposes Undo. I am changing the driver to click Undo immediately in a new signed-in context and verify the terminal Undo receipt before the final Tags list/detail captures.

kayg commented

2026-10-07 07:50:22 +00:00

Author

Owner

The fresh-context Undo flow completed and restored the source Tag; the run advanced to the final surface screenshots. At phone width, Calendar sidebar capture then timed out because its locator resolved 57 times to a hidden responsive copy of the Tags section after the Show sidebar action. I am selecting the visible section instance and rerunning the complete screenshot matrix.

kayg commented

2026-10-07 08:07:04 +00:00

Author

Owner

The visible-section selector still timed out because the phone Tags section is inside the Navigation dialog, not an aside. The Show sidebar action opens that dialog. The desktop shell and phone dialog use the same section[aria-labelledby="calendar-tags"]; I am querying that shared section and waiting for the Navigation dialog at phone widths.

The visible-section selector still timed out because the phone Tags section is inside the Navigation dialog, not an `aside`. The Show sidebar action opens that dialog. The desktop shell and phone dialog use the same `section[aria-labelledby="calendar-tags"]`; I am querying that shared section and waiting for the Navigation dialog at phone widths.

kayg commented

2026-10-07 08:22:16 +00:00

Author

Owner

With the phone Navigation selector fixed, the full run passed progress and Undo screenshots, completed the reverse receipt and reached the post-Undo assertion. That assertion counted matching links across the entire page and found one renamed link. I am scoping it to the Tags card and checking /api/v1/tags separately to distinguish the real list from responsive navigation copies.

With the phone Navigation selector fixed, the full run passed progress and Undo screenshots, completed the reverse receipt and reached the post-Undo assertion. That assertion counted matching links across the entire page and found one renamed link. I am scoping it to the Tags card and checking `/api/v1/tags` separately to distinguish the real list from responsive navigation copies.

kayg commented

2026-10-07 08:42:32 +00:00

Author

Owner

The scoped check now reads the real /api/v1/tags response after the reverse receipt reaches completed. It still contains tagperf/renamed. The previous page-wide selector was unrelated to this: the server Index itself reports the renamed Tag. I am adding the reverse receipt's conflict count and per-Tag item counts to the next run before fixing the responsible path.

The scoped check now reads the real `/api/v1/tags` response after the reverse receipt reaches `completed`. It still contains `tagperf/renamed`. The previous page-wide selector was unrelated to this: the server Index itself reports the renamed Tag. I am adding the reverse receipt's conflict count and per-Tag item counts to the next run before fixing the responsible path.

kayg commented

2026-10-07 09:23:27 +00:00

Author

Owner

The retained #1186 fixture showed why Undo reported conflicts. The first differing Notes had date and last edited fields added while the server was running its post-startup Note date migration; the journal captured bytes before those fields existed. The run with screenshots had 127 conflicts, while the shorter diagnostic run had 27, matching migration progress. Undo preserved those later writes as designed. I am adding managed dates to the real fixture before server restart so this performance and UI flow measures Tag rewrite work without racing the independent Note migration.

The retained #1186 fixture showed why Undo reported conflicts. The first differing Notes had `date` and `last edited` fields added while the server was running its post-startup Note date migration; the journal captured bytes before those fields existed. The run with screenshots had 127 conflicts, while the shorter diagnostic run had 27, matching migration progress. Undo preserved those later writes as designed. I am adding managed dates to the real fixture before server restart so this performance and UI flow measures Tag rewrite work without racing the independent Note migration.

kayg commented

2026-10-07 09:46:54 +00:00

Author

Owner

The failed sidebar screenshot was a fixture mismatch. The Calendar sidebar only renders Tags in the area/ namespace (areaTagLeaves in apps/web/src/lib/components/tagTree.ts), while the performance fixture used tagperf/source. The shared Tags list and detail correctly included that Tag, but Calendar had no matching area Tag. I updated the E2E and benchmark fixture to use area/tagperf/source and its renamed and destination siblings so the screenshots exercise the real Calendar surface.

The failed sidebar screenshot was a fixture mismatch. The Calendar sidebar only renders Tags in the `area/` namespace (`areaTagLeaves` in `apps/web/src/lib/components/tagTree.ts`), while the performance fixture used `tagperf/source`. The shared Tags list and detail correctly included that Tag, but Calendar had no matching area Tag. I updated the E2E and benchmark fixture to use `area/tagperf/source` and its renamed and destination siblings so the screenshots exercise the real Calendar surface.

kayg commented

2026-10-07 10:46:38 +00:00

Author

Owner

Progress for #1186 on job/tagperf-1186 (head b54a24255895cec4582ee8a0288a7dae752ffa72).

bun run check passed with 0 errors and 1 pre-existing warning in packages/ui/src/components/calendar/AttachmentDeck.svelte.
The full web suite passed with 2 workers: 287 test files and 2,037 tests. The default-worker run hit timeouts in unrelated tests; the full rerun passed.
The production E2E rewrite completed for 1,000 Notes. Exact Undo restored all 1,000 Notes with 0 conflicts. The production screenshot matrix has 30 macOS images (Tags list, Tag detail, sidebar, progress and Undo toast; 390/820/1440 px; light/dark). Archives are attached below.
The shared perf VM is reserved by another benchmark until its stated slot ends. The final three interleaved 1,000-target rename/merge rounds remain pending.

Screenshot archives: Tags list, Tag detail, sidebar, progress, Undo toast.

Progress for #1186 on `job/tagperf-1186` (head `b54a24255895cec4582ee8a0288a7dae752ffa72`). - `bun run check` passed with 0 errors and 1 pre-existing warning in `packages/ui/src/components/calendar/AttachmentDeck.svelte`. - The full web suite passed with 2 workers: 287 test files and 2,037 tests. The default-worker run hit timeouts in unrelated tests; the full rerun passed. - The production E2E rewrite completed for 1,000 Notes. Exact Undo restored all 1,000 Notes with 0 conflicts. The production screenshot matrix has 30 macOS images (Tags list, Tag detail, sidebar, progress and Undo toast; 390/820/1440 px; light/dark). Archives are attached below. - The shared perf VM is reserved by another benchmark until its stated slot ends. The final three interleaved 1,000-target rename/merge rounds remain pending. Screenshot archives: [Tags list](https://git.kayg.org/attachments/fd0fe2ed-6bc1-4b8a-a4cd-7b2e56891a08), [Tag detail](https://git.kayg.org/attachments/b62bd84b-ed8a-4c3c-9a61-c3a60dc8446c), [sidebar](https://git.kayg.org/attachments/4b4d0a9d-87c0-441b-bc24-5646254b44c3), [progress](https://git.kayg.org/attachments/1fc195d0-f26d-4b12-b665-25ab7f9edfc4), [Undo toast](https://git.kayg.org/attachments/9dc7e74a-2597-4b2d-826d-bbfae8d504be).

kayg commented

2026-10-07 12:36:46 +00:00

Author

Owner

Merged origin/dev at 8f89c1458 after preserving its #1197 Note-room source preflight and publication hooks in both the legacy and batched Tag rewrite paths. Batched replay republishes an already-written Markdown revision before advancing its cursor, so a stop between file commit and room publication does not leave an open Note stale. The perf ratchet combines this branch's updates with dev's four fewer unresolved-call sites: total 22534. Evidence: cargo fmt --check passed; calternal-tags Clippy passed; 52 calternal-tags tests passed; perf-lint reported 0 violations and 22534 scoped exceptions; bun run check reports 0 errors and 1 existing warning. Final tests and performance run are in progress.

Merged origin/dev at 8f89c1458 after preserving its #1197 Note-room source preflight and publication hooks in both the legacy and batched Tag rewrite paths. Batched replay republishes an already-written Markdown revision before advancing its cursor, so a stop between file commit and room publication does not leave an open Note stale. The perf ratchet combines this branch's updates with dev's four fewer unresolved-call sites: total 22534. Evidence: cargo fmt --check passed; calternal-tags Clippy passed; 52 calternal-tags tests passed; perf-lint reported 0 violations and 22534 scoped exceptions; bun run check reports 0 errors and 1 existing warning. Final tests and performance run are in progress.

kayg commented

2026-10-07 12:48:51 +00:00

Author

Owner

Corrected the extrapolation in docs/perf/2026-10-07-tag-rewrite-1186.md: 180.151s × 1000/70 = 2573.59s for rename; 180.193s × 1000/70 = 2574.19s for merge. The separate 10× median targets are 257.36s and 257.42s. The prior 2571s estimate was low by about 2.6s. Commit: 1e0c3a1e4.

Corrected the extrapolation in docs/perf/2026-10-07-tag-rewrite-1186.md: 180.151s × 1000/70 = 2573.59s for rename; 180.193s × 1000/70 = 2574.19s for merge. The separate 10× median targets are 257.36s and 257.42s. The prior 2571s estimate was low by about 2.6s. Commit: 1e0c3a1e4.

kayg commented

2026-10-07 13:58:37 +00:00

Author

Owner

Extra calternal-collab verification: Clippy passed. The full default-thread test run completed 132 tests and had two elapsed-timeout failures: each_source_writer_keeps_concurrent_typing waited 10s for its injected publication point, and history_preflight_drops_idle_room_after_error waited 5s for room cleanup. The 96-case writer/lock matrix passed. Both failed tests passed alone with --test-threads=1 (34.35s and 2.11s). This suggests host scheduling pressure in the parallel run; no test expectation or production code was changed. The required calternal-tags and calternal-fs gates pass.

kayg commented

2026-10-07 14:07:11 +00:00

Author

Owner

Merged-release round 10 (fresh 1,000-Note Home, one destination, 16 Tag readers): rename 46.410s; merge 35.659s. Other-request p95 was 312.22ms during rename and 319.70ms during merge. The matching 12s idle profile had 192 requests, 0 errors and p95 249.10ms at load averages 4.51/3.03/3.07 before the run. Quota probe: 6.76ms; 16-walk estimate: 108.16ms. This sample is well below the 10× duration thresholds, but p95 is above 50ms even at idle; I will report medians after rounds 11 and 12. Raw profiles committed as e5d67ee6f.

Merged-release round 10 (fresh 1,000-Note Home, one destination, 16 Tag readers): rename 46.410s; merge 35.659s. Other-request p95 was 312.22ms during rename and 319.70ms during merge. The matching 12s idle profile had 192 requests, 0 errors and p95 249.10ms at load averages 4.51/3.03/3.07 before the run. Quota probe: 6.76ms; 16-walk estimate: 108.16ms. This sample is well below the 10× duration thresholds, but p95 is above 50ms even at idle; I will report medians after rounds 11 and 12. Raw profiles committed as e5d67ee6f.

kayg commented

2026-10-07 14:17:44 +00:00

Author

Owner

Round 11 is committed as 838ebbd73 in docs/perf/runs/tag-rewrite-1186-round11.json. On a fresh 1,000-Note Home, rename completed in 44.205 s and merge in 30.337 s. Sixteen authenticated Tag readers returned no errors; their p95 was 343.68 ms during rename and 548.84 ms during merge, above the 50 ms target. The idle profile was already 249.10 ms p95 at load average 4.51/3.03/3.07, so the final three-run report will compare these values without attributing all latency to the rewrite. Quota usage probe was 7.74 ms; 16 batch walks estimate to 123.84 ms per operation.

Round 11 is committed as `838ebbd73` in `docs/perf/runs/tag-rewrite-1186-round11.json`. On a fresh 1,000-Note Home, rename completed in 44.205 s and merge in 30.337 s. Sixteen authenticated Tag readers returned no errors; their p95 was 343.68 ms during rename and 548.84 ms during merge, above the 50 ms target. The idle profile was already 249.10 ms p95 at load average 4.51/3.03/3.07, so the final three-run report will compare these values without attributing all latency to the rewrite. Quota usage probe was 7.74 ms; 16 batch walks estimate to 123.84 ms per operation.

kayg commented

2026-10-07 14:27:09 +00:00

Author

Owner

Final report — #1186

Head: aca393266eb2eda3cbeffcb75dbc7ab40533d5ea

Performance

The measured before values cover 70 source Notes and one destination Note: rename 180.151 s and merge 180.193 s. Linear estimates for 1,000 Notes are 2,573.59 s and 2,574.19 s. These are not full-size before measurements. They set 10x targets of 257.36 s for rename and 257.42 s for merge.

Three interleaved runs used a fresh 1,000-Note Home each time, one destination Note, a 1 GB quota and 16 authenticated Tag readers at one request per second each. The final release server was built locally from the merged source and copied to the perf VM. It completed every operation with 1,000 targets and no concurrent-read errors.

Run	Rename (s)	Rename reader p95 (ms)	Merge (s)	Merge reader p95 (ms)
10	46.410	312.22	35.659	319.70
11	44.205	343.68	30.337	548.84
12	44.798	308.52	27.964	502.81
Median	44.798	312.22	30.337	502.81

The rename median is 57.45x faster than its estimated before time. The merge median is 84.85x faster. Both pass the 10x time target. The other-request p95 target was below 50 ms. It was not met: the measured medians were 312.22 ms during rename and 502.81 ms during merge. An idle profile on the same Home measured 249.10 ms p95 across 192 requests with no errors. The old p95 values came from a 70-Note Home, so they are not a like-for-like comparison. The final report and raw profiles are in docs/perf/2026-10-07-tag-rewrite-1186.md and docs/perf/runs/tag-rewrite-1186-{round10,round11,round12,idle-merged}.json.

Quota usage probes were 6.76, 7.74 and 8.81 ms; the median was 7.74 ms. Sixteen walks estimate to 123.84 ms per operation, about 0.3% of rename and 0.4% of merge. I left the quota path unchanged because this cost is small beside the measured operation times.

Production E2E and screenshots

The production E2E driver passed with 1,000 Notes, completed rename, and exact Undo. It produced 30 fresh macOS captures: Tags list, Tag detail, sidebar, progress and Undo toast at phone (390 px), tablet (820 px) and desktop (1440 px), in light and dark themes. The PNGs are local at artifacts/tag-rewrite-1186-final/ and are not committed. The screenshot archives already attached to this issue are:

Gates

Commands ran after merging origin/dev. Output excerpts are verbatim.

$ cargo fmt --check
(exit 0; no output)

$ cargo clippy -p calternal-tags --all-targets -- -D warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.69s

$ cargo test -p calternal-tags
test result: ok. 52 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 60.10s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

$ cargo clippy -p calternal-fs --all-targets -- -D warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.03s

$ cargo test -p calternal-fs
test result: ok. 100 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 74.26s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.25s
test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 54.21s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

$ cargo clippy -p calternal-collab --all-targets -- -D warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 7m 22s

$ cargo test -p calternal-collab
test result: FAILED. 132 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 460.55s

$ cargo test -p calternal-collab --lib session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing -- --exact --test-threads=1
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 34.35s

$ cargo test -p calternal-collab --lib session::live_history_write_tests::history_preflight_drops_idle_room_after_error -- --exact --test-threads=1
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 2.11s

$ cd apps/web && bun run check
perf-lint: PASS; 0 violations; 22534 scoped exceptions
svelte-check found 0 errors and 1 warning in 1 file

$ cd apps/web && bun run test --maxWorkers=2
 Test Files  287 passed (287)
      Tests  2038 passed (2038)
   Duration  843.84s (transform 29%, import 26%, environment 24%, tests 15%, setup 5%)

$ cd apps/web && bun run build
Compressed 917 static variants; saved 21489111 bytes.

$ cargo build --release -p calternal-server
    Finished `release` profile [optimized] target(s) in 23m 45s

$ production tag-rewrite-1186 E2E driver
#1186 E2E: launching headless Chromium
#1186 E2E: Chromium is ready; starting the release server
#1186 E2E: server is ready; registering the screenshot User
#1186 E2E: seeding the 1,000-Note Home
#1186 E2E: fixture is seeded; restarting the release server
#1186 E2E: rename source_items=0 renamed_items=1000
#1186 E2E: completed receipt can_undo=true
#1186 E2E: Undo conflicts=0 source_items=1000 renamed_items=0

$ cargo clean
     Removed 22775 files, 15.8GiB total

The two full calternal-collab failures were timeouts in the concurrent typing and idle-room cleanup tests. Both passed when run alone with one test thread. No assertions or expectations changed. bun run check reports one existing empty CSS ruleset warning in packages/ui/src/components/calendar/AttachmentDeck.svelte:1055.

UX gaps closed

Re-ran the production rename and exact Undo flow. All 1,000 targets moved and returned.
Captured all five requested views across the complete width, theme and macOS matrix.

UX gaps left

Other-request p95 remains above 50 ms. The measurements do not isolate host load from Tag list cost.
A default-thread full calternal-collab test run had two timeout failures; the two focused reruns passed.

Decisions

Keep the batch quota walk. The measured estimate is under 0.5% of operation time.
Use the measured 70-Note before values with linear estimates for 1,000 Notes. No complete 1,000-Note before median exists.

For the merge round

Run tests/adversarial/run.sh against the local server. It must cover the full XUser, authorization and robustness matrices for the merged branch. The verification policy defers those matrices from this job.

## Final report — #1186 Head: `aca393266eb2eda3cbeffcb75dbc7ab40533d5ea` ### Performance The measured before values cover 70 source Notes and one destination Note: rename 180.151 s and merge 180.193 s. Linear estimates for 1,000 Notes are 2,573.59 s and 2,574.19 s. These are not full-size before measurements. They set 10x targets of 257.36 s for rename and 257.42 s for merge. Three interleaved runs used a fresh 1,000-Note Home each time, one destination Note, a 1 GB quota and 16 authenticated Tag readers at one request per second each. The final release server was built locally from the merged source and copied to the perf VM. It completed every operation with 1,000 targets and no concurrent-read errors. | Run | Rename (s) | Rename reader p95 (ms) | Merge (s) | Merge reader p95 (ms) | | --- | ---: | ---: | ---: | ---: | | 10 | 46.410 | 312.22 | 35.659 | 319.70 | | 11 | 44.205 | 343.68 | 30.337 | 548.84 | | 12 | 44.798 | 308.52 | 27.964 | 502.81 | | Median | 44.798 | 312.22 | 30.337 | 502.81 | The rename median is 57.45x faster than its estimated before time. The merge median is 84.85x faster. Both pass the 10x time target. The other-request p95 target was below 50 ms. It was not met: the measured medians were 312.22 ms during rename and 502.81 ms during merge. An idle profile on the same Home measured 249.10 ms p95 across 192 requests with no errors. The old p95 values came from a 70-Note Home, so they are not a like-for-like comparison. The final report and raw profiles are in `docs/perf/2026-10-07-tag-rewrite-1186.md` and `docs/perf/runs/tag-rewrite-1186-{round10,round11,round12,idle-merged}.json`. Quota usage probes were 6.76, 7.74 and 8.81 ms; the median was 7.74 ms. Sixteen walks estimate to 123.84 ms per operation, about 0.3% of rename and 0.4% of merge. I left the quota path unchanged because this cost is small beside the measured operation times. ### Production E2E and screenshots The production E2E driver passed with 1,000 Notes, completed rename, and exact Undo. It produced 30 fresh macOS captures: Tags list, Tag detail, sidebar, progress and Undo toast at phone (390 px), tablet (820 px) and desktop (1440 px), in light and dark themes. The PNGs are local at `artifacts/tag-rewrite-1186-final/` and are not committed. The screenshot archives already attached to this issue are: - [Tags list](https://git.kayg.org/attachments/fd0fe2ed-6bc1-4b8a-a4cd-7b2e56891a08) - [Tag detail](https://git.kayg.org/attachments/b62bd84b-ed8a-4c3c-9a61-c3a60dc8446c) - [Sidebar](https://git.kayg.org/attachments/4b4d0a9d-87c0-441b-bc24-5646254b44c3) - [Progress](https://git.kayg.org/attachments/1fc195d0-f26d-4b12-b665-25ab7f9edfc4) - [Undo toast](https://git.kayg.org/attachments/9dc7e74a-2597-4b2d-826d-bbfae8d504be) ### Gates Commands ran after merging `origin/dev`. Output excerpts are verbatim. ```text $ cargo fmt --check (exit 0; no output) $ cargo clippy -p calternal-tags --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.69s $ cargo test -p calternal-tags test result: ok. 52 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 60.10s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s $ cargo clippy -p calternal-fs --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.03s $ cargo test -p calternal-fs test result: ok. 100 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 74.26s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.25s test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 54.21s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s $ cargo clippy -p calternal-collab --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 7m 22s $ cargo test -p calternal-collab test result: FAILED. 132 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 460.55s $ cargo test -p calternal-collab --lib session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing -- --exact --test-threads=1 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 34.35s $ cargo test -p calternal-collab --lib session::live_history_write_tests::history_preflight_drops_idle_room_after_error -- --exact --test-threads=1 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 2.11s $ cd apps/web && bun run check perf-lint: PASS; 0 violations; 22534 scoped exceptions svelte-check found 0 errors and 1 warning in 1 file $ cd apps/web && bun run test --maxWorkers=2 Test Files 287 passed (287) Tests 2038 passed (2038) Duration 843.84s (transform 29%, import 26%, environment 24%, tests 15%, setup 5%) $ cd apps/web && bun run build Compressed 917 static variants; saved 21489111 bytes. $ cargo build --release -p calternal-server Finished `release` profile [optimized] target(s) in 23m 45s $ production tag-rewrite-1186 E2E driver #1186 E2E: launching headless Chromium #1186 E2E: Chromium is ready; starting the release server #1186 E2E: server is ready; registering the screenshot User #1186 E2E: seeding the 1,000-Note Home #1186 E2E: fixture is seeded; restarting the release server #1186 E2E: rename source_items=0 renamed_items=1000 #1186 E2E: completed receipt can_undo=true #1186 E2E: Undo conflicts=0 source_items=1000 renamed_items=0 $ cargo clean Removed 22775 files, 15.8GiB total ``` The two full `calternal-collab` failures were timeouts in the concurrent typing and idle-room cleanup tests. Both passed when run alone with one test thread. No assertions or expectations changed. `bun run check` reports one existing empty CSS ruleset warning in `packages/ui/src/components/calendar/AttachmentDeck.svelte:1055`. ### UX gaps closed - Re-ran the production rename and exact Undo flow. All 1,000 targets moved and returned. - Captured all five requested views across the complete width, theme and macOS matrix. ### UX gaps left - Other-request p95 remains above 50 ms. The measurements do not isolate host load from Tag list cost. - A default-thread full `calternal-collab` test run had two timeout failures; the two focused reruns passed. ### Decisions - Keep the batch quota walk. The measured estimate is under 0.5% of operation time. - Use the measured 70-Note before values with linear estimates for 1,000 Notes. No complete 1,000-Note before median exists. ### For the merge round Run `tests/adversarial/run.sh` against the local server. It must cover the full XUser, authorization and robustness matrices for the merged branch. The verification policy defers those matrices from this job.

kayg commented

2026-10-07 14:36:35 +00:00

Author

Owner

Continuing #1186 on job/tagperf-1186, head aca393266, base 0419170fe456b58614befcb12d9c9d255118f1c0. New target: durable Index intent at acceptance, bounded source replay, immediate Note reads and exact Undo. Existing batch writer already uses filesystem-wide flushes; current acceptance only enqueues and Index updates follow file batches. No pushes or deploys.

Continuing #1186 on `job/tagperf-1186`, head `aca393266`, base `0419170fe456b58614befcb12d9c9d255118f1c0`. New target: durable Index intent at acceptance, bounded source replay, immediate Note reads and exact Undo. Existing batch writer already uses filesystem-wide flushes; current acceptance only enqueues and Index updates follow file batches. No pushes or deploys.

kayg commented

2026-10-07 15:27:45 +00:00

Author

Owner

Progress: committed the compact sidebar count-slot fix as 503868bec; focused web tests: Test Files 2 passed (2) / Tests 11 passed (11). Backend acceptance regressions passed: test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 52 filtered out; finished in 30.67s. The queue-only acceptance regression failed first with old memberships still present. The new path uses one FULL SQLite transaction for source witnesses, the queue receipt, Tags memberships and Notes memberships. Markdown editor reads finish their pending source before returning an ETag. The existing legacy receipt test found that saved batches can start off the normal 64-file grid; the inverse now follows the actual saved journals. Final gates and release measurements remain pending. No performance target is claimed yet.

Progress: committed the compact sidebar count-slot fix as `503868bec`; focused web tests: `Test Files 2 passed (2)` / `Tests 11 passed (11)`. Backend acceptance regressions passed: `test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 52 filtered out; finished in 30.67s`. The queue-only acceptance regression failed first with old memberships still present. The new path uses one FULL SQLite transaction for source witnesses, the queue receipt, Tags memberships and Notes memberships. Markdown editor reads finish their pending source before returning an ETag. The existing legacy receipt test found that saved batches can start off the normal 64-file grid; the inverse now follows the actual saved journals. Final gates and release measurements remain pending. No performance target is claimed yet.

kayg commented

2026-10-07 16:08:43 +00:00

Author

Owner

#1186 checkpoint

Branch job/tagperf-1186, durable intent commit 56d97d94f, immediate UI refresh commit 9bfa982a7. git fetch origin and the one requested git merge origin/dev completed: Already up to date. Tags migration 7 is free on origin/dev.

Acceptance now commits the owner-scoped source identity/hash witnesses, queue receipt and Tags/Notes memberships in one FULL SQLite transaction. It performs no Markdown reads. Bounded worker batches use the existing checked calternal-fs staging and syncfs protocol, retaining exact Home journals until completion. Notes reads finish their pending source before returning a real-byte ETag.

Review found and fixes cover: plan without checkpoint after crash; Undo before the first worker lease; replacement Note identity at an old path; moved Notes; accepted inverse projection after reconcile; retry after a lost acceptance response. A further fix now blocks a new operation while a cancelled/dead acceptance still owns source changes, and resolves identities for inverse batches as well.

Verification evidence before those last focused fixes:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 37.95s
test result: ok. 59 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 182.11s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 52 filtered out; finished in 51.23s
Test Files  2 passed (2)
Tests  17 passed (17)

Production web build exited 0. Final per-crate gates and web check/test are running. Release measurements and screenshot capture remain; no timing target is claimed yet.

Decision: use the existing two syncfs barriers per source batch (staged data before the filesystem write-ahead record, then namespace publication before checkpoint). Directory-only fsync cannot make the staged file contents durable. WebDAV/CalDAV may briefly read old physical bytes; this is recorded in DESIGN §2.

#1186 checkpoint Branch `job/tagperf-1186`, durable intent commit `56d97d94f`, immediate UI refresh commit `9bfa982a7`. `git fetch origin` and the one requested `git merge origin/dev` completed: `Already up to date.` Tags migration 7 is free on origin/dev. Acceptance now commits the owner-scoped source identity/hash witnesses, queue receipt and Tags/Notes memberships in one FULL SQLite transaction. It performs no Markdown reads. Bounded worker batches use the existing checked calternal-fs staging and syncfs protocol, retaining exact Home journals until completion. Notes reads finish their pending source before returning a real-byte ETag. Review found and fixes cover: plan without checkpoint after crash; Undo before the first worker lease; replacement Note identity at an old path; moved Notes; accepted inverse projection after reconcile; retry after a lost acceptance response. A further fix now blocks a new operation while a cancelled/dead acceptance still owns source changes, and resolves identities for inverse batches as well. Verification evidence before those last focused fixes: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 37.95s test result: ok. 59 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 182.11s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 52 filtered out; finished in 51.23s Test Files 2 passed (2) Tests 17 passed (17) ``` Production web build exited 0. Final per-crate gates and web check/test are running. Release measurements and screenshot capture remain; no timing target is claimed yet. Decision: use the existing two syncfs barriers per source batch (staged data before the filesystem write-ahead record, then namespace publication before checkpoint). Directory-only fsync cannot make the staged file contents durable. WebDAV/CalDAV may briefly read old physical bytes; this is recorded in DESIGN §2.

kayg commented

2026-10-07 16:57:37 +00:00

Author

Owner

#1186 recovery finding and correction

The durable intent must also cover early cancellation, a lost Home checkpoint, and a Note moved while its journal is pending. The current recovery changes keep the original stable Note identity, journal the current path, create a missing checkpoint independently of the plan, and block a second forward intent until a cancelled accepted intent is resolved. Undo restores only matching byte revisions and consumes its original receipt. A newer edit is preserved, including an edit that restores a Tag after an earlier skipped journal entry.

The focused regressions also include actual subprocess SIGKILL at the journal and durable-replacement boundaries, restart and exact inverse replay. The full touched-crate gates are running. The read-only review found no remaining crash, data-loss or security blocker; this is not a passing-gate or timing claim.

cargo clippy -p calternal-plugin-notes --all-targets -- -D warnings finished successfully. The exact performance guard refresh passes:

perf-lint: PASS; 0 violations; 22532 scoped exceptions

The ratchet fell from 22534 to 22532; no threshold was raised. The two Notes read boundaries explicitly remain content reads with bounded-source proof references. Release timing and screenshot measurements remain outstanding. I interrupted the cold release build to let the required crate gates run before resuming it.

#1186 recovery finding and correction The durable intent must also cover early cancellation, a lost Home checkpoint, and a Note moved while its journal is pending. The current recovery changes keep the original stable Note identity, journal the current path, create a missing checkpoint independently of the plan, and block a second forward intent until a cancelled accepted intent is resolved. Undo restores only matching byte revisions and consumes its original receipt. A newer edit is preserved, including an edit that restores a Tag after an earlier skipped journal entry. The focused regressions also include actual subprocess SIGKILL at the journal and durable-replacement boundaries, restart and exact inverse replay. The full touched-crate gates are running. The read-only review found no remaining crash, data-loss or security blocker; this is not a passing-gate or timing claim. `cargo clippy -p calternal-plugin-notes --all-targets -- -D warnings` finished successfully. The exact performance guard refresh passes: ``` perf-lint: PASS; 0 violations; 22532 scoped exceptions ``` The ratchet fell from 22534 to 22532; no threshold was raised. The two Notes read boundaries explicitly remain content reads with bounded-source proof references. Release timing and screenshot measurements remain outstanding. I interrupted the cold release build to let the required crate gates run before resuming it.

kayg commented

2026-10-07 17:18:21 +00:00

Author

Owner

#1186 checkpoint at 0b76f61a6

The branch now has separate commits for compact sidebar progress, durable-intent acceptance and replay, immediate accepted-membership refresh, the revised benchmark, production screenshot coverage, and the measurement method. Recovery and pending-editor-read corrections await their final crate gates before commit.

The Notes test build is still compiling its test target. At 19:18 CEST its rustc process had run for about 20 minutes and consumed about 103 CPU seconds. The host load average was about 80. The cold release build was paused to release the Cargo lock for these gates. No new release timing or screenshot evidence exists yet; the previous 44.798-second Rename median remains the old-worker baseline, not evidence for the new target.

The web test run now uses two workers and a 30-second command-line timeout because default five-second timeouts failed under this load. No assertion or existing test expectation was changed. The corrected performance guard reports zero violations and 22532 scoped exceptions. Final Svelte checking and full tests remain running.

#1186 checkpoint at 0b76f61a6 The branch now has separate commits for compact sidebar progress, durable-intent acceptance and replay, immediate accepted-membership refresh, the revised benchmark, production screenshot coverage, and the measurement method. Recovery and pending-editor-read corrections await their final crate gates before commit. The Notes test build is still compiling its test target. At 19:18 CEST its rustc process had run for about 20 minutes and consumed about 103 CPU seconds. The host load average was about 80. The cold release build was paused to release the Cargo lock for these gates. No new release timing or screenshot evidence exists yet; the previous 44.798-second Rename median remains the old-worker baseline, not evidence for the new target. The web test run now uses two workers and a 30-second command-line timeout because default five-second timeouts failed under this load. No assertion or existing test expectation was changed. The corrected performance guard reports zero violations and 22532 scoped exceptions. Final Svelte checking and full tests remain running.

kayg commented

2026-10-07 17:42:29 +00:00

Author

Owner

#1186 finding: per-Note source callbacks must not restore a whole intent

An accepted 1,000-Note intent stays present while the source worker emits Note updates. The public Tags index_markdown hook originally restored all 1,000 candidate memberships after each individual Note projection. That could repeat the large membership update 1,000 times.

The correction scopes an incremental callback to its own path. Full reconciliation still restores the complete intent. Inverse parsing also happens before the SQLite writer transaction. A regression models three stale projections and proves one callback repairs one Note, while full reconciliation repairs all three. This bounds the SQL update work to the changed source; intent deserialization still reads the packed journal.

New viewer-Share acceptance coverage passes:

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 62 filtered out; finished in 3.07s

The new quota test already proved an exact one-byte-over-limit refusal without any source or usage change. Its final Undo setup used the wrong stored queue state; it is corrected to dead. The SIGKILL fixture now keeps a stable executable copy, because another Cargo gate can replace the running test binary. Both corrections and the scoped projection regression are being rerun. No timing or READY claim yet.

#1186 finding: per-Note source callbacks must not restore a whole intent An accepted 1,000-Note intent stays present while the source worker emits Note updates. The public Tags `index_markdown` hook originally restored all 1,000 candidate memberships after each individual Note projection. That could repeat the large membership update 1,000 times. The correction scopes an incremental callback to its own path. Full reconciliation still restores the complete intent. Inverse parsing also happens before the SQLite writer transaction. A regression models three stale projections and proves one callback repairs one Note, while full reconciliation repairs all three. This bounds the SQL update work to the changed source; intent deserialization still reads the packed journal. New viewer-Share acceptance coverage passes: ``` test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 62 filtered out; finished in 3.07s ``` The new quota test already proved an exact one-byte-over-limit refusal without any source or usage change. Its final Undo setup used the wrong stored queue state; it is corrected to `dead`. The SIGKILL fixture now keeps a stable executable copy, because another Cargo gate can replace the running test binary. Both corrections and the scoped projection regression are being rerun. No timing or READY claim yet.

kayg referenced this issue

2026-10-07 18:30:03 +00:00

Audit: server and bulk operations that are far slower than they should be #1374

kayg commented

2026-10-07 18:44:06 +00:00

Author

Owner

#1186 tagfast: NOT READY FOR MERGE

Head: 46037c0fb on job/tagperf-1186. Started from aca393266. Fetched and merged origin/dev once; it was already up to date. No push or deploy. The job reached its four-hour limit; the remaining verification is listed below.

Built:

One FULL SQLite acceptance transaction stores the source witnesses, queue receipt and Tag/Notes membership projection before file work.
Bounded source replay, exact Markdown Undo, restart recovery, stable Note identity/path resolution and protection of newer edits.
On-demand editor reads and new collaboration room baselines adopt pending source changes under the User writer guard.
Incremental Note callbacks restore only their own membership, rather than updating the complete intent after every Note. Inverse parsing precedes the SQLite writer transaction.
Compact sidebar progress, immediate accepted membership refresh, and separate acceptance/Index/durability/Undo benchmark fields.
Production screenshot harness coverage for Tags, sidebar and Notes at the owner widths and themes, with macOS emulation. This harness was extended but not run against the new release binary.

Files:
crates/calternal-tags/src/{lib,index,rewrite}.rs, crates/calternal-tags/migrations/0007_tag_rewrite_acceptance.sql, crates/plugins/notes/src/{lib,store}.rs, crates/calternal-collab/src/session.rs, apps/web/src/lib/components/TagRow.svelte, apps/web/src/lib/stores/tagRewrite.svelte.ts, its .test.ts, apps/web/e2e/tag-rewrite-1186.mjs, bench/tag-rewrite-1186.py, contracts/perf/{registry,exceptions,adoption-1058,ratchet}.json, docs/DESIGN.md, docs/perf/2026-10-07-tag-rewrite-1186.md.

Gate output, verbatim result lines (full logs remain under artifacts/tagfast/):

cd apps/web && bun run build — passed:

✓ built in 7m 10s
✓ built in 97ms
✓ built in 13m 10s

cargo fmt --check — exit 0, no output.

Tags clippy:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.98s

Tags full test, with the shared compiler wrapper bypassed after its deleted-temp-directory failure:

test result: ok. 63 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 787.30s

Notes clippy:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 27m 02s

Notes full unit and integration tests:

test result: ok. 317 passed; 0 failed; 4 ignored; 0 measured; 0 filtered out; finished in 1908.08s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 10.99s

Collaboration clippy:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 16s

Collaboration full test — FAILED:

test result: FAILED. 133 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 441.53s

The failure is session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing, waiting ten seconds for the Tag writer publication hook. A one-thread focused retry also timed out. No expectation was changed:

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 133 filtered out; finished in 70.19s

Related source/typing integration tests pass:

test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 126 filtered out; finished in 34.75s

Server clippy:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 46s

Server test: interrupted during compilation at the job limit; no passing result.

bun run check — exit 0:

perf-lint: PASS; 0 violations; 22532 scoped exceptions
svelte-check found 0 errors and 1 warning in 1 file

The warning is the existing empty .attachment-row:focus ruleset in AttachmentDeck.svelte. The exact performance ratchet decreased from 22534 to 22532; no threshold increased.

bun run test --maxWorkers=2 --testTimeout=30000 — passed:

 Test Files  287 passed (287)
      Tests  2039 passed (2039)
   Start at  19:08:14
   Duration  2509.80s (transform 52%, import 16%, environment 14%, tests 12%, setup 5%)

The initial runs hit existing five-second timeouts under host load around 80. The final command changes the execution timeout only; assertions and existing expectations remain unchanged.

Correctness coverage: actual SIGKILL at journal and durable-publication boundaries, a randomized interruption window during application, old-or-new file checks, restart/resume, exact Undo, newer edits, moved/replaced Notes, an exact one-byte quota refusal, and viewer Share isolation. The read-only review found no crash/data-loss/security blocker in the recovery diff.

Performance and known gaps:

No new release measurement was completed. The release build remained cold while crate gates consumed the job budget. The previous VM medians remain 44.798 s Rename, 30.337 s Merge, and other-request p95 312.22/502.81 ms. Those are the previous worker, with storage unspecified here; they are not SSD results for this implementation.
The 500 ms click p95, 2 s SSD durability, 50 ms other-request p95 and corresponding Undo targets are UNVERIFIED. No HDD run occurred. The VM was free when checked; lack of the new binary prevented a measured run.
Undo with any structured Sidecar still uses the legacy queue path. Its immediate inverse Index intent is not implemented. This is a gap against the complete design.
Already-open Note rooms update when their source batch publishes; their 500 ms bound is not established. New reads/rooms adopt pending content on demand.
The existing Root quota reservation still walks the Home once per batch, including on-demand publication. Larger-Home cost remains unmeasured; removing that scan needs a safe quota accounting follow-up.
No screenshot set or icon alignment evidence was captured. No visual quality claim is made.
The repeated collaboration publication-hook timeout remains unresolved; it is not a passing gate.

Expected cost: Rename acceptance handles 1,000 Index witnesses in one transaction with no Markdown reads. Source publication handles 1,000 Notes in 16 batches of 64, with at most eight independent file tasks. Undo reads the exact byte journals with the same concurrency bound before accepting its inverse. Per-Note projection repairs update one source, but still deserialize the packed intent.

UX gaps closed: compact count-slot progress, accepted Tags refresh before source completion, pre-existing merge destination memberships survive Undo refresh, and editor reads receive real durable revisions.
UX gaps left: mixed-source Undo intent, the already-open-room timing bound, and the missing complete screenshot walkthrough.

Decisions: reuse Root's two syncfs barriers, rather than directory-only fsync; keep exact byte journals before publication; finish an editor's requested source before returning its ETag; serialize a cancelled accepted intent until Retry/Undo resolves it; scope incremental projection repair to one path. WebDAV/CalDAV can briefly see old bytes, as recorded in DESIGN §2.

Remaining work / for the merge round or a continuation:

Diagnose the unchanged collaboration ten-second publication-hook failure; do not weaken its assertion. Run cargo test -p calternal-collab -- --test-threads=4 after the fix.
Finish cargo test -p calternal-server -- --test-threads=4.
Implement durable inverse acceptance for structured sources and verify live room visibility timing.
Build cargo build --release -p calternal-server -p calternal-tags --bin calternal-server --example tag_rewrite_1186_seed here. Use the private fixture setup and locked run prototypes under artifacts/tagfast/; never print their credential file. Run SSD and HDD phases only through perf-run calternal tagfast 20 -- <command>, with HDD using bench/hdd-emu.sh run-limited. Record five samples per operation and actual browser click p95.
Rebuild the SPA and run CALTERNAL_SERVER_BIN=<release binary> TAG_REWRITE_SEED_BIN=<release seeder> TAG_REWRITE_SCREENSHOTS_DIR=artifacts/tagfast TMPDIR=<worktree>/target/tmp node apps/web/e2e/tag-rewrite-1186.mjs. Review real during/after evidence at 390/820/1440, light/dark, macOS. A fast worker may finish before a new review context opens; repeat real operations as needed to capture each active phase.
Merge round: tests/adversarial/run.sh against the combined real server must prove malformed/oversized input, authorization, quota and cross-plugin behaviour. Full adversarial matrices were not run in this performance job, per the verification policy.

Cleanup: own remaining server build work was stopped. Cargo clean and deletion of the web build output are running; their final result is appended below.

#1186 tagfast: NOT READY FOR MERGE Head: `46037c0fb` on `job/tagperf-1186`. Started from `aca393266`. Fetched and merged `origin/dev` once; it was already up to date. No push or deploy. The job reached its four-hour limit; the remaining verification is listed below. Built: - One FULL SQLite acceptance transaction stores the source witnesses, queue receipt and Tag/Notes membership projection before file work. - Bounded source replay, exact Markdown Undo, restart recovery, stable Note identity/path resolution and protection of newer edits. - On-demand editor reads and new collaboration room baselines adopt pending source changes under the User writer guard. - Incremental Note callbacks restore only their own membership, rather than updating the complete intent after every Note. Inverse parsing precedes the SQLite writer transaction. - Compact sidebar progress, immediate accepted membership refresh, and separate acceptance/Index/durability/Undo benchmark fields. - Production screenshot harness coverage for Tags, sidebar and Notes at the owner widths and themes, with macOS emulation. This harness was extended but not run against the new release binary. Files: `crates/calternal-tags/src/{lib,index,rewrite}.rs`, `crates/calternal-tags/migrations/0007_tag_rewrite_acceptance.sql`, `crates/plugins/notes/src/{lib,store}.rs`, `crates/calternal-collab/src/session.rs`, `apps/web/src/lib/components/TagRow.svelte`, `apps/web/src/lib/stores/tagRewrite.svelte.ts`, its `.test.ts`, `apps/web/e2e/tag-rewrite-1186.mjs`, `bench/tag-rewrite-1186.py`, `contracts/perf/{registry,exceptions,adoption-1058,ratchet}.json`, `docs/DESIGN.md`, `docs/perf/2026-10-07-tag-rewrite-1186.md`. Gate output, verbatim result lines (full logs remain under `artifacts/tagfast/`): `cd apps/web && bun run build` — passed: ``` ✓ built in 7m 10s ✓ built in 97ms ✓ built in 13m 10s ``` `cargo fmt --check` — exit 0, no output. Tags clippy: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.98s ``` Tags full test, with the shared compiler wrapper bypassed after its deleted-temp-directory failure: ``` test result: ok. 63 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 787.30s ``` Notes clippy: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 27m 02s ``` Notes full unit and integration tests: ``` test result: ok. 317 passed; 0 failed; 4 ignored; 0 measured; 0 filtered out; finished in 1908.08s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 10.99s ``` Collaboration clippy: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 16s ``` Collaboration full test — FAILED: ``` test result: FAILED. 133 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 441.53s ``` The failure is `session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing`, waiting ten seconds for the Tag writer publication hook. A one-thread focused retry also timed out. No expectation was changed: ``` test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 133 filtered out; finished in 70.19s ``` Related source/typing integration tests pass: ``` test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 126 filtered out; finished in 34.75s ``` Server clippy: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 46s ``` Server test: interrupted during compilation at the job limit; no passing result. `bun run check` — exit 0: ``` perf-lint: PASS; 0 violations; 22532 scoped exceptions svelte-check found 0 errors and 1 warning in 1 file ``` The warning is the existing empty `.attachment-row:focus` ruleset in AttachmentDeck.svelte. The exact performance ratchet decreased from 22534 to 22532; no threshold increased. `bun run test --maxWorkers=2 --testTimeout=30000` — passed: ``` Test Files 287 passed (287) Tests 2039 passed (2039) Start at 19:08:14 Duration 2509.80s (transform 52%, import 16%, environment 14%, tests 12%, setup 5%) ``` The initial runs hit existing five-second timeouts under host load around 80. The final command changes the execution timeout only; assertions and existing expectations remain unchanged. Correctness coverage: actual SIGKILL at journal and durable-publication boundaries, a randomized interruption window during application, old-or-new file checks, restart/resume, exact Undo, newer edits, moved/replaced Notes, an exact one-byte quota refusal, and viewer Share isolation. The read-only review found no crash/data-loss/security blocker in the recovery diff. Performance and known gaps: - No new release measurement was completed. The release build remained cold while crate gates consumed the job budget. The previous VM medians remain 44.798 s Rename, 30.337 s Merge, and other-request p95 312.22/502.81 ms. Those are the previous worker, with storage unspecified here; they are not SSD results for this implementation. - The 500 ms click p95, 2 s SSD durability, 50 ms other-request p95 and corresponding Undo targets are UNVERIFIED. No HDD run occurred. The VM was free when checked; lack of the new binary prevented a measured run. - Undo with any structured Sidecar still uses the legacy queue path. Its immediate inverse Index intent is not implemented. This is a gap against the complete design. - Already-open Note rooms update when their source batch publishes; their 500 ms bound is not established. New reads/rooms adopt pending content on demand. - The existing Root quota reservation still walks the Home once per batch, including on-demand publication. Larger-Home cost remains unmeasured; removing that scan needs a safe quota accounting follow-up. - No screenshot set or icon alignment evidence was captured. No visual quality claim is made. - The repeated collaboration publication-hook timeout remains unresolved; it is not a passing gate. Expected cost: Rename acceptance handles 1,000 Index witnesses in one transaction with no Markdown reads. Source publication handles 1,000 Notes in 16 batches of 64, with at most eight independent file tasks. Undo reads the exact byte journals with the same concurrency bound before accepting its inverse. Per-Note projection repairs update one source, but still deserialize the packed intent. UX gaps closed: compact count-slot progress, accepted Tags refresh before source completion, pre-existing merge destination memberships survive Undo refresh, and editor reads receive real durable revisions. UX gaps left: mixed-source Undo intent, the already-open-room timing bound, and the missing complete screenshot walkthrough. Decisions: reuse Root's two syncfs barriers, rather than directory-only fsync; keep exact byte journals before publication; finish an editor's requested source before returning its ETag; serialize a cancelled accepted intent until Retry/Undo resolves it; scope incremental projection repair to one path. WebDAV/CalDAV can briefly see old bytes, as recorded in DESIGN §2. Remaining work / for the merge round or a continuation: 1. Diagnose the unchanged collaboration ten-second publication-hook failure; do not weaken its assertion. Run `cargo test -p calternal-collab -- --test-threads=4` after the fix. 2. Finish `cargo test -p calternal-server -- --test-threads=4`. 3. Implement durable inverse acceptance for structured sources and verify live room visibility timing. 4. Build `cargo build --release -p calternal-server -p calternal-tags --bin calternal-server --example tag_rewrite_1186_seed` here. Use the private fixture setup and locked run prototypes under `artifacts/tagfast/`; never print their credential file. Run SSD and HDD phases only through `perf-run calternal tagfast 20 -- <command>`, with HDD using `bench/hdd-emu.sh run-limited`. Record five samples per operation and actual browser click p95. 5. Rebuild the SPA and run `CALTERNAL_SERVER_BIN=<release binary> TAG_REWRITE_SEED_BIN=<release seeder> TAG_REWRITE_SCREENSHOTS_DIR=artifacts/tagfast TMPDIR=<worktree>/target/tmp node apps/web/e2e/tag-rewrite-1186.mjs`. Review real during/after evidence at 390/820/1440, light/dark, macOS. A fast worker may finish before a new review context opens; repeat real operations as needed to capture each active phase. 6. Merge round: `tests/adversarial/run.sh` against the combined real server must prove malformed/oversized input, authorization, quota and cross-plugin behaviour. Full adversarial matrices were not run in this performance job, per the verification policy. Cleanup: own remaining server build work was stopped. Cargo clean and deletion of the web build output are running; their final result is appended below.

kayg commented

2026-10-07 18:45:12 +00:00

Author

Owner

#1186 final cleanup: head 46037c0fbba8e4c1186f57e7243d20ef6c4ba373, clean worktree. Own server compilation was stopped; web build output was removed. Cargo clean output, verbatim:

     Removed 31225 files, 24.8GiB total

The final report above remains NOT READY FOR MERGE. Performance targets, screenshot evidence, complete server tests, mixed-source inverse intent and the collaboration publication-hook timeout remain outstanding. No push or deploy.

#1186 final cleanup: head `46037c0fbba8e4c1186f57e7243d20ef6c4ba373`, clean worktree. Own server compilation was stopped; web build output was removed. Cargo clean output, verbatim: ``` Removed 31225 files, 24.8GiB total ``` The final report above remains **NOT READY FOR MERGE**. Performance targets, screenshot evidence, complete server tests, mixed-source inverse intent and the collaboration publication-hook timeout remain outstanding. No push or deploy.

kayg commented

2026-10-08 02:02:46 +00:00

Author

Owner

Continuing tagfast2 on job/tagperf-1186 at 46037c0fbb. Read the contracts and predecessor report. First checking the unchanged concurrent-typing publication regression against origin/dev; then removing repeated quota walks before release measurements. No pushes or deployment.

Continuing tagfast2 on job/tagperf-1186 at 46037c0fbba8e4c1186f57e7243d20ef6c4ba373. Read the contracts and predecessor report. First checking the unchanged concurrent-typing publication regression against origin/dev; then removing repeated quota walks before release measurements. No pushes or deployment.

kayg commented

2026-10-08 02:09:16 +00:00

Author

Owner

Finding: reserve_batch_write_quota scans Home once per batch. The continuation now prepares an intent-wide source byte allowance from candidate sources only (bounded reads) and exchanges its reservation for actual bytes after each durable commit. This keeps unrelated writes quota-aware between batches. Regression coverage is being added for partial commits and exhausted allowances. The required origin/dev merge is committed at 5e2ee2aa6; it preserves both sidebar context menus (#1222) and rewrite progress. Exact performance pins refresh to 22532, with no threshold increase.

Finding: reserve_batch_write_quota scans Home once per batch. The continuation now prepares an intent-wide source byte allowance from candidate sources only (bounded reads) and exchanges its reservation for actual bytes after each durable commit. This keeps unrelated writes quota-aware between batches. Regression coverage is being added for partial commits and exhausted allowances. The required origin/dev merge is committed at 5e2ee2aa6; it preserves both sidebar context menus (#1222) and rewrite progress. Exact performance pins refresh to 22532, with no threshold increase.

kayg commented

2026-10-08 02:10:16 +00:00

Author

Owner

Blocker isolation: the unchanged each_source_writer_keeps_concurrent_typing test PASSES on the origin/dev source snapshot (all seven writer families, including Tag rename and Undo). Result: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out. The branch reproduction is compiling. The branch uses two filesystem-wide syncfs barriers even for a singleton Tag edit, unlike the baseline checked per-file writer; that is the leading publication-delay suspect, to be confirmed by the unchanged test.

kayg commented

2026-10-08 02:13:50 +00:00

Author

Owner

The current continuation test also passes unchanged (14.72 s), so the predecessor timeout did not reproduce under this host load. The source snapshot passes in 13.47 s. I am reducing the singleton writer’s filesystem-wide flush coupling, without moving the publication hook or changing assertions/timeouts, then running the full collaboration suite. Filesystem quota gates have passed Clippy and unit tests (101 passed, 1 ignored); integration tests are finishing.

kayg commented

2026-10-08 02:33:32 +00:00

Author

Owner

Quota slice committed at bc25beff6. All 63 Tags tests pass unchanged, including mixed Markdown/XMP/folder sources, legacy partial journals, restart, moved identities and exact one-byte quota refusal. Filesystem gates: 102 unit tests pass (1 ignored), plus integration groups 1/1/53/2 pass. Singleton crash injection passes at every journal step, preserving source and Version. The full collaboration run has passed the unchanged concurrent-typing test; its 96-case writer-lock matrix is still running. Web full suite: 297 passed files, 2119 passed tests.

Quota slice committed at bc25beff6. All 63 Tags tests pass unchanged, including mixed Markdown/XMP/folder sources, legacy partial journals, restart, moved identities and exact one-byte quota refusal. Filesystem gates: 102 unit tests pass (1 ignored), plus integration groups 1/1/53/2 pass. Singleton crash injection passes at every journal step, preserving source and Version. The full collaboration run has passed the unchanged concurrent-typing test; its 96-case writer-lock matrix is still running. Web full suite: 297 passed files, 2119 passed tests.

kayg commented

2026-10-08 02:49:47 +00:00

Author

Owner

Finding: the server upgrade test fails only because its exact pending frontier omits calternal-tags migration 0007 added by this issue. The focused failure lists identical left/right frontiers except that one entry. I am adding that receipt to the pending and final-frontier pins; deployed SQL, original receipt equality, data-preservation assertions and timeouts stay unchanged.

kayg commented

2026-10-08 02:52:46 +00:00

Author

Owner

Server gate evidence: initial full suite completed (270 passed, 2 failed, 12 ignored). One failure is the exact Tags migration pin, now corrected. The other is SLOW-only: startup_search_reconciliation_keeps_calendar_writes_responsive_on_large_home reported Calendar write p95=17496 ms versus its unchanged 10000 ms budget during a 20000-file startup. No assertion or timeout was weakened. Final clippy and the focused migration regression are running; the final full run follows.

kayg commented

2026-10-08 03:25:27 +00:00

Author

Owner

Progress at 2b42e12d6: final calternal-server suite completed successfully: test result: ok. 272 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 339.53s; integration groups each passed (37.94s and 0.37s). Source quota admission is now once per worker attempt. Important remaining limitation: Home byte journals/checkpoints still go through ordinary Root writes, whose quota checks scan the Home; these are still per-batch O(Home) costs and must not be described as eliminated. VM measurements are queued through perf-run; no unlocked measurements or builds ran there.

Progress at 2b42e12d6: final calternal-server suite completed successfully: test result: ok. 272 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 339.53s; integration groups each passed (37.94s and 0.37s). Source quota admission is now once per worker attempt. Important remaining limitation: Home byte journals/checkpoints still go through ordinary Root writes, whose quota checks scan the Home; these are still per-batch O(Home) costs and must not be described as eliminated. VM measurements are queued through perf-run; no unlocked measurements or builds ran there.

kayg commented

2026-10-08 03:59:14 +00:00

Author

Owner

VM continuation finding: the first paired 1,000-Note production-SPA sample completed on the branch in 22,887.9 ms, with first paint at 900.3 ms and concurrent Tag-list p95 35.09 ms (384 reads, no errors). The shared origin/dev release did not finish within the 600 s cap and had no renamed Tag paint within 5 s. These are one sample per variant, not the required three-sample result. The source reservation is now once per worker attempt, but ordinary durable Tag journal/progress writes still invoke Root Home quota checks. This remaining O(Home) work and the missed paint/durability targets keep the branch NOT READY. Browser review also found that an initially hidden item-menu surface loses synchronous keyboard focus inside the phone Navigation dialog; a shared Menu focus option now uses the existing post-placement focus path. Focused tests passed; production-build review is next.

kayg commented

2026-10-08 04:29:44 +00:00

Author

Owner

Finding from the #1374 rerun, branch head 94b056bfe9b49d24aee6da2412a88dd9f955f935, measured on the perf VM with the shared release server from d395b6e25:

Renaming the Tag across 100 Notes did not acknowledge or become visible before the 120,000 ms harness timeout. The durable lower bound is 120,186.76 ms, against the 500 ms visibility target. The fixture had 100 tagged Notes. This is not a successful completion time. The call path is crates/calternal-tags/src/lib.rs::rename_tag into the rewrite worker in crates/calternal-tags/src/rewrite.rs; this confirms the existing #1186 cause, so I am not filing a duplicate.

Finding from the #1374 rerun, branch head `94b056bfe9b49d24aee6da2412a88dd9f955f935`, measured on the perf VM with the shared release server from `d395b6e25`: Renaming the Tag across 100 Notes did not acknowledge or become visible before the 120,000 ms harness timeout. The durable lower bound is 120,186.76 ms, against the 500 ms visibility target. The fixture had 100 tagged Notes. This is not a successful completion time. The call path is `crates/calternal-tags/src/lib.rs::rename_tag` into the rewrite worker in `crates/calternal-tags/src/rewrite.rs`; this confirms the existing #1186 cause, so I am not filing a duplicate.

kayg commented

2026-10-08 04:30:52 +00:00

Author

Owner

Quota continuation: commit 45ec97eaf adds Root::write_from_batch_allowance. Normal file/Version durability remains; journal bytes debit the admitted ledger without walking the Home. The full calternal-fs suite passes (104 unit tests, 1 ignored, plus integrations). New tests cover staged-source budget preservation, oversized journal refusal before install, allowance exhaustion, cross-Home rejection and a reduced quota. The Tags worker now admits candidate source bytes plus serialized journal/checkpoint bounds once, before its first batch write. An initial combined admission changed the projected value in accepted_quota_refusal_is_exact_and_preserves_sources; the assertion was not changed. The implementation now retains the exact source-only refusal amount when sources alone exceed quota, using the same admission measurement. Its focused retry passes (1 passed; 62 filtered; 7.39 s). Full Tags/server gates are running. The 1,000-Note VM profile sampled SQLite VM execution (6.95%) and ONNX GEMM (6.54%) as the largest leaf symbols; background work is part of this fixture. The 5,000-Note phase is queued behind another project job that declared a 50-minute VM lease.

Quota continuation: commit 45ec97eaf adds Root::write_from_batch_allowance. Normal file/Version durability remains; journal bytes debit the admitted ledger without walking the Home. The full calternal-fs suite passes (104 unit tests, 1 ignored, plus integrations). New tests cover staged-source budget preservation, oversized journal refusal before install, allowance exhaustion, cross-Home rejection and a reduced quota. The Tags worker now admits candidate source bytes plus serialized journal/checkpoint bounds once, before its first batch write. An initial combined admission changed the projected value in accepted_quota_refusal_is_exact_and_preserves_sources; the assertion was not changed. The implementation now retains the exact source-only refusal amount when sources alone exceed quota, using the same admission measurement. Its focused retry passes (1 passed; 62 filtered; 7.39 s). Full Tags/server gates are running. The 1,000-Note VM profile sampled SQLite VM execution (6.95%) and ONNX GEMM (6.54%) as the largest leaf symbols; background work is part of this fixture. The 5,000-Note phase is queued behind another project job that declared a 50-minute VM lease.

kayg commented

2026-10-08 04:32:08 +00:00

Author

Owner

Correction to my previous comment, branch head 95223dde0194a91a9b7783f4e814e1943e3384ce: the rename request returned HTTP 200 and was acknowledged in 382.30 ms. The visibility poll did not observe the new Tag before its 120,186.76 ms deadline. The rewrite status returned completed in the following completion check, but that check did not record an exact visible-state time. The measured visibility is therefore a lower bound, not an unacknowledged request.

The checkpoint row's action_acknowledged:false was a harness bug: the timeout callback hard-coded that field. I fixed the runner so future rows record whether the request returned (commit 95223dde0). This already-running process loaded the earlier code. Its summary row records the HTTP 200 and 382.30 ms response time; the final CSV will include an explicit correction row.

Correction to my previous comment, branch head `95223dde0194a91a9b7783f4e814e1943e3384ce`: the rename request returned HTTP 200 and was acknowledged in 382.30 ms. The visibility poll did not observe the new Tag before its 120,186.76 ms deadline. The rewrite status returned `completed` in the following completion check, but that check did not record an exact visible-state time. The measured visibility is therefore a lower bound, not an unacknowledged request. The checkpoint row's `action_acknowledged:false` was a harness bug: the timeout callback hard-coded that field. I fixed the runner so future rows record whether the request returned (commit `95223dde0`). This already-running process loaded the earlier code. Its summary row records the HTTP 200 and 382.30 ms response time; the final CSV will include an explicit correction row.

kayg commented

2026-10-08 05:06:29 +00:00

Author

Owner

Final full server gate completed and did not pass. The unchanged 20,000-file Calendar write profile failed in its isolated subprocess:

search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12260 p95_ms=12541 p95_budget_ms=10000
test result: FAILED. 271 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 195.25s

A focused retry with the documented SERVER_WORKER_STACK_SIZE=4 MiB also failed only its latency budget:

search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=10012 p95_ms=10391 p95_budget_ms=10000
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 283 filtered out; finished in 52.93s

One direct diagnostic invocation initially omitted RUST_MIN_STACK and overflowed the default 2 MiB test stack. wire.rs documents that deep live-app future and sets 4 MiB in the normal isolated runner (#549). The retry used that existing configuration and did not crash. No assertion, fixture or timeout was changed. This is SLOW evidence, not a clean final server gate.

The final web suite passed:

 Test Files  297 passed (297)
      Tests  2124 passed (2124)
   Duration  517.08s (transform 36%, environment 22%, import 21%, tests 16%, setup 5%)

Final full server gate completed and did not pass. The unchanged 20,000-file Calendar write profile failed in its isolated subprocess: ```text search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12260 p95_ms=12541 p95_budget_ms=10000 test result: FAILED. 271 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 195.25s ``` A focused retry with the documented SERVER_WORKER_STACK_SIZE=4 MiB also failed only its latency budget: ```text search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=10012 p95_ms=10391 p95_budget_ms=10000 test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 283 filtered out; finished in 52.93s ``` One direct diagnostic invocation initially omitted RUST_MIN_STACK and overflowed the default 2 MiB test stack. wire.rs documents that deep live-app future and sets 4 MiB in the normal isolated runner (#549). The retry used that existing configuration and did not crash. No assertion, fixture or timeout was changed. This is SLOW evidence, not a clean final server gate. The final web suite passed: ```text Test Files 297 passed (297) Tests 2124 passed (2124) Duration 517.08s (transform 36%, environment 22%, import 21%, tests 16%, setup 5%) ```

kayg commented

2026-10-08 05:08:18 +00:00

Author

Owner

The final focused concurrent-typing publication test passed after the shared quota change, with its assertion and timeout unchanged:

test session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 15.37s

The same test passed on the origin/dev snapshot (13.47 s) and on the original job head before my changes (14.72 s). The earlier failure was not reproduced. The complete collab suite passed earlier in this continuation; the final shared-FS change was followed by this focused publication regression. No claim is made that host I/O coupling caused the predecessor failure.

Performance lint now reports:

perf-lint: PASS; 0 violations; 22532 scoped exceptions

Commit 7d7841ba8 refreshes exact live hashes and synchronizes existing #1058 adoption entries. The ratchet count is unchanged.

The final focused concurrent-typing publication test passed after the shared quota change, with its assertion and timeout unchanged: ```text test session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 15.37s ``` The same test passed on the origin/dev snapshot (13.47 s) and on the original job head before my changes (14.72 s). The earlier failure was not reproduced. The complete collab suite passed earlier in this continuation; the final shared-FS change was followed by this focused publication regression. No claim is made that host I/O coupling caused the predecessor failure. Performance lint now reports: ```text perf-lint: PASS; 0 violations; 22532 scoped exceptions ``` Commit 7d7841ba8 refreshes exact live hashes and synchronizes existing #1058 adoption entries. The ratchet count is unchanged.

kayg commented

2026-10-08 05:34:33 +00:00

Author

Owner

The final quota binary is now being measured under perf-run calternal on the perf VM. Its SHA-256 is 50bb7fc56527c88f3edda5d0d06f2a620e2d2c85ee97ee25e12dbc89cb25adea. The common saved production SPA includes early validated Rename projection. Its later sidebar focus-only fix is reviewed separately in production screenshots.

First final 1,000-Note sample (99 Hz cycles profiling enabled):

Browser paint: 102.8 ms.
Durable source completion: 30,364.89 ms; all 1,000 source files verified.
Other Tag-list reads: p95 43.36 ms, no errors.
CPU: 21.75 seconds; peak RSS: 287,318,016 bytes.

This sample meets the paint and Tag-list p95 targets. It misses the 2-second durability target. The fixture has a cold Notes Index and startup work overlaps the rewrite; this is not an all-route steady-state claim.

First final 5,000-Note sample: paint 110 ms, durable receipt incomplete at 120,002.61 ms, Tag-list read p95 146 ms, peak RSS 373,035,008 bytes. The incomplete time is a lower bound.

The second profile has ONNX matrix work at 8.23%, sqlite3VdbeExec at 6.15%, malloc at 2.17% and SQLite JSON translation at 2.09% of leaf samples. These samples cannot separate the rewrite from startup work. No third optimization/measurement round is started. The remaining paired samples are running in the same locked phase. Status remains NOT READY because the owner durability target is not met.

The final quota binary is now being measured under perf-run calternal on the perf VM. Its SHA-256 is 50bb7fc56527c88f3edda5d0d06f2a620e2d2c85ee97ee25e12dbc89cb25adea. The common saved production SPA includes early validated Rename projection. Its later sidebar focus-only fix is reviewed separately in production screenshots. First final 1,000-Note sample (99 Hz cycles profiling enabled): - Browser paint: 102.8 ms. - Durable source completion: 30,364.89 ms; all 1,000 source files verified. - Other Tag-list reads: p95 43.36 ms, no errors. - CPU: 21.75 seconds; peak RSS: 287,318,016 bytes. This sample meets the paint and Tag-list p95 targets. It misses the 2-second durability target. The fixture has a cold Notes Index and startup work overlaps the rewrite; this is not an all-route steady-state claim. First final 5,000-Note sample: paint 110 ms, durable receipt incomplete at 120,002.61 ms, Tag-list read p95 146 ms, peak RSS 373,035,008 bytes. The incomplete time is a lower bound. The second profile has ONNX matrix work at 8.23%, sqlite3VdbeExec at 6.15%, malloc at 2.17% and SQLite JSON translation at 2.09% of leaf samples. These samples cannot separate the rewrite from startup work. No third optimization/measurement round is started. The remaining paired samples are running in the same locked phase. Status remains NOT READY because the owner durability target is not met.

kayg commented

2026-10-08 06:00:38 +00:00

Author

Owner

NOT READY — a3db565a48b148f79f5dcee3b50fa721a24259f9

Built

Reuse one worker admission for source bytes, serialized journals and bounded checkpoints. Each durable batch consumes that allowance. Worker journal/checkpoint writes no longer scan the Home. Root identity, staged budgets and reduced quotas remain checked.
Use scoped file/parent durability for singleton batches; retain bounded filesystem barriers and replay journals for multi-file batches. New regression tests cover allowance exhaustion, pending sources, cross-Home use, quota shrink and singleton crash recovery.
Paint a validated Rename before its acceptance response. Roll back only that attempt on failure. Reuse the shared focus helper, Menu/FloatingSurface and ItemActionMenu. Keep menus inside Navigation focus scope and stop late Calendar focus restores. Restore Rename focus through the stable section/list after keyed wrappers disappear.
Extend the existing profile with production-SPA paint, disk source checks, capped-run evidence and saved asset selection. Fix fixture migration setup and deployed Tags migration receipt pins.

Files: crates/calternal-fs/src/{root,quota,write}.rs; crates/calternal-tags/src/rewrite.rs and its existing seeder; crates/calternal-server/src/upgrade_tests.rs; TagRow, ItemActionMenu, inputModality and tagRewrite store/tests in apps/web/src/lib; shared TimeGrid, Menu and FloatingSurface in packages/ui; apps/web/e2e/tag-rewrite-1186.mjs; bench/tag-rewrite-1186{.py,-browser.mjs}; exact performance contracts; DESIGN durability notes; docs/perf/2026-10-08-tagfast2-1186.md and both raw round profiles.

Shared controls used: existing TagRow inline field/count slot, ItemActionMenu, Menu/MenuItem, FloatingSurface, ConfirmSheet, Pill and the shared focus helper. No new visual style was added in this continuation.

Measurements

Final release SHA-256: 50bb7fc56527c88f3edda5d0d06f2a620e2d2c85ee97ee25e12dbc89cb25adea. Built locally; copied to the perf VM. Every measurement ran under perf-run calternal. Load averages are in the committed raw profiles. No build ran on that VM.

Final size	Browser paint p50 / p95	Durable p50 / p95	Concurrent Tag-list p95 by run	Largest RSS
1,000 Notes	114.4 / 117.4 ms	19,828.70 / 30,364.89 ms	43.36 / 29.53 / 79.97 ms	282.85 MiB
5,000 Notes	110.0 / 112.3 ms	One completed at 78,952.12 ms; two exceeded 120 s	146.00 / 145.55 / 131.55 ms	365.30 MiB

Operation p95 uses nearest rank across three samples; it is the largest sample. All concurrent reads had zero errors. Completed operations verified every source file. Capped times remain lower bounds, not completed samples.

The earlier shared-release baseline had three incomplete 1,000-Note runs and three incomplete 5,000-Note runs. First samples had 600 s caps; later ones had 120 s caps. The initial 1,000-Note baseline/job pairs were interleaved. Final 1,000-Note reruns preceded the final 5,000-Note job samples. The first 5,000-Note baseline used the older SPA; later baseline/job samples shared the saved final SPA. That SPA includes early optimism and menu focus fixes, but precedes the final sidebar wrapper-only focus fix shown in the attached captures.

The fixture has a cold Notes Index and startup work overlaps the rewrite. Sixteen readers send one Tag-list request per second each. Other route classes, a request storm and an already indexed Home were not measured. The generic Tag-route number in docs/perf/baseline.json is a different workload. No completed baseline median or speedup is claimed.

The second profile has ONNX matrix work at 8.23%, SQLite VM work at 6.15%, malloc at 2.17% and SQLite JSON translation at 2.09% of leaf samples. It does not isolate the rewrite from startup work. No third optimization round was started.

Expected worker cost: one O(Home) admission walk per attempt, O(N) candidate reads with eight workers, and ceil(N / 64) source batches (16 / 79 here). Ordinary acceptance writes retain their existing checked quota path.

Publication failure

The exact concurrent-typing test passed on origin/dev (13.47 s), the starting job head (14.72 s), and after the final shared-FS change (15.37 s), with the assertion and timeout unchanged. The complete collab suite passed earlier. The predecessor failure did not reproduce; no root cause is claimed.

UX gaps closed

Phone keyboard menus now stay inside Navigation and get focus after placement.
Mouse and opening-shortcut Rename fields have no programmatic ring.
Late Calendar restores no longer steal focus from menus/dialogs.
A keyed Rename no longer moves phone focus to Search and leaves a ring.
Rename projects across existing Tag surfaces before acceptance and rolls back its own failure.
Review captures dismiss the real update notice before Navigation makes outside toasts inert.

Known gaps / UX gaps left

The 2-second durability target is not met. The 50 ms concurrent-read target is not met consistently at 1,000 Notes and fails at 5,000 Notes.
The full original Notes editor + standard visible Undo flow did not finish before the four-hour cap. Six sidebar Rename operations reached completed receipts and produced all 24 requested captures. Five review resets completed through exact server Undo; the last reset/full remaining flow was stopped at the cap. These captures are not a passing full end-to-end suite.
No fresh final 1,000-Note baseline pairs, mixed-route storm, large unrelated Home, large source bodies or native Mac interop were run. Measurement limitations are recorded above.
The final full server gate failed its unchanged 20,000-file Calendar latency budget. A correct-stack focused retry also failed only that budget. No assertions or timeouts were relaxed.

Decisions

Use conservative serialized journal/checkpoint bounds in the existing allowance. Fresh worker attempts re-admit from remaining sources and saved journals. Report a source-only quota refusal exactly when source growth alone exceeds the limit; keep the full refusal when metadata alone makes it exceed the limit.
Use scoped singleton barriers under DESIGN §2; multi-file barriers remain intact.
Track pre-acceptance UI projection ownership separately from the durable receipt. Persist progress/Undo only after acceptance.
Extend shared menu placement/focus options and guard late Calendar focus rather than add local styling or keyboard motion branches.
Retain right-censored benchmarks and skip Undo only for fresh one-sample profiling fixtures. Do not infer completed medians from caps.
Update the deployed Tags migration frontier to 7 and total receipts to 25 because this issue adds that migration. These are the only old expected values changed; typing/performance assertions and timeouts are unchanged.

Gates (verbatim excerpts)

cargo fmt --check: exit 0, no output. Full logs as run are attached in gates-verbatim.log.gz, including the exact command, complete output and stage notes.

cargo clippy -p calternal-fs --all-targets -- -D warnings; cargo test -p calternal-fs:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.82s
test result: ok. 104 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 71.03s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.17s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.87s
test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 48.21s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo clippy -p calternal-tags --all-targets -- -D warnings; cargo test -p calternal-tags:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.02s
test result: ok. 63 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 212.62s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo clippy -p calternal-collab --all-targets -- -D warnings; cargo test -p calternal-collab:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 29s
test result: ok. 134 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 416.33s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.43s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 21.26s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.35s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 77.00s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.16s
test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.44s
test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 11.78s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.39s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.43s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 11.59s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.53s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 89.38s
test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.61s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 16.05s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo clippy -p calternal-server --all-targets -- -D warnings; cargo test -p calternal-server -- --test-threads=4:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 43s
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 283 filtered out; finished in 36.56s
test result: FAILED. 271 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 195.25s

The final collab focused regression:

test session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 15.37s

Final cd apps/web && bun run check:

perf-lint: PASS; 0 violations; 22532 scoped exceptions
svelte-check found 0 errors and 4 warnings in 2 files

Complete web suite before the final wrapper-only focus fix, then that fix's focused regression:

 Test Files  297 passed (297)
      Tests  2124 passed (2124)
   Duration  517.08s (transform 36%, environment 22%, import 21%, tests 16%, setup 5%)
 Test Files  1 passed (1)
      Tests  8 passed (8)
   Duration  25.88s (transform 80%, import 7%, tests 6%, environment 5%, setup 2%)

Final production build:

Compressed 914 static variants; saved 21514803 bytes.

Scoped Notes adversarial round and final-release Tags boundaries:

Cross-User classification gate: 429 operations classified
Generated entry point classification: 1206 tools classified
Admin coverage: 53 reviewed operations; contract and Rust guards agree
Notes exact-path lookup: 8 hostile paths, 16 concurrent reads, 0 findings
PASS durable Tag route boundaries

The full server gate's unchanged child profile reported:

search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12260 p95_ms=12541 p95_budget_ms=10000

The focused retry used the documented 4 MiB worker stack:

search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=10012 p95_ms=10391 p95_budget_ms=10000
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 283 filtered out; finished in 52.93s

One earlier direct diagnostic invocation omitted that stack setting and overflowed the default 2 MiB test stack. The normal subprocess runner already supplies 4 MiB; the correctly configured retry did not crash. The gate remains failed; this is not a passing result.

For the merge round / continuation

bash tests/adversarial/run.sh: full authz, concurrency, cross-plugin and robustness matrices.
cd apps/web && bun run test --maxWorkers=2: combined branch suite including the final wrapper regression.
node apps/web/e2e/tag-rewrite-1186.mjs: finish the real Notes/visible Undo path and comparisons with matching production server/web builds. Do not weaken its assertions or timeouts.
cargo test -p calternal-server -- --test-threads=4: resolve/verify the existing 20,000-file latency profile and complete the full server gate.
Continue #1186 with an already indexed Home, mixed requests and source/Index publication profiling. Preserve durable recovery and concurrent typing. The performance targets remain open.

Evidence attachments

All 24 sidebar PNGs use the actual production SPA and real server at 390/820/1440, light/dark, with macOS emulated. They cover mouse opening, Shift+F10 opening, real pending/running progress and completed count. No screenshot was committed. Claude/orchestrator remains the visual reviewer.

NOT READY — a3db565a48b148f79f5dcee3b50fa721a24259f9

NOT READY — `a3db565a48b148f79f5dcee3b50fa721a24259f9` ## Built - Reuse one worker admission for source bytes, serialized journals and bounded checkpoints. Each durable batch consumes that allowance. Worker journal/checkpoint writes no longer scan the Home. Root identity, staged budgets and reduced quotas remain checked. - Use scoped file/parent durability for singleton batches; retain bounded filesystem barriers and replay journals for multi-file batches. New regression tests cover allowance exhaustion, pending sources, cross-Home use, quota shrink and singleton crash recovery. - Paint a validated Rename before its acceptance response. Roll back only that attempt on failure. Reuse the shared focus helper, Menu/FloatingSurface and ItemActionMenu. Keep menus inside Navigation focus scope and stop late Calendar focus restores. Restore Rename focus through the stable section/list after keyed wrappers disappear. - Extend the existing profile with production-SPA paint, disk source checks, capped-run evidence and saved asset selection. Fix fixture migration setup and deployed Tags migration receipt pins. Files: `crates/calternal-fs/src/{root,quota,write}.rs`; `crates/calternal-tags/src/rewrite.rs` and its existing seeder; `crates/calternal-server/src/upgrade_tests.rs`; TagRow, ItemActionMenu, inputModality and tagRewrite store/tests in `apps/web/src/lib`; shared TimeGrid, Menu and FloatingSurface in `packages/ui`; `apps/web/e2e/tag-rewrite-1186.mjs`; `bench/tag-rewrite-1186{.py,-browser.mjs}`; exact performance contracts; DESIGN durability notes; `docs/perf/2026-10-08-tagfast2-1186.md` and both raw round profiles. Shared controls used: existing TagRow inline field/count slot, ItemActionMenu, Menu/MenuItem, FloatingSurface, ConfirmSheet, Pill and the shared focus helper. No new visual style was added in this continuation. ## Measurements Final release SHA-256: `50bb7fc56527c88f3edda5d0d06f2a620e2d2c85ee97ee25e12dbc89cb25adea`. Built locally; copied to the perf VM. Every measurement ran under `perf-run calternal`. Load averages are in the committed raw profiles. No build ran on that VM. | Final size | Browser paint p50 / p95 | Durable p50 / p95 | Concurrent Tag-list p95 by run | Largest RSS | | --- | --- | --- | --- | --- | | 1,000 Notes | 114.4 / 117.4 ms | 19,828.70 / 30,364.89 ms | 43.36 / 29.53 / 79.97 ms | 282.85 MiB | | 5,000 Notes | 110.0 / 112.3 ms | One completed at 78,952.12 ms; two exceeded 120 s | 146.00 / 145.55 / 131.55 ms | 365.30 MiB | Operation p95 uses nearest rank across three samples; it is the largest sample. All concurrent reads had zero errors. Completed operations verified every source file. Capped times remain lower bounds, not completed samples. The earlier shared-release baseline had three incomplete 1,000-Note runs and three incomplete 5,000-Note runs. First samples had 600 s caps; later ones had 120 s caps. The initial 1,000-Note baseline/job pairs were interleaved. Final 1,000-Note reruns preceded the final 5,000-Note job samples. The first 5,000-Note baseline used the older SPA; later baseline/job samples shared the saved final SPA. That SPA includes early optimism and menu focus fixes, but precedes the final sidebar wrapper-only focus fix shown in the attached captures. The fixture has a cold Notes Index and startup work overlaps the rewrite. Sixteen readers send one Tag-list request per second each. Other route classes, a request storm and an already indexed Home were not measured. The generic Tag-route number in `docs/perf/baseline.json` is a different workload. No completed baseline median or speedup is claimed. The second profile has ONNX matrix work at 8.23%, SQLite VM work at 6.15%, malloc at 2.17% and SQLite JSON translation at 2.09% of leaf samples. It does not isolate the rewrite from startup work. No third optimization round was started. Expected worker cost: one O(Home) admission walk per attempt, O(N) candidate reads with eight workers, and ceil(N / 64) source batches (16 / 79 here). Ordinary acceptance writes retain their existing checked quota path. ## Publication failure The exact concurrent-typing test passed on origin/dev (13.47 s), the starting job head (14.72 s), and after the final shared-FS change (15.37 s), with the assertion and timeout unchanged. The complete collab suite passed earlier. The predecessor failure did not reproduce; no root cause is claimed. ## UX gaps closed - Phone keyboard menus now stay inside Navigation and get focus after placement. - Mouse and opening-shortcut Rename fields have no programmatic ring. - Late Calendar restores no longer steal focus from menus/dialogs. - A keyed Rename no longer moves phone focus to Search and leaves a ring. - Rename projects across existing Tag surfaces before acceptance and rolls back its own failure. - Review captures dismiss the real update notice before Navigation makes outside toasts inert. ## Known gaps / UX gaps left - The 2-second durability target is not met. The 50 ms concurrent-read target is not met consistently at 1,000 Notes and fails at 5,000 Notes. - The full original Notes editor + standard visible Undo flow did not finish before the four-hour cap. Six sidebar Rename operations reached completed receipts and produced all 24 requested captures. Five review resets completed through exact server Undo; the last reset/full remaining flow was stopped at the cap. These captures are not a passing full end-to-end suite. - No fresh final 1,000-Note baseline pairs, mixed-route storm, large unrelated Home, large source bodies or native Mac interop were run. Measurement limitations are recorded above. - The final full server gate failed its unchanged 20,000-file Calendar latency budget. A correct-stack focused retry also failed only that budget. No assertions or timeouts were relaxed. ## Decisions - Use conservative serialized journal/checkpoint bounds in the existing allowance. Fresh worker attempts re-admit from remaining sources and saved journals. Report a source-only quota refusal exactly when source growth alone exceeds the limit; keep the full refusal when metadata alone makes it exceed the limit. - Use scoped singleton barriers under DESIGN §2; multi-file barriers remain intact. - Track pre-acceptance UI projection ownership separately from the durable receipt. Persist progress/Undo only after acceptance. - Extend shared menu placement/focus options and guard late Calendar focus rather than add local styling or keyboard motion branches. - Retain right-censored benchmarks and skip Undo only for fresh one-sample profiling fixtures. Do not infer completed medians from caps. - Update the deployed Tags migration frontier to 7 and total receipts to 25 because this issue adds that migration. These are the only old expected values changed; typing/performance assertions and timeouts are unchanged. ## Gates (verbatim excerpts) `cargo fmt --check`: exit 0, no output. Full logs as run are attached in `gates-verbatim.log.gz`, including the exact command, complete output and stage notes. `cargo clippy -p calternal-fs --all-targets -- -D warnings`; `cargo test -p calternal-fs`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.82s test result: ok. 104 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 71.03s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.17s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.87s test result: ok. 53 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 48.21s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo clippy -p calternal-tags --all-targets -- -D warnings`; `cargo test -p calternal-tags`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.02s test result: ok. 63 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 212.62s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo clippy -p calternal-collab --all-targets -- -D warnings`; `cargo test -p calternal-collab`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 29s test result: ok. 134 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 416.33s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.43s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 21.26s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.35s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 77.00s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.16s test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.44s test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 11.78s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.39s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.43s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 11.59s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.53s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 89.38s test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.08s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.61s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 16.05s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo clippy -p calternal-server --all-targets -- -D warnings`; `cargo test -p calternal-server -- --test-threads=4`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 43s test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 283 filtered out; finished in 36.56s test result: FAILED. 271 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 195.25s ``` The final collab focused regression: ```text test session::external_persistence_race_tests::each_source_writer_keeps_concurrent_typing ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 133 filtered out; finished in 15.37s ``` Final `cd apps/web && bun run check`: ```text perf-lint: PASS; 0 violations; 22532 scoped exceptions svelte-check found 0 errors and 4 warnings in 2 files ``` Complete web suite before the final wrapper-only focus fix, then that fix's focused regression: ```text Test Files 297 passed (297) Tests 2124 passed (2124) Duration 517.08s (transform 36%, environment 22%, import 21%, tests 16%, setup 5%) Test Files 1 passed (1) Tests 8 passed (8) Duration 25.88s (transform 80%, import 7%, tests 6%, environment 5%, setup 2%) ``` Final production build: ```text Compressed 914 static variants; saved 21514803 bytes. ``` Scoped Notes adversarial round and final-release Tags boundaries: ```text Cross-User classification gate: 429 operations classified Generated entry point classification: 1206 tools classified Admin coverage: 53 reviewed operations; contract and Rust guards agree Notes exact-path lookup: 8 hostile paths, 16 concurrent reads, 0 findings PASS durable Tag route boundaries ``` The full server gate's unchanged child profile reported: ```text search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12260 p95_ms=12541 p95_budget_ms=10000 ``` The focused retry used the documented 4 MiB worker stack: ```text search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=10012 p95_ms=10391 p95_budget_ms=10000 test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 283 filtered out; finished in 52.93s ``` One earlier direct diagnostic invocation omitted that stack setting and overflowed the default 2 MiB test stack. The normal subprocess runner already supplies 4 MiB; the correctly configured retry did not crash. The gate remains failed; this is not a passing result. ## For the merge round / continuation - `bash tests/adversarial/run.sh`: full authz, concurrency, cross-plugin and robustness matrices. - `cd apps/web && bun run test --maxWorkers=2`: combined branch suite including the final wrapper regression. - `node apps/web/e2e/tag-rewrite-1186.mjs`: finish the real Notes/visible Undo path and comparisons with matching production server/web builds. Do not weaken its assertions or timeouts. - `cargo test -p calternal-server -- --test-threads=4`: resolve/verify the existing 20,000-file latency profile and complete the full server gate. - Continue #1186 with an already indexed Home, mixed requests and source/Index publication profiling. Preserve durable recovery and concurrent typing. The performance targets remain open. ## Evidence attachments All 24 sidebar PNGs use the actual production SPA and real server at 390/820/1440, light/dark, with macOS emulated. They cover mouse opening, Shift+F10 opening, real pending/running progress and completed count. No screenshot was committed. Claude/orchestrator remains the visual reviewer. [sidebar-mouse-rename-desktop-dark-mac.png](https://git.kayg.org/attachments/1ab3d806-8517-4068-a1b9-9e1311688135) [sidebar-mouse-rename-desktop-light-mac.png](https://git.kayg.org/attachments/4fcd8268-6973-43d9-ba53-4319e5b15151) [sidebar-mouse-rename-phone-dark-mac.png](https://git.kayg.org/attachments/2dc6efd7-5709-4388-b484-26e71837885f) [sidebar-mouse-rename-phone-light-mac.png](https://git.kayg.org/attachments/d877ca0b-6f54-4b5c-ae72-cb87173197d7) [sidebar-mouse-rename-tablet-dark-mac.png](https://git.kayg.org/attachments/88321bca-6800-4f14-976a-aaca73be2f51) [sidebar-mouse-rename-tablet-light-mac.png](https://git.kayg.org/attachments/1fcdfe6d-0eb9-4e74-87de-1590c57a1a9f) [sidebar-shortcut-rename-desktop-dark-mac.png](https://git.kayg.org/attachments/b6398580-080f-49f9-b491-962ef5e83c4b) [sidebar-shortcut-rename-desktop-light-mac.png](https://git.kayg.org/attachments/73a58b40-7c03-4592-b7a9-dcaf94f234ff) [sidebar-shortcut-rename-phone-dark-mac.png](https://git.kayg.org/attachments/f7db073e-d8af-42a2-80be-7680423109aa) [sidebar-shortcut-rename-phone-light-mac.png](https://git.kayg.org/attachments/c73b9236-b779-4735-af0c-6220c2e85c36) [sidebar-shortcut-rename-tablet-dark-mac.png](https://git.kayg.org/attachments/04d12846-abd2-4e4b-bb5d-7bdb56da0ac9) [sidebar-shortcut-rename-tablet-light-mac.png](https://git.kayg.org/attachments/ae4ff39f-878e-47b8-978f-dfafa239e49b) [sidebar-progress-desktop-dark-mac.png](https://git.kayg.org/attachments/84b60081-63dc-456e-b821-b90a4df4e4da) [sidebar-progress-desktop-light-mac.png](https://git.kayg.org/attachments/cd59c506-22df-45b6-b67a-3dc61968ecf3) [sidebar-progress-phone-dark-mac.png](https://git.kayg.org/attachments/a48ab7b7-82b1-4285-be37-107f177a6867) [sidebar-progress-phone-light-mac.png](https://git.kayg.org/attachments/cdf0ca38-333a-47fe-b8c4-348f62005094) [sidebar-progress-tablet-dark-mac.png](https://git.kayg.org/attachments/99ac548d-1328-40af-888f-3c8c85f675f1) [sidebar-progress-tablet-light-mac.png](https://git.kayg.org/attachments/af990954-4a4b-4941-bb4e-430b409b53d6) [sidebar-done-desktop-light-mac.png](https://git.kayg.org/attachments/9963115e-2d1c-4de6-b2d6-cfb116a208c3) [sidebar-done-phone-dark-mac.png](https://git.kayg.org/attachments/9b565aa7-1ace-4812-9a95-96b89babf98c) [sidebar-done-phone-light-mac.png](https://git.kayg.org/attachments/299c62bb-a273-484a-9198-cd0eea238e25) [sidebar-done-tablet-dark-mac.png](https://git.kayg.org/attachments/2fd69082-f4aa-4c63-b8d9-3a7b43151106) [sidebar-done-tablet-light-mac.png](https://git.kayg.org/attachments/15294b59-9cc1-408c-8df5-92b68f43c165) [gates-verbatim.log.gz](https://git.kayg.org/attachments/37aed551-8b65-403e-8e69-6f6e663d5cd8) [sidebar-done-desktop-dark-mac.png](https://git.kayg.org/attachments/369162f6-484d-4874-90f9-d41cd40cf0aa) NOT READY — `a3db565a48b148f79f5dcee3b50fa721a24259f9`

kayg commented

2026-10-08 13:04:00 +00:00

Author

Owner

Independent read-only data-safety review started on job/tagperf-1186 at a3db565a48. Reviewing the diff and running independent real-server probes; no author reports or result files used. No tracked edits or ref changes. Evidence: artifacts/tagrev-findings.md.

Independent read-only data-safety review started on job/tagperf-1186 at a3db565a48b148f79f5dcee3b50fa721a24259f9. Reviewing the diff and running independent real-server probes; no author reports or result files used. No tracked edits or ref changes. Evidence: artifacts/tagrev-findings.md.

kayg commented

2026-10-08 13:53:14 +00:00

Author

Owner

Independent review progress: head a3db565a48 remains unchanged. The reviewed server build, production web build and cargo fmt --check passed. The first 14,010-Note fixture setup timed out before rename (120 s, then 1,200 s), so these runs are not rename evidence. The current probe seeds a 1,000-Note set and a 5,000-Note set, and reuses them with fresh byte baselines for crash and failure tests. About 4,000 Notes are indexed so far. No tracked edits, commits or ref changes. Evidence accumulates in artifacts/tagrev-findings.md. A Playwright timeout exposed a disposable test session in the first run; it was scrubbed from artifacts and its test Instance was deleted to invalidate it. No production credentials were involved.

Independent review progress: head a3db565a48b148f79f5dcee3b50fa721a24259f9 remains unchanged. The reviewed server build, production web build and cargo fmt --check passed. The first 14,010-Note fixture setup timed out before rename (120 s, then 1,200 s), so these runs are not rename evidence. The current probe seeds a 1,000-Note set and a 5,000-Note set, and reuses them with fresh byte baselines for crash and failure tests. About 4,000 Notes are indexed so far. No tracked edits, commits or ref changes. Evidence accumulates in artifacts/tagrev-findings.md. A Playwright timeout exposed a disposable test session in the first run; it was scrubbed from artifacts and its test Instance was deleted to invalidate it. No production credentials were involved.

kayg commented

2026-10-08 14:26:08 +00:00

Author

Owner

P1 — Accepted rename at quota blocks Note access and Undo

Evidence: TMPDIR=$PWD/target/tmp node artifacts/tagrev-quota.mjs against the reviewed server binary. Create 10 Notes tagged reviewquota/a; set the User quota to exact Home size (2,130 bytes); restart to reload policy; POST /api/v1/tags/rewrites (reviewquota/a → reviewquota/z, confirmed). Acceptance returned 200/pending. Worker returned failed; GET an affected Note returned 500; POST receipt Undo returned 500/internal/tag operation failed. Every source was byte-for-byte unchanged. Raising quota to 1,000,000,000 and restarting permitted Undo (200, completed, 10/10 exact bytes).

Impact: quota exhaustion can strand an accepted operation and make existing Note reads unavailable. This is a correctness failure, not a slow-host finding. Source content loss was not observed. Final projection read showed the old tag; do not claim permanent new projection from this case.

Repro and machine-readable evidence: artifacts/tagrev-quota.mjs; artifacts/tagrev-quota-results.json; artifacts/tagrev-quota.log.

## P1 — Accepted rename at quota blocks Note access and Undo Evidence: `TMPDIR=$PWD/target/tmp node artifacts/tagrev-quota.mjs` against the reviewed server binary. Create 10 Notes tagged `reviewquota/a`; set the User quota to exact Home size (2,130 bytes); restart to reload policy; POST `/api/v1/tags/rewrites` (`reviewquota/a` → `reviewquota/z`, confirmed). Acceptance returned 200/pending. Worker returned failed; GET an affected Note returned 500; POST receipt Undo returned 500/internal/tag operation failed. Every source was byte-for-byte unchanged. Raising quota to 1,000,000,000 and restarting permitted Undo (200, completed, 10/10 exact bytes). Impact: quota exhaustion can strand an accepted operation and make existing Note reads unavailable. This is a correctness failure, not a slow-host finding. Source content loss was not observed. Final projection read showed the old tag; do not claim permanent new projection from this case. Repro and machine-readable evidence: artifacts/tagrev-quota.mjs; artifacts/tagrev-quota-results.json; artifacts/tagrev-quota.log.

kayg commented

2026-10-08 14:36:19 +00:00

Author

Owner

P1 — Failed rewrite leaves the early UI projection inconsistent with files

Real production browser UI, 256 Notes: rename reviewfive/z to reviewfail/z; after 64 new sources, SIGSTOP the server, chmod the disposable Notes folder 0500, SIGCONT, then restore permissions after terminal failure. Status: failed/current=64/total=256/can_undo=true. Sources: 64 new, 192 old. GET Tags projected reviewfail/z and its child for all 256 Notes. Browser: old row=0, new row=1, Undo button=1, failure toast visible. The UI did not roll back the early projection to source truth. Screenshot: artifacts/tagrev-focused-midway-failure.png.

Click the actual failure toast Undo: completed 256/256, every source restored byte-exact to its pre-rename value. This case demonstrates the projection inconsistency, not failed Undo or source loss.

Repro: TMPDIR=$PWD/target/tmp node artifacts/tagrev-focused.mjs; evidence: artifacts/tagrev-focused-results.json and artifacts/tagrev-focused.log. The same runner also proves all three SIGKILL windows on 128 Notes (0/128, journal before publication, 64/128 durable prefix), exact convergence, consistent final projection and idempotent retry.

## P1 — Failed rewrite leaves the early UI projection inconsistent with files Real production browser UI, 256 Notes: rename `reviewfive/z` to `reviewfail/z`; after 64 new sources, SIGSTOP the server, chmod the disposable Notes folder 0500, SIGCONT, then restore permissions after terminal failure. Status: failed/current=64/total=256/can_undo=true. Sources: 64 new, 192 old. GET Tags projected `reviewfail/z` and its child for all 256 Notes. Browser: old row=0, new row=1, Undo button=1, failure toast visible. The UI did not roll back the early projection to source truth. Screenshot: artifacts/tagrev-focused-midway-failure.png. Click the actual failure toast Undo: completed 256/256, every source restored byte-exact to its pre-rename value. This case demonstrates the projection inconsistency, not failed Undo or source loss. Repro: `TMPDIR=$PWD/target/tmp node artifacts/tagrev-focused.mjs`; evidence: artifacts/tagrev-focused-results.json and artifacts/tagrev-focused.log. The same runner also proves all three SIGKILL windows on 128 Notes (0/128, journal before publication, 64/128 durable prefix), exact convergence, consistent final projection and idempotent retry.

kayg commented

2026-10-08 16:32:50 +00:00

Author

Owner

tagfix3 started on job/tagperf-1186, base a3db565a48. Reading the complete independent review and its probes; controlled 128–256 Note reproductions precede fixes. No pushes or deployments. Will report each of the four safety items separately with source-byte and job evidence.

tagfix3 started on job/tagperf-1186, base a3db565a48b148f79f5dcee3b50fa721a24259f9. Reading the complete independent review and its probes; controlled 128–256 Note reproductions precede fixes. No pushes or deployments. Will report each of the four safety items separately with source-byte and job evidence.

kayg commented

2026-10-08 16:38:59 +00:00

Author

Owner

Controlled pre-fix run of items 3/4: 128 owner Notes plus one Daily note and 10 untouched foreign Notes with the same Tag, reviewed binary a3db565a4. Removed the review runner's deliberate baseline/busy writes to foreign Note 0; waited for BOTH live markers to echo to the other collaborator before inspecting files. Rename completed 129/129, conflicts=0; both markers persisted and both editors retained both markers; 127 plain Notes matched exact expected bytes; Journal baseline and concurrent append persisted. All 10/10 foreign sources stayed byte-identical and kept the old Tag; foreign receipt GET returned 404. artifacts/tagfix3-live-results.json. The original 10th foreign body was the runner's explicitly edited latency target, not an owner rename target. Original missing marker 1 had only local visibility (peer=false), so it was not proven acknowledged; controlled peer-confirmed case has no loss. Cross-user regression will also cover Undo.

Controlled pre-fix run of items 3/4: 128 owner Notes plus one Daily note and 10 untouched foreign Notes with the same Tag, reviewed binary a3db565a4. Removed the review runner's deliberate baseline/busy writes to foreign Note 0; waited for BOTH live markers to echo to the other collaborator before inspecting files. Rename completed 129/129, conflicts=0; both markers persisted and both editors retained both markers; 127 plain Notes matched exact expected bytes; Journal baseline and concurrent append persisted. All 10/10 foreign sources stayed byte-identical and kept the old Tag; foreign receipt GET returned 404. artifacts/tagfix3-live-results.json. The original 10th foreign body was the runner's explicitly edited latency target, not an owner rename target. Original missing marker 1 had only local visibility (peer=false), so it was not proven acknowledged; controlled peer-confirmed case has no loss. Cross-user regression will also cover Undo.

kayg commented

2026-10-08 16:40:22 +00:00

Author

Owner

Item 1 independently reproduced on 128 Notes using the reviewed binary: SIGSTOP at 64 rewritten files, chmod Notes to 0500, resume. Receipt failed at 64/128 with Undo. Actual files: 64 old, 64 new; GET Tags showed new=128, old absent. An additional terminal GET of an untouched Note returned 200 but rewrote that file to the new Tag despite the failed receipt. artifacts/tagfix3-failure-before.log. The fix must therefore disable editor-side publication as well as repair both Tag and Notes memberships; keeping only the failure toast is insufficient.

kayg commented

2026-10-08 16:42:50 +00:00

Author

Owner

Item 2: restarted the retained 5,000-Note disposable Instance (same reviewed binary, no new fixture or load generator). The exact job moved from leased/attempts=3/max_attempts=3/current=4992/total=5001 to dead with the same progress and last_error. artifacts/tagfix3-lease-recovery.log. This proves expired final-attempt recovery already works: a claimed third attempt is not a finished third attempt. No permanently leased job reproduced. The original generic 'tag Index operation failed' does not record a SQLx class/code, so its underlying Index error cannot be recovered from that field; adding content-free class/code diagnostics rather than inventing a cause. The new source-backed repair also handles terminal orphan/cancelled intents, with Undo retained.

kayg commented

2026-10-08 16:58:52 +00:00

Author

Owner

Item 4 denominator correction: the original metric is foreign.notes.slice(1).filter(...).length; its maximum is nine, and it reported nine. It excludes Note 0 by construction. That Note was deliberately edited by the foreign User in 20 baseline and up to 20 busy conditional PUTs. The retained source keeps the original body and old Tag plus the runner's B baseline/B busy five lines; no owner-rename replacement appears. The new controlled run compares the full ten-element array with no foreign writes and reports 10/10 exact. This is NOT REPRODUCED, not a cross-User data change.

Item 4 denominator correction: the original metric is `foreign.notes.slice(1).filter(...).length`; its maximum is nine, and it reported nine. It excludes Note 0 by construction. That Note was deliberately edited by the foreign User in 20 baseline and up to 20 busy conditional PUTs. The retained source keeps the original body and old Tag plus the runner's B baseline/B busy five lines; no owner-rename replacement appears. The new controlled run compares the full ten-element array with no foreign writes and reports 10/10 exact. This is NOT REPRODUCED, not a cross-User data change.

kayg commented

2026-10-08 17:03:31 +00:00

Author

Owner

Safety slice committed: b764854b9. Failed attempts now switch their retained intent to source-backed memberships and repair the saved candidates in bounded batches before settlement. Both Tags and Notes rows are repaired; terminal Note reads stop on-demand publication. Startup and the receipt route repair exhausted orphan/cancelled intents, retaining Retry and exact Undo. A weak keyed mutex shares repair across concurrent receipt polls. GET Tags still performs no source IO. No migration or dependency was added.

Focused crate gates (verbatim):
cargo fmt --check: exit 0, no output.
cargo clippy -p calternal-tags --all-targets -- -D warnings:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 18.70s

cargo test -p calternal-tags -- --test-threads=4:

test result: ok. 66 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 180.13s

All new regressions passed: read-only failure at 64/128 with both projections matching files and exact Undo; exhausted final orphan lease with visible failed receipt/Undo; same-Tag foreign Home exact through forward and inverse. The first run found a test-fixture error in the new foreign-Undo test (the fixture had not settled its forward queue lease); fixed the fixture's queue lifecycle, with no existing expectation changed. Server gates and the real-server regression follow.

Safety slice committed: b764854b9. Failed attempts now switch their retained intent to source-backed memberships and repair the saved candidates in bounded batches before settlement. Both Tags and Notes rows are repaired; terminal Note reads stop on-demand publication. Startup and the receipt route repair exhausted orphan/cancelled intents, retaining Retry and exact Undo. A weak keyed mutex shares repair across concurrent receipt polls. GET Tags still performs no source IO. No migration or dependency was added. Focused crate gates (verbatim): `cargo fmt --check`: exit 0, no output. `cargo clippy -p calternal-tags --all-targets -- -D warnings`: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 18.70s ``` `cargo test -p calternal-tags -- --test-threads=4`: ``` test result: ok. 66 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 180.13s ``` All new regressions passed: read-only failure at 64/128 with both projections matching files and exact Undo; exhausted final orphan lease with visible failed receipt/Undo; same-Tag foreign Home exact through forward and inverse. The first run found a test-fixture error in the new foreign-Undo test (the fixture had not settled its forward queue lease); fixed the fixture's queue lifecycle, with no existing expectation changed. Server gates and the real-server regression follow.

kayg commented

2026-10-08 17:05:55 +00:00

Author

Owner

Web test gate passed (verbatim):

      Tests  2129 passed (2129)
   Duration  787.22s (transform 27%, environment 27%, import 23%, tests 17%, setup 6%)

bun run check has zero Svelte errors so far, but perf-lint found exact hashes moved in Notes read consumers of the changed shared Tag helper. Refreshing only the existing live hashes; no rule, limit or ratchet increase. The earlier attempt to put source repair inside GET Tags was removed: repair belongs in the failed worker/startup/receipt paths so ordinary Tag lists do no source IO.

Web test gate passed (verbatim): ``` Tests 2129 passed (2129) Duration 787.22s (transform 27%, environment 27%, import 23%, tests 17%, setup 6%) ``` `bun run check` has zero Svelte errors so far, but perf-lint found exact hashes moved in Notes read consumers of the changed shared Tag helper. Refreshing only the existing live hashes; no rule, limit or ratchet increase. The earlier attempt to put source repair inside GET Tags was removed: repair belongs in the failed worker/startup/receipt paths so ordinary Tag lists do no source IO.

kayg commented

2026-10-08 17:14:16 +00:00

Author

Owner

Perf-pin refresh committed separately: the two existing read_visible_source call sites in Notes get_note and open_file now resolve as io.unresolved-call instead of io.content-read after the shared helper change. Same sites and hashes; the exact #1058 checkpoint was refreshed. Total exceptions remain 22,532; per-rule counts move by -2/+2 to match live entries. No scanner rule or source contract was weakened.

Verbatim validation:

perf-lint: PASS; 0 violations; 22532 scoped exceptions
----------------------------------------------------------------------
Ran 140 tests in 0.085s

OK

Perf-pin refresh committed separately: the two existing `read_visible_source` call sites in Notes `get_note` and `open_file` now resolve as `io.unresolved-call` instead of `io.content-read` after the shared helper change. Same sites and hashes; the exact #1058 checkpoint was refreshed. Total exceptions remain 22,532; per-rule counts move by -2/+2 to match live entries. No scanner rule or source contract was weakened. Verbatim validation: ``` perf-lint: PASS; 0 violations; 22532 scoped exceptions ---------------------------------------------------------------------- Ran 140 tests in 0.085s OK ```

kayg commented

2026-10-08 17:30:23 +00:00

Author

Owner

Controlled terminal recovery evidence for tagfix3 (#1186), source fix b764854b9, regression 22eda56b3:

The new real-server fault regression passed: 128 Notes, stopped after 64, read-only Notes folder; terminal failed receipt with Undo; GET Tags and the mounted UI show 64 old/64 new; GET untouched Note does not rewrite it; all 10 foreign Notes exact; foreign receipt/Undo 404; Undo restores all 128 owner Notes exactly. Log: artifacts/tagfix3-real-failure.log.
Startup against a disposable COPY of the retained 5,000-Note fixture passed in 39,316 ms on this loaded local host (debug build; diagnostic, not a release performance comparison). The retained sources contain 3 old/4,997 new Tags. Repaired rows match those counts; all 5,000 source hashes unchanged. Job remains dead at attempt 3/3, progress 4992/5001. Log: artifacts/tagfix3-lease-after.log.
An expired third lease is terminal under the reviewed binary too. Thus the observation of attempts=3 while leased did not establish a stuck queue: attempts count claims, and the review stopped the server before expiry recovery. The historical SQLx cause is UNKNOWN: the retained error stores only a generic string. New code logs only failure class and SQLite code, not User values.
Captures use the real production UI, Mac platform emulation, phone 390/tablet 820/desktop 1440, light and dark: artifacts/tagfix3-failure-captures/. The CLI has no issue-attachment operation; the screenshots remain in the worktree for orchestrator visual review.

Controlled terminal recovery evidence for tagfix3 (#1186), source fix b764854b9, regression 22eda56b3: - The new real-server fault regression passed: 128 Notes, stopped after 64, read-only Notes folder; terminal failed receipt with Undo; GET Tags and the mounted UI show 64 old/64 new; GET untouched Note does not rewrite it; all 10 foreign Notes exact; foreign receipt/Undo 404; Undo restores all 128 owner Notes exactly. Log: artifacts/tagfix3-real-failure.log. - Startup against a disposable COPY of the retained 5,000-Note fixture passed in 39,316 ms on this loaded local host (debug build; diagnostic, not a release performance comparison). The retained sources contain 3 old/4,997 new Tags. Repaired rows match those counts; all 5,000 source hashes unchanged. Job remains dead at attempt 3/3, progress 4992/5001. Log: artifacts/tagfix3-lease-after.log. - An expired third lease is terminal under the reviewed binary too. Thus the observation of attempts=3 while leased did not establish a stuck queue: attempts count claims, and the review stopped the server before expiry recovery. The historical SQLx cause is UNKNOWN: the retained error stores only a generic string. New code logs only failure class and SQLite code, not User values. - Captures use the real production UI, Mac platform emulation, phone 390/tablet 820/desktop 1440, light and dark: artifacts/tagfix3-failure-captures/. The CLI has no issue-attachment operation; the screenshots remain in the worktree for orchestrator visual review.

kayg commented

2026-10-08 17:31:03 +00:00

Author

Owner

Controlled collaborative/cross-User follow-up, source fix b764854b9:

128 regular Notes plus one Daily note completed at 129/129 with one conflict. Both collaborative markers were visibly echoed to the other editor before final checks; both persisted in the file and both editor views. The original sentinel and all original body lines remained.
Ten conditional body PUTs returned success; lost edits=0; edited-body mismatches=0. The Journal retained baseline and concurrent writes. The remaining 117 ordinary Notes matched expected bytes.
No foreign writes were sent. All 10 foreign same-Tag Notes stayed byte-identical and kept the old Tag; the foreign receipt read was 404. Logs/results: artifacts/tagfix3-live-after.log and artifacts/tagfix3-live-after-results.json.
Item 3: NOT REPRODUCED for acknowledged edits. The original runner proved peer echo only for marker 0, not marker 1. A local DOM echo alone did not establish server acceptance. Both markers are now explicitly peer-confirmed. This establishes preservation for the controlled scenario, not arbitrary concurrency.
Item 4: NOT REPRODUCED. The original “untouched” count used notes.slice(1), so 9 was its maximum. Its excluded Note 0 had exactly 20 intentional baseline PUT markers plus 20 intentional busy PUT markers sent by that foreign User; the original body and old Tag remained. That explains the tenth body exactly. A new Rust regression and the real-server fault/Undo probe also compare all 10 foreign Notes byte-for-byte.

Controlled collaborative/cross-User follow-up, source fix b764854b9: - 128 regular Notes plus one Daily note completed at 129/129 with one conflict. Both collaborative markers were visibly echoed to the other editor before final checks; both persisted in the file and both editor views. The original sentinel and all original body lines remained. - Ten conditional body PUTs returned success; lost edits=0; edited-body mismatches=0. The Journal retained baseline and concurrent writes. The remaining 117 ordinary Notes matched expected bytes. - No foreign writes were sent. All 10 foreign same-Tag Notes stayed byte-identical and kept the old Tag; the foreign receipt read was 404. Logs/results: artifacts/tagfix3-live-after.log and artifacts/tagfix3-live-after-results.json. - Item 3: NOT REPRODUCED for acknowledged edits. The original runner proved peer echo only for marker 0, not marker 1. A local DOM echo alone did not establish server acceptance. Both markers are now explicitly peer-confirmed. This establishes preservation for the controlled scenario, not arbitrary concurrency. - Item 4: NOT REPRODUCED. The original “untouched” count used notes.slice(1), so 9 was its maximum. Its excluded Note 0 had exactly 20 intentional baseline PUT markers plus 20 intentional busy PUT markers sent by that foreign User; the original body and old Tag remained. That explains the tenth body exactly. A new Rust regression and the real-server fault/Undo probe also compare all 10 foreign Notes byte-for-byte.

kayg commented

2026-10-08 17:35:48 +00:00

Author

Owner

Capture coverage correction: the first post-fix fixture used reviewfail/*, which populated the Tags page but not the area-Tag sidebar. Its source/API/Undo checks passed, but that capture did not prove the sidebar claim. The probe now uses area/reviewfail/* and requires the actual mounted sidebar leaves before capture. I am rebuilding the evidence server after the earlier cleanup and will report that result separately. No implementation behavior or existing test expectation changed.

Server gate result: 271 passed, 1 failed, 12 ignored. The only failure is the existing 20,000-file startup Calendar-write latency test: p50=17,320ms, p95=17,874ms against10,000ms. This is a SLOW-only local-host finding. The assertion is unchanged; no repeated full suite or timing-budget increase. Full output: artifacts/tagfix3-server-test.log.

Capture coverage correction: the first post-fix fixture used `reviewfail/*`, which populated the Tags page but not the area-Tag sidebar. Its source/API/Undo checks passed, but that capture did not prove the sidebar claim. The probe now uses `area/reviewfail/*` and requires the actual mounted sidebar leaves before capture. I am rebuilding the evidence server after the earlier cleanup and will report that result separately. No implementation behavior or existing test expectation changed. Server gate result: 271 passed, 1 failed, 12 ignored. The only failure is the existing 20,000-file startup Calendar-write latency test: p50=17,320ms, p95=17,874ms against10,000ms. This is a SLOW-only local-host finding. The assertion is unchanged; no repeated full suite or timing-budget increase. Full output: artifacts/tagfix3-server-test.log.

kayg commented

2026-10-08 17:50:03 +00:00

Author

Owner

tagfix3 — Tag rename data safety (#1186)

Branch: job/tagperf-1186. Head: be82a149a5.
Merged origin/dev once before final gates (7970a90e7). No push or deploy.

Built

Failed Tag rewrites retain the completed source prefix and exact Undo. Their Tag and Notes projections now match source files. A failed or cancelled Note read cannot publish more rename writes. Startup and terminal receipts resume interrupted projection repair. Repair reads only saved candidates, in batches of 64 with at most eight concurrent reads. GET Tags remains an Index-only read.

Changed files:

crates/calternal-tags/src/rewrite.rs: source-backed repair, durable repair flags, safe SQLx class/code diagnostics, three regressions.
crates/calternal-tags/src/lib.rs: startup recovery and owned terminal-receipt repair.
tests/adversarial/tag-rewrite-failure-1186.mjs: focused real-server fault, source counts, mounted UI counts, hostile requests, foreign User checks and exact Undo.
bench/tag-rewrite-1186.py: 20 steady terminal reads and a 16-request burst, at most eight readers; separate latency/CPU/RSS sample.
contracts/perf/{exceptions,adoption-1058,ratchet}.json: exact refresh of two existing Notes consumer classifications. The exception total remains 22,532. No rule changed.

Item results

CONFIRMED+FIXED. Before the fix, a 128-Note run failed after 64 writes. Files were old 64/new 64; GET Tags reported new 128 and hid old. Reading an untouched Note also renamed it after failure. After the fix, the terminal failed receipt offers Undo; files, GET Tags and mounted sidebar/page rows are old 64/new 64. A terminal Note read leaves its bytes unchanged. Undo restores all 128 owner Notes exactly. All10 foreign Notes remain exact. Three Rust regressions cover failed prefixes, expired final leases and foreign Homes.
NOT REPRODUCED: permanent leased state. The retained 5,000-Note job became dead at attempt 3/3 when the reviewed server restarted and recovered its expired lease. Attempts count claims; a leased third attempt has not exhausted its claim until expiry or settlement. The original SQLx root cause remains UNKNOWN: its saved error contains only “tag Index operation failed.” The fix records safe SQLx class and SQLite code for a future failure. On a copy of the retained fixture, the new server reached readiness in 39,316 ms on the loaded local host and repaired old 3/new 4,997 to match actual files; zero of 5,000 source hashes changed. The job remained visibly dead at 4992/5001. The expired-final-lease Rust regression proves failed receipt, retained Undo and resumable repair.
NOT REPRODUCED: acknowledged live-edit loss. The original runner confirmed peer echo for marker 0 only; marker 1 had only local DOM visibility. Controlled runs before and after the fix confirm peer echo for BOTH markers. Both survive in the source and both editor views. The post-fix run also preserves 10 acknowledged conditional PUTs and a concurrent Journal write. It completes 129/129 with one conflict, lost edits0, edited-body mismatches0 and ordinary-source mismatches0. This proves the controlled scenario, not every concurrency schedule.
NOT REPRODUCED: foreign rewrite. The original count used notes.slice(1) and could report at most9 untouched bodies. The excluded tenth Note received 20 baseline PUT markers plus 20 busy PUT markers, deliberately sent by that foreign User; its old Tag remained. After the frontmatter separator, its body equals the original fixture body plus those 40 submitted markers in order, byte-for-byte. In both controlled runs with those writes removed, all 10 foreign bodies are exact. The new Rust and real-server Undo regressions also compare all 10 foreign Notes. Foreign receipt and Undo requests return 404.

UX gaps closed

The failure view shows actual old/new counts. Failed Note reads stop applying rename writes. Undo restores exact bytes. No new UI styles or controls: existing TagRow, Tags page/sidebar and shared failure Toast show the corrected API state.

Six production-build captures use Mac platform emulation:390,820,1440px, light and dark, without a page reload concealing stale rows. Files: artifacts/tagfix3-failure-captures/failure-{390,820,1440}-{light,dark}.png. Visual review belongs to the orchestrator. The Forgejo CLI exposes no attachment operation, so captures remain in this worktree for attachment.

Decisions

Keep committed source edits and repair Derived data instead of automatic rollback. This preserves later edits and leaves explicit Undo available.
Store two backward-compatible flags in the existing accepted journal; no migration.
Use candidate-only repair under existing source-writer locks. A weak keyed mutex coalesces terminal repair requests. Ordinary writers can enter between batches.
Repair at worker failure, startup and owned terminal-receipt reads. Keep filesystem IO out of GET Tags.

Expected repair cost: O(saved candidate bytes + Tag rows), ceil(N/64) batches, eight concurrent reads. It never scans unrelated Home sources. Steady Tag reads retain their existing Index query cost.

Gates

cargo fmt --check: exit 0, no output.

cargo clippy -p calternal-tags --all-targets -- -D warnings:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 18.70s

cargo test -p calternal-tags:

test result: ok. 66 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 180.13s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo clippy -p calternal-server --all-targets -- -D warnings:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 52s

cargo test -p calternal-server: exit 101; one SLOW-only existing timing assertion failed. No assertion changed.

search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=17320 p95_ms=17874 p95_budget_ms=10000
Calendar write p95 17.874184064s exceeded 10s
test result: FAILED. 271 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 222.06s
error: test failed, to rerun pass `-p calternal-server --bin calternal-server`

cd apps/web && bun run check: exit 0.

perf-lint: PASS; 0 violations; 22532 scoped exceptions
svelte-check found 0 errors and 4 warnings in 2 files

cd apps/web && bun run test -- --maxWorkers=2: exit 0.

 Test Files  297 passed (297)
      Tests  2129 passed (2129)

scripts/perf-lint test: exit 0.

----------------------------------------------------------------------
Ran 140 tests in 0.085s

OK

Focused real-server #1186 regression: PASS.

{"result":"PASS","sources":128,"prefix":64,"old":64,"new":64,"foreign_exact":10,"undo_exact":128}
```.
`bench/tag-rewrite-1186.py`: Python syntax PASS.
Production web builds passed. No existing test expectation was changed.


## Evidence

- `artifacts/tagfix3-failure-before.log`: controlled pre-fix failure.
- `artifacts/tagfix3-real-failure-sidebar-final.log`: post-fix focused fault with mounted row assertions.
- `artifacts/tagfix3-lease-recovery.log`: reviewed-binary expired-lease recovery.
- `artifacts/tagfix3-lease-after.log`: copied retained 5,000-Note startup repair.
- `artifacts/tagfix3-live-results.json` and `artifacts/tagfix3-live-after-results.json`: peer-confirmed collaborative runs.
- `artifacts/tagfix3-{tags-clippy-final,tags-test-final,server-clippy,server-test,web-check-ready,web-test}.log`: full gate logs.

## Known gaps and performance

The historical generic Index error cannot identify its SQLx cause. The controlled probes do not prove every concurrent schedule. The original review's separate full-quota Undo observation is outside these four items and was not reverified here. No claim is made that all of #1186 meets its performance target.

The new steady-read profile is syntax-checked but unmeasured. The perf VM was occupied (latest holder: calternal mailsel-1449-candidate-a-resources,45 min reservation). No benchmark ran without its lock. The 39.3 s copied-fixture startup is a local debug diagnostic, not a release baseline comparison. `docs/perf/baseline.json` has `tag_rename_525` on three tagged sources (p50=2529.87ms,p95=6741.14ms), which is not comparable to this repair. The prior 1,000-Note release profile in `docs/perf/2026-10-08-tagfast2-1186.md` reports durable median19,828.7ms and concurrent Tag-read p95=29.53–79.97ms. No new regression is established by unlike workloads.

## For the merge round

- `cargo test -p calternal-server -- --test-threads=4` must run on the combined build and confirm the existing startup responsiveness budget. The job reports its measured SLOW-only failure above.
- Full e2e/adversarial matrices remain merge-round work. Run the repository's full XUser/authz/robustness matrices on the combined server to prove combined route contracts. The focused #1186 regression already ran here.
- On the perf VM, with its real fixture/controller environment and copied release build, run `perf-run calternal tagfix3 15 -- python3 bench/tag-rewrite-1186.py`; compare the new terminal-read latency/CPU/RSS windows at 1,000 Notes and a fresh 5,000-Note fixture. Record load inside the reservation. Do not compile there. Browser timing, if requested, requires the locked real macOS VM; no new browser-timing claim is made here.

Cleanup: Own Cargo target cleaned; production web build and .svelte-kit/output removed. Worktree clean. Ignored captures and evidence retained. Removed 8648 files, 7.3GiB total.

# tagfix3 — Tag rename data safety (#1186) Branch: `job/tagperf-1186`. Head: be82a149a5f94e019ea308adb58a540c2b3b1cd1. Merged `origin/dev` once before final gates (`7970a90e7`). No push or deploy. ## Built Failed Tag rewrites retain the completed source prefix and exact Undo. Their Tag and Notes projections now match source files. A failed or cancelled Note read cannot publish more rename writes. Startup and terminal receipts resume interrupted projection repair. Repair reads only saved candidates, in batches of 64 with at most eight concurrent reads. GET Tags remains an Index-only read. Changed files: - `crates/calternal-tags/src/rewrite.rs`: source-backed repair, durable repair flags, safe SQLx class/code diagnostics, three regressions. - `crates/calternal-tags/src/lib.rs`: startup recovery and owned terminal-receipt repair. - `tests/adversarial/tag-rewrite-failure-1186.mjs`: focused real-server fault, source counts, mounted UI counts, hostile requests, foreign User checks and exact Undo. - `bench/tag-rewrite-1186.py`: 20 steady terminal reads and a 16-request burst, at most eight readers; separate latency/CPU/RSS sample. - `contracts/perf/{exceptions,adoption-1058,ratchet}.json`: exact refresh of two existing Notes consumer classifications. The exception total remains 22,532. No rule changed. ## Item results 1. **CONFIRMED+FIXED.** Before the fix, a 128-Note run failed after 64 writes. Files were old 64/new 64; GET Tags reported new 128 and hid old. Reading an untouched Note also renamed it after failure. After the fix, the terminal failed receipt offers Undo; files, GET Tags and mounted sidebar/page rows are old 64/new 64. A terminal Note read leaves its bytes unchanged. Undo restores all 128 owner Notes exactly. All10 foreign Notes remain exact. Three Rust regressions cover failed prefixes, expired final leases and foreign Homes. 2. **NOT REPRODUCED: permanent leased state.** The retained 5,000-Note job became dead at attempt 3/3 when the reviewed server restarted and recovered its expired lease. Attempts count claims; a leased third attempt has not exhausted its claim until expiry or settlement. The original SQLx root cause remains UNKNOWN: its saved error contains only “tag Index operation failed.” The fix records safe SQLx class and SQLite code for a future failure. On a copy of the retained fixture, the new server reached readiness in 39,316 ms on the loaded local host and repaired old 3/new 4,997 to match actual files; zero of 5,000 source hashes changed. The job remained visibly dead at 4992/5001. The expired-final-lease Rust regression proves failed receipt, retained Undo and resumable repair. 3. **NOT REPRODUCED: acknowledged live-edit loss.** The original runner confirmed peer echo for marker 0 only; marker 1 had only local DOM visibility. Controlled runs before and after the fix confirm peer echo for BOTH markers. Both survive in the source and both editor views. The post-fix run also preserves 10 acknowledged conditional PUTs and a concurrent Journal write. It completes 129/129 with one conflict, lost edits0, edited-body mismatches0 and ordinary-source mismatches0. This proves the controlled scenario, not every concurrency schedule. 4. **NOT REPRODUCED: foreign rewrite.** The original count used `notes.slice(1)` and could report at most9 untouched bodies. The excluded tenth Note received 20 baseline PUT markers plus 20 busy PUT markers, deliberately sent by that foreign User; its old Tag remained. After the frontmatter separator, its body equals the original fixture body plus those 40 submitted markers in order, byte-for-byte. In both controlled runs with those writes removed, all 10 foreign bodies are exact. The new Rust and real-server Undo regressions also compare all 10 foreign Notes. Foreign receipt and Undo requests return 404. ## UX gaps closed The failure view shows actual old/new counts. Failed Note reads stop applying rename writes. Undo restores exact bytes. No new UI styles or controls: existing TagRow, Tags page/sidebar and shared failure Toast show the corrected API state. Six production-build captures use Mac platform emulation:390,820,1440px, light and dark, without a page reload concealing stale rows. Files: `artifacts/tagfix3-failure-captures/failure-{390,820,1440}-{light,dark}.png`. Visual review belongs to the orchestrator. The Forgejo CLI exposes no attachment operation, so captures remain in this worktree for attachment. ## Decisions - Keep committed source edits and repair Derived data instead of automatic rollback. This preserves later edits and leaves explicit Undo available. - Store two backward-compatible flags in the existing accepted journal; no migration. - Use candidate-only repair under existing source-writer locks. A weak keyed mutex coalesces terminal repair requests. Ordinary writers can enter between batches. - Repair at worker failure, startup and owned terminal-receipt reads. Keep filesystem IO out of GET Tags. Expected repair cost: O(saved candidate bytes + Tag rows), ceil(N/64) batches, eight concurrent reads. It never scans unrelated Home sources. Steady Tag reads retain their existing Index query cost. ## Gates `cargo fmt --check`: exit 0, no output. `cargo clippy -p calternal-tags --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 18.70s ``` `cargo test -p calternal-tags`: ```text test result: ok. 66 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 180.13s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo clippy -p calternal-server --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 52s ``` `cargo test -p calternal-server`: exit 101; one SLOW-only existing timing assertion failed. No assertion changed. ```text search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=17320 p95_ms=17874 p95_budget_ms=10000 Calendar write p95 17.874184064s exceeded 10s test result: FAILED. 271 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 222.06s error: test failed, to rerun pass `-p calternal-server --bin calternal-server` ``` `cd apps/web && bun run check`: exit 0. ```text perf-lint: PASS; 0 violations; 22532 scoped exceptions svelte-check found 0 errors and 4 warnings in 2 files ``` `cd apps/web && bun run test -- --maxWorkers=2`: exit 0. ```text Test Files 297 passed (297) Tests 2129 passed (2129) ``` `scripts/perf-lint test`: exit 0. ```text ---------------------------------------------------------------------- Ran 140 tests in 0.085s OK ``` Focused real-server #1186 regression: PASS. ```text {"result":"PASS","sources":128,"prefix":64,"old":64,"new":64,"foreign_exact":10,"undo_exact":128} ```. `bench/tag-rewrite-1186.py`: Python syntax PASS. Production web builds passed. No existing test expectation was changed. ## Evidence - `artifacts/tagfix3-failure-before.log`: controlled pre-fix failure. - `artifacts/tagfix3-real-failure-sidebar-final.log`: post-fix focused fault with mounted row assertions. - `artifacts/tagfix3-lease-recovery.log`: reviewed-binary expired-lease recovery. - `artifacts/tagfix3-lease-after.log`: copied retained 5,000-Note startup repair. - `artifacts/tagfix3-live-results.json` and `artifacts/tagfix3-live-after-results.json`: peer-confirmed collaborative runs. - `artifacts/tagfix3-{tags-clippy-final,tags-test-final,server-clippy,server-test,web-check-ready,web-test}.log`: full gate logs. ## Known gaps and performance The historical generic Index error cannot identify its SQLx cause. The controlled probes do not prove every concurrent schedule. The original review's separate full-quota Undo observation is outside these four items and was not reverified here. No claim is made that all of #1186 meets its performance target. The new steady-read profile is syntax-checked but unmeasured. The perf VM was occupied (latest holder: calternal mailsel-1449-candidate-a-resources,45 min reservation). No benchmark ran without its lock. The 39.3 s copied-fixture startup is a local debug diagnostic, not a release baseline comparison. `docs/perf/baseline.json` has `tag_rename_525` on three tagged sources (p50=2529.87ms,p95=6741.14ms), which is not comparable to this repair. The prior 1,000-Note release profile in `docs/perf/2026-10-08-tagfast2-1186.md` reports durable median19,828.7ms and concurrent Tag-read p95=29.53–79.97ms. No new regression is established by unlike workloads. ## For the merge round - `cargo test -p calternal-server -- --test-threads=4` must run on the combined build and confirm the existing startup responsiveness budget. The job reports its measured SLOW-only failure above. - Full e2e/adversarial matrices remain merge-round work. Run the repository's full XUser/authz/robustness matrices on the combined server to prove combined route contracts. The focused #1186 regression already ran here. - On the perf VM, with its real fixture/controller environment and copied release build, run `perf-run calternal tagfix3 15 -- python3 bench/tag-rewrite-1186.py`; compare the new terminal-read latency/CPU/RSS windows at 1,000 Notes and a fresh 5,000-Note fixture. Record load inside the reservation. Do not compile there. Browser timing, if requested, requires the locked real macOS VM; no new browser-timing claim is made here. Cleanup: Own Cargo target cleaned; production web build and .svelte-kit/output removed. Worktree clean. Ignored captures and evidence retained. Removed 8648 files, 7.3GiB total.

kayg commented

2026-10-09 03:41:31 +00:00

Author

Owner

tagperf4 started on job/tagperf-1186, base e1672b919c. Scope: full-quota Undo on 128 Notes and three interleaved origin/dev/candidate performance pairs at 1,000 and 5,000 Notes. No UI code changes. Prior quota evidence reports 500 on Note read and Undo with unchanged sources; current admission reserves remaining intent bytes before publication. Will test both initial exhaustion and quota reduction after a durable prefix.

tagperf4 started on job/tagperf-1186, base e1672b919cd97c92a6c55ff2a3142392fa875a29. Scope: full-quota Undo on 128 Notes and three interleaved origin/dev/candidate performance pairs at 1,000 and 5,000 Notes. No UI code changes. Prior quota evidence reports 500 on Note read and Undo with unchanged sources; current admission reserves remaining intent bytes before publication. Will test both initial exhaustion and quota reduction after a durable prefix.

kayg commented

2026-10-09 03:49:37 +00:00

Author

Owner

tagperf4 finding: Tags maps calternal_fs::Error::QuotaExceeded to HTTP 500 and records only "tag filesystem operation failed" in worker errors (crates/calternal-tags/src/lib.rs). The job changes this to HTTP 507 with "Home storage quota reached; free space and retry" for admission and queue diagnostics. The new 128-Note real-server probe reached a 64-Note committed prefix against the shared origin/dev binary. At exact Home quota, Undo returned 200 then failed after a partial inverse; this is baseline evidence, not a verdict on the reviewed branch. The regression now checks whole source versions and source-backed Tag counts, and requires byte-exact Undo after quota restoration. Reviewed-server compilation and focused unit tests are running. No UI code changed.

Performance setup: copied origin/dev cc1af3c12 release to the perf VM; reused fresh disposable copies of the existing 1,000/5,000-Note fixtures. Added an independent User and a 4 Hz Note-write sampler, and direct CDP input with separate sidebar/Tags-page paint clocks. Initial run hit the old browser harness 30-second cap and is retained as a failed setup attempt. Corrected run holds perf-run. macOS reservation is occupied (holder start 2026-10-08T22:01:55Z, stated 60 minutes); no foreign reservation was removed.

tagperf4 finding: Tags maps calternal_fs::Error::QuotaExceeded to HTTP 500 and records only "tag filesystem operation failed" in worker errors (crates/calternal-tags/src/lib.rs). The job changes this to HTTP 507 with "Home storage quota reached; free space and retry" for admission and queue diagnostics. The new 128-Note real-server probe reached a 64-Note committed prefix against the shared origin/dev binary. At exact Home quota, Undo returned 200 then failed after a partial inverse; this is baseline evidence, not a verdict on the reviewed branch. The regression now checks whole source versions and source-backed Tag counts, and requires byte-exact Undo after quota restoration. Reviewed-server compilation and focused unit tests are running. No UI code changed. Performance setup: copied origin/dev cc1af3c12 release to the perf VM; reused fresh disposable copies of the existing 1,000/5,000-Note fixtures. Added an independent User and a 4 Hz Note-write sampler, and direct CDP input with separate sidebar/Tags-page paint clocks. Initial run hit the old browser harness 30-second cap and is retained as a failed setup attempt. Corrected run holds perf-run. macOS reservation is occupied (holder start 2026-10-08T22:01:55Z, stated 60 minutes); no foreign reservation was removed.

kayg commented

2026-10-09 04:16:21 +00:00

Author

Owner

tagperf4 progress: head 8c42a48f0; origin/dev cc1af3c12 merged in a77e71920. Full-quota real-server regression PASS: 128 Notes, 64-Note durable prefix, exact Home quota, Undo 507 with clear storage message, source-backed counts 64/64, both prefixes readable, then 128/128 byte-exact Undo after quota restoration. Three focused Rust quota tests passed. The shared fixture refactor also passed the existing real-server failure probe: foreign_exact=10 and undo_exact=128.

Post-merge web gates, verbatim:

perf-lint: PASS; 0 violations; 22532 scoped exceptions
svelte-check found 0 errors and 4 warnings in 2 files
 Test Files  301 passed (301)
      Tests  2178 passed (2178)

Merge decisions: retain both defer-initial-focus and restore-focus surface props; retain dev's quiet programmatic focus rule (#1391) with the existing explicit quiet-focus options type. Its one old default-focus expectation now follows #1391's false policy; this is a merge resolution of that explicit behavior change. The upgrade receipt count combines #1186 and dev's #1422: 26 new receipts, not either side's 25. Tags migration 0007 is still free on origin/dev (last deployed Tags migration is 0006). Exact performance pins refreshed with the existing generator; no rule changed; count remains 22,532.

Performance: release candidate is building locally; final per-crate Rust gates are started. Corrected baseline 1,000-Note sample is running with perf-run on the perf VM. Setup errors are retained and excluded: old browser cap; wrong Note-write endpoint (405); second User Note not yet indexed. Corrected functional preflight proves both mounted views and conditional Note PUT 200; local durations were discarded. Mac reservation remains occupied and has not been removed. No push, deploy or UI feature/style edit.

tagperf4 progress: head 8c42a48f0; origin/dev cc1af3c12 merged in a77e71920. Full-quota real-server regression PASS: 128 Notes, 64-Note durable prefix, exact Home quota, Undo 507 with clear storage message, source-backed counts 64/64, both prefixes readable, then 128/128 byte-exact Undo after quota restoration. Three focused Rust quota tests passed. The shared fixture refactor also passed the existing real-server failure probe: foreign_exact=10 and undo_exact=128. Post-merge web gates, verbatim: ``` perf-lint: PASS; 0 violations; 22532 scoped exceptions svelte-check found 0 errors and 4 warnings in 2 files Test Files 301 passed (301) Tests 2178 passed (2178) ``` Merge decisions: retain both defer-initial-focus and restore-focus surface props; retain dev's quiet programmatic focus rule (#1391) with the existing explicit quiet-focus options type. Its one old default-focus expectation now follows #1391's false policy; this is a merge resolution of that explicit behavior change. The upgrade receipt count combines #1186 and dev's #1422: 26 new receipts, not either side's 25. Tags migration 0007 is still free on origin/dev (last deployed Tags migration is 0006). Exact performance pins refreshed with the existing generator; no rule changed; count remains 22,532. Performance: release candidate is building locally; final per-crate Rust gates are started. Corrected baseline 1,000-Note sample is running with perf-run on the perf VM. Setup errors are retained and excluded: old browser cap; wrong Note-write endpoint (405); second User Note not yet indexed. Corrected functional preflight proves both mounted views and conditional Note PUT 200; local durations were discarded. Mac reservation remains occupied and has not been removed. No push, deploy or UI feature/style edit.

kayg referenced this issue

2026-10-09 04:28:26 +00:00

Server reaches descriptor limit during Tag rewrite and independent Note writes #1497

kayg commented

2026-10-09 04:28:27 +00:00

Author

Owner

During #1186 tagperf4 measurement, origin/dev cc1af3c12 release reached 1,024 open descriptors on a fresh disposable Home (1,000 Notes, sixteen 1 Hz Tag readers and one independent-User conditional Note-body writer). The worker stopped at 381/1,000 after three attempts. One independent write returned 500. The private server log contains three Too many open files / os error 24 occurrences. The public receipt reported a generic filesystem failure. Storage was 889,992 bytes against a 1 GB quota; disk free space was 24 GB.

This is not a SLOW-only finding. Sources remain on disk; full source consistency was not verified in this diagnostic run. Candidate behavior is not yet measured. Reproduction is bench/tag-rewrite-1186-paired.sh baseline 1000 1 with the explicit ulimit -n 65536 line removed, under perf-run. Fixtures are disposable; credentials are private and excluded from evidence.

The valid A/B comparison will use a service-like descriptor limit of 65,536 for both servers. It records peak/average descriptors, and stops immediately on failed receipts. The diagnostic profile is excluded from latency comparisons.

During #1186 tagperf4 measurement, origin/dev cc1af3c12 release reached 1,024 open descriptors on a fresh disposable Home (1,000 Notes, sixteen 1 Hz Tag readers and one independent-User conditional Note-body writer). The worker stopped at 381/1,000 after three attempts. One independent write returned 500. The private server log contains three `Too many open files` / `os error 24` occurrences. The public receipt reported a generic filesystem failure. Storage was 889,992 bytes against a 1 GB quota; disk free space was 24 GB. This is not a SLOW-only finding. Sources remain on disk; full source consistency was not verified in this diagnostic run. Candidate behavior is not yet measured. Reproduction is `bench/tag-rewrite-1186-paired.sh baseline 1000 1` with the explicit `ulimit -n 65536` line removed, under perf-run. Fixtures are disposable; credentials are private and excluded from evidence. The valid A/B comparison will use a service-like descriptor limit of 65,536 for both servers. It records peak/average descriptors, and stops immediately on failed receipts. The diagnostic profile is excluded from latency comparisons.

kayg commented

2026-10-09 04:39:19 +00:00

Author

Owner

Head 174ea93f9. Merged origin/dev cc1af3c12 once. All requested final gates pass. Verbatim summaries:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.09s
test result: ok. 68 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 196.38s
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 10m 25s
test result: ok. 273 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 280.84s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.66s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s
perf-lint: PASS; 0 violations; 22532 scoped exceptions
svelte-check found 0 errors and 4 warnings in 2 files
 Test Files  301 passed (301)
      Tests  2178 passed (2178)

cargo fmt --check emitted no output and exited 0. Tags and server Clippy/tests each exited 0. The full-quota real-server probe also passed against the merged release build: 64/128 durable prefix; Undo 507 with Home storage quota reached; free space and retry; refused Undo changed no files; after raising quota, Undo restored all 128 byte-exact.

The default SSH descriptor-limit failure is filed as #1497; the private log confirms os error 24 three times. Both comparison builds now have the same explicit 65,536 descriptor limit. The first 1,000-Note baseline remains active; three interleaved pairs at each size are queued under separate perf-run reservations. macOS GPU confirmation remains unavailable because the shared Mac reservation is occupied.

Head 174ea93f9. Merged origin/dev cc1af3c12 once. All requested final gates pass. Verbatim summaries: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.09s test result: ok. 68 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 196.38s Finished `dev` profile [unoptimized + debuginfo] target(s) in 10m 25s test result: ok. 273 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 280.84s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.66s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s perf-lint: PASS; 0 violations; 22532 scoped exceptions svelte-check found 0 errors and 4 warnings in 2 files Test Files 301 passed (301) Tests 2178 passed (2178) ``` `cargo fmt --check` emitted no output and exited 0. Tags and server Clippy/tests each exited 0. The full-quota real-server probe also passed against the merged release build: 64/128 durable prefix; Undo 507 with `Home storage quota reached; free space and retry`; refused Undo changed no files; after raising quota, Undo restored all 128 byte-exact. The default SSH descriptor-limit failure is filed as #1497; the private log confirms `os error 24` three times. Both comparison builds now have the same explicit 65,536 descriptor limit. The first 1,000-Note baseline remains active; three interleaved pairs at each size are queued under separate perf-run reservations. macOS GPU confirmation remains unavailable because the shared Mac reservation is occupied.

kayg commented

2026-10-09 04:46:10 +00:00

Author

Owner

First paired result committed at c91caf1c7; baseline cc1af3c12 and candidate merged Rust source a77e71920, identical frozen e1672b919 production SPA.

1,000 Notes, pair 1	Tags page	Sidebar	Source completion	Tag-read p95	Other-User write p95	load5 start/end
origin/dev	>5,000 ms	>5,000 ms	>900.00 s (capped)	46.98 ms	571.91 ms	1.95 / 6.22
candidate	115.00 ms	115.00 ms	25.70 s	56.42 ms	230.56 ms	6.38 / 6.29

The candidate's first sample meets the 500 ms target in both views. Neither read nor write route returned an error. The candidate terminal receipt and every source Note's final Tag bytes were verified. Both builds use a 65,536 descriptor budget; peak descriptors were baseline 3,543 / candidate 303. The baseline did not complete in the 900-second cap; its result is a lower bound, not a completion latency. UI clock: trusted CDP Enter to first rAF with the new name. Linux headless Chromium with macOS platform emulation, not confirmed real Mac GPU timing. HTTP/1.1 request queue time baseline 3.20 ms / candidate 16.90 ms.

Numbers and method live in docs/perf/2026-10-09-tagperf4-1186.md, raw docs/perf/runs/tag-rewrite-1186-tagperf4.json, and scoped metrics.tag_rename_1186_tagperf4 in docs/perf/baseline.json. The Index check occurs after completion and is explicitly not claimed as earliest API visibility.

Remaining ten measurements are queued. Another job now holds the shared perf reservation; no unlocked measurements run. All final gates and merged-release full-quota Undo verification already pass.

First paired result committed at c91caf1c7; baseline cc1af3c12 and candidate merged Rust source a77e71920, identical frozen e1672b919 production SPA. | 1,000 Notes, pair 1 | Tags page | Sidebar | Source completion | Tag-read p95 | Other-User write p95 | load5 start/end | | --- | ---: | ---: | ---: | ---: | ---: | --- | | origin/dev | >5,000 ms | >5,000 ms | >900.00 s (capped) | 46.98 ms | 571.91 ms | 1.95 / 6.22 | | candidate | 115.00 ms | 115.00 ms | 25.70 s | 56.42 ms | 230.56 ms | 6.38 / 6.29 | The candidate's first sample meets the 500 ms target in both views. Neither read nor write route returned an error. The candidate terminal receipt and every source Note's final Tag bytes were verified. Both builds use a 65,536 descriptor budget; peak descriptors were baseline 3,543 / candidate 303. The baseline did not complete in the 900-second cap; its result is a lower bound, not a completion latency. UI clock: trusted CDP Enter to first rAF with the new name. Linux headless Chromium with macOS platform emulation, not confirmed real Mac GPU timing. HTTP/1.1 request queue time baseline 3.20 ms / candidate 16.90 ms. Numbers and method live in `docs/perf/2026-10-09-tagperf4-1186.md`, raw `docs/perf/runs/tag-rewrite-1186-tagperf4.json`, and scoped `metrics.tag_rename_1186_tagperf4` in `docs/perf/baseline.json`. The Index check occurs after completion and is explicitly not claimed as earliest API visibility. Remaining ten measurements are queued. Another job now holds the shared perf reservation; no unlocked measurements run. All final gates and merged-release full-quota Undo verification already pass.

kayg commented

2026-10-09 05:01:45 +00:00

Author

Owner

Head 41f51c176. The initial 5-second UI observer was enough to assess the 500 ms target, but could only retain a lower bound for a slow baseline. The observer now continues independently for the receipt window, with a content-free /paints snapshot. Receipt polling and read/write measurement stop at the terminal receipt; a separate two-second post-completion UI allowance covers the normal 500 ms progress poll. The first pair and the already-running second baseline keep their original 5-second UI lower bounds; no collected result is rewritten as an exact time.

The updated browser controller passed its local functional preflight (both real views; actual conditional other-User write). Local durations are discarded. The next candidate VM phase will exercise the separate snapshot. No production UI code changed. Error-mapping function comments were reviewed and their storage/retry invariants added in 9b2d0b411; formatting passes. Rust behavior remains the already-gated merged release source.

The second baseline is active under perf-run; remaining phases remain interleaved on fresh fixtures. Mac reservation remains occupied. All requested gates and the merged-release full-quota Undo probe pass, as recorded above.

Head 41f51c176. The initial 5-second UI observer was enough to assess the 500 ms target, but could only retain a lower bound for a slow baseline. The observer now continues independently for the receipt window, with a content-free `/paints` snapshot. Receipt polling and read/write measurement stop at the terminal receipt; a separate two-second post-completion UI allowance covers the normal 500 ms progress poll. The first pair and the already-running second baseline keep their original 5-second UI lower bounds; no collected result is rewritten as an exact time. The updated browser controller passed its local functional preflight (both real views; actual conditional other-User write). Local durations are discarded. The next candidate VM phase will exercise the separate snapshot. No production UI code changed. Error-mapping function comments were reviewed and their storage/retry invariants added in 9b2d0b411; formatting passes. Rust behavior remains the already-gated merged release source. The second baseline is active under perf-run; remaining phases remain interleaved on fresh fixtures. Mac reservation remains occupied. All requested gates and the merged-release full-quota Undo probe pass, as recorded above.

kayg referenced this issue

2026-10-09 05:19:35 +00:00

Tags read p95 rises during batched rewrite compared with dev #1499

kayg commented

2026-10-09 05:19:36 +00:00

Author

Owner

The #1186 tagperf4 A/B profile found higher GET /api/v1/tags p95 during the candidate rewrite. Both builds run on perf-test-69-63 under perf-run, with the same frozen e1672b919 production SPA, fresh 1,000-Note Homes, sixteen 1 Hz Tag readers, and one separate-User conditional Note-body writer (250 ms delay after each successful write).

Baseline is origin/dev cc1af3c12; candidate Rust source is a77e71920 (quota error fix plus merged dev). All requested code gates pass. Both servers have a 65,536 descriptor budget. Neither read nor write route returned an error in these pairs.

Pair	Baseline Tag-read p95 ms	Candidate Tag-read p95 ms	Baseline completion s	Candidate completion s	Baseline load5 start/end	Candidate load5 start/end
1	46.98	56.42	>900.00	25.70	1.95 / 6.22	6.38 / 6.29
2	47.58	84.20	869.06	25.83	4.69 / 5.79	5.65 / 5.59
3	49.50	113.17	>900.02	25.76	5.51 / 6.03	5.96 / 5.86

Median per-run p95: baseline 47.58 ms, candidate 84.20 ms (+76.97%). The route-p95 regression threshold in bench/compare.py is 10%. These are medians of per-run p95, not a pooled request percentile.

Interpretation limit: candidate request windows are about 26 seconds; baseline windows are about 15 minutes. Startup background work overlaps both, but the short candidate windows give startup work more weight. This observation does not isolate a steady-state route regression or its cause. It is still the requested during-rewrite measurement and must not be hidden. Candidate other-User write p95 improves in all three pairs, and every candidate UI clock meets 500 ms.

Raw profiles and the scoped baseline are in docs/perf/runs/tag-rewrite-1186-tagperf4.json and docs/perf/baseline.json; method and table are in docs/perf/2026-10-09-tagperf4-1186.md. The 5,000-Note pairs are running; their results will be added. Performance findings do not block a merge under the owner rule.

The #1186 tagperf4 A/B profile found higher GET /api/v1/tags p95 during the candidate rewrite. Both builds run on perf-test-69-63 under perf-run, with the same frozen e1672b919 production SPA, fresh 1,000-Note Homes, sixteen 1 Hz Tag readers, and one separate-User conditional Note-body writer (250 ms delay after each successful write). Baseline is origin/dev cc1af3c12; candidate Rust source is a77e71920 (quota error fix plus merged dev). All requested code gates pass. Both servers have a 65,536 descriptor budget. Neither read nor write route returned an error in these pairs. | Pair | Baseline Tag-read p95 ms | Candidate Tag-read p95 ms | Baseline completion s | Candidate completion s | Baseline load5 start/end | Candidate load5 start/end | | ---: | ---: | ---: | ---: | ---: | --- | --- | | 1 | 46.98 | 56.42 | >900.00 | 25.70 | 1.95 / 6.22 | 6.38 / 6.29 | | 2 | 47.58 | 84.20 | 869.06 | 25.83 | 4.69 / 5.79 | 5.65 / 5.59 | | 3 | 49.50 | 113.17 | >900.02 | 25.76 | 5.51 / 6.03 | 5.96 / 5.86 | Median per-run p95: baseline 47.58 ms, candidate 84.20 ms (+76.97%). The route-p95 regression threshold in bench/compare.py is 10%. These are medians of per-run p95, not a pooled request percentile. Interpretation limit: candidate request windows are about 26 seconds; baseline windows are about 15 minutes. Startup background work overlaps both, but the short candidate windows give startup work more weight. This observation does not isolate a steady-state route regression or its cause. It is still the requested during-rewrite measurement and must not be hidden. Candidate other-User write p95 improves in all three pairs, and every candidate UI clock meets 500 ms. Raw profiles and the scoped baseline are in docs/perf/runs/tag-rewrite-1186-tagperf4.json and docs/perf/baseline.json; method and table are in docs/perf/2026-10-09-tagperf4-1186.md. The 5,000-Note pairs are running; their results will be added. Performance findings do not block a merge under the owner rule.

kayg commented

2026-10-09 05:55:54 +00:00

Author

Owner

The first tagperf4 5,000-Note candidate sample shows a cross-User write stall. Tags page/sidebar names appear at 195.40 ms, but POST /api/v1/tags/rewrites accepts only after 37,101.40 ms. One separate-User conditional Note-body PUT exceeds its 30-second transport cap (30,027.68 ms, status -1). The writer then stops because its ETag outcome is unknown. Five writes were attempted; four returned 200 and one timed out. Do not call the timeout a successful write p95.

The forward receipt completes at 128,188.45 ms, every source Note has its expected final Tag, and Tag reads return no errors (p95 189.66 ms). Server peak descriptors are 306; this is not the earlier 1,024-descriptor failure. load5 start/end is 4.16 / 5.23.

Candidate Rust source a77e71920, binary SHA-256 05cb9d89046ddc5ad5a410c9623c1b986a173add05239118235534a12139bdfd. The run uses perf-run on perf-test-69-63 and a fresh disposable fixture with a second real User/Note. Frozen SPA e1672b919. Caller start_rewrite holds the namespace/User locks through acceptance (crates/calternal-tags/src/lib.rs:3052). project_accepted joins json_each witnesses with tags before staging memberships (rewrite.rs); query-plan cost is a lead, not a measured cause. The observation does not isolate lock wait from database work or startup work.

The profile now allows 120 seconds for an independent write so it can record a slow success and continue the ETag chain. It separately reports successful-write p95, attempt maximum, counts and errors. It never resubmits an unknown timed-out write. The original sample is retained as a diagnostic; a corrected full pair is needed for successful other-User p95. No production algorithm changes are made by this measurement update.

The first tagperf4 5,000-Note candidate sample shows a cross-User write stall. Tags page/sidebar names appear at 195.40 ms, but POST /api/v1/tags/rewrites accepts only after 37,101.40 ms. One separate-User conditional Note-body PUT exceeds its 30-second transport cap (30,027.68 ms, status -1). The writer then stops because its ETag outcome is unknown. Five writes were attempted; four returned 200 and one timed out. Do not call the timeout a successful write p95. The forward receipt completes at 128,188.45 ms, every source Note has its expected final Tag, and Tag reads return no errors (p95 189.66 ms). Server peak descriptors are 306; this is not the earlier 1,024-descriptor failure. load5 start/end is 4.16 / 5.23. Candidate Rust source a77e71920, binary SHA-256 05cb9d89046ddc5ad5a410c9623c1b986a173add05239118235534a12139bdfd. The run uses perf-run on perf-test-69-63 and a fresh disposable fixture with a second real User/Note. Frozen SPA e1672b919. Caller start_rewrite holds the namespace/User locks through acceptance (crates/calternal-tags/src/lib.rs:3052). project_accepted joins json_each witnesses with tags before staging memberships (rewrite.rs); query-plan cost is a lead, not a measured cause. The observation does not isolate lock wait from database work or startup work. The profile now allows 120 seconds for an independent write so it can record a slow success and continue the ETag chain. It separately reports successful-write p95, attempt maximum, counts and errors. It never resubmits an unknown timed-out write. The original sample is retained as a diagnostic; a corrected full pair is needed for successful other-User p95. No production algorithm changes are made by this measurement update.

kayg commented

2026-10-09 06:38:04 +00:00

Author

Owner

Head 4997fb926: real macOS arm64 confirmation now passes at both sizes. Headful Chromium reports Apple Paravirtual Metal, GPU compositing enabled and rasterization enabled. Trusted CDP Enter to the first frame containing the new name in BOTH sidebar and Tags page: 1,000 Notes 214.80 ms; 5,000 Notes 306.80 ms. HTTP/1.1 request queue 0.80/0.60 ms. These use the local candidate release through a NetBird SSH reverse tunnel, so they are separate rendering confirmation, not substituted into the perf-VM backend pairs. Both local macvm.lock flock and remote /tmp/mac-bench.lock were held; own downloads are removed. No trust settings or foreign accounts changed. Evidence: docs/perf/runs/tag-rewrite-1186-tagperf4-mac.json.

Ten of twelve interleaved perf-VM profiles are collected. The third 5,000-Note baseline and candidate wait for the shared perf reservation. Second candidate completes in 137.39 s, UI 100.40 ms, read p95 211.48 ms. Its other-User writer completed four requests then returned 503 after a 35.54 s stall; successful-write p95 125.98 ms only describes those four successes. Added evidence to #1501. The first 5,000-Note candidate had a 30 s timeout, so its successful-write p95 remains unavailable; it is not mislabeled as 30,027 ms successful latency.

Head `4997fb926`: real macOS arm64 confirmation now passes at both sizes. Headful Chromium reports Apple Paravirtual Metal, GPU compositing enabled and rasterization enabled. Trusted CDP Enter to the first frame containing the new name in BOTH sidebar and Tags page: 1,000 Notes 214.80 ms; 5,000 Notes 306.80 ms. HTTP/1.1 request queue 0.80/0.60 ms. These use the local candidate release through a NetBird SSH reverse tunnel, so they are separate rendering confirmation, not substituted into the perf-VM backend pairs. Both local macvm.lock flock and remote /tmp/mac-bench.lock were held; own downloads are removed. No trust settings or foreign accounts changed. Evidence: `docs/perf/runs/tag-rewrite-1186-tagperf4-mac.json`. Ten of twelve interleaved perf-VM profiles are collected. The third 5,000-Note baseline and candidate wait for the shared perf reservation. Second candidate completes in 137.39 s, UI 100.40 ms, read p95 211.48 ms. Its other-User writer completed four requests then returned 503 after a 35.54 s stall; successful-write p95 125.98 ms only describes those four successes. Added evidence to #1501. The first 5,000-Note candidate had a 30 s timeout, so its successful-write p95 remains unavailable; it is not mislabeled as 30,027 ms successful latency.

kayg commented

2026-10-09 07:28:49 +00:00

Author

Owner

Head e1f7e492524e50323e7d4d105b9c819ad21d5130 on job/tagperf-1186. No pushes, deploys, issue closes or merge into dev were performed.

Built: full-quota Undo returns 507 with Home storage quota reached; free space and retry, preserves source bytes and journals, and can be retried after space is freed. The new Rust regression and real-server probe cover a 64-Note durable prefix out of 128. Refused Undo changes no Note or Tag count; after raising quota, Undo restores all 128 Notes byte-exact. The merged release build passes this probe. Existing focused publication/foreign-User failure probes also pass.

The paired profile measures trusted CDP Enter to first rAF containing the new Tag in the sidebar and Tags page, background receipt completion, sixteen bounded Tag readers, and a separate User conditional Note-body writer. The writer waits 250 ms after each response and reuses its ETag. CPU, RSS, descriptors, HTTP protocol, queue time and load5 are retained. Fresh fixtures and identical e1672b919 production SPA assets isolate server builds. origin/dev is pinned at cc1af3c12; candidate Rust behavior is from a77e71920. Later Rust changes are doc comments only.

Performance measurements run on perf-test-69-63 under separate perf-run reservations. The candidate binary SHA-256 is 05cb9d89046ddc5ad5a410c9623c1b986a173add05239118235534a12139bdfd; baseline is 6c25a4023cebe0e6baf1230270af5d864c9931b8bda9eec9d4bca77178357fb2. Builds run on the build host, not the perf VM. These are unthrottled Linux headless Chromium timings at 1440 × 900 with macOS platform emulation. Separate reserved headful macOS Apple Metal checks with the candidate embedded SPA showed both views at 214.80 ms for 1,000 Notes and 306.80 ms for 5,000 Notes. Their server ran locally through a NetBird SSH tunnel, so they are rendering confirmations rather than perf-VM backend pairs. HTTP/1.1 queue time was 0.80/0.60 ms. GPU compositing and rasterization were enabled; own downloads/processes and both reservations were cleaned up.

Notes	Pair	Build	Tags page ms	Sidebar ms	Completion s	Tag-read p95 ms	Other-User write attempt p95 ms	load5 start/end
1,000	1	origin/dev	>5,000	>5,000	>900.00	46.98	571.91	1.95 / 6.22
1,000	1	candidate	115.00	115.00	25.70	56.42	230.56	6.38 / 6.29
1,000	2	origin/dev	>5,000	>5,000	869.06	47.58	471.26	4.69 / 5.79
1,000	2	candidate	331.90	331.90	25.83	84.20	283.42	5.65 / 5.59
1,000	3	origin/dev	>900,000	>900,000	>900.02	49.50	606.03	5.51 / 6.03
1,000	3	candidate	130.30	130.30	25.76	113.17	331.74	5.96 / 5.86
5,000	1	origin/dev	>900,000	>900,000	>900.02	252.62	819.00	5.80 / 7.31
5,000	1	candidate	195.40	195.40	128.19	189.66	30027.68 (timeout)	4.16 / 5.23
5,000	2	origin/dev	>900,000	>900,000	>900.04	232.80	744.93	2.78 / 6.15
5,000	2	candidate	100.40	100.40	137.39	211.48	35536.03 (503)	3.60 / 4.38
5,000	3	origin/dev	>900,000	>900,000	>900.04	240.36	699.93	4.03 / 6.98
5,000	3	candidate	435.20	435.20	154.36	229.12	285.37	6.78 / 6.80

Write attempt p95 includes failed requests; a timeout is a client wait cap, not a successful write or completed server latency. The writer stops on an error; successful percentiles cover only requests before that stop. A > value is a measured lower bound, not a completed latency. The initial pair and second baseline used a five-second UI observation window. Later profiles keep the observer active through the 900-second receipt window. Index verification occurs after the terminal receipt and is not claimed as earliest API visibility. Server measurement ends before the separate two-second final UI poll allowance. No capped or failed sample is turned into an exact completion time.

Expected cost: O(candidate source bytes), one Home quota walk per worker attempt, and ceil(N / 64) durable source batches with eight file-IO workers and one Index transaction per batch. This job changes storage error reporting and the profile; it does not change the worker algorithm.

Files:

crates/calternal-tags/src/lib.rs, crates/calternal-tags/src/rewrite.rs.
tests/adversarial/tag-rewrite-quota-1186.mjs, tag-rewrite-fixture-1186.mjs, tag-rewrite-failure-1186.mjs.
bench/tag-rewrite-1186.py, tag-rewrite-1186-browser.mjs, tag-rewrite-1186-fixture.py, tag-rewrite-1186-paired.sh.
docs/perf/2026-10-09-tagperf4-1186.md, docs/perf/runs/tag-rewrite-1186-tagperf4.json, tag-rewrite-1186-tagperf4-mac.json, scoped metrics.tag_rename_1186_tagperf4 in docs/perf/baseline.json.

Gates (verbatim summaries; all exit 0):

cargo fmt --check emits no output and exits 0. It also passes after the final Rust doc-comment addition. The commands use CARGO_BUILD_JOBS=4, CARGO_INCREMENTAL=0, CARGO_PROFILE_DEV_DEBUG=line-tables-only and the worktree TMPDIR.

cargo clippy -p calternal-tags --all-targets -- -D warnings

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.09s

cargo test -p calternal-tags -- --test-threads=4

    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 06s
test result: ok. 68 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 196.38s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo clippy -p calternal-server --all-targets -- -D warnings

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 10m 25s

cargo test -p calternal-server -- --test-threads=4

    Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 56s
test result: ok. 273 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 280.84s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.66s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s

cd apps/web && bun run check

perf-lint: PASS; 0 violations; 22532 scoped exceptions
svelte-check found 0 errors and 4 warnings in 2 files

cd apps/web && bun run test --maxWorkers=2

 Test Files  301 passed (301)
      Tests  2178 passed (2178)

Known gaps:

The perf-VM pairs remain Linux headless timings; separate real macOS GPU rendering confirmation passes at both sizes. The Mac check does not measure background completion or independent-write p95.
The existing Undo toast keeps its general failure text. The API now gives the storage cause and retry action; the Background Work error formatter gives the same cause. Production UI code was not changed beyond required merge conflict resolution.
Background completion is 25.70–25.83 s at 1,000 Notes and 128.19–154.36 s at 5,000 Notes. The 1,000-Note Tag-read p95 regression is filed as #1499; startup work overlaps both builds, but short candidate windows give it more weight. These measurements do not isolate steady-state route cost.
The first 5,000-Note candidate write probe stops on a 30-second timeout. Successful-write p95 is unavailable for that diagnostic; its 37.10-second acceptance and cross-User stall are filed as #1501. The second candidate uses a 120-second cap and confirms HTTP 503 after a 35.54-second independent-write stall. Its successful p95 covers only four writes. The third candidate completes 286 writes with no errors, p95 285.37 ms and maximum 40,675.46 ms. It confirms the initial stall even when the write succeeds. Later profiles separate successful p95 from failed attempts and retain maximum stalls.
The default SSH 1,024-descriptor diagnostic hit os error 24 in origin/dev and one other-User write returned 500. It is filed as #1497 and excluded from comparisons. Valid comparisons use the same explicit 65,536 limit for both builds.

UX gaps closed: full-quota refusal has a clear API message and keeps all Note bytes consistent; retry can restore every Note exactly. UX gaps left: the existing Undo toast does not state the storage cause. No visible controls or styles were added.

Decisions:

Keep DESIGN §5 quota checks for restores and Versions. Refuse Undo with 507; do not bypass quota or delete Versions. Force a deterministic full-Home state after one 64-Note batch in the disposable probe.
Pin origin/dev and freeze the same production SPA for both builds. Use fresh 1,000/5,000-Note fixtures, sixteen readers with a one-second post-response delay, a conditional writer with a 250 ms delay, and a 900-second receipt cap. Capped measurements remain bounds.
Use a 65,536 descriptor budget for the comparison and file the default-limit failure separately. This is a profile choice, not a verified production service setting.
Keep dev #1391 quiet programmatic focus during the required merge. The branch focus test expectation changes from undefined to false because the merged owner behavior explicitly changed. Combine both sides of menu focus options. The upgrade test count changes from 25 to 26 because both pending migrations must apply. Exact perf pins were regenerated with 22,532 scoped exceptions and zero violations; no rule or ratchet was weakened.

For the merge round:

ADVERSARIAL_SERVER_BIN=<combined-server> tests/adversarial/run-split.sh: run the combined-branch XUser/authz/hostile-input matrices. This job runs its focused quota and publication regressions only.
Under both Mac reservation guards, run TAG_REWRITE_HEADLESS=0 node bench/tag-rewrite-1186-browser.mjs with the matching SPA, private disposable installation credential and tunneled server. Confirm the UI clock on real macOS GPU with trusted CDP Enter. Keep downloads under the job directory and remove them after the run.

No new dependencies or migrations were added. Tags migration 0007 was free on the fetched origin/dev before final gates.

All six candidate name-appearance samples meet 500 ms in both views: 115.00–331.90 ms at 1,000 Notes and 100.40–435.20 ms at 5,000 Notes. All six completed receipts pass source-byte verification. Five baseline receipts reach the 900-second cap; one completes in 869.06 seconds. The 1,000-Note median per-run Tag-read p95 is 84.20 versus 47.58 ms (+76.97%, #1499). At 5,000 Notes it is 211.48 versus 240.36 ms. All twelve profiles have zero Tag-read errors. The independent-write stall/error in #1501 remains unresolved; meeting the name-appearance target does not resolve it.

Resource and write-failure evidence:

These values cover the measured rewrite window. Incomplete samples stop at
the receipt cap; their CPU is not the full operation cost. RSS uses MiB.

Notes	Pair	Build	Outcome	CPU s	Mean RSS MiB	Peak RSS MiB
1,000	1	baseline	incomplete	845.10	309.8	352.2
1,000	1	candidate	completed	28.71	220.4	278.9
1,000	2	baseline	completed	886.59	315.7	364.4
1,000	2	candidate	completed	33.38	231.0	288.5
1,000	3	baseline	incomplete	818.87	319.3	352.9
1,000	3	candidate	completed	30.75	231.6	293.4
5,000	1	baseline	incomplete	1398.68	363.9	387.0
5,000	1	candidate	completed	298.38	348.0	383.5
5,000	2	baseline	incomplete	1369.02	386.8	444.7
5,000	2	candidate	completed	318.31	350.9	384.6
5,000	3	baseline	incomplete	1379.18	395.7	448.6
5,000	3	candidate	completed	353.04	350.2	381.8

Independent-write failures

The writer stops on an error. It does not replay a write with an unknown
outcome. Successful p95 covers only the requests before that stop.
The following attempt p95 includes the failed request. A transport timeout
is a cap, not a successful response or a completed server latency.

Notes	Pair	Build	Successful/attempted	Attempt p95 ms	Failure
5,000	1	candidate	4/5	30027.68	transport timeout
5,000	2	candidate	4/5	35536.03	503

Cleanup: cargo clean completed with Removed 27726 files, 17.1GiB total. The local production build directory was deleted. Own Mac downloads and processes were removed, and both Mac locks were released. Private perf-VM fixtures and public profiles remain in the job directory for reproduction; each measured server and controller exited with its reservation. No secret was displayed.

Head `e1f7e492524e50323e7d4d105b9c819ad21d5130` on `job/tagperf-1186`. No pushes, deploys, issue closes or merge into dev were performed. Built: full-quota Undo returns 507 with `Home storage quota reached; free space and retry`, preserves source bytes and journals, and can be retried after space is freed. The new Rust regression and real-server probe cover a 64-Note durable prefix out of 128. Refused Undo changes no Note or Tag count; after raising quota, Undo restores all 128 Notes byte-exact. The merged release build passes this probe. Existing focused publication/foreign-User failure probes also pass. The paired profile measures trusted CDP Enter to first rAF containing the new Tag in the sidebar and Tags page, background receipt completion, sixteen bounded Tag readers, and a separate User conditional Note-body writer. The writer waits 250 ms after each response and reuses its ETag. CPU, RSS, descriptors, HTTP protocol, queue time and load5 are retained. Fresh fixtures and identical e1672b919 production SPA assets isolate server builds. origin/dev is pinned at cc1af3c12; candidate Rust behavior is from a77e71920. Later Rust changes are doc comments only. Performance measurements run on perf-test-69-63 under separate perf-run reservations. The candidate binary SHA-256 is `05cb9d89046ddc5ad5a410c9623c1b986a173add05239118235534a12139bdfd`; baseline is `6c25a4023cebe0e6baf1230270af5d864c9931b8bda9eec9d4bca77178357fb2`. Builds run on the build host, not the perf VM. These are unthrottled Linux headless Chromium timings at 1440 × 900 with macOS platform emulation. Separate reserved headful macOS Apple Metal checks with the candidate embedded SPA showed both views at 214.80 ms for 1,000 Notes and 306.80 ms for 5,000 Notes. Their server ran locally through a NetBird SSH tunnel, so they are rendering confirmations rather than perf-VM backend pairs. HTTP/1.1 queue time was 0.80/0.60 ms. GPU compositing and rasterization were enabled; own downloads/processes and both reservations were cleaned up. | Notes | Pair | Build | Tags page ms | Sidebar ms | Completion s | Tag-read p95 ms | Other-User write attempt p95 ms | load5 start/end | | ---: | ---: | --- | ---: | ---: | ---: | ---: | ---: | --- | | 1,000 | 1 | origin/dev | >5,000 | >5,000 | >900.00 | 46.98 | 571.91 | 1.95 / 6.22 | | 1,000 | 1 | candidate | 115.00 | 115.00 | 25.70 | 56.42 | 230.56 | 6.38 / 6.29 | | 1,000 | 2 | origin/dev | >5,000 | >5,000 | 869.06 | 47.58 | 471.26 | 4.69 / 5.79 | | 1,000 | 2 | candidate | 331.90 | 331.90 | 25.83 | 84.20 | 283.42 | 5.65 / 5.59 | | 1,000 | 3 | origin/dev | >900,000 | >900,000 | >900.02 | 49.50 | 606.03 | 5.51 / 6.03 | | 1,000 | 3 | candidate | 130.30 | 130.30 | 25.76 | 113.17 | 331.74 | 5.96 / 5.86 | | 5,000 | 1 | origin/dev | >900,000 | >900,000 | >900.02 | 252.62 | 819.00 | 5.80 / 7.31 | | 5,000 | 1 | candidate | 195.40 | 195.40 | 128.19 | 189.66 | 30027.68 (timeout) | 4.16 / 5.23 | | 5,000 | 2 | origin/dev | >900,000 | >900,000 | >900.04 | 232.80 | 744.93 | 2.78 / 6.15 | | 5,000 | 2 | candidate | 100.40 | 100.40 | 137.39 | 211.48 | 35536.03 (503) | 3.60 / 4.38 | | 5,000 | 3 | origin/dev | >900,000 | >900,000 | >900.04 | 240.36 | 699.93 | 4.03 / 6.98 | | 5,000 | 3 | candidate | 435.20 | 435.20 | 154.36 | 229.12 | 285.37 | 6.78 / 6.80 | Write attempt p95 includes failed requests; a timeout is a client wait cap, not a successful write or completed server latency. The writer stops on an error; successful percentiles cover only requests before that stop. A `>` value is a measured lower bound, not a completed latency. The initial pair and second baseline used a five-second UI observation window. Later profiles keep the observer active through the 900-second receipt window. Index verification occurs after the terminal receipt and is not claimed as earliest API visibility. Server measurement ends before the separate two-second final UI poll allowance. No capped or failed sample is turned into an exact completion time. Expected cost: O(candidate source bytes), one Home quota walk per worker attempt, and ceil(N / 64) durable source batches with eight file-IO workers and one Index transaction per batch. This job changes storage error reporting and the profile; it does not change the worker algorithm. Files: - `crates/calternal-tags/src/lib.rs`, `crates/calternal-tags/src/rewrite.rs`. - `tests/adversarial/tag-rewrite-quota-1186.mjs`, `tag-rewrite-fixture-1186.mjs`, `tag-rewrite-failure-1186.mjs`. - `bench/tag-rewrite-1186.py`, `tag-rewrite-1186-browser.mjs`, `tag-rewrite-1186-fixture.py`, `tag-rewrite-1186-paired.sh`. - `docs/perf/2026-10-09-tagperf4-1186.md`, `docs/perf/runs/tag-rewrite-1186-tagperf4.json`, `tag-rewrite-1186-tagperf4-mac.json`, scoped `metrics.tag_rename_1186_tagperf4` in `docs/perf/baseline.json`. Gates (verbatim summaries; all exit 0): `cargo fmt --check` emits no output and exits 0. It also passes after the final Rust doc-comment addition. The commands use CARGO_BUILD_JOBS=4, CARGO_INCREMENTAL=0, CARGO_PROFILE_DEV_DEBUG=line-tables-only and the worktree TMPDIR. `cargo clippy -p calternal-tags --all-targets -- -D warnings` ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.09s ``` `cargo test -p calternal-tags -- --test-threads=4` ``` Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 06s test result: ok. 68 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 196.38s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo clippy -p calternal-server --all-targets -- -D warnings` ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 10m 25s ``` `cargo test -p calternal-server -- --test-threads=4` ``` Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 56s test result: ok. 273 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 280.84s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 33.66s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s ``` `cd apps/web && bun run check` ``` perf-lint: PASS; 0 violations; 22532 scoped exceptions svelte-check found 0 errors and 4 warnings in 2 files ``` `cd apps/web && bun run test --maxWorkers=2` ``` Test Files 301 passed (301) Tests 2178 passed (2178) ``` Known gaps: - The perf-VM pairs remain Linux headless timings; separate real macOS GPU rendering confirmation passes at both sizes. The Mac check does not measure background completion or independent-write p95. - The existing Undo toast keeps its general failure text. The API now gives the storage cause and retry action; the Background Work error formatter gives the same cause. Production UI code was not changed beyond required merge conflict resolution. - Background completion is 25.70–25.83 s at 1,000 Notes and 128.19–154.36 s at 5,000 Notes. The 1,000-Note Tag-read p95 regression is filed as #1499; startup work overlaps both builds, but short candidate windows give it more weight. These measurements do not isolate steady-state route cost. - The first 5,000-Note candidate write probe stops on a 30-second timeout. Successful-write p95 is unavailable for that diagnostic; its 37.10-second acceptance and cross-User stall are filed as #1501. The second candidate uses a 120-second cap and confirms HTTP 503 after a 35.54-second independent-write stall. Its successful p95 covers only four writes. The third candidate completes 286 writes with no errors, p95 285.37 ms and maximum 40,675.46 ms. It confirms the initial stall even when the write succeeds. Later profiles separate successful p95 from failed attempts and retain maximum stalls. - The default SSH 1,024-descriptor diagnostic hit os error 24 in origin/dev and one other-User write returned 500. It is filed as #1497 and excluded from comparisons. Valid comparisons use the same explicit 65,536 limit for both builds. UX gaps closed: full-quota refusal has a clear API message and keeps all Note bytes consistent; retry can restore every Note exactly. UX gaps left: the existing Undo toast does not state the storage cause. No visible controls or styles were added. Decisions: - Keep DESIGN §5 quota checks for restores and Versions. Refuse Undo with 507; do not bypass quota or delete Versions. Force a deterministic full-Home state after one 64-Note batch in the disposable probe. - Pin origin/dev and freeze the same production SPA for both builds. Use fresh 1,000/5,000-Note fixtures, sixteen readers with a one-second post-response delay, a conditional writer with a 250 ms delay, and a 900-second receipt cap. Capped measurements remain bounds. - Use a 65,536 descriptor budget for the comparison and file the default-limit failure separately. This is a profile choice, not a verified production service setting. - Keep dev #1391 quiet programmatic focus during the required merge. The branch focus test expectation changes from undefined to false because the merged owner behavior explicitly changed. Combine both sides of menu focus options. The upgrade test count changes from 25 to 26 because both pending migrations must apply. Exact perf pins were regenerated with 22,532 scoped exceptions and zero violations; no rule or ratchet was weakened. For the merge round: - `ADVERSARIAL_SERVER_BIN=<combined-server> tests/adversarial/run-split.sh`: run the combined-branch XUser/authz/hostile-input matrices. This job runs its focused quota and publication regressions only. - Under both Mac reservation guards, run `TAG_REWRITE_HEADLESS=0 node bench/tag-rewrite-1186-browser.mjs` with the matching SPA, private disposable installation credential and tunneled server. Confirm the UI clock on real macOS GPU with trusted CDP Enter. Keep downloads under the job directory and remove them after the run. No new dependencies or migrations were added. Tags migration 0007 was free on the fetched origin/dev before final gates. All six candidate name-appearance samples meet 500 ms in both views: 115.00–331.90 ms at 1,000 Notes and 100.40–435.20 ms at 5,000 Notes. All six completed receipts pass source-byte verification. Five baseline receipts reach the 900-second cap; one completes in 869.06 seconds. The 1,000-Note median per-run Tag-read p95 is 84.20 versus 47.58 ms (+76.97%, #1499). At 5,000 Notes it is 211.48 versus 240.36 ms. All twelve profiles have zero Tag-read errors. The independent-write stall/error in #1501 remains unresolved; meeting the name-appearance target does not resolve it. Resource and write-failure evidence: These values cover the measured rewrite window. Incomplete samples stop at the receipt cap; their CPU is not the full operation cost. RSS uses MiB. | Notes | Pair | Build | Outcome | CPU s | Mean RSS MiB | Peak RSS MiB | | ---: | ---: | --- | --- | ---: | ---: | ---: | | 1,000 | 1 | baseline | incomplete | 845.10 | 309.8 | 352.2 | | 1,000 | 1 | candidate | completed | 28.71 | 220.4 | 278.9 | | 1,000 | 2 | baseline | completed | 886.59 | 315.7 | 364.4 | | 1,000 | 2 | candidate | completed | 33.38 | 231.0 | 288.5 | | 1,000 | 3 | baseline | incomplete | 818.87 | 319.3 | 352.9 | | 1,000 | 3 | candidate | completed | 30.75 | 231.6 | 293.4 | | 5,000 | 1 | baseline | incomplete | 1398.68 | 363.9 | 387.0 | | 5,000 | 1 | candidate | completed | 298.38 | 348.0 | 383.5 | | 5,000 | 2 | baseline | incomplete | 1369.02 | 386.8 | 444.7 | | 5,000 | 2 | candidate | completed | 318.31 | 350.9 | 384.6 | | 5,000 | 3 | baseline | incomplete | 1379.18 | 395.7 | 448.6 | | 5,000 | 3 | candidate | completed | 353.04 | 350.2 | 381.8 | ## Independent-write failures The writer stops on an error. It does not replay a write with an unknown outcome. Successful p95 covers only the requests before that stop. The following attempt p95 includes the failed request. A transport timeout is a cap, not a successful response or a completed server latency. | Notes | Pair | Build | Successful/attempted | Attempt p95 ms | Failure | | ---: | ---: | --- | --- | ---: | --- | | 5,000 | 1 | candidate | 4/5 | 30027.68 | transport timeout | | 5,000 | 2 | candidate | 4/5 | 35536.03 | 503 | Cleanup: `cargo clean` completed with ` Removed 27726 files, 17.1GiB total`. The local production build directory was deleted. Own Mac downloads and processes were removed, and both Mac locks were released. Private perf-VM fixtures and public profiles remain in the job directory for reproduction; each measured server and controller exited with its reservation. No secret was displayed.

kayg commented

2026-10-09 07:31:59 +00:00

Author

Owner

tagfix5 starts on job/tagperf-1186, base/head e1f7e49252. Read #1501, #1499 and tagperf4 final evidence. Trace namespace lock, per-User writer lock and SQLite acceptance projection before fixing cross-User stalls. Preserve FULL acceptance and all publication/Undo recovery tests. No pushes or deployment.

tagfix5 starts on job/tagperf-1186, base/head e1f7e492524e50323e7d4d105b9c819ad21d5130. Read #1501, #1499 and tagperf4 final evidence. Trace namespace lock, per-User writer lock and SQLite acceptance projection before fixing cross-User stalls. Preserve FULL acceptance and all publication/Undo recovery tests. No pushes or deployment.

kayg commented

2026-10-09 07:40:34 +00:00

Author

Owner

Measured #1501 cause in the Rust test build: the former acceptance query holds the FULL authority SQLite write transaction while a different writer obtains its ordinary pool connection in 0.098 ms, then waits 5006.733 ms in BEGIN IMMEDIATE and returns SQLite BUSY. The diagnostic requests no User/namespace mutex, so SQLite write-lock contention alone is sufficient. Notes projection uses this same BEGIN IMMEDIATE path and bounded BUSY retries (crates/plugins/notes/src/store.rs), which explains repeated five-second waits and eventual 503. The namespace lock is global but a plain conditional Note-body write does not acquire it; its Notes mutex is per User. The current fix removes the acceptance namespace lock, saves intent only, stages 64 witnesses per background transaction, and forces witness-first owner+path indexed probes. The 5,000-Note admission regression passes while the namespace lock is held. Full safety gates and real-server/paired evidence are in progress.

kayg commented

2026-10-09 07:56:45 +00:00

Author

Owner

Head after the required origin/dev merge: ef625e246c. Real-server #1501 regression passed on 2,000 Notes with two real Users: acceptance 156.92 ms, 20/20 successful conditional writes, other-User p95 46.11 ms and max 69.37 ms; the forward receipt completed. The former-query Rust test build isolated writer-pool checkout (0.098 ms) from SQLite BEGIN IMMEDIATE BUSY wait (5006.733 ms), while the authority projection took 37783.513 ms. No User or namespace mutex was requested by this diagnostic writer. Tags safety suite: 69 passed, 0 failed, one time-boxed diagnostic ignored by default; the diagnostic was run separately and passed. Web: 305 test files / 2218 tests passed; check has zero errors, three existing warnings. Final merged Rust gates and reserved 5,000-Note A/B samples remain in progress. Merge decisions: combine optional parent proof capture with dev directory proof pins; retain Tags 0007 plus dev Calendar/Mail/Share migration frontiers (34 pending receipts), preserve both perf records, refresh only exact live perf scopes (22260) without changing rules. No new migration or dependency.

Head after the required origin/dev merge: ef625e246c210af1ce9296b9b6bea5f90337ba77. Real-server #1501 regression passed on 2,000 Notes with two real Users: acceptance 156.92 ms, 20/20 successful conditional writes, other-User p95 46.11 ms and max 69.37 ms; the forward receipt completed. The former-query Rust test build isolated writer-pool checkout (0.098 ms) from SQLite BEGIN IMMEDIATE BUSY wait (5006.733 ms), while the authority projection took 37783.513 ms. No User or namespace mutex was requested by this diagnostic writer. Tags safety suite: 69 passed, 0 failed, one time-boxed diagnostic ignored by default; the diagnostic was run separately and passed. Web: 305 test files / 2218 tests passed; check has zero errors, three existing warnings. Final merged Rust gates and reserved 5,000-Note A/B samples remain in progress. Merge decisions: combine optional parent proof capture with dev directory proof pins; retain Tags 0007 plus dev Calendar/Mail/Share migration frontiers (34 pending receipts), preserve both perf records, refresh only exact live perf scopes (22260) without changing rules. No new migration or dependency.

kayg commented

2026-10-09 08:56:04 +00:00

Author

Owner

The final release binary passed the real-server two-User regression under a reserved perf-run session on perf-test-69-63 (load5 4.01). User A renamed a Tag on 2,000 Notes while User B made 20 conditional Note-body PUTs. All 20 succeeded. Acceptance: 149.601 ms; other-User write p95: 311.465 ms; maximum: 330.594 ms. The test enforces an independent 1-second cap for every write and verifies completed receipt and final bytes. No timeout assertion was weakened.

Two 5,000-Note candidate samples now completed: acceptance 246.8 / 393.8 ms; other-User write maxima 511.36 / 445.66 ms; durable completion 109.17 / 120.65 s. The third paired sample is running. Tag-read p95 is mixed (196.00 / 234.42 ms against origin/dev 215.81 / 203.31 ms), so #1499 is not marked resolved; a matched 1,000-Note pair is queued.

The headful real-macOS production Tags view showed the new Tag on page and sidebar 340.6 ms after trusted CDP Enter. HTTP/1.1 queue was 0.5 ms. This uses a remote browser with a local-server tunnel; server response timing includes the shared build host. Mac job directory and reservation lock are verified removed.

The final release binary passed the real-server two-User regression under a reserved perf-run session on perf-test-69-63 (load5 4.01). User A renamed a Tag on 2,000 Notes while User B made 20 conditional Note-body PUTs. All 20 succeeded. Acceptance: 149.601 ms; other-User write p95: 311.465 ms; maximum: 330.594 ms. The test enforces an independent 1-second cap for every write and verifies completed receipt and final bytes. No timeout assertion was weakened. Two 5,000-Note candidate samples now completed: acceptance 246.8 / 393.8 ms; other-User write maxima 511.36 / 445.66 ms; durable completion 109.17 / 120.65 s. The third paired sample is running. Tag-read p95 is mixed (196.00 / 234.42 ms against origin/dev 215.81 / 203.31 ms), so #1499 is not marked resolved; a matched 1,000-Note pair is queued. The headful real-macOS production Tags view showed the new Tag on page and sidebar 340.6 ms after trusted CDP Enter. HTTP/1.1 queue was 0.5 ms. This uses a remote browser with a local-server tunnel; server response timing includes the shared build host. Mac job directory and reservation lock are verified removed.

kayg commented

2026-10-09 09:31:18 +00:00

Author

Owner

Final lock review found a receipt-retry lock inversion: start_rewrite retained the requesting User writer mutex while calling rewrite_status; a saved failed/orphan receipt can run repair_accepted, which acquires that same mutex. Admission now releases the guard before either saved-receipt branch. A focused orphan-retry regression passed and checks failed status, stable receipt identity, source-backed names, Undo and exact bytes. Final crate checks are running.

The first parallel suite then exposed interference in this job's 5,000-Note timing fixture: it shared the process-wide User mutex with other fixtures. The test now uses its own User/Home. Its one-second timeout and all assertions are unchanged; no existing product expectation was weakened. The final isolated fixture and suite are being checked.

Final lock review found a receipt-retry lock inversion: start_rewrite retained the requesting User writer mutex while calling rewrite_status; a saved failed/orphan receipt can run repair_accepted, which acquires that same mutex. Admission now releases the guard before either saved-receipt branch. A focused orphan-retry regression passed and checks failed status, stable receipt identity, source-backed names, Undo and exact bytes. Final crate checks are running. The first parallel suite then exposed interference in this job's 5,000-Note timing fixture: it shared the process-wide User mutex with other fixtures. The test now uses its own User/Home. Its one-second timeout and all assertions are unchanged; no existing product expectation was weakened. The final isolated fixture and suite are being checked.

kayg commented

2026-10-09 10:12:51 +00:00

Author

Owner

tagfix5 / #1186 / #1501 — final report

Head: 4c4734791c on job/tagperf-1186. Start: e1f7e49252. Merged origin/dev once at 9dc65437e8 before gates. No push, deploy or merge into dev/main.

Built

Acceptance records the immutable intent and queue receipt in a FULL transaction; forward projection is deferred to the background worker.
Projection uses indexed witness seeks, at most 64 source paths per transaction, and yields after each commit. Per-User admission locks remain keyed by User; acceptance does not acquire the root namespace lock.
Saved exact membership names make child-to-parent and nested namespace replay idempotent. Legacy receipts remain readable. Distinct item counts do not count each membership twice.
Added a real-server two-User test with >=2,000 Notes and 20 conditional writes, each capped at 1 second; no unknown outcome is retried.

Files changed by this job

crates/calternal-tags/src/{lib.rs,rewrite.rs}; docs/DESIGN.md
tests/adversarial/tag-rewrite-cross-user-1501.mjs
bench/tag-rewrite-1186-paired.sh (isolated run root)
docs/perf/2026-10-09-tagfix5-1501.md, docs/perf/runs/tag-rewrite-1186-tagfix5.json, docs/perf/baseline.json
Merge resolutions: calternal-fs/src/write.rs; calternal-server/src/upgrade_tests.rs; exact live contracts/perf pins. Both directory-proof and batch-proof invariants are retained. Server upgrade frontiers include Tags 7 and the new dev migrations. The merged frontier-count assertion is 34 (dev 33 plus this branch’s existing Tags migration); content assertions are retained. No new migration was added by tagfix5.

Cause
A test-build diagnostic measured writer-pool checkout 0.098 ms, then BEGIN IMMEDIATE wait 5,006.733 ms with SQLITE_BUSY, while the former authority projection held its write transaction for 37,783.513 ms. The competing writer requested neither namespace nor User lock. The normal Note-body route uses a User-keyed mutex and retries SQLite busy writes. The blocking resource was SQLite's writer, not pool checkout or a cross-User mutex.

Performance (three 5,000-Note pairs on perf-test-69-63)
Frozen baseline: origin/dev 9dc65437e; frozen forward-path candidate source: 9f24ac50a; production SPA: e1672b919. Every measured run held perf-run's single shared lock. The later 6b000c795 retry-lock fix does not change the measured new-intent forward path. Raw records preserve hashes, CPU/RSS, actual run order and load5 in docs/perf/runs/tag-rewrite-1186-tagfix5.json.

Pair	Build	Acceptance ms	UI page/sidebar ms	Completion s	Other-User write p95/max ms	Tag read p95 ms	load5 before/after
1	origin/dev	490.20	>900,376	>900.05	649.23 / 1814.89	215.81	3.05 / 7.10
1	candidate	246.80	90.80	109.17	197.57 / 511.36	196.00	2.17 / 3.47
2	origin/dev	146.10	>900,003.80	>900.05	854.48 / 2670.44	203.31	1.08 / 6.72
2	candidate	393.80	138.30	120.65	233.29 / 445.66	234.42	2.69 / 4.01
3	origin/dev	260.10	>900,626.40	>900.01	635.40 / 2443.00	208.33	3.68 / 6.78
3	candidate	365.50	194.00	111.95	223.17 / 418.92	197.99	5.75 / 6.27

All candidate samples completed and verified every source. Their 309/327/303 independent writes had zero errors. No capped baseline is represented as completed latency. Expected cost: accepted witness rows times <=2 exact membership-name seeks per projection table, bounded transactions of <=64 source paths, candidate source bytes plus one Home quota walk per worker attempt. Candidate CPU: 232.99/259.90/238.29 s; average RSS: 373946317/387788114/371558625 bytes; peak: 411766784/422440960/405184512 bytes. Baseline CPU: 1400.15/1416.29/1386.69 s across its longer windows; raw record includes all RSS and descriptor values.

#1499 / UX gap left: the matched 1,000-Note pair still shows a Tag-read tail regression: p95 47.36 -> 60.81 ms (+28.4%), p50 22.12 -> 20.47 ms. Candidate completed in 24.249 s; baseline was incomplete at 847/1000 after 900 s. Terminal candidate ordinary-read p95 was 29.04 ms. The reader-pool recursive prefix grouping still does work per request; shorter startup-overlapped windows weight initial bursts more heavily. CPU/query and window weighting are supported explanations, not isolated attribution. #1499 remains open, with its numbers posted. This is a performance gap, not a claimed resolution.

Safety and UX gaps closed
The 71-test Tags suite covers subprocess SIGKILL windows, retry, failed-prefix repair, concurrent source edits, byte-exact Undo and quota. The new overlapping namespace replay test passed. Final review also fixed an admission/repair self-deadlock on saved receipt retries: release the User mutex before terminal repair. Its regression verifies stable receipt identity, failure, source-backed projection, Undo and unchanged source bytes. The new 5,000-Note timing fixture uses a separate User to avoid unrelated parallel fixture mutex waits; its one-second bound is unchanged. The real-server quota probe passed at a 64/128 prefix: refused Undo returned 507 without changing files; raising quota allowed exact Undo. The real-server failure probe passed with 64 old/64 new Notes, exact foreign bytes (10 Notes), cross-User receipt/Undo 404 and exact Undo of 128 Notes. Malformed receipt and rename probes returned 4xx. The separate two-User probe passed under perf-run: acceptance 149.601 ms, 20/20 writes succeeded, p95 311.465 ms, max 330.594 ms, load5 4.01. Final receipt and other-User bytes were checked.

No visible control or CSS was added. Existing TagRow and ItemActionMenu remain the UI controls. Headful macOS Chromium 155.0.8059.39 showed page/sidebar Tag projection at 340.6 ms after trusted CDP Enter; HTTP/1.1 queue 0.5 ms. Observed response 790.1 ms includes the tunnel and shared build host. This confirms rendering, not the reserved-server acceptance bound. Mac browser, download directory and /tmp/mac-bench.lock were removed and verified.

Gates (verbatim summaries)
cargo fmt --check: exit 0, no output.
cargo clippy -p <crate> --all-targets -- -D warnings and cargo test -p <crate> for calternal-fs, calternal-tags and calternal-server.

calternal-fs-clippy-final
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.17s

calternal-tags-clippy-last
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 58s

calternal-server-clippy-last
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 51s

calternal-fs-test-final
    Finished `test` profile [unoptimized + debuginfo] target(s) in 11.15s
test result: ok. 105 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 52.97s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.22s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.13s
test result: ok. 54 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 317.31s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-tags-test-last
    Finished `test` profile [unoptimized + debuginfo] target(s) in 34.85s
test result: ok. 71 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 170.78s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-server-test-last
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 27s
test result: ok. 274 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 251.60s

server-integration-exact
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 00s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.44s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.34s

web-check-exact
perf-lint: PASS; 0 violations; 22260 scoped exceptions
svelte-check found 0 errors and 3 warnings in 1 file

web-test-last
 Test Files  305 passed (305)
      Tests  2218 passed (2218)
   Duration  204.89s (environment 28%, transform 25%, import 24%, tests 17%, setup 5%)

Final server unit command: cargo test -p calternal-server -- --test-threads=4 (274 passed). Final integration command after exact pin refresh: cargo test -p calternal-server --test perf_guards --test private_index_permissions (both passed). The first retry-check server run had a startup/backfill timeout; its failing group passed alone, then all server units passed. That run also exposed a stale performance exception fingerprint after the retry-body change. Only the existing live start_rewrite:mutation hash was refreshed in exceptions/adoption; no rule, limit or ratchet changed. The final integration checks and bun run check both passed with 22,260 scoped exceptions. The web test command passed all 305 files / 2,218 tests. These corrected failures are retained in artifacts.

All Rust commands used CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_INCREMENTAL=0 and CARGO_BUILD_JOBS=4. Web gates were cd apps/web && bun run check and bun run test --maxWorkers=2. The ignored diagnostic was run separately and passed. Only the documented projection staging boundary changed two existing test setups; their assertions were retained. Existing test expectations were not weakened.

Decisions

Keep the existing HTTP 200 pending receipt contract rather than change all clients to 202. Response status does not change acceptance work.
Reuse the existing 64-source batch bound for projection. Keep all memberships for one path in one batch.
Store each original Tag spelling in the durable witness and add serde defaults for old receipts. This prevents overlapping namespace replay from transforming a projected Tag twice.
Admission keeps this User's writer mutex for Index planning and the short intent commit. It no longer holds a root namespace lock or runs projection in that transaction.

Known gaps / UX gaps left

The comparative profile uses a 65,536 descriptor limit; #1497 remains a production limit issue.
Two local release probes exceeded the first write's 1-second timing cap on the shared build host. They remain failed timing diagnostics, not successful samples. The reserved strict real-server regression passed without changing its bound.
Initial perf attempts failed before measurement because of missing browser helper/broken supplied runtime symlink. They do not count as paired results.
The new forward-path measurements use the frozen 9f24ac50a release binary; the subsequent saved-receipt mutex-release fix is covered by focused/full Rust checks and leaves that measured forward path unchanged.
GPU renderer metadata was not retained from the real Mac run. No new visual styling was changed in this job.

For the merge round

bash tests/adversarial/run.sh — full combined authz/robustness/XUser coverage.
CALTERNAL_SERVER_BIN=<merge binary> CALTERNAL_E2E_ASSET_OVERRIDE=1 bun tests/adversarial/tag-rewrite-cross-user-1501.mjs — repeat the strict User-isolation contract on the combined build.
cd apps/web && bun run test:e2e (use the configured shared-server harness) — full combined UI behaviour; the focused probes and full web unit suite passed here.
The orchestrator owns full workspace gates and deployment checks. No push, deploy or merge into dev/main occurred.

Cleanup
cargo clean first removed 28,022 files / 17.3 GiB. After the late retry checks it removed 18,096 files / 12.3 GiB. apps/web/build was deleted again after its temporary restoration for RustEmbed. Worktree changes are committed. The own perf-VM run directory, binaries and disposable data were also removed.

Diagnostic disclosure
A disposable test session cookie appeared in a local transport-error trace. That disposable Instance was destroyed, so the session is invalid. The local trace was redacted and the new regression now hides transport headers. No live owner credential was read or printed.

tagfix5 / #1186 / #1501 — final report Head: 4c4734791ceeb13e2ba66b9fd58a8c9b45457551 on job/tagperf-1186. Start: e1f7e492524e50323e7d4d105b9c819ad21d5130. Merged origin/dev once at 9dc65437e86c74efe92d723f9a99774246e60650 before gates. No push, deploy or merge into dev/main. Built - Acceptance records the immutable intent and queue receipt in a FULL transaction; forward projection is deferred to the background worker. - Projection uses indexed witness seeks, at most 64 source paths per transaction, and yields after each commit. Per-User admission locks remain keyed by User; acceptance does not acquire the root namespace lock. - Saved exact membership names make child-to-parent and nested namespace replay idempotent. Legacy receipts remain readable. Distinct item counts do not count each membership twice. - Added a real-server two-User test with >=2,000 Notes and 20 conditional writes, each capped at 1 second; no unknown outcome is retried. Files changed by this job - crates/calternal-tags/src/{lib.rs,rewrite.rs}; docs/DESIGN.md - tests/adversarial/tag-rewrite-cross-user-1501.mjs - bench/tag-rewrite-1186-paired.sh (isolated run root) - docs/perf/2026-10-09-tagfix5-1501.md, docs/perf/runs/tag-rewrite-1186-tagfix5.json, docs/perf/baseline.json - Merge resolutions: calternal-fs/src/write.rs; calternal-server/src/upgrade_tests.rs; exact live contracts/perf pins. Both directory-proof and batch-proof invariants are retained. Server upgrade frontiers include Tags 7 and the new dev migrations. The merged frontier-count assertion is 34 (dev 33 plus this branch’s existing Tags migration); content assertions are retained. No new migration was added by tagfix5. Cause A test-build diagnostic measured writer-pool checkout 0.098 ms, then BEGIN IMMEDIATE wait 5,006.733 ms with SQLITE_BUSY, while the former authority projection held its write transaction for 37,783.513 ms. The competing writer requested neither namespace nor User lock. The normal Note-body route uses a User-keyed mutex and retries SQLite busy writes. The blocking resource was SQLite's writer, not pool checkout or a cross-User mutex. Performance (three 5,000-Note pairs on perf-test-69-63) Frozen baseline: origin/dev 9dc65437e; frozen forward-path candidate source: 9f24ac50a; production SPA: e1672b919. Every measured run held perf-run's single shared lock. The later 6b000c795 retry-lock fix does not change the measured new-intent forward path. Raw records preserve hashes, CPU/RSS, actual run order and load5 in docs/perf/runs/tag-rewrite-1186-tagfix5.json. | Pair | Build | Acceptance ms | UI page/sidebar ms | Completion s | Other-User write p95/max ms | Tag read p95 ms | load5 before/after | | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | | 1 | origin/dev | 490.20 | >900,376 | >900.05 | 649.23 / 1814.89 | 215.81 | 3.05 / 7.10 | | 1 | candidate | 246.80 | 90.80 | 109.17 | 197.57 / 511.36 | 196.00 | 2.17 / 3.47 | | 2 | origin/dev | 146.10 | >900,003.80 | >900.05 | 854.48 / 2670.44 | 203.31 | 1.08 / 6.72 | | 2 | candidate | 393.80 | 138.30 | 120.65 | 233.29 / 445.66 | 234.42 | 2.69 / 4.01 | | 3 | origin/dev | 260.10 | >900,626.40 | >900.01 | 635.40 / 2443.00 | 208.33 | 3.68 / 6.78 | | 3 | candidate | 365.50 | 194.00 | 111.95 | 223.17 / 418.92 | 197.99 | 5.75 / 6.27 | All candidate samples completed and verified every source. Their 309/327/303 independent writes had zero errors. No capped baseline is represented as completed latency. Expected cost: accepted witness rows times <=2 exact membership-name seeks per projection table, bounded transactions of <=64 source paths, candidate source bytes plus one Home quota walk per worker attempt. Candidate CPU: 232.99/259.90/238.29 s; average RSS: 373946317/387788114/371558625 bytes; peak: 411766784/422440960/405184512 bytes. Baseline CPU: 1400.15/1416.29/1386.69 s across its longer windows; raw record includes all RSS and descriptor values. #1499 / UX gap left: the matched 1,000-Note pair still shows a Tag-read tail regression: p95 47.36 -> 60.81 ms (+28.4%), p50 22.12 -> 20.47 ms. Candidate completed in 24.249 s; baseline was incomplete at 847/1000 after 900 s. Terminal candidate ordinary-read p95 was 29.04 ms. The reader-pool recursive prefix grouping still does work per request; shorter startup-overlapped windows weight initial bursts more heavily. CPU/query and window weighting are supported explanations, not isolated attribution. #1499 remains open, with its numbers posted. This is a performance gap, not a claimed resolution. Safety and UX gaps closed The 71-test Tags suite covers subprocess SIGKILL windows, retry, failed-prefix repair, concurrent source edits, byte-exact Undo and quota. The new overlapping namespace replay test passed. Final review also fixed an admission/repair self-deadlock on saved receipt retries: release the User mutex before terminal repair. Its regression verifies stable receipt identity, failure, source-backed projection, Undo and unchanged source bytes. The new 5,000-Note timing fixture uses a separate User to avoid unrelated parallel fixture mutex waits; its one-second bound is unchanged. The real-server quota probe passed at a 64/128 prefix: refused Undo returned 507 without changing files; raising quota allowed exact Undo. The real-server failure probe passed with 64 old/64 new Notes, exact foreign bytes (10 Notes), cross-User receipt/Undo 404 and exact Undo of 128 Notes. Malformed receipt and rename probes returned 4xx. The separate two-User probe passed under perf-run: acceptance 149.601 ms, 20/20 writes succeeded, p95 311.465 ms, max 330.594 ms, load5 4.01. Final receipt and other-User bytes were checked. No visible control or CSS was added. Existing TagRow and ItemActionMenu remain the UI controls. Headful macOS Chromium 155.0.8059.39 showed page/sidebar Tag projection at 340.6 ms after trusted CDP Enter; HTTP/1.1 queue 0.5 ms. Observed response 790.1 ms includes the tunnel and shared build host. This confirms rendering, not the reserved-server acceptance bound. Mac browser, download directory and /tmp/mac-bench.lock were removed and verified. Gates (verbatim summaries) `cargo fmt --check`: exit 0, no output. `cargo clippy -p <crate> --all-targets -- -D warnings` and `cargo test -p <crate>` for calternal-fs, calternal-tags and calternal-server. ```text calternal-fs-clippy-final Finished `dev` profile [unoptimized + debuginfo] target(s) in 10.17s calternal-tags-clippy-last Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 58s calternal-server-clippy-last Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 51s calternal-fs-test-final Finished `test` profile [unoptimized + debuginfo] target(s) in 11.15s test result: ok. 105 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 52.97s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.22s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.13s test result: ok. 54 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 317.31s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s calternal-tags-test-last Finished `test` profile [unoptimized + debuginfo] target(s) in 34.85s test result: ok. 71 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 170.78s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s calternal-server-test-last Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 27s test result: ok. 274 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 251.60s server-integration-exact Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 00s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.44s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.34s web-check-exact perf-lint: PASS; 0 violations; 22260 scoped exceptions svelte-check found 0 errors and 3 warnings in 1 file web-test-last Test Files 305 passed (305) Tests 2218 passed (2218) Duration 204.89s (environment 28%, transform 25%, import 24%, tests 17%, setup 5%) ``` Final server unit command: `cargo test -p calternal-server -- --test-threads=4` (274 passed). Final integration command after exact pin refresh: `cargo test -p calternal-server --test perf_guards --test private_index_permissions` (both passed). The first retry-check server run had a startup/backfill timeout; its failing group passed alone, then all server units passed. That run also exposed a stale performance exception fingerprint after the retry-body change. Only the existing live `start_rewrite:mutation` hash was refreshed in exceptions/adoption; no rule, limit or ratchet changed. The final integration checks and `bun run check` both passed with 22,260 scoped exceptions. The web test command passed all 305 files / 2,218 tests. These corrected failures are retained in artifacts. All Rust commands used CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_INCREMENTAL=0 and CARGO_BUILD_JOBS=4. Web gates were `cd apps/web && bun run check` and `bun run test --maxWorkers=2`. The ignored diagnostic was run separately and passed. Only the documented projection staging boundary changed two existing test setups; their assertions were retained. Existing test expectations were not weakened. Decisions - Keep the existing HTTP 200 pending receipt contract rather than change all clients to 202. Response status does not change acceptance work. - Reuse the existing 64-source batch bound for projection. Keep all memberships for one path in one batch. - Store each original Tag spelling in the durable witness and add serde defaults for old receipts. This prevents overlapping namespace replay from transforming a projected Tag twice. - Admission keeps this User's writer mutex for Index planning and the short intent commit. It no longer holds a root namespace lock or runs projection in that transaction. Known gaps / UX gaps left - The comparative profile uses a 65,536 descriptor limit; #1497 remains a production limit issue. - Two local release probes exceeded the first write's 1-second timing cap on the shared build host. They remain failed timing diagnostics, not successful samples. The reserved strict real-server regression passed without changing its bound. - Initial perf attempts failed before measurement because of missing browser helper/broken supplied runtime symlink. They do not count as paired results. - The new forward-path measurements use the frozen 9f24ac50a release binary; the subsequent saved-receipt mutex-release fix is covered by focused/full Rust checks and leaves that measured forward path unchanged. - GPU renderer metadata was not retained from the real Mac run. No new visual styling was changed in this job. For the merge round - `bash tests/adversarial/run.sh` — full combined authz/robustness/XUser coverage. - `CALTERNAL_SERVER_BIN=<merge binary> CALTERNAL_E2E_ASSET_OVERRIDE=1 bun tests/adversarial/tag-rewrite-cross-user-1501.mjs` — repeat the strict User-isolation contract on the combined build. - `cd apps/web && bun run test:e2e` (use the configured shared-server harness) — full combined UI behaviour; the focused probes and full web unit suite passed here. - The orchestrator owns full workspace gates and deployment checks. No push, deploy or merge into dev/main occurred. Cleanup `cargo clean` first removed 28,022 files / 17.3 GiB. After the late retry checks it removed 18,096 files / 12.3 GiB. apps/web/build was deleted again after its temporary restoration for RustEmbed. Worktree changes are committed. The own perf-VM run directory, binaries and disposable data were also removed. Diagnostic disclosure A disposable test session cookie appeared in a local transport-error trace. That disposable Instance was destroyed, so the session is invalid. The local trace was redacted and the new regression now hides transport headers. No live owner credential was read or printed.

kayg referenced this issue

2026-10-09 10:12:52 +00:00

Large Tag acceptance stalls another User Note write beyond 30 seconds #1501

kayg referenced this issue

2026-10-09 11:13:24 +00:00

Merge round 36 #1526

kayg referenced this issue from a commit

2026-10-09 11:32:38 +00:00

Merge job/tagperf-1186 (#1186)

kayg referenced this issue

2026-10-09 12:46:52 +00:00

Merge round 36 #1526