PERF: isolate WebDAV file reads from the global mutation queue (#663)

kayg commented

2026-10-02 13:11:39 +00:00

Owner

Context: #663 rule 8, sync work must not block interactive reads. Source evidence at c4a61e8cf; confirmed at job/merge-round-7a 2f4482ded.

Evidence:

crates/plugins/files/src/dav.rs:74 file stat takes Root::lock_mutation before its Index hash read and final stat.
:157 open_read takes the same lock before opening a file, reading indexed_hash, and checking its inode fingerprint. It holds the lock across awaited Index work. It returns an open descriptor and releases the lock before body streaming; streaming is not held under this lock.
crates/calternal-fs/src/root.rs:263 documents and implements lock_mutation as one mutex shared across all handles for a Data directory. This is not per path or per User.
The same provider's create_dir/move_path/copy_path takes this mutex for mutation. The upload publication and filesystem scan paths also use this shared authority. Separate SQLite reader pools do not bypass a wait that happens before the query.

Reasoned impact: a WebDAV GET/HEAD/file PROPFIND from one User can wait for an unrelated User's write or sync mutation. Conversely, a reader keeps the global mutation queue occupied while it awaits its Index hash. The coherence check is intentional; simply removing the lock can pair an old descriptor with a new ETag and break conditional reads.

Concrete fix: define a revision snapshot/open invariant and validate/retry against that revision with bounded attempts, or use an appropriate per-item read/publication lock. Keep the descriptor, size and ETag from one content revision and preserve confinement and write preconditions. Remove unrelated-path and unrelated-User coupling, not the revision check.

Tests: hold a mutation for User A; an existing file GET/HEAD for User B must finish without waiting for it. Race replace/rename/delete against reads; every successful body/ETag/length must describe one revision and conditionals must stay correct. Keep litmus/locks and Finder replay tests. Extend bench/webdav-lock-476.py with interleaved reads during writes, with one and two Users, p50/p95, CPU/RSS.

Duplicate search: read #476, which owns per-User WebDAV write serialization throughput. This is the global read dependency, a different lock and acceptance criterion. #699 covers UI request parsing; this is the DAV provider's read/publication coherence. No measured timing or authorization defect is claimed.

Context: #663 rule 8, sync work must not block interactive reads. Source evidence at c4a61e8cf; confirmed at job/merge-round-7a 2f4482ded. Evidence: - crates/plugins/files/src/dav.rs:74 file stat takes Root::lock_mutation before its Index hash read and final stat. - :157 open_read takes the same lock before opening a file, reading indexed_hash, and checking its inode fingerprint. It holds the lock across awaited Index work. It returns an open descriptor and releases the lock before body streaming; streaming is not held under this lock. - crates/calternal-fs/src/root.rs:263 documents and implements lock_mutation as one mutex shared across all handles for a Data directory. This is not per path or per User. - The same provider's create_dir/move_path/copy_path takes this mutex for mutation. The upload publication and filesystem scan paths also use this shared authority. Separate SQLite reader pools do not bypass a wait that happens before the query. Reasoned impact: a WebDAV GET/HEAD/file PROPFIND from one User can wait for an unrelated User's write or sync mutation. Conversely, a reader keeps the global mutation queue occupied while it awaits its Index hash. The coherence check is intentional; simply removing the lock can pair an old descriptor with a new ETag and break conditional reads. Concrete fix: define a revision snapshot/open invariant and validate/retry against that revision with bounded attempts, or use an appropriate per-item read/publication lock. Keep the descriptor, size and ETag from one content revision and preserve confinement and write preconditions. Remove unrelated-path and unrelated-User coupling, not the revision check. Tests: hold a mutation for User A; an existing file GET/HEAD for User B must finish without waiting for it. Race replace/rename/delete against reads; every successful body/ETag/length must describe one revision and conditionals must stay correct. Keep litmus/locks and Finder replay tests. Extend bench/webdav-lock-476.py with interleaved reads during writes, with one and two Users, p50/p95, CPU/RSS. Duplicate search: read #476, which owns per-User WebDAV write serialization throughput. This is the global read dependency, a different lock and acceptance criterion. #699 covers UI request parsing; this is the DAV provider's read/publication coherence. No measured timing or authorization defect is claimed.

kayg referenced this issue

2026-10-02 13:16:50 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-02 13:18:04 +00:00

PERF: move WAL checkpoint work off interactive commits (#663) #823

kayg referenced this issue

2026-10-02 13:18:09 +00:00

BLOCKER: make acknowledged Security state survive power loss (#663) #824

kayg referenced this issue

2026-10-02 13:19:07 +00:00

PERF: make Mail folder page SQL work proportional to its window (#663) #825

kayg referenced this issue

2026-10-02 13:19:27 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-02 13:22:24 +00:00

PERF: publish Search generations without whole-manifest writer transactions (#663) #832

kayg commented

2026-10-02 13:25:25 +00:00

Author

Owner

Starting implementation for this issue on branch job/ioperf, based at 2f4482ded066d9c5d9c59130377907f7fd2916c9 (job/merge-round-7a). I have read the issue and the #663 audit evidence. I am inspecting the job/hddsql-549 changes before editing shared startup, filesystem, and database paths. I will add a regression test and a focused benchmark for each performance path, then report measured results and crate gates here.

Starting implementation for this issue on branch `job/ioperf`, based at `2f4482ded066d9c5d9c59130377907f7fd2916c9` (`job/merge-round-7a`). I have read the issue and the #663 audit evidence. I am inspecting the `job/hddsql-549` changes before editing shared startup, filesystem, and database paths. I will add a regression test and a focused benchmark for each performance path, then report measured results and crate gates here.

kayg commented

2026-10-02 13:34:17 +00:00

Author

Owner

Confirmed in crates/plugins/files/src/dav.rs: file stat and open_read acquire the Root-wide mutation mutex while they await Index reads. This couples unrelated Users' WebDAV reads to mutations. I am removing that dependency only with a revision check that binds the open descriptor, size, modified time and ETag to one indexed fingerprint.

Confirmed in `crates/plugins/files/src/dav.rs`: file `stat` and `open_read` acquire the Root-wide mutation mutex while they await Index reads. This couples unrelated Users' WebDAV reads to mutations. I am removing that dependency only with a revision check that binds the open descriptor, size, modified time and ETag to one indexed fingerprint.

kayg referenced this issue

2026-10-02 17:27:10 +00:00

PERF: Files startup reconciliation holds the shared mutation lock across a whole Home (#663) #750

kayg referenced this issue

2026-10-02 17:27:25 +00:00

PERF: skip full pre-migration Index snapshots on unchanged restarts (#663) #748

kayg referenced this issue

2026-10-02 17:27:25 +00:00

PERF: wake the job worker on queue changes instead of empty one-second writes (#663) #764

kayg commented

2026-10-02 17:27:26 +00:00

Author

Owner

ioperf final report

Built changes for #748, #750, #764, #800 and #806. Head: d43e7829a3e0c8a483cc45bdbb965af5f5d1c713.

Changes

#748: validate pending migrations before taking a startup snapshot; unchanged restarts skip it.
#750: list and hash one folder outside the Root mutation lock, then validate generations and publish the prepared Index changes.
#764: idle workers subscribe to queue changes and wake for due-job and lease deadlines; the server fallback is 30 seconds.
#800: DAV stat/open reads use indexed hashes without holding the Root mutation lock and verify full file fingerprints.
#806: the server owns one bounded recursive Home watcher and forwards external hints through the shared Root change stream. Overflow triggers a coalesced rebuild, including open collaboration rooms.

Files

crates/calternal-db/src/migrations.rs, crates/calternal-db/src/worker.rs, crates/calternal-server/src/wire.rs, crates/plugins/files/src/dav.rs, crates/plugins/files/src/index.rs, crates/plugins/files/src/lib.rs, crates/calternal-fs/src/lib.rs, crates/calternal-fs/src/quota.rs, crates/calternal-fs/src/root.rs, crates/calternal-fs/tests/storage.rs, crates/calternal-search/src/indexer.rs, crates/calternal-collab/src/session.rs.

Gate output

cargo fmt --all --check: exit 0; no output.

cargo clippy --offline -p calternal-db --all-targets -- -D warnings:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 55s

RUST_TEST_THREADS=1 cargo test --offline -p calternal-db:

running 27 tests
test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 68.51s

running 17 tests
test result: ok. 16 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 25.17s

running 1 test
test sqlite_pools_bound_readers_and_page_cache ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.80s

Doc-tests calternal_db
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Focused Files regressions:

running 4 tests
test dav::tests::open_read_does_not_wait_for_the_home_mutation_lock ... ok
test dav::tests::stat_does_not_wait_for_the_home_mutation_lock ... ok
test tests::reconcile_hash_does_not_hold_mutation_lock_across_homes ... ok
test tests::public_password_rejection_does_not_wait_for_data_mutation_lock ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 156 filtered out; finished in 4.16s

bun run check:

$ node scripts/check-user-storage.mjs && node scripts/check-type-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
User browser caches use userStorage; only documented device/public-link exceptions remain.
Text sizes and UI shape values use shared role tokens.
UI transitions and animation options use shared motion tokens or documented exceptions.
Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/ioperf/apps/web
Getting Svelte diagnostics...

svelte-check found 0 errors and 0 warnings

Known gaps

The full Files test command was interrupted with exit 130 when the shared target stayed blocked on I/O. The focused four tests passed.
Final clippy/test gates for Files, server, fs, search and collaboration were not completed. The #806 server/filesystem integration is therefore not compile-verified in this run.
bun run test, adversarial probing, and the requested per-feature performance profiles and measurements were not completed.
cargo clean and removal of the generated web build output remain undone.
No UI changed, so UX gaps and screenshots are N/A.

Decisions

The #764 cross-process fallback is 30 seconds; local queue hints and durable deadlines provide prompt wakeups.
Watcher overflow recovery waits for a 100 ms quiet window before repeating a full repair.
Folder reconciliation reuses the existing eight-attempt policy for changed snapshots.
External watcher events carry an explicit source on the existing Root change stream, so only external changes publish the Home feed.

No UX gaps were introduced. Commits are on job/ioperf; no push or deploy was made.

## ioperf final report Built changes for #748, #750, #764, #800 and #806. Head: `d43e7829a3e0c8a483cc45bdbb965af5f5d1c713`. ### Changes - #748: validate pending migrations before taking a startup snapshot; unchanged restarts skip it. - #750: list and hash one folder outside the Root mutation lock, then validate generations and publish the prepared Index changes. - #764: idle workers subscribe to queue changes and wake for due-job and lease deadlines; the server fallback is 30 seconds. - #800: DAV stat/open reads use indexed hashes without holding the Root mutation lock and verify full file fingerprints. - #806: the server owns one bounded recursive Home watcher and forwards external hints through the shared Root change stream. Overflow triggers a coalesced rebuild, including open collaboration rooms. ### Files `crates/calternal-db/src/migrations.rs`, `crates/calternal-db/src/worker.rs`, `crates/calternal-server/src/wire.rs`, `crates/plugins/files/src/dav.rs`, `crates/plugins/files/src/index.rs`, `crates/plugins/files/src/lib.rs`, `crates/calternal-fs/src/lib.rs`, `crates/calternal-fs/src/quota.rs`, `crates/calternal-fs/src/root.rs`, `crates/calternal-fs/tests/storage.rs`, `crates/calternal-search/src/indexer.rs`, `crates/calternal-collab/src/session.rs`. ### Gate output `cargo fmt --all --check`: exit 0; no output. `cargo clippy --offline -p calternal-db --all-targets -- -D warnings`: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 55s ``` `RUST_TEST_THREADS=1 cargo test --offline -p calternal-db`: ``` running 27 tests test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 68.51s running 17 tests test result: ok. 16 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 25.17s running 1 test test sqlite_pools_bound_readers_and_page_cache ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.80s Doc-tests calternal_db running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Focused Files regressions: ``` running 4 tests test dav::tests::open_read_does_not_wait_for_the_home_mutation_lock ... ok test dav::tests::stat_does_not_wait_for_the_home_mutation_lock ... ok test tests::reconcile_hash_does_not_hold_mutation_lock_across_homes ... ok test tests::public_password_rejection_does_not_wait_for_data_mutation_lock ... ok test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 156 filtered out; finished in 4.16s ``` `bun run check`: ``` $ node scripts/check-user-storage.mjs && node scripts/check-type-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json User browser caches use userStorage; only documented device/public-link exceptions remain. Text sizes and UI shape values use shared role tokens. UI transitions and animation options use shared motion tokens or documented exceptions. Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/ioperf/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings ``` ### Known gaps - The full Files test command was interrupted with exit 130 when the shared target stayed blocked on I/O. The focused four tests passed. - Final clippy/test gates for Files, server, fs, search and collaboration were not completed. The #806 server/filesystem integration is therefore not compile-verified in this run. - `bun run test`, adversarial probing, and the requested per-feature performance profiles and measurements were not completed. - `cargo clean` and removal of the generated web build output remain undone. - No UI changed, so UX gaps and screenshots are N/A. ### Decisions - The #764 cross-process fallback is 30 seconds; local queue hints and durable deadlines provide prompt wakeups. - Watcher overflow recovery waits for a 100 ms quiet window before repeating a full repair. - Folder reconciliation reuses the existing eight-attempt policy for changed snapshots. - External watcher events carry an explicit source on the existing Root change stream, so only external changes publish the Home feed. No UX gaps were introduced. Commits are on `job/ioperf`; no push or deploy was made.

kayg referenced this issue

2026-10-02 17:27:27 +00:00

PERF: recursive Home watcher registration repeats full directory walks before readiness (#663) #806

Rows
Columns

PERF: isolate WebDAV file reads from the global mutation queue (#663) #800

ioperf final report

Changes

Files

Gate output

Known gaps

Decisions