kayg/calternal

Watch

Fork 0

PERF: recursive Home watcher registration repeats full directory walks before readiness (#663) #806

New issue

Open

opened 2026-10-02 13:12:34 +00:00 by kayg · 3 comments

kayg commented

2026-10-02 13:12:34 +00:00

Owner

Context: perf-arch-io startup audit, #663 rule 8. Product source origin/dev c4a61e8cf090170f35b1bed3350d9de20c83ecd5. Rechecked merge-round-7a 2f4482ded066d9c5d9c59130377907f7fd2916c9: independent recursive watcher setup remains.

Before HTTP binds, Search opens its recursive watcher (crates/calternal-search/src/indexer.rs:504, start_watcher:2597). Later Hub::new sets a second recursive watch on the same users tree (crates/calternal-collab/src/session.rs:703). The server also starts watch_home_changes (crates/calternal-server/src/wire.rs:5538-5554), whose recursive watch runs inside a spawned Tokio task. On Linux these are separate inotify watcher instances over every User's Home, even when no live Note room exists.

Verified from the locally installed notify 8.2.0 source, the exact Cargo.lock version at :4868: src/inotify.rs:400-413 uses WalkDir to enumerate the recursive tree and install each watch; watch_inner:547-562 waits synchronously on rx.recv for that entire registration. This is source evidence, not an inference from the API name. No dependency version changed.

Reasoned impact: Search and collaboration each add a complete directory enumeration to pre-listener startup; the third enumeration can block a Tokio worker. Directory count and cold metadata IO determine cost, even if all file bodies are cached. Deferring Search CONTENT reconciliation until after bind does not defer recursive watcher setup. This is a candidate pre-listener phase for #549's hddsql investigation, NOT an attribution of its 300 s readiness timeout. No new timing is claimed.

Concrete fix: reuse one server-owned Home watcher/event stream and its bounded overflow-to-reconcile mechanism for Search, Files and collaboration. Live rooms can consume filtered changed paths. Alternatively, move registration off Tokio and stage it after listener readiness with a startup reconcile barrier that covers writes during registration. Keep no-lost-change guarantees; do not just omit existing watches or disable restore repair. Avoid a new generic helper if the shared watcher already has the needed hook.

Regression tests: record watcher registrations for many nested folders and verify only one recursive registration; delay registration and prove health/readiness remain responsive if registration is deferred; make a file change during registration and verify Files/Search/live Note convergence; fill the bounded event channel and prove full reconcile is requested. Measure cold and warm startup phases for the 100k nested-file fixture on the locked HDD emulator, plus request latency during registration.

Duplicate check: all open/closed titles searched; #549 owns startup measurement, #695 owns Search work priority, #470 covers offline Photos discovery, and #325/#343 concern runtime storms. None specifies duplicate pre-listener recursive registrations. Coordinate startup instrumentation with job/hddsql-549. This issue owns watcher reuse/registration scheduling. No measured DoS or security blocker is asserted.

Context: perf-arch-io startup audit, #663 rule 8. Product source origin/dev `c4a61e8cf090170f35b1bed3350d9de20c83ecd5`. Rechecked merge-round-7a `2f4482ded066d9c5d9c59130377907f7fd2916c9`: independent recursive watcher setup remains. Before HTTP binds, Search opens its recursive watcher (`crates/calternal-search/src/indexer.rs:504`, `start_watcher:2597`). Later `Hub::new` sets a second recursive watch on the same users tree (`crates/calternal-collab/src/session.rs:703`). The server also starts `watch_home_changes` (`crates/calternal-server/src/wire.rs:5538-5554`), whose recursive watch runs inside a spawned Tokio task. On Linux these are separate inotify watcher instances over every User's Home, even when no live Note room exists. Verified from the locally installed notify 8.2.0 source, the exact Cargo.lock version at :4868: `src/inotify.rs:400-413` uses WalkDir to enumerate the recursive tree and install each watch; `watch_inner:547-562` waits synchronously on rx.recv for that entire registration. This is source evidence, not an inference from the API name. No dependency version changed. Reasoned impact: Search and collaboration each add a complete directory enumeration to pre-listener startup; the third enumeration can block a Tokio worker. Directory count and cold metadata IO determine cost, even if all file bodies are cached. Deferring Search CONTENT reconciliation until after bind does not defer recursive watcher setup. This is a candidate pre-listener phase for #549's hddsql investigation, NOT an attribution of its 300 s readiness timeout. No new timing is claimed. Concrete fix: reuse one server-owned Home watcher/event stream and its bounded overflow-to-reconcile mechanism for Search, Files and collaboration. Live rooms can consume filtered changed paths. Alternatively, move registration off Tokio and stage it after listener readiness with a startup reconcile barrier that covers writes during registration. Keep no-lost-change guarantees; do not just omit existing watches or disable restore repair. Avoid a new generic helper if the shared watcher already has the needed hook. Regression tests: record watcher registrations for many nested folders and verify only one recursive registration; delay registration and prove health/readiness remain responsive if registration is deferred; make a file change during registration and verify Files/Search/live Note convergence; fill the bounded event channel and prove full reconcile is requested. Measure cold and warm startup phases for the 100k nested-file fixture on the locked HDD emulator, plus request latency during registration. Duplicate check: all open/closed titles searched; #549 owns startup measurement, #695 owns Search work priority, #470 covers offline Photos discovery, and #325/#343 concern runtime storms. None specifies duplicate pre-listener recursive registrations. Coordinate startup instrumentation with job/hddsql-549. This issue owns watcher reuse/registration scheduling. No measured DoS or security blocker is asserted.

kayg referenced this issue

2026-10-02 13:17:40 +00:00

PERF: tab switch to fully painted on a hard disk with heavy data (Calendar much slower than Photos); stress, measure, fix #549

kayg referenced this issue

2026-10-02 13:18:27 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg commented

2026-10-02 13:25:26 +00:00

Author

Owner

Starting implementation for this issue on branch job/ioperf, based at 2f4482ded066d9c5d9c59130377907f7fd2916c9 (job/merge-round-7a). I have read the issue and the #663 audit evidence. I am inspecting the job/hddsql-549 changes before editing shared startup, filesystem, and database paths. I will add a regression test and a focused benchmark for each performance path, then report measured results and crate gates here.

Starting implementation for this issue on branch `job/ioperf`, based at `2f4482ded066d9c5d9c59130377907f7fd2916c9` (`job/merge-round-7a`). I have read the issue and the #663 audit evidence. I am inspecting the `job/hddsql-549` changes before editing shared startup, filesystem, and database paths. I will add a regression test and a focused benchmark for each performance path, then report measured results and crate gates here.

kayg commented

2026-10-02 13:34:19 +00:00

Author

Owner

Confirmed three recursive registrations over users/: Search's start_watcher, Hub::new collaboration watcher, and the server's watch_home_changes. On Linux each notify recursive registration walks the tree synchronously. I am routing Search and collaboration through the server-owned watcher while preserving overflow-to-reconcile behavior.

Confirmed three recursive registrations over `users/`: Search's `start_watcher`, `Hub::new` collaboration watcher, and the server's `watch_home_changes`. On Linux each notify recursive registration walks the tree synchronously. I am routing Search and collaboration through the server-owned watcher while preserving overflow-to-reconcile behavior.

kayg referenced this issue

2026-10-02 17:27:10 +00:00

PERF: Files startup reconciliation holds the shared mutation lock across a whole Home (#663) #750

kayg referenced this issue

2026-10-02 17:27:25 +00:00

PERF: skip full pre-migration Index snapshots on unchanged restarts (#663) #748

kayg referenced this issue

2026-10-02 17:27:25 +00:00

PERF: wake the job worker on queue changes instead of empty one-second writes (#663) #764

kayg referenced this issue

2026-10-02 17:27:26 +00:00

PERF: isolate WebDAV file reads from the global mutation queue (#663) #800

kayg commented

2026-10-02 17:27:27 +00:00

Author

Owner

ioperf final report

Built changes for #748, #750, #764, #800 and #806. Head: d43e7829a3e0c8a483cc45bdbb965af5f5d1c713.

Changes

#748: validate pending migrations before taking a startup snapshot; unchanged restarts skip it.
#750: list and hash one folder outside the Root mutation lock, then validate generations and publish the prepared Index changes.
#764: idle workers subscribe to queue changes and wake for due-job and lease deadlines; the server fallback is 30 seconds.
#800: DAV stat/open reads use indexed hashes without holding the Root mutation lock and verify full file fingerprints.
#806: the server owns one bounded recursive Home watcher and forwards external hints through the shared Root change stream. Overflow triggers a coalesced rebuild, including open collaboration rooms.

Files

crates/calternal-db/src/migrations.rs, crates/calternal-db/src/worker.rs, crates/calternal-server/src/wire.rs, crates/plugins/files/src/dav.rs, crates/plugins/files/src/index.rs, crates/plugins/files/src/lib.rs, crates/calternal-fs/src/lib.rs, crates/calternal-fs/src/quota.rs, crates/calternal-fs/src/root.rs, crates/calternal-fs/tests/storage.rs, crates/calternal-search/src/indexer.rs, crates/calternal-collab/src/session.rs.

Gate output

cargo fmt --all --check: exit 0; no output.

cargo clippy --offline -p calternal-db --all-targets -- -D warnings:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 55s

RUST_TEST_THREADS=1 cargo test --offline -p calternal-db:

running 27 tests
test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 68.51s

running 17 tests
test result: ok. 16 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 25.17s

running 1 test
test sqlite_pools_bound_readers_and_page_cache ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.80s

Doc-tests calternal_db
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Focused Files regressions:

running 4 tests
test dav::tests::open_read_does_not_wait_for_the_home_mutation_lock ... ok
test dav::tests::stat_does_not_wait_for_the_home_mutation_lock ... ok
test tests::reconcile_hash_does_not_hold_mutation_lock_across_homes ... ok
test tests::public_password_rejection_does_not_wait_for_data_mutation_lock ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 156 filtered out; finished in 4.16s

bun run check:

$ node scripts/check-user-storage.mjs && node scripts/check-type-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
User browser caches use userStorage; only documented device/public-link exceptions remain.
Text sizes and UI shape values use shared role tokens.
UI transitions and animation options use shared motion tokens or documented exceptions.
Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/ioperf/apps/web
Getting Svelte diagnostics...

svelte-check found 0 errors and 0 warnings

Known gaps

The full Files test command was interrupted with exit 130 when the shared target stayed blocked on I/O. The focused four tests passed.
Final clippy/test gates for Files, server, fs, search and collaboration were not completed. The #806 server/filesystem integration is therefore not compile-verified in this run.
bun run test, adversarial probing, and the requested per-feature performance profiles and measurements were not completed.
cargo clean and removal of the generated web build output remain undone.
No UI changed, so UX gaps and screenshots are N/A.

Decisions

The #764 cross-process fallback is 30 seconds; local queue hints and durable deadlines provide prompt wakeups.
Watcher overflow recovery waits for a 100 ms quiet window before repeating a full repair.
Folder reconciliation reuses the existing eight-attempt policy for changed snapshots.
External watcher events carry an explicit source on the existing Root change stream, so only external changes publish the Home feed.

No UX gaps were introduced. Commits are on job/ioperf; no push or deploy was made.

## ioperf final report Built changes for #748, #750, #764, #800 and #806. Head: `d43e7829a3e0c8a483cc45bdbb965af5f5d1c713`. ### Changes - #748: validate pending migrations before taking a startup snapshot; unchanged restarts skip it. - #750: list and hash one folder outside the Root mutation lock, then validate generations and publish the prepared Index changes. - #764: idle workers subscribe to queue changes and wake for due-job and lease deadlines; the server fallback is 30 seconds. - #800: DAV stat/open reads use indexed hashes without holding the Root mutation lock and verify full file fingerprints. - #806: the server owns one bounded recursive Home watcher and forwards external hints through the shared Root change stream. Overflow triggers a coalesced rebuild, including open collaboration rooms. ### Files `crates/calternal-db/src/migrations.rs`, `crates/calternal-db/src/worker.rs`, `crates/calternal-server/src/wire.rs`, `crates/plugins/files/src/dav.rs`, `crates/plugins/files/src/index.rs`, `crates/plugins/files/src/lib.rs`, `crates/calternal-fs/src/lib.rs`, `crates/calternal-fs/src/quota.rs`, `crates/calternal-fs/src/root.rs`, `crates/calternal-fs/tests/storage.rs`, `crates/calternal-search/src/indexer.rs`, `crates/calternal-collab/src/session.rs`. ### Gate output `cargo fmt --all --check`: exit 0; no output. `cargo clippy --offline -p calternal-db --all-targets -- -D warnings`: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 55s ``` `RUST_TEST_THREADS=1 cargo test --offline -p calternal-db`: ``` running 27 tests test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 68.51s running 17 tests test result: ok. 16 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 25.17s running 1 test test sqlite_pools_bound_readers_and_page_cache ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.80s Doc-tests calternal_db running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Focused Files regressions: ``` running 4 tests test dav::tests::open_read_does_not_wait_for_the_home_mutation_lock ... ok test dav::tests::stat_does_not_wait_for_the_home_mutation_lock ... ok test tests::reconcile_hash_does_not_hold_mutation_lock_across_homes ... ok test tests::public_password_rejection_does_not_wait_for_data_mutation_lock ... ok test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 156 filtered out; finished in 4.16s ``` `bun run check`: ``` $ node scripts/check-user-storage.mjs && node scripts/check-type-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json User browser caches use userStorage; only documented device/public-link exceptions remain. Text sizes and UI shape values use shared role tokens. UI transitions and animation options use shared motion tokens or documented exceptions. Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/ioperf/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings ``` ### Known gaps - The full Files test command was interrupted with exit 130 when the shared target stayed blocked on I/O. The focused four tests passed. - Final clippy/test gates for Files, server, fs, search and collaboration were not completed. The #806 server/filesystem integration is therefore not compile-verified in this run. - `bun run test`, adversarial probing, and the requested per-feature performance profiles and measurements were not completed. - `cargo clean` and removal of the generated web build output remain undone. - No UI changed, so UX gaps and screenshots are N/A. ### Decisions - The #764 cross-process fallback is 30 seconds; local queue hints and durable deadlines provide prompt wakeups. - Watcher overflow recovery waits for a 100 ms quiet window before repeating a full repair. - Folder reconciliation reuses the existing eight-attempt policy for changed snapshots. - External watcher events carry an explicit source on the existing Root change stream, so only external changes publish the Home feed. No UX gaps were introduced. Commits are on `job/ioperf`; no push or deploy was made.

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal#806

No description provided.

Rows
Columns