PERF: Files rule 1 — serve interactive reads from committed precomputed projections (#663) #699

New issue

Open

opened 2026-10-02 05:36:55 +00:00 by kayg · 0 comments

kayg commented

2026-10-02 05:36:55 +00:00

Owner

Context: #663 and DESIGN §58 rule 1. Scope: Files interactive metadata/text reads from a committed projection. Shared primitives are owned only by #665–#668.

Evidence at c4a61e8cf0: GET /api/v1/files/download?inline=true, crates/plugins/files/src/lib.rs:3664. Indexed folder list already passes the projection-only read path, but Quick Look TextView fetches downloadUrl (viewer.ts:16), streaming Markdown source from disk under the mutation lock. UI text preview must use a committed indexed body; keep original-byte download semantics separate.

Expected:
Serve interactive header/text/root data from the Index or a precomputed committed server projection, without reading/parsing Markdown or calling an external provider. Keep the filesystem as source of truth and the Index rebuildable. Authorize each read, including share revoke and plugin disable. Use the WAL reader pool; a held writer must not block last committed data. Keep original binary/download byte semantics and conditional writes intact. The Files rule-1 issue owns the generic indexed file-text preview; Notes rule-1 owns Note/collaboration body publication. Photos reuses the Settings rule-1 User preference projection and only adds authorized root/media fields. Do not implement competing text or preferences caches.

Tests:

Instrument a UI read and require zero Markdown read/parse/external calls; held writer returns last complete projection. A changed file/preference publishes its revision and fields atomically, after ingest. Index rebuild restores data; renamed stable IDs and deleted/share-revoked items are correct.
Verify two-User isolation, byte bounds for text and original download streaming/precondition tests. A Note open/preview must not write the source (#661).
Extend bench hot path and #549/#641 profiles: ≥5 production/HDD locked VM cold/warm samples, median/p95/max, CPU/RSS, 10k items plus burst. Preserve warm cached-open ≤100 ms and first usable 10k view ≤1.5 s. Counts/background work are in separate rule-8 issues.

Measurements: representative locked production endpoint results and exact source/binary scope are posted on #663. Small-list API reads do not prove full-view budgets.

Reuse: #555, #549, Files Index/body ETags/signed pages, #641 active blaze-surfaces decoded-image/viewer work; integrate that branch before changing related UI. #642 owns Settings resources. This issue does not duplicate active traversal work.

Acceptance: existing tests/expectations stay unchanged; per-crate gates and calternal-server gates for routes/contracts, one time-boxed real-server regression/adversarial round. Preserve calternal-fs and single writer. UI adaptations need keyboard/touch/screen-reader coverage and production captures at 390/820/1440 in both themes; keep shared motion.

Representative measurement from the audit (not a full-view budget result):
/api/v1/files/entries?limit=100, five serial requests after fixture readiness, all HTTP 200. Median/p95/max 87.8/1069.4/1069.4 ms; five-request burst median/p95/max 2033.2/2034.0/2034.0 ms. Serial window server CPU 1090 ms, RSS 443625472 bytes; no ETag on these sampled responses. Load inside lock 3.32/1.98/0.88.
Shared release server source cc25c441b7a974185622a1dee853cf38686d2b67, binary SHA-256 2f3567d91c34839851247bc0acbc25a56aaacd14dca269b8f0342ddf83447ed9; embedded production SPA; Chromium browser on the build host through SSH/HTTPS. Server/Home/Index on perf VM HDD emulator: direct-I/O loop, 8 ms read/write dm-delay, 200 IOPS and 150 MiB/s caps. Every measured phase held flock -w 14400 /root/perf.lock. Qualification QD1 115.3 IOPS/8.028 ms median, QD16 200.7 IOPS/96.993 ms. Fixture: 366 Daily notes, 10,980 Logs, 100 Files/Photos, 20 Notes/Tasks, three Budgets and 100 transactions; Mail empty, Admin one User.
Structural source evidence above is the newer audit base, not the measured binary revision. No claim that these revisions are equivalent. The baseline in docs/perf/baseline.json uses another fixture/build/transport; no regression ratio is valid here. See #663 for matching baseline endpoint values and coverage gaps.

Context: #663 and DESIGN §58 rule 1. Scope: Files interactive metadata/text reads from a committed projection. Shared primitives are owned only by #665–#668. Evidence at c4a61e8cf090170f35b1bed3350d9de20c83ecd5: `GET /api/v1/files/download?inline=true`, `crates/plugins/files/src/lib.rs:3664`. Indexed folder list already passes the projection-only read path, but Quick Look TextView fetches downloadUrl (viewer.ts:16), streaming Markdown source from disk under the mutation lock. UI text preview must use a committed indexed body; keep original-byte download semantics separate. Expected: Serve interactive header/text/root data from the Index or a precomputed committed server projection, without reading/parsing Markdown or calling an external provider. Keep the filesystem as source of truth and the Index rebuildable. Authorize each read, including share revoke and plugin disable. Use the WAL reader pool; a held writer must not block last committed data. Keep original binary/download byte semantics and conditional writes intact. The Files rule-1 issue owns the generic indexed file-text preview; Notes rule-1 owns Note/collaboration body publication. Photos reuses the Settings rule-1 User preference projection and only adds authorized root/media fields. Do not implement competing text or preferences caches. Tests: - Instrument a UI read and require zero Markdown read/parse/external calls; held writer returns last complete projection. A changed file/preference publishes its revision and fields atomically, after ingest. Index rebuild restores data; renamed stable IDs and deleted/share-revoked items are correct. - Verify two-User isolation, byte bounds for text and original download streaming/precondition tests. A Note open/preview must not write the source (#661). - Extend bench hot path and #549/#641 profiles: ≥5 production/HDD locked VM cold/warm samples, median/p95/max, CPU/RSS, 10k items plus burst. Preserve warm cached-open ≤100 ms and first usable 10k view ≤1.5 s. Counts/background work are in separate rule-8 issues. Measurements: representative locked production endpoint results and exact source/binary scope are posted on #663. Small-list API reads do not prove full-view budgets. Reuse: #555, #549, Files Index/body ETags/signed pages, #641 active blaze-surfaces decoded-image/viewer work; integrate that branch before changing related UI. #642 owns Settings resources. This issue does not duplicate active traversal work. Acceptance: existing tests/expectations stay unchanged; per-crate gates and calternal-server gates for routes/contracts, one time-boxed real-server regression/adversarial round. Preserve calternal-fs and single writer. UI adaptations need keyboard/touch/screen-reader coverage and production captures at 390/820/1440 in both themes; keep shared motion. Representative measurement from the audit (not a full-view budget result): `/api/v1/files/entries?limit=100`, five serial requests after fixture readiness, all HTTP 200. Median/p95/max 87.8/1069.4/1069.4 ms; five-request burst median/p95/max 2033.2/2034.0/2034.0 ms. Serial window server CPU 1090 ms, RSS 443625472 bytes; no ETag on these sampled responses. Load inside lock 3.32/1.98/0.88. Shared release server source `cc25c441b7a974185622a1dee853cf38686d2b67`, binary SHA-256 `2f3567d91c34839851247bc0acbc25a56aaacd14dca269b8f0342ddf83447ed9`; embedded production SPA; Chromium browser on the build host through SSH/HTTPS. Server/Home/Index on perf VM HDD emulator: direct-I/O loop, 8 ms read/write dm-delay, 200 IOPS and 150 MiB/s caps. Every measured phase held `flock -w 14400 /root/perf.lock`. Qualification QD1 115.3 IOPS/8.028 ms median, QD16 200.7 IOPS/96.993 ms. Fixture: 366 Daily notes, 10,980 Logs, 100 Files/Photos, 20 Notes/Tasks, three Budgets and 100 transactions; Mail empty, Admin one User. Structural source evidence above is the newer audit base, not the measured binary revision. No claim that these revisions are equivalent. The baseline in docs/perf/baseline.json uses another fixture/build/transport; no regression ratio is valid here. See #663 for matching baseline endpoint values and coverage gaps.

kayg referenced this issue

2026-10-02 05:46:53 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-02 13:11:39 +00:00

PERF: isolate WebDAV file reads from the global mutation queue (#663) #800

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal#699

No description provided.

Rows
Columns