kayg/calternal

Watch

Fork 0

PERF: Calendar rule 2 — bound lists by rows and bytes with signed keyset pages (#663) #678

New issue

Open

opened 2026-10-02 05:36:25 +00:00 by kayg · 1 comment

kayg commented

2026-10-02 05:36:25 +00:00

Owner

Context: #663 matrix and DESIGN §58 rule 2. Scope: Calendar must bound lists by rows and bytes with signed keyset pages. Shared client machinery belongs only to #665–#668.

Evidence at c4a61e8cf0:

Endpoint: GET /api/v1/calendar/range.
Source: crates/plugins/calendar/src/view.rs:35. A 366-day bound is not a 100-item/byte page. External items can reach 20,000; details and counts share the response. items.rs already supplies a cursor for saved items.
Representative endpoint numbers, server/client revisions, fixture size, lock/HDD qualification and limits are in the #663 production/HDD table. Those numbers do not prove this rule passes; the structural gap above is separate evidence.

Expected and regression tests:
At 10k items (and largest realistic plugin data), every response has ≤100 item rows and an explicit total byte cap; nested arrays count toward both. Signed cursors bind User, view/filter/sort/zone and keyset position. Edits between pages do not skip or repeat unchanged items; malformed/cross-User/replayed scope cursors cannot change authorization. No OFFSET and no startup whole-corpus drain. Client retains first page plus one lookahead and a bounded resident row/byte window. Preserve deep links, selection, Copy link and current list order. Test large values and variable row size at the byte boundary, empty pages, filter switches, deleted cursor anchor and cache eviction.

Performance test:
Extend the existing bench profile for this hot path and the #549/#641 harness. Use production builds on root@10.69.69.63, bench/hdd-emu.sh and flock -w 14400 /root/perf.lock. Record load inside the lock; ≥5 samples, median/p95/max, average CPU/RSS and one realistic large-data/burst case. Separate warm, cold, accepted and durable boundaries. First usable 10k view ≤1.5 s; cached open/warm return ≤100 ms and accepted action ≤150 ms where applicable. Compare only a matching baseline in docs/perf/baseline.json; missing profiles require a new recorded baseline, not a made-up comparison.

Reuse/ownership:

Reuse #555 userStorage, #549 route caches, Files signed keysets/change feed and the existing Db reader_pool. #641 owns blaze measurements; #642 owns Settings opening; #640 owns Mail layouts; #639 owns linked Note opening. Integrate their active/completed branches before changing related code.

Acceptance:
Existing tests and status expectations stay intact. Run the per-crate gates (and calternal-server for route/contract changes), web gates if changed, and one time-boxed real-server regression/adversarial round for any new API contract. Preserve calternal-fs as the only filesystem interface and the server as the only writer. UI changes need pointer/touch/keyboard/screen-reader coverage and real-production captures at 390/820/1440 in both themes. Do not change shared motion for keyboard input.

Representative measurement from the audit (not a full-view budget result):
/api/v1/calendar/range?from=2026-09-18&to=2026-10-02&tz=UTC, five serial requests after fixture readiness, all HTTP 200. Median/p95/max 64.2/518.9/518.9 ms; five-request burst median/p95/max 107.6/123.7/123.7 ms. Serial window server CPU 290 ms, RSS 286105600 bytes; no ETag on these sampled responses. Load inside lock 3.1/1.9/0.84.
Shared release server source cc25c441b7a974185622a1dee853cf38686d2b67, binary SHA-256 2f3567d91c34839851247bc0acbc25a56aaacd14dca269b8f0342ddf83447ed9; embedded production SPA; Chromium browser on the build host through SSH/HTTPS. Server/Home/Index on perf VM HDD emulator: direct-I/O loop, 8 ms read/write dm-delay, 200 IOPS and 150 MiB/s caps. Every measured phase held flock -w 14400 /root/perf.lock. Qualification QD1 115.3 IOPS/8.028 ms median, QD16 200.7 IOPS/96.993 ms. Fixture: 366 Daily notes, 10,980 Logs, 100 Files/Photos, 20 Notes/Tasks, three Budgets and 100 transactions; Mail empty, Admin one User.
Structural source evidence above is the newer audit base, not the measured binary revision. No claim that these revisions are equivalent. The baseline in docs/perf/baseline.json uses another fixture/build/transport; no regression ratio is valid here. See #663 for matching baseline endpoint values and coverage gaps.

Context: #663 matrix and DESIGN §58 rule 2. Scope: Calendar must bound lists by rows and bytes with signed keyset pages. Shared client machinery belongs only to #665–#668. Evidence at c4a61e8cf090170f35b1bed3350d9de20c83ecd5: - Endpoint: `GET /api/v1/calendar/range`. - Source: `crates/plugins/calendar/src/view.rs:35`. A 366-day bound is not a 100-item/byte page. External items can reach 20,000; details and counts share the response. items.rs already supplies a cursor for saved items. - Representative endpoint numbers, server/client revisions, fixture size, lock/HDD qualification and limits are in the #663 production/HDD table. Those numbers do not prove this rule passes; the structural gap above is separate evidence. Expected and regression tests: At 10k items (and largest realistic plugin data), every response has ≤100 item rows and an explicit total byte cap; nested arrays count toward both. Signed cursors bind User, view/filter/sort/zone and keyset position. Edits between pages do not skip or repeat unchanged items; malformed/cross-User/replayed scope cursors cannot change authorization. No OFFSET and no startup whole-corpus drain. Client retains first page plus one lookahead and a bounded resident row/byte window. Preserve deep links, selection, Copy link and current list order. Test large values and variable row size at the byte boundary, empty pages, filter switches, deleted cursor anchor and cache eviction. Performance test: Extend the existing bench profile for this hot path and the #549/#641 harness. Use production builds on root@10.69.69.63, bench/hdd-emu.sh and flock -w 14400 /root/perf.lock. Record load inside the lock; ≥5 samples, median/p95/max, average CPU/RSS and one realistic large-data/burst case. Separate warm, cold, accepted and durable boundaries. First usable 10k view ≤1.5 s; cached open/warm return ≤100 ms and accepted action ≤150 ms where applicable. Compare only a matching baseline in docs/perf/baseline.json; missing profiles require a new recorded baseline, not a made-up comparison. Reuse/ownership: Reuse #555 userStorage, #549 route caches, Files signed keysets/change feed and the existing Db reader_pool. #641 owns blaze measurements; #642 owns Settings opening; #640 owns Mail layouts; #639 owns linked Note opening. Integrate their active/completed branches before changing related code. Acceptance: Existing tests and status expectations stay intact. Run the per-crate gates (and calternal-server for route/contract changes), web gates if changed, and one time-boxed real-server regression/adversarial round for any new API contract. Preserve calternal-fs as the only filesystem interface and the server as the only writer. UI changes need pointer/touch/keyboard/screen-reader coverage and real-production captures at 390/820/1440 in both themes. Do not change shared motion for keyboard input. Representative measurement from the audit (not a full-view budget result): `/api/v1/calendar/range?from=2026-09-18&to=2026-10-02&tz=UTC`, five serial requests after fixture readiness, all HTTP 200. Median/p95/max 64.2/518.9/518.9 ms; five-request burst median/p95/max 107.6/123.7/123.7 ms. Serial window server CPU 290 ms, RSS 286105600 bytes; no ETag on these sampled responses. Load inside lock 3.1/1.9/0.84. Shared release server source `cc25c441b7a974185622a1dee853cf38686d2b67`, binary SHA-256 `2f3567d91c34839851247bc0acbc25a56aaacd14dca269b8f0342ddf83447ed9`; embedded production SPA; Chromium browser on the build host through SSH/HTTPS. Server/Home/Index on perf VM HDD emulator: direct-I/O loop, 8 ms read/write dm-delay, 200 IOPS and 150 MiB/s caps. Every measured phase held `flock -w 14400 /root/perf.lock`. Qualification QD1 115.3 IOPS/8.028 ms median, QD16 200.7 IOPS/96.993 ms. Fixture: 366 Daily notes, 10,980 Logs, 100 Files/Photos, 20 Notes/Tasks, three Budgets and 100 transactions; Mail empty, Admin one User. Structural source evidence above is the newer audit base, not the measured binary revision. No claim that these revisions are equivalent. The baseline in docs/perf/baseline.json uses another fixture/build/transport; no regression ratio is valid here. See #663 for matching baseline endpoint values and coverage gaps.

kayg referenced this issue

2026-10-02 05:46:53 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-02 13:23:22 +00:00

Log attachments: deleted file still shown, attachments not live, batch send not live, Log order, raw camera names #427

kayg commented

2026-10-02 13:34:15 +00:00

Author

Owner

SQLite audit #663. Base c4a61e8cf0; queued round-7a 2f4482ded0.

Additional database evidence: cache/store.rs:455-478 range candidates use owner/calendar joins, two all_day branches, nullable range bounds and ORDER BY CASE WHEN range_start IS NULL THEN 0 ELSE 1 END,range_start DESC,id LIMIT 5001. With 100k finite synthetic Events, a narrow late window returning one Event visits owner/calendar history via calendar_events_range and uses a temporary sort: about 1,500,000 VM instructions after ANALYZE. Base and round-7a agree. A range index does not prove bounded overlap work. Check separate finite/unknown recurrence streams and prove a bounded seek/merge; preserve unbounded series, recurring exceptions, long Events and all-day/timed boundaries. Test large past/future history and NULL spans; measure candidate counts before a locked HDD run. Reuse #678/#679.

Figures are local Python SQLite 3.53.3 query work in 1,000-instruction callback units, not production latency. Rust bundles 3.51.3. Recheck production plans before implementation; no product edit or assertion change.

SQLite audit #663. Base c4a61e8cf090170f35b1bed3350d9de20c83ecd5; queued round-7a 2f4482ded066d9c5d9c59130377907f7fd2916c9. Additional database evidence: cache/store.rs:455-478 range candidates use owner/calendar joins, two all_day branches, nullable range bounds and ORDER BY CASE WHEN range_start IS NULL THEN 0 ELSE 1 END,range_start DESC,id LIMIT 5001. With 100k finite synthetic Events, a narrow late window returning one Event visits owner/calendar history via calendar_events_range and uses a temporary sort: about 1,500,000 VM instructions after ANALYZE. Base and round-7a agree. A range index does not prove bounded overlap work. Check separate finite/unknown recurrence streams and prove a bounded seek/merge; preserve unbounded series, recurring exceptions, long Events and all-day/timed boundaries. Test large past/future history and NULL spans; measure candidate counts before a locked HDD run. Reuse #678/#679. Figures are local Python SQLite 3.53.3 query work in 1,000-instruction callback units, not production latency. Rust bundles 3.51.3. Recheck production plans before implementation; no product edit or assertion change.

kayg referenced this issue

2026-10-02 13:39:12 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-03 12:51:32 +00:00

Issue hygiene sweep 2026-10-03 #1008

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal#678

No description provided.

Rows
Columns