kayg/calternal

Watch

Fork 0

PERF: target external Note changes to one collaboration room (#663) #769

New issue

Open

opened 2026-10-02 13:10:12 +00:00 by kayg · 2 comments

kayg commented

2026-10-02 13:10:12 +00:00

Owner

Context: #663 work-per-change audit, rules 5 and 8. Evidence at c4a61e8cf; confirmed in job/merge-round-7a 2f4482ded, including the #634 changes.

Evidence:

crates/calternal-collab/src/session.rs:1113 external_change collects every room for the User (:1116–1121).
:1124 takes each room flush_lock; :1127 calls collab_read for each room. It compares note.path to the changed path only at :1132, after the read.
crates/plugins/notes/src/lib.rs:3158 (round-7a; :2800 at base) collab_read queries identity-to-path, reads the Note file, splits the body and hashes all source bytes. It is not a cheap path lookup.
session.rs:691 consumes Notes plugin events and :739 consumes filesystem watcher events through the same external_change. :1173 also exposes the direct notification path. Repeated hints are not combined before those reads.

Reasoned impact: one external write with R loaded rooms costs R Index path lookups and R full Note reads/hashes before it selects one room. Unrelated Notes are read even for a self-save whose ETag is unchanged. Flush locks on unrelated rooms can delay processing. Event bursts cost O(events × loaded rooms × Note bytes), instead of work for the changed Note alone. This is a source-derived cost, not a measured p95.

Concrete fix: resolve a confined (User, path) to the existing stable Note identity once, then fetch only that room. Maintain or reuse a path-to-room index with correct rename/delete updates if needed. Combine duplicate pending external hints per Note. Preserve external/live merge ordering, If-Match and flush-before-ack guarantees from #634/#382. Keep missed-event recovery bounded.

Tests: load many rooms, update one path, and count source reads/path queries. Only the target room may read its source. Cover rename, deletion, self-save, repeated watcher/plugin hints, a locked unrelated room and simultaneous local/external edits. Preserve the existing journal_race/shared_notes/untouched_bytes tests. Extend collaboration performance evidence for 1/10/100 loaded rooms.

Duplicate search: #265 owns the large-Note initial sync timeout, #634 duplicate block reconciliation, #382 external edit integrity. This finding covers the unrelated-room scan before any block merge; it needs a separate performance fix. No data loss or crash was demonstrated.

Context: #663 work-per-change audit, rules 5 and 8. Evidence at c4a61e8cf; confirmed in job/merge-round-7a 2f4482ded, including the #634 changes. Evidence: - crates/calternal-collab/src/session.rs:1113 external_change collects every room for the User (:1116–1121). - :1124 takes each room flush_lock; :1127 calls collab_read for each room. It compares note.path to the changed path only at :1132, after the read. - crates/plugins/notes/src/lib.rs:3158 (round-7a; :2800 at base) collab_read queries identity-to-path, reads the Note file, splits the body and hashes all source bytes. It is not a cheap path lookup. - session.rs:691 consumes Notes plugin events and :739 consumes filesystem watcher events through the same external_change. :1173 also exposes the direct notification path. Repeated hints are not combined before those reads. Reasoned impact: one external write with R loaded rooms costs R Index path lookups and R full Note reads/hashes before it selects one room. Unrelated Notes are read even for a self-save whose ETag is unchanged. Flush locks on unrelated rooms can delay processing. Event bursts cost O(events × loaded rooms × Note bytes), instead of work for the changed Note alone. This is a source-derived cost, not a measured p95. Concrete fix: resolve a confined (User, path) to the existing stable Note identity once, then fetch only that room. Maintain or reuse a path-to-room index with correct rename/delete updates if needed. Combine duplicate pending external hints per Note. Preserve external/live merge ordering, If-Match and flush-before-ack guarantees from #634/#382. Keep missed-event recovery bounded. Tests: load many rooms, update one path, and count source reads/path queries. Only the target room may read its source. Cover rename, deletion, self-save, repeated watcher/plugin hints, a locked unrelated room and simultaneous local/external edits. Preserve the existing journal_race/shared_notes/untouched_bytes tests. Extend collaboration performance evidence for 1/10/100 loaded rooms. Duplicate search: #265 owns the large-Note initial sync timeout, #634 duplicate block reconciliation, #382 external edit integrity. This finding covers the unrelated-room scan before any block merge; it needs a separate performance fix. No data loss or crash was demonstrated.

kayg referenced this issue

2026-10-02 13:11:35 +00:00

PERF: bound the collaboration filesystem event queue (#663) #761

kayg referenced this issue

2026-10-02 13:16:50 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-02 13:19:27 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg commented

2026-10-02 13:25:38 +00:00

Author

Owner

Starting work on job/notesperf, based on job/merge-round-7a at 2f4482ded066d9c5d9c59130377907f7fd2916c9. I will address the issue with focused changes and regression coverage, then report the final head SHA and verbatim gate output here.

Starting work on `job/notesperf`, based on `job/merge-round-7a` at `2f4482ded066d9c5d9c59130377907f7fd2916c9`. I will address the issue with focused changes and regression coverage, then report the final head SHA and verbatim gate output here.

kayg commented

2026-10-02 19:10:47 +00:00

Author

Owner

External changes now resolve the indexed stable Note ID once and read only that active room. The small public helper calternal_plugin_notes::collab_id_for_path exposes the Index lookup to the Collab crate; the watcher queue coalesces path aliases by User and ID. The regression test passed: cargo test -p calternal-collab --lib external_change_skips_unrelated_locked_rooms (1 passed). Commit: 750b36129.

External changes now resolve the indexed stable Note ID once and read only that active room. The small public helper `calternal_plugin_notes::collab_id_for_path` exposes the Index lookup to the Collab crate; the watcher queue coalesces path aliases by User and ID. The regression test passed: `cargo test -p calternal-collab --lib external_change_skips_unrelated_locked_rooms` (1 passed). Commit: 750b36129.

kayg referenced this issue

2026-10-02 21:09:16 +00:00

PERF: mount Notes before optional syntax grammars load (#663) #747

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal#769

No description provided.

Rows
Columns