kayg/calternal

Watch

Fork 0

Reminders CalDAV: initial sync-collection takes 17 s on a small task set #82

New issue

Closed

opened 2026-09-25 04:28:15 +00:00 by kayg · 4 comments

kayg commented

2026-09-25 04:28:15 +00:00

Owner

Adversarial round 1 twice reported Reminders initial sync :: SLOW 17–18 s status 207 (main f8e93b7, and the #78 branch) for the Reminders CalDAV collection added in #48. The probe's test user has only a handful of tasks, so a 17 s initial sync-collection REPORT is too slow even with the VM loaded (other endpoints stayed at 1–6 s).

Find what the initial sync does (probably a full Markdown scan or projection rebuild per request, or per-item queries), make the initial sync read from the reminder projection/Index with bounded queries, and add a timing assertion (e.g. 2,000 tasks initial sync < 1 s on a release build). Apple Reminders does an initial sync on every new device, and large task sets will make this worse.

Adversarial round 1 twice reported `Reminders initial sync :: SLOW 17–18 s status 207` (main `f8e93b7`, and the #78 branch) for the Reminders CalDAV collection added in #48. The probe's test user has only a handful of tasks, so a 17 s initial `sync-collection` REPORT is too slow even with the VM loaded (other endpoints stayed at 1–6 s). Find what the initial sync does (probably a full Markdown scan or projection rebuild per request, or per-item queries), make the initial sync read from the reminder projection/Index with bounded queries, and add a timing assertion (e.g. 2,000 tasks initial sync < 1 s on a release build). Apple Reminders does an initial sync on every new device, and large task sets will make this worse.

kayg commented

2026-09-25 04:40:32 +00:00

Author

Owner

Starting work on job/reminders-sync at base 9561cc8b69508145709ce409995848f525171834. I am measuring the release sync-collection path at 10, 200, and 2,000 tasks before changing the reminder projection, then I will optimize the measured slow path and add the issue's timing regression.

Starting work on `job/reminders-sync` at base `9561cc8b69508145709ce409995848f525171834`. I am measuring the release `sync-collection` path at 10, 200, and 2,000 tasks before changing the reminder projection, then I will optimize the measured slow path and add the issue's timing regression.

kayg commented

2026-09-25 04:55:57 +00:00

Author

Owner

Finding from the release REPORT harness: initial sync makes one indexed list query, then the DAV protocol calls RemindersProvider::get once per active UID. The measured get counts were 10, 200, and 2,000 for those collection sizes. Release REPORT timings were 9.9 ms, 37.8 ms, and 728.1 ms in the first run (3.0 ms, 59.4 ms, and 671.9 ms in the instrumented run); the 2,000-task sample is under the suggested one-second budget but leaves little margin. This reproduces the unbounded query count, though not the reported 17-second latency with a handful of tasks. I am changing initial sync to reuse the already-loaded projection rows while preserving the current hrefs, ETags, and sync token.

Finding from the release REPORT harness: initial sync makes one indexed list query, then the DAV protocol calls `RemindersProvider::get` once per active UID. The measured `get` counts were 10, 200, and 2,000 for those collection sizes. Release REPORT timings were 9.9 ms, 37.8 ms, and 728.1 ms in the first run (3.0 ms, 59.4 ms, and 671.9 ms in the instrumented run); the 2,000-task sample is under the suggested one-second budget but leaves little margin. This reproduces the unbounded query count, though not the reported 17-second latency with a handful of tasks. I am changing initial sync to reuse the already-loaded projection rows while preserving the current hrefs, ETags, and sync token.

kayg commented

2026-09-25 05:32:59 +00:00

Author

Owner

Finished on job/reminders-sync, head af77387757fd0a3dba7a043e34e82aff54daf22a.

The release REPORT baseline was 9.9 ms / 37.8 ms / 728.1 ms at 10 / 200 / 2,000 Tasks. The regression observed 10 / 200 / 2,000 provider get calls. After reusing the indexed snapshot, the final release run measured 2.3 ms / 2.4 ms / 28.1 ms, with zero per-item get calls and the <1 s 2,000-Task budget passing. Hrefs, escaped ETags, and the sync token are checked against the provider projection.

Gate results:

cargo fmt --check: exit 0, no output.
cargo clippy --all-targets -- -D warnings: Finished dev profile [unoptimized + debuginfo] target(s) in 27.28s.
Notes tests: test result: ok. 50 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 14.93s.
CalDAV tests: test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s.
bash tests/adversarial/run.sh: ==== FINDINGS 0 and ==== ROUND 2 FINDINGS 0; the Reminders section did not report SLOW.
cargo test --workspace: test result: FAILED. 7 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.92s. The failure is calternal-collab test unrepresentable_update_is_rejected_and_room_keeps_saving, at its room-unload assertion. The same test failed when run alone (test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 7 filtered out; finished in 4.38s). The Notes and CalDAV crate suites passed separately.

Decision not specified in the design doc: ReminderChanges.initial_resources carries the initial snapshot from the existing projection query. Its items list stays empty for an initial sync; incremental changes keep the existing UID/tombstone flow. The DAV response format and token remain unchanged.

The local release harness did not reproduce the reported 17–18 s delay with a handful of Tasks. It did reproduce the per-UID query growth, which is removed by this change.

Finished on `job/reminders-sync`, head `af77387757fd0a3dba7a043e34e82aff54daf22a`. The release REPORT baseline was 9.9 ms / 37.8 ms / 728.1 ms at 10 / 200 / 2,000 Tasks. The regression observed 10 / 200 / 2,000 provider `get` calls. After reusing the indexed snapshot, the final release run measured 2.3 ms / 2.4 ms / 28.1 ms, with zero per-item `get` calls and the <1 s 2,000-Task budget passing. Hrefs, escaped ETags, and the sync token are checked against the provider projection. Gate results: - `cargo fmt --check`: exit 0, no output. - `cargo clippy --all-targets -- -D warnings`: `Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.28s`. - Notes tests: `test result: ok. 50 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 14.93s`. - CalDAV tests: `test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s`. - `bash tests/adversarial/run.sh`: `==== FINDINGS 0` and `==== ROUND 2 FINDINGS 0`; the Reminders section did not report `SLOW`. - `cargo test --workspace`: `test result: FAILED. 7 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.92s`. The failure is `calternal-collab` test `unrepresentable_update_is_rejected_and_room_keeps_saving`, at its room-unload assertion. The same test failed when run alone (`test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 7 filtered out; finished in 4.38s`). The Notes and CalDAV crate suites passed separately. Decision not specified in the design doc: `ReminderChanges.initial_resources` carries the initial snapshot from the existing projection query. Its `items` list stays empty for an initial sync; incremental changes keep the existing UID/tombstone flow. The DAV response format and token remain unchanged. The local release harness did not reproduce the reported 17–18 s delay with a handful of Tasks. It did reproduce the per-UID query growth, which is removed by this change.

kayg referenced this issue from a commit

2026-09-25 05:35:15 +00:00

Merge job/reminders-sync: initial Reminders sync reads the indexed snapshot (#82)

kayg commented

2026-10-01 05:09:01 +00:00

Author

Owner

Completed on dev in 4257fe7d7c (Merge job/reminders-sync: initial Reminders sync reads the indexed snapshot (#82)).

Completed on dev in 4257fe7d7c629ca0c9ed27bb603ed613eb66a9d6 (Merge job/reminders-sync: initial Reminders sync reads the indexed snapshot (#82)).

kayg closed this issue

2026-10-01 05:09:02 +00:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal#82

No description provided.

Rows
Columns