SYNC: Reminders.app creates are duplicated (17 Reminders → 34 Tasks); SQLite pool timeouts during the burst

kayg commented

2026-10-01 07:17:08 +00:00

Owner

Found by mac-393 round 4 (2026-10-01; real macOS Reminders.app on the Mac VM against a local calternal server)

Matrix cell R8: creating 17 Reminders natively in Reminders.app produced 34 Task records with distinct Task IDs in calternal: every Reminder was duplicated. This is the sync-collision / data-integrity class (a merge blocker for anything touching it).
Also seen: the Webcal create attempt timed out while the local server logged SQLite pool timeouts.
Job (Sol medium: sync correctness):

Reproduce deterministically without the Mac: replay the captured Reminders.app traffic for R8 (tests/apple/ fixtures, the mac-393 evidence on #393) against the server and find why one Reminder becomes two Tasks. Likely suspects (verify): a PUT followed by a second PUT/PROPPATCH with a different href or UID handling; the client retrying after a slow response (SQLite pool timeout) while the first write had already committed (non-idempotent create); a UID → Task ID mapping that does not dedupe on UID; the #531 anchor/ID change.
Fix: a create keyed by the iCalendar UID (per collection) is idempotent. A repeated PUT with the same UID updates instead of inserting; If-None-Match: * conflicts return 412. Add a regression test with the replayed traffic, plus a retry-after-timeout case.
SQLite pool timeouts during a burst of creates: find the long-held connection or transaction (N+1 inside a write transaction? the writer held across file I/O?). Fix, or file a separate PERF issue with numbers (link #573).
Re-verify on the Mac VM (flock the Mac lock): 17 native Reminders → exactly 17 Tasks, across 3 runs.
Per-crate gates (calternal-dav, plugins/notes, plugins/calendar as touched). Time limit 4 h.

## Found by mac-393 round 4 (2026-10-01; real macOS Reminders.app on the Mac VM against a local calternal server) Matrix cell **R8**: creating **17 Reminders natively in Reminders.app** produced **34 Task records with distinct Task IDs** in calternal: every Reminder was duplicated. This is the sync-collision / data-integrity class (a merge blocker for anything touching it). Also seen: the Webcal create attempt timed out while the local server logged **SQLite pool timeouts**. **Job (Sol medium: sync correctness):** 1. Reproduce deterministically without the Mac: replay the captured Reminders.app traffic for R8 (`tests/apple/` fixtures, the mac-393 evidence on #393) against the server and find why one Reminder becomes two Tasks. Likely suspects (verify): a PUT followed by a second PUT/PROPPATCH with a different href or UID handling; the client retrying after a slow response (SQLite pool timeout) while the first write had already committed (non-idempotent create); a UID → Task ID mapping that does not dedupe on UID; the #531 anchor/ID change. 2. Fix: a create keyed by the iCalendar **UID** (per collection) is idempotent. A repeated PUT with the same UID updates instead of inserting; If-None-Match: * conflicts return 412. Add a regression test with the replayed traffic, plus a retry-after-timeout case. 3. SQLite pool timeouts during a burst of creates: find the long-held connection or transaction (N+1 inside a write transaction? the writer held across file I/O?). Fix, or file a separate PERF issue with numbers (link #573). 4. Re-verify on the Mac VM (`flock` the Mac lock): 17 native Reminders → exactly 17 Tasks, across 3 runs. Per-crate gates (calternal-dav, plugins/notes, plugins/calendar as touched). Time limit 4 h.

remdup-585-evidence.zip

4.2 KiB

native-three-runs.png

701 KiB

remdup-585-final-evidence.zip

39 KiB

kayg commented

2026-10-01 07:19:28 +00:00

Author

Owner

Starting #585 on job/remdup-585, base/head cc25c441b7a974185622a1dee853cf38686d2b67. Read CLAUDE.md, CONTEXT.md and DESIGN §30/§31/§33/§40/§41/§46/§47. The #393 round-4 report says the second batch used new UIDs, so a same-UID retry alone does not explain R8. Retained wire captures exist in the mac-393 worktree. I will compare their bodies and server projection, add replay coverage, then validate on the locked Mac. No push or deployment.

Starting #585 on `job/remdup-585`, base/head `cc25c441b7a974185622a1dee853cf38686d2b67`. Read CLAUDE.md, CONTEXT.md and DESIGN §30/§31/§33/§40/§41/§46/§47. The #393 round-4 report says the second batch used new UIDs, so a same-UID retry alone does not explain R8. Retained wire captures exist in the mac-393 worktree. I will compare their bodies and server projection, add replay coverage, then validate on the locked Mac. No push or deployment.

kayg commented

2026-10-01 07:26:42 +00:00

Author

Owner

Captured R8 finding: 1790831467537-00025 through 1790831487439-00041 are 17 creates. 1790831667128-00045 through 1790831703203-00061 are another 17 creates with different VTODO UIDs and new CREATED timestamps (for example yearly changes from FC319F3B-0DBA-4868-A4DE-39800E0BAED0 to AE684453-4EE6-4071-A597-9AAEA5612D2C). All use If-None-Match: * and return 201. Exchange 00042 returns the first 17 original hrefs and their PUT ETags unchanged; there is no intervening content GET/multiget in the retained capture before the second create batch. No repeated-UID PUT or 412 is present in this sequence. Thus replaying all 34 writes must retain 34 independent identities; deduplication by title would discard legitimate Tasks. Native verification is needed to isolate why Apple emitted the new batch.

Separate interruption risk found in the provider: the per-User guard belongs to the request future, and creation installs Markdown before index_source acquires the SQLite writer. Dropping that future at this boundary leaves a content file without its UID projection. For a non-UUID client UID, retry generates a fresh Markdown UUID and can install a second file. A test now holds the writer, waits for the file installation, cancels the request and retries. Build is in progress. The fix will preserve the mutation and guard through the Index commit when the HTTP caller stops waiting.

Decision: retain distinct UIDs as distinct Tasks. No title-based deduplication or format change.

Captured R8 finding: `1790831467537-00025` through `1790831487439-00041` are 17 creates. `1790831667128-00045` through `1790831703203-00061` are another 17 creates with **different VTODO UIDs and new CREATED timestamps** (for example yearly changes from FC319F3B-0DBA-4868-A4DE-39800E0BAED0 to AE684453-4EE6-4071-A597-9AAEA5612D2C). All use `If-None-Match: *` and return 201. Exchange 00042 returns the first 17 original hrefs and their PUT ETags unchanged; there is no intervening content GET/multiget in the retained capture before the second create batch. No repeated-UID PUT or 412 is present in this sequence. Thus replaying all 34 writes must retain 34 independent identities; deduplication by title would discard legitimate Tasks. Native verification is needed to isolate why Apple emitted the new batch. Separate interruption risk found in the provider: the per-User guard belongs to the request future, and creation installs Markdown before `index_source` acquires the SQLite writer. Dropping that future at this boundary leaves a content file without its UID projection. For a non-UUID client UID, retry generates a fresh Markdown UUID and can install a second file. A test now holds the writer, waits for the file installation, cancels the request and retries. Build is in progress. The fix will preserve the mutation and guard through the Index commit when the HTTP caller stops waiting. Decision: retain distinct UIDs as distinct Tasks. No title-based deduplication or format change.

kayg commented

2026-10-01 07:32:35 +00:00

Author

Owner

The deterministic interruption regression failed on the base provider:

thread 'retry_after_timeout_during_index_commit_creates_one_task' (1710948) panicked at crates/plugins/notes/tests/apple_replay.rs:274:5:
assertion failed: matches!(retry, Err(calternal_dav::protocol::DavError::Precondition))
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 3.72s

The test holds the sole SQLite writer, waits for the content file to appear, aborts the caller, releases the writer and repeats If-None-Match: * with the same non-UUID UID. The base accepts the retry because the first UID has no Index row yet. The patch acquires admission in the caller, then owns the User guard and the file/Index operation in a Tokio task. It carries current_agent_turn_scope() into the task so write attribution is retained. A cancelled caller waiting for admission does not enqueue a detached mutation. Validation is running. This is a separate defect from the captured R8 sequence, whose second batch has new UIDs.

The deterministic interruption regression failed on the base provider: ``` thread 'retry_after_timeout_during_index_commit_creates_one_task' (1710948) panicked at crates/plugins/notes/tests/apple_replay.rs:274:5: assertion failed: matches!(retry, Err(calternal_dav::protocol::DavError::Precondition)) test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 3.72s ``` The test holds the sole SQLite writer, waits for the content file to appear, aborts the caller, releases the writer and repeats `If-None-Match: *` with the same non-UUID UID. The base accepts the retry because the first UID has no Index row yet. The patch acquires admission in the caller, then owns the User guard and the file/Index operation in a Tokio task. It carries `current_agent_turn_scope()` into the task so write attribution is retained. A cancelled caller waiting for admission does not enqueue a detached mutation. Validation is running. This is a separate defect from the captured R8 sequence, whose second batch has new UIDs.

kayg commented

2026-10-01 07:44:34 +00:00

Author

Owner

Committed the cancellation fix and R8 fixtures at 4f3a95eda. The full Notes slice gates passed:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 38s
test result: ok. 131 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 71.01s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.04s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo fmt --check exited 0 with no output. The R8 test creates 17 distinct UIDs, retries each with If-None-Match: * (412), updates each title with its ETag, and verifies 17 Task rows with unchanged IDs. The interruption test now verifies one file and one Task after a dropped caller and same-UID retry. A first version exceeded the debug test thread stack; boxing the large writer future fixed it, and the default-stack gates above passed.

The raw long-title Apple fixtures have trailing spaces inside folded content lines. These are captured data, not code whitespace; retained unchanged. The perf VM lock was unavailable on the bounded preflight, so measurement will be labelled local. Mac EventKit discovery works; its only current list is local, so the disposable branch-server account needs installation before the three native runs. Server build is in progress.

Committed the cancellation fix and R8 fixtures at `4f3a95eda`. The full Notes slice gates passed: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 38s test result: ok. 131 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 71.01s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.04s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo fmt --check` exited 0 with no output. The R8 test creates 17 distinct UIDs, retries each with `If-None-Match: *` (412), updates each title with its ETag, and verifies 17 Task rows with unchanged IDs. The interruption test now verifies one file and one Task after a dropped caller and same-UID retry. A first version exceeded the debug test thread stack; boxing the large writer future fixed it, and the default-stack gates above passed. The raw long-title Apple fixtures have trailing spaces inside folded content lines. These are captured data, not code whitespace; retained unchanged. The perf VM lock was unavailable on the bounded preflight, so measurement will be labelled local. Mac EventKit discovery works; its only current list is local, so the disposable branch-server account needs installation before the three native runs. Server build is in progress.

kayg referenced this issue from a commit

2026-10-01 07:48:58 +00:00

Keep admitted Reminder PUTs alive through Index commit

kayg referenced this issue from a commit

2026-10-01 07:48:58 +00:00

Measure Reminder create bursts and check same-UID retries

kayg commented

2026-10-01 07:57:42 +00:00

Author

Owner

Pool investigation found two source-size loops inside the Notes writer transaction. tasks_store::index_source begins the sole writer transaction before reminder_projection. That projection hashes the entire Markdown text once per Task and calls inline_completed_date, which scans the entire Note once per inline Task. A 10,000-Task Note therefore hashes/scans the same source 10,000 times while every other SQLite writer waits. File I/O is outside this transaction; this is CPU work plus per-row SQL in the transaction.

I will reuse one BLAKE3 prefix state (preserving the exact existing ETag bytes), collect inline completion dates once through the existing Task-line parser, and compute the DAV projection before acquiring the SQLite writer. The alias filter drops only Reference rows, which the DAV projection already skips, so moving that computation does not change the projection. The existing API benchmark will measure the large-corpus create burst. Remaining per-row SQL costs, if they still cause timeouts, will be filed separately with numbers and linked to #573.

Pool investigation found two source-size loops inside the Notes writer transaction. `tasks_store::index_source` begins the sole writer transaction before `reminder_projection`. That projection hashes the entire Markdown text once per Task and calls `inline_completed_date`, which scans the entire Note once per inline Task. A 10,000-Task Note therefore hashes/scans the same source 10,000 times while every other SQLite writer waits. File I/O is outside this transaction; this is CPU work plus per-row SQL in the transaction. I will reuse one BLAKE3 prefix state (preserving the exact existing ETag bytes), collect inline completion dates once through the existing Task-line parser, and compute the DAV projection before acquiring the SQLite writer. The alias filter drops only Reference rows, which the DAV projection already skips, so moving that computation does not change the projection. The existing API benchmark will measure the large-corpus create burst. Remaining per-row SQL costs, if they still cause timeouts, will be filed separately with numbers and linked to #573.

kayg commented

2026-10-01 08:11:48 +00:00

Author

Owner

#585 writer preparation finding: index_source acquired the sole SQLite writer before reminder_projection, which hashed the full Note per Task and scanned the full Note per inline completion lookup. The change prepares source-only projections before writer admission, clones one BLAKE3 prefix, and collects completion dates once through the existing Index parser. Tests compare the old and new fingerprint bytes and exercise repair of the old prefix-collision completion value.

Small public addition: expose calternal_notes_core::tasks::parse_task_line without changing its behavior, so DAV and the Index share their Task grammar instead of duplicating it. Core gates are included. Local server measurement follows; the perf VM lock was occupied at the bounded preflight.

#585 writer preparation finding: `index_source` acquired the sole SQLite writer before `reminder_projection`, which hashed the full Note per Task and scanned the full Note per inline completion lookup. The change prepares source-only projections before writer admission, clones one BLAKE3 prefix, and collects completion dates once through the existing Index parser. Tests compare the old and new fingerprint bytes and exercise repair of the old prefix-collision completion value. Small public addition: expose `calternal_notes_core::tasks::parse_task_line` without changing its behavior, so DAV and the Index share their Task grammar instead of duplicating it. Core gates are included. Local server measurement follows; the perf VM lock was occupied at the bounded preflight.

kayg referenced this issue

2026-10-01 08:13:08 +00:00

CalDAV Reminders reads lose VTODO completion state #558

kayg commented

2026-10-01 08:21:28 +00:00

Author

Owner

#585 adjacent correctness finding: the old completion lookup used substring anchors. A pending ^child could receive the dated completion of ^childlong. The shared parser now matches exact block IDs in one source scan. A regression reproduces the collision and a second regression simulates an old Index row, verifies a repair emits one sync change, and verifies the ETag changes.

Decision: preserve source-prefix hashing bytes, but version inline Reminder fingerprints with the parsed completion value. Retaining the old ETag during a repair could leave Apple using cached incorrect bytes. File Task fingerprints remain unchanged; identities and sync epochs remain stable. No schema migration or new dependency is needed.

#585 adjacent correctness finding: the old completion lookup used substring anchors. A pending `^child` could receive the dated completion of `^childlong`. The shared parser now matches exact block IDs in one source scan. A regression reproduces the collision and a second regression simulates an old Index row, verifies a repair emits one sync change, and verifies the ETag changes. Decision: preserve source-prefix hashing bytes, but version inline Reminder fingerprints with the parsed completion value. Retaining the old ETag during a repair could leave Apple using cached incorrect bytes. File Task fingerprints remain unchanged; identities and sync epochs remain stable. No schema migration or new dependency is needed.

kayg commented

2026-10-01 08:39:00 +00:00

Author

Owner

Projection slice committed at 9282f570c5599d2e058ae06e2a60e24e4c9d9484. Notes/core gates passed. The real production web build completed. No dependency or schema change. Live server replay, local performance numbers and locked Mac validation follow.

clippy-notes-final.log
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 45s
test-notes-final.log
    Finished `test` profile [unoptimized + debuginfo] target(s) in 9m 34s
test result: ok. 133 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 135.87s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 18.16s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
clippy-core-final.log
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 24s
test-core-final.log
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 31s
test result: ok. 515 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.78s
test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 9.26s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.24s
test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Projection slice committed at `9282f570c5599d2e058ae06e2a60e24e4c9d9484`. Notes/core gates passed. The real production web build completed. No dependency or schema change. Live server replay, local performance numbers and locked Mac validation follow. ```text clippy-notes-final.log Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 45s test-notes-final.log Finished `test` profile [unoptimized + debuginfo] target(s) in 9m 34s test result: ok. 133 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 135.87s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 18.16s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s clippy-core-final.log Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 24s test-core-final.log Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 31s test result: ok. 515 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.78s test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 9.26s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.24s test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ```

kayg referenced this issue

2026-10-01 09:05:32 +00:00

PERF: Reminder create burst exceeds 180 s with a 10k-Task Note #602

kayg commented

2026-10-01 09:10:09 +00:00

Author

Owner

Live local findings on cd6aa2caee:

Full captured R8 replay: 34 distinct captured UIDs produced 34 Tasks. All 34 conditional retries returned 412. Both sync responses listed the correct 17 resources but omitted the requested getcontenttype. This proves the provider UID collision hypothesis does not explain the original distinct-UID batch.
The retry probe's new 17-request storm encountered nine legitimate 429 rate-limit responses during the one broad DAV round. Other observed findings were SLOW-only. The new check now accepts credential throttling with its rate_limited code, still requires at least one provider 412, and keeps the Task integrity assertion. No baseline test expectation changed. Focused replay passed: 17 conflicts, 0 throttled, exactly one Task and successful cleanup.
Remaining create latency is tracked in #602, linked to #573, with local CPU/RSS and bounded request numbers. No matching baseline exists and the profile recorded zero SQLite pool timeouts.
Clarification: reminder_resources.etag stores a source fingerprint for sync change detection. tasks_dav::decode computes the actual HTTP ETag from canonical iCalendar bytes. The new inline completion version applies to source fingerprints; unchanged GET representations keep their HTTP ETags. Comments now state that distinction.
Mac account validation is not a pass: HTTP discovery stopped at its unauthenticated 401; TLS setup and certificate authorization are still in progress. No native create batch has been applied.

Live local findings on cd6aa2caeef2330f8c8d7f7234da111a89629817: - Full captured R8 replay: 34 distinct captured UIDs produced 34 Tasks. All 34 conditional retries returned 412. Both sync responses listed the correct 17 resources but omitted the requested getcontenttype. This proves the provider UID collision hypothesis does not explain the original distinct-UID batch. - The retry probe's new 17-request storm encountered nine legitimate 429 rate-limit responses during the one broad DAV round. Other observed findings were SLOW-only. The new check now accepts credential throttling with its rate_limited code, still requires at least one provider 412, and keeps the Task integrity assertion. No baseline test expectation changed. Focused replay passed: 17 conflicts, 0 throttled, exactly one Task and successful cleanup. - Remaining create latency is tracked in #602, linked to #573, with local CPU/RSS and bounded request numbers. No matching baseline exists and the profile recorded zero SQLite pool timeouts. - Clarification: reminder_resources.etag stores a source fingerprint for sync change detection. tasks_dav::decode computes the actual HTTP ETag from canonical iCalendar bytes. The new inline completion version applies to source fingerprints; unchanged GET representations keep their HTTP ETags. Comments now state that distinction. - Mac account validation is not a pass: HTTP discovery stopped at its unauthenticated 401; TLS setup and certificate authorization are still in progress. No native create batch has been applied.

kayg commented

2026-10-01 09:34:00 +00:00

Author

Owner

Mac verification is waiting for administrator authentication. The disposable HTTP account failed DAAccountValidationDomain:100 after one unauthenticated PROPFIND 401. The TLS lab is ready, but sudo -n reports a password is required, and user-domain security add-trusted-cert reports The authorization was denied since no user interaction was possible. The root certificate profile entered installation but did not install or validate. I requested the approved private credential file path (not the password) and am releasing the Mac lock while final gates and the sync-property regression run. No native create batch has been submitted, and three-run acceptance is not claimed.

Remaining latency is recorded in #602, with the #573 link and exact local CPU/RSS numbers. The new retry probe passed in a focused real-server run with 17×412 and exactly one Task. The one broad DAV round's new-test failures were legitimate 429 throttling; no baseline expectation was changed.

Mac verification is waiting for administrator authentication. The disposable HTTP account failed DAAccountValidationDomain:100 after one unauthenticated PROPFIND 401. The TLS lab is ready, but `sudo -n` reports `a password is required`, and user-domain `security add-trusted-cert` reports `The authorization was denied since no user interaction was possible`. The root certificate profile entered installation but did not install or validate. I requested the approved private credential file path (not the password) and am releasing the Mac lock while final gates and the sync-property regression run. No native create batch has been submitted, and three-run acceptance is not claimed. Remaining latency is recorded in #602, with the #573 link and exact local CPU/RSS numbers. The new retry probe passed in a focused real-server run with 17×412 and exactly one Task. The one broad DAV round's new-test failures were legitimate 429 throttling; no baseline expectation was changed.

kayg commented

2026-10-01 09:36:21 +00:00

Author

Owner

The R8 sync selector fixture now records Apple's getetag + getcontenttype request with its instance token removed (176e60fb5). The confirmed omission is fixed through the existing property renderer for both initial and incremental Reminders sync. DAV and Notes clippy passed; DAV tests passed; Notes tests are running. The replay adds assertions for 17 initial content types and one incremental content type without changing prior assertions.

The Mac authorization attempt was cancelled; the test certificate was removed from the user keychain and the installed-profile list still contains only the pre-existing managed profile. The Mac lock is released while the credential-path request is pending. No native Reminders were created.

The R8 sync selector fixture now records Apple's `getetag` + `getcontenttype` request with its instance token removed (176e60fb5). The confirmed omission is fixed through the existing property renderer for both initial and incremental Reminders sync. DAV and Notes clippy passed; DAV tests passed; Notes tests are running. The replay adds assertions for 17 initial content types and one incremental content type without changing prior assertions. The Mac authorization attempt was cancelled; the test certificate was removed from the user keychain and the installed-profile list still contains only the pre-existing managed profile. The Mac lock is released while the credential-path request is pending. No native Reminders were created.

kayg commented

2026-10-01 10:25:08 +00:00

Author

Owner

Head 0516d0a9c: the real branch server replay now returns 17 requested getcontenttype properties for each R8 sync batch. All 34 captured PUTs return 201 because they contain 34 distinct UIDs. All 34 conditional retries return 412; the Index contains 34 Tasks and 34 unique resource UIDs. This verifies the MIME response repair and same-UID isolation, but does not prove the original native duplication is resolved.

Original native history: after normalizing the memory address text in EventKit recurrence descriptions, every field of the first 17 native Reminders is unchanged after refresh. The second batch has 17 new native IDs and 17 new iCalendar UIDs. The original first batch uses CREATED 05:11:05Z; the second uses 05:14:25Z. A sync response and two Home PROPFINDs occur between them; no GET or multiget fetch occurs. Do not dedupe distinct UIDs by title.

Local debug sync profile (shared host; no matching create baseline): serial create p50/p95 854.88/7540.54 ms, 17 creates and 17 conditional retries. Initial sync of 17 Tasks: p50/p95 7.65/8.25 ms. Initial sync of 10,017 Tasks: p50/p95 977.27/986.02 ms, mean RSS 474898747 bytes, peak 542289920 bytes, CPU 6.28 seconds over 3.076 seconds. Every requested content type is present. No SQLite pool timeout was logged. The earlier slow create burst remains PERF #602, linked to #573.

Mac account setup: an absolute HTTP principal URL allowed the User profile to install, but native discovery still exposes only the local Reminders list. The client probes discovery and receives 401 without a Basic-auth retry. The TLS route needs administrator authentication for the disposable CA; the approved private credential path is still unavailable. No native batch has been submitted, and the required three 17-to-17 runs remain unverified. All Mac operations hold the shared Mac lock.

Head `0516d0a9c`: the real branch server replay now returns 17 requested `getcontenttype` properties for each R8 sync batch. All 34 captured PUTs return 201 because they contain 34 distinct UIDs. All 34 conditional retries return 412; the Index contains 34 Tasks and 34 unique resource UIDs. This verifies the MIME response repair and same-UID isolation, but does not prove the original native duplication is resolved. Original native history: after normalizing the memory address text in EventKit recurrence descriptions, every field of the first 17 native Reminders is unchanged after refresh. The second batch has 17 new native IDs and 17 new iCalendar UIDs. The original first batch uses CREATED 05:11:05Z; the second uses 05:14:25Z. A sync response and two Home PROPFINDs occur between them; no GET or multiget fetch occurs. Do not dedupe distinct UIDs by title. Local debug sync profile (shared host; no matching create baseline): serial create p50/p95 854.88/7540.54 ms, 17 creates and 17 conditional retries. Initial sync of 17 Tasks: p50/p95 7.65/8.25 ms. Initial sync of 10,017 Tasks: p50/p95 977.27/986.02 ms, mean RSS 474898747 bytes, peak 542289920 bytes, CPU 6.28 seconds over 3.076 seconds. Every requested content type is present. No SQLite pool timeout was logged. The earlier slow create burst remains PERF #602, linked to #573. Mac account setup: an absolute HTTP principal URL allowed the User profile to install, but native discovery still exposes only the local Reminders list. The client probes discovery and receives 401 without a Basic-auth retry. The TLS route needs administrator authentication for the disposable CA; the approved private credential path is still unavailable. No native batch has been submitted, and the required three 17-to-17 runs remain unverified. All Mac operations hold the shared Mac lock.

kayg commented

2026-10-01 10:59:21 +00:00

Author

Owner

Head 0516d0a9c1cbcb49b5b16ac6e037c0e818338a4e. All final Rust gates passed (notes-core, DAV, Notes, server). The repeated Apple replay check also passed 3/3 after its prior waiting process was terminated. Gate summaries will be quoted verbatim in the final report.

The locked Mac lab now verifies TLS trust. Native discovery exposes the writable remote Reminders collection. One disposable API Task was used for discovery, then deleted with 204 before the native test. The first native apply submitted exactly 17 items once. The real server now has exactly 17 Tasks and 17 unique resource UIDs, all with the run1 prefix. Native reads and another Calendar refresh are in progress; no three-run acceptance is claimed yet. No second apply was sent for run1.

Head `0516d0a9c1cbcb49b5b16ac6e037c0e818338a4e`. All final Rust gates passed (notes-core, DAV, Notes, server). The repeated Apple replay check also passed 3/3 after its prior waiting process was terminated. Gate summaries will be quoted verbatim in the final report. The locked Mac lab now verifies TLS trust. Native discovery exposes the writable remote Reminders collection. One disposable API Task was used for discovery, then deleted with 204 before the native test. The first native apply submitted exactly 17 items once. The real server now has exactly 17 Tasks and 17 unique resource UIDs, all with the run1 prefix. Native reads and another Calendar refresh are in progress; no three-run acceptance is claimed yet. No second apply was sent for run1.

kayg commented

2026-10-01 11:16:52 +00:00

Author

Owner

Built (#585):

Keep an admitted Reminder PUT alive through the file and Index commit when its HTTP wait is cancelled. Retain the User writer lock and Agent attribution. Waiting requests remain cancellable.
Reuse one source hash prefix and one completion-date scan before SQLite writer admission. Match exact block IDs. Version inline source fingerprints so a completion repair emits one sync change.
Return the resource properties that Apple requests in initial and incremental Reminders sync REPORTs. Reuse the existing DAV property renderer, including missing-property responses.
Keep the exact R8 request fixtures and add tests for 17 creates, conditional retries, identity-preserving edits, cancelled-response retries, and sync content types.
Add a local performance profile for serial creates, a large create burst and both sync workload sizes. Extend the real-server adversarial retry probe.

Files: crates/plugins/notes/src/tasks_dav.rs, tasks_store.rs and tests/apple_replay.rs; crates/calternal-notes-core/src/tasks/mod.rs and line.rs; crates/calternal-dav/src/protocol.rs and tests/fixtures/macos27/r8/; bench/reminders_create.py and .md; tests/adversarial/attack.py.

Decisions:

Keep separate UIDs as separate Tasks. The R8 capture contains 34 distinct VTODO UIDs. Title equality cannot define identity.
Keep the current conditional-write and UID/href contract. Do not rotate Task IDs or the sync epoch.
Expose the existing Task-line parser for the DAV projection. This is a small public addition to notes-core; its grammar is unchanged.
Version the inline Index source fingerprint for completion repair. The HTTP ETag still comes from canonical iCalendar bytes.
Use the existing property renderer for sync REPORTs. Older callers without a prop selector keep the ETag default.
Measure locally because the perf VM lock was occupied. The build is debug and the host is shared. There is no matching Reminders-create baseline. Do not call these numbers a release regression. File remaining burst latency separately as #602 and link #573.

Evidence:

Real server: 34 captured distinct-UID PUTs return 201; 34 conditional retries return 412; 34 Tasks and 34 unique UIDs. Both sync batches contain 17 resources and 17 requested content types.
Focused real-server adversarial check: 17 conflicts, 0 throttled, one Task. The one broad round found only SLOW responses and legitimate 429 throttling in the new retry check. The new probe accepts 429 only with the rate_limited code and still requires a provider 412. Existing expectations were not changed.
Original native history retains all 17 original IDs and all their fields after refresh; the extra 17 have new native IDs and new UIDs. Normalize recurrence-description memory addresses for the comparison. This does not establish why Apple creates the second batch.
Sanitized replay, native-history, performance and probe evidence: https://git.kayg.org/attachments/b0e47b83-6176-4ae0-b523-880842eec101

Performance (local debug):

Latest serial: 17 creates + 17 retries; p50/p95 854.88/7540.54 ms; mean CPU 33.03%; mean/peak RSS 321487502/371187712 bytes.
Sync 17 Tasks: p50/p95 7.65/8.25 ms, three responses.
Sync 10017 Tasks: p50/p95 977.27/986.02 ms, three responses; CPU 6.28s over 3.076s; mean/peak RSS 474898747/542289920 bytes. Every content type is present.
Earlier large create burst: 17 attempts, one completed create/retry and 16 timeouts at the 180s request bound. Successful create 172502.28ms is one sample, not a representative p95. Attempt p95 including retry/timeouts 296047.52ms; CPU 181.17s over 296.056s; mean/peak RSS 660167645/794935296 bytes. The final Index had 10019 Tasks: 10000 corpus, 17 serial and two admitted burst creates. Timed-out attempts are not counted as successful creates. No SQLite pool timeout was logged. Remaining whole-User reconciliation and per-row SQL are suspects, not a proven root cause. PERF #602: #602

Known gaps:

The original distinct-UID R8 duplication is not yet proved fixed. Same-UID retry isolation and the sync response repair are verified independently.
Three native runs passed. This does not isolate the original trigger that made Apple submit a second set of distinct UIDs.
The original pool timeout did not recur. The create burst remains slow; see #602. No release build comparison was measured.
Final Rust gates passed. A repeated Apple replay process was terminated while waiting on the build lock; its rerun passed all three tests.

No pushes or deploys. One required origin/dev merge was completed before final gates. No dependency or database migration changes. No app UI source changes.

Head: 0516d0a9c1cbcb49b5b16ac6e037c0e818338a4e

Gate output (verbatim terminal summary lines):

fmt-final.log

clippy-core-final.log

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 24s

test-core-final.log

    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 31s
test result: ok. 515 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.78s
test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 9.26s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.24s
test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

clippy-dav-final.log

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 32m 59s

test-dav-final.log

    Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 16s
test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s
test result: ok. 36 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

clippy-notes-final.log

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 49s

test-notes-final.log

    Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 58s
test result: ok. 133 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 232.83s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 14.50s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

clippy-server-final.log

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 13m 13s

test-server-final.log

    Finished `test` profile [unoptimized + debuginfo] target(s) in 14m 51s
test result: ok. 93 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out; finished in 19.51s

test-apple-final.log

    Finished `test` profile [unoptimized + debuginfo] target(s) in 33.96s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 5.63s

cargo fmt --check: exit 0, no output. Python profile and adversarial syntax checks passed. Real production web build passed for server embedding; no web source changed.

Native acceptance (Mac lock held): the original 17-item native fixture was applied once per run with unique run prefixes. Run1 retained all 17 native IDs after 236 seconds and refresh: 17 Tasks, 17 UIDs. Run2 retained all 17 after 215 seconds: 34 cumulative Tasks/Uids, 17 per run. Run3 retained all 17 after 215 seconds: 51 cumulative Tasks/Uids, 17 per run. No new native IDs appeared within any run. Each run includes one completed Reminder, so the UI shows 48 incomplete items after all three. A disposable API discovery Task was deleted before run1; it is outside these counts. The original distinct-UID trigger remains unknown.

Native UI evidence: https://git.kayg.org/attachments/992f2a6e-cbf4-4e65-a208-45e58f505082

Cleanup: Cargo removed 16868 files, 9.1GiB total. Web build output was deleted. The Mac account and CA profiles were removed. System certificate lookup returns not found, and the admin trust store reports no Trust Settings. Own Mac helper files, private credential copies and disposable server data were removed. The lab server, proxy and tunnel were stopped. The Mac lock was released.

Final evidence (native IDs, three verdicts and gate logs): https://git.kayg.org/attachments/27baffe4-46cf-44d6-b1ea-1d51860d7d47

Built (#585): - Keep an admitted Reminder PUT alive through the file and Index commit when its HTTP wait is cancelled. Retain the User writer lock and Agent attribution. Waiting requests remain cancellable. - Reuse one source hash prefix and one completion-date scan before SQLite writer admission. Match exact block IDs. Version inline source fingerprints so a completion repair emits one sync change. - Return the resource properties that Apple requests in initial and incremental Reminders sync REPORTs. Reuse the existing DAV property renderer, including missing-property responses. - Keep the exact R8 request fixtures and add tests for 17 creates, conditional retries, identity-preserving edits, cancelled-response retries, and sync content types. - Add a local performance profile for serial creates, a large create burst and both sync workload sizes. Extend the real-server adversarial retry probe. Files: crates/plugins/notes/src/tasks_dav.rs, tasks_store.rs and tests/apple_replay.rs; crates/calternal-notes-core/src/tasks/mod.rs and line.rs; crates/calternal-dav/src/protocol.rs and tests/fixtures/macos27/r8/; bench/reminders_create.py and .md; tests/adversarial/attack.py. Decisions: - Keep separate UIDs as separate Tasks. The R8 capture contains 34 distinct VTODO UIDs. Title equality cannot define identity. - Keep the current conditional-write and UID/href contract. Do not rotate Task IDs or the sync epoch. - Expose the existing Task-line parser for the DAV projection. This is a small public addition to notes-core; its grammar is unchanged. - Version the inline Index source fingerprint for completion repair. The HTTP ETag still comes from canonical iCalendar bytes. - Use the existing property renderer for sync REPORTs. Older callers without a prop selector keep the ETag default. - Measure locally because the perf VM lock was occupied. The build is debug and the host is shared. There is no matching Reminders-create baseline. Do not call these numbers a release regression. File remaining burst latency separately as #602 and link #573. Evidence: - Real server: 34 captured distinct-UID PUTs return 201; 34 conditional retries return 412; 34 Tasks and 34 unique UIDs. Both sync batches contain 17 resources and 17 requested content types. - Focused real-server adversarial check: 17 conflicts, 0 throttled, one Task. The one broad round found only SLOW responses and legitimate 429 throttling in the new retry check. The new probe accepts 429 only with the rate_limited code and still requires a provider 412. Existing expectations were not changed. - Original native history retains all 17 original IDs and all their fields after refresh; the extra 17 have new native IDs and new UIDs. Normalize recurrence-description memory addresses for the comparison. This does not establish why Apple creates the second batch. - Sanitized replay, native-history, performance and probe evidence: https://git.kayg.org/attachments/b0e47b83-6176-4ae0-b523-880842eec101 Performance (local debug): - Latest serial: 17 creates + 17 retries; p50/p95 854.88/7540.54 ms; mean CPU 33.03%; mean/peak RSS 321487502/371187712 bytes. - Sync 17 Tasks: p50/p95 7.65/8.25 ms, three responses. - Sync 10017 Tasks: p50/p95 977.27/986.02 ms, three responses; CPU 6.28s over 3.076s; mean/peak RSS 474898747/542289920 bytes. Every content type is present. - Earlier large create burst: 17 attempts, one completed create/retry and 16 timeouts at the 180s request bound. Successful create 172502.28ms is one sample, not a representative p95. Attempt p95 including retry/timeouts 296047.52ms; CPU 181.17s over 296.056s; mean/peak RSS 660167645/794935296 bytes. The final Index had 10019 Tasks: 10000 corpus, 17 serial and two admitted burst creates. Timed-out attempts are not counted as successful creates. No SQLite pool timeout was logged. Remaining whole-User reconciliation and per-row SQL are suspects, not a proven root cause. PERF #602: https://git.kayg.org/kayg/calternal/issues/602 Known gaps: - The original distinct-UID R8 duplication is not yet proved fixed. Same-UID retry isolation and the sync response repair are verified independently. - Three native runs passed. This does not isolate the original trigger that made Apple submit a second set of distinct UIDs. - The original pool timeout did not recur. The create burst remains slow; see #602. No release build comparison was measured. - Final Rust gates passed. A repeated Apple replay process was terminated while waiting on the build lock; its rerun passed all three tests. No pushes or deploys. One required origin/dev merge was completed before final gates. No dependency or database migration changes. No app UI source changes. Head: `0516d0a9c1cbcb49b5b16ac6e037c0e818338a4e` Gate output (verbatim terminal summary lines): fmt-final.log ```text ``` clippy-core-final.log ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 24s ``` test-core-final.log ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 31s test result: ok. 515 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.78s test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 9.26s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.11s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.24s test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` clippy-dav-final.log ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 32m 59s ``` test-dav-final.log ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 16s test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s test result: ok. 36 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` clippy-notes-final.log ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 49s ``` test-notes-final.log ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 58s test result: ok. 133 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 232.83s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 14.50s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` clippy-server-final.log ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 13m 13s ``` test-server-final.log ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 14m 51s test result: ok. 93 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out; finished in 19.51s ``` test-apple-final.log ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 33.96s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 5.63s ``` `cargo fmt --check`: exit 0, no output. Python profile and adversarial syntax checks passed. Real production web build passed for server embedding; no web source changed. Native acceptance (Mac lock held): the original 17-item native fixture was applied once per run with unique run prefixes. Run1 retained all 17 native IDs after 236 seconds and refresh: 17 Tasks, 17 UIDs. Run2 retained all 17 after 215 seconds: 34 cumulative Tasks/Uids, 17 per run. Run3 retained all 17 after 215 seconds: 51 cumulative Tasks/Uids, 17 per run. No new native IDs appeared within any run. Each run includes one completed Reminder, so the UI shows 48 incomplete items after all three. A disposable API discovery Task was deleted before run1; it is outside these counts. The original distinct-UID trigger remains unknown. Native UI evidence: https://git.kayg.org/attachments/992f2a6e-cbf4-4e65-a208-45e58f505082 Cleanup: Cargo removed 16868 files, 9.1GiB total. Web build output was deleted. The Mac account and CA profiles were removed. System certificate lookup returns not found, and the admin trust store reports no Trust Settings. Own Mac helper files, private credential copies and disposable server data were removed. The lab server, proxy and tunnel were stopped. The Mac lock was released. Final evidence (native IDs, three verdicts and gate logs): https://git.kayg.org/attachments/27baffe4-46cf-44d6-b1ea-1d51860d7d47

kayg commented

2026-10-01 19:05:28 +00:00

Author

Owner

Data point from the Apple interop run on the macOS 27 VM, 2026-10-01 (lab server from dev at 687ff7031, one fresh profile-installed CalDAV account, no other CalDAV account on the Mac):

AppleScript created 21 Reminders in one batch (20 plain titles plus one Unicode/RTL title with notes).
Proxy log: 25 PUT …/reminders/*.ics → 201 (21 creates with If-None-Match: *, the rest If-Match updates of the same UIDs). No PUT used a new UID for an existing title.
Task API: 24 Tasks (3 earlier + 21 new), no duplicate titles.
After a Calendar "Refresh Calendars" and a Reminders relaunch: 0 further PUTs, still 24 Tasks and 24 Reminders on the Mac.

So the duplication did not reproduce at this size on a clean account. The round-4 run had an older account history on the same VM; a leftover second account or stale local store is a likely factor. Full results: docs/research/apple-interop-2026-10-02.md in the macdav-verify worktree (not committed).

Data point from the Apple interop run on the macOS 27 VM, 2026-10-01 (lab server from `dev` at `687ff7031`, one fresh profile-installed CalDAV account, no other CalDAV account on the Mac): - AppleScript created 21 Reminders in one batch (20 plain titles plus one Unicode/RTL title with notes). - Proxy log: 25 `PUT …/reminders/*.ics` → 201 (21 creates with `If-None-Match: *`, the rest `If-Match` updates of the same UIDs). No PUT used a new UID for an existing title. - Task API: 24 Tasks (3 earlier + 21 new), no duplicate titles. - After a Calendar "Refresh Calendars" and a Reminders relaunch: 0 further PUTs, still 24 Tasks and 24 Reminders on the Mac. So the duplication did **not** reproduce at this size on a clean account. The round-4 run had an older account history on the same VM; a leftover second account or stale local store is a likely factor. Full results: `docs/research/apple-interop-2026-10-02.md` in the `macdav-verify` worktree (not committed).

kayg referenced this issue

2026-10-03 12:51:31 +00:00

Issue hygiene sweep 2026-10-03 #1008

Rows
Columns

SYNC: Reminders.app creates are duplicated (17 Reminders → 34 Tasks); SQLite pool timeouts during the burst #585

Found by mac-393 round 4 (2026-10-01; real macOS Reminders.app on the Mac VM against a local calternal server)