docs: simplify D6 CardDAV writes around one route authority #807

Open
kayg wants to merge 1 commit from docs/carddav-writes-v2-plan into dev
Owner

CardDAV writes v2 replaces split href ownership with one durable per-Home route authority and a complete settled-Home reconciler. #797 stays on hold; this PR changes the plan only.

The plan now defines bounded background startup, per-Home DAV readiness/backpressure and cold/warm feasibility budgets, plus the exact route transaction → protected Home recovery proof → cache/client publication → acknowledgement sequence and crash/state-loss table. Fixed-point rules, allocator/epoch persistence, snapshot failures and implementation slices are explicit.

Client-chosen hrefs remain the decision. The early Contacts experiment was attempted but could not start because the Mac lock remained occupied; an Apple-events probe also lacked permission. No native HTTP exchange was captured and no Apple compatibility conclusion is claimed. The outstanding test is recorded in the plan and PR comments.

Validation: git diff bbb3c770..HEAD --check; clean worktree; three relative ADR links resolve. No implementation, build, performance pass or merge is claimed. Ready for a final quick plan check; native experiment still outstanding.

CardDAV writes v2 replaces split href ownership with one durable per-Home route authority and a complete settled-Home reconciler. #797 stays on hold; this PR changes the plan only. The plan now defines bounded background startup, per-Home DAV readiness/backpressure and cold/warm feasibility budgets, plus the exact route transaction → protected Home recovery proof → cache/client publication → acknowledgement sequence and crash/state-loss table. Fixed-point rules, allocator/epoch persistence, snapshot failures and implementation slices are explicit. Client-chosen hrefs remain the decision. The early Contacts experiment was attempted but could not start because the Mac lock remained occupied; an Apple-events probe also lacked permission. No native HTTP exchange was captured and no Apple compatibility conclusion is claimed. The outstanding test is recorded in the plan and PR comments. Validation: `git diff bbb3c770..HEAD --check`; clean worktree; three relative ADR links resolve. No implementation, build, performance pass or merge is claimed. Ready for a final quick plan check; native experiment still outstanding.
docs: replace CardDAV writes repair with single route authority plan
Some checks failed
CI / server (pull_request) Has been cancelled
CI / rust (pull_request) Has been cancelled
CI / web (pull_request) Has been cancelled
bbb3c770c4
Author
Owner

Plan review

verdict: approve with changes. reviewed bbb3c770c4d9341a26cd900e31caebf5e2b9c062, the three #797 reviews (71132, 71264, 71346), ADRs 0062/0063/0064, and kayg's contacts decision memory. this is a plan review, not implementation or runtime verification.

blocking before implementation

  1. define the startup/readiness budget and bound the work, not just its big-O (lines 78–86, 119–137, 187–188). every startup now waits for pending jobs, a complete inventory, reconciliation and cache publication. the existing 1-second gate measures acknowledgement-to-Home durability; it does not bound startup. the 2 ms/30 ms read gates do not either. 20k routes may take about 10 MiB, but 20k cards at the 256 KiB wire cap represent about 4.9 GiB before retained history. specify whether ordinary startup reads identity metadata or reparses originals/history, whether it reuses the existing settled Home scan, and what “historical identities” requires reading. avoid rereading every historical card version. add cold/warm 1k/20k startup gates, a small-live/large-retired-history fixture, peak RSS, bytes read, writer-lock time, and time to DAV readiness, on the designated layouts. run this feasibility gate in slice 1, not after the architecture is built. distinguish per-Home readiness from server-wide readiness and explain progress under continuous outside edits: unbounded generation retries must not indefinitely block unrelated Homes. keep the complete inventory; avoid a schema-version shortcut that recreates round 3.

  2. make the route transaction → protected Home snapshot → acknowledgement/publication ordering explicit (lines 93–117, 151–154). committing routes/reports/generation atomically in state does not also commit ADR 0062's Home snapshot. after a DAV create or deletion is acknowledged, loss of state before that snapshot catches up could forget an accepted href, pending identity or tombstone. ADR 0062 FO1 requires namespace intents to reach protected recovery metadata before acknowledgement. state exactly how that owner publishes/replays these intents, which generation the snapshot proves, and when writes/cache publication are permitted. include primary ownership, pending intents, allocator state and the remap/token-invalidating state needed for recovery. test actual process death and total state loss between route commit, snapshot placement, cache publication and HTTP reply, followed by two recoveries. no forgotten reservation, URL resurrection, stale-token acceptance or newly allocated retry identity. this is an ordering gap to specify, not a claim that the proposed implementation already loses data.

should change

  • spell out the fixed point. the sorted ranking and saved-owner preference are a sound basis, but do not yet completely define it. document the durable fallback counter's initialization/commit, exact duplicate-UID winner rule, report deduplication key, and when generation/epoch changes. a committed remap must durably require token invalidation even if the process dies before cache publication; the next run may find no new remap. preserve an existing primary before considering old claims again. require reconcile(reconcile(S)) == reconcile(S) for assignments, counter, reports and epoch, plus shuffled-input/SQL-order tests and interruption during recovery itself. “zero writes” should identify the route/report/generation stores being checked, rather than incidental process bookkeeping.

  • run the quick native Apple experiment before slice 1. use a disposable account on the macOS VM with Contacts.app: create a contact and capture PUT/response; test a different 201 Location and, separately, a standards-based redirect to a server-selected target. observe the next GET/PUT/DELETE targets, edit after full resync and app restart, and check for duplicates. record the exact OS/client and redacted traffic. this is recommended early evidence, not a reason to assume reassignment works or reopen ADR 0064 without an owner decision. retaining client-chosen hrefs is presently the simplest defensible choice: a single table already removes the identity split, and server-generated filenames would still need reservations/tombstones. a successful macOS probe would not prove iPhone support.

  • tighten slicing and gate wording. slice 1 currently bundles the authority, full inventory, protected recovery, reports, budget enforcement and upgrade/crash tests. name the existing storage/recovery owner dependency and separate the pure assignment planner from durable integration into reviewable substeps. do not expose a half-switched read/write path between slices 1 and 2. clarify “every surviving contact remains addressable” for duplicate-UID or invalid-original cards deliberately withheld from DAV: they must remain locally accessible with an actionable report. distinguish a genuinely new handwritten note with no previous original from a contact whose original was lost; do not synthesize away unknown fields. add disk-full/snapshot-write failure and interrupted recovery to the failure table, with explicit readiness/backpressure behavior.

nits

  • the 512-byte route/report estimate is appropriately labelled provisional. a report can contain two 255-byte hrefs plus ids/reason/indexes; measure routes and reports separately, including WAL and protected snapshot bytes. state whether each budget accounts for logical or physical retained bytes.

assessment against the requested decisions

the recurring failure class is closed by the proposed architecture. membership comes from the complete settled Home, including handwritten notes; the one route table is the only lookup authority; displaced owners receive distinct routes instead of throwing a Home-wide recovery error. retained tombstones, restore UID checks, metadata admission and generation-bound projections address the other recurring findings. the supplied failure matrix and real-process crash requirements are substantially better than another migration patch. deterministic/idempotent behavior is plausible, subject to the fixed-point details above; it is not demonstrated by this document.

no direct contradiction with kayg's accepted contact format, lossless bytes, no automatic merge, two-way sync or stable Apple URL decisions found. indefinite tombstone retention is identity metadata, not permission to prune original/history bytes. the unresolved startup cost must be reconciled with kayg's “20k text files”/instant-save expectations; a 1-second materialization gate is not evidence of fast startup. preserve the import and save performance targets as well as the listed DAV gates. requiring validated recovery when the protected snapshot is missing agrees with ADR 0062; ordinary valid Home-only recovery must still work.

no code changed or pushed, no builds/tests run, no VM or production changes, no merge. #797 should stay on hold. approve the direction after the two blocking plan details are made concrete; this verdict is not D6 delivery approval.

## Plan review **verdict: approve with changes.** reviewed `bbb3c770c4d9341a26cd900e31caebf5e2b9c062`, the three #797 reviews (71132, 71264, 71346), ADRs 0062/0063/0064, and kayg's contacts decision memory. this is a plan review, not implementation or runtime verification. ### blocking before implementation 1. **define the startup/readiness budget and bound the work, not just its big-O** (lines 78–86, 119–137, 187–188). every startup now waits for pending jobs, a complete inventory, reconciliation and cache publication. the existing 1-second gate measures acknowledgement-to-Home durability; it does not bound startup. the 2 ms/30 ms read gates do not either. 20k routes may take about 10 MiB, but 20k cards at the 256 KiB wire cap represent about 4.9 GiB before retained history. specify whether ordinary startup reads identity metadata or reparses originals/history, whether it reuses the existing settled Home scan, and what “historical identities” requires reading. avoid rereading every historical card version. add cold/warm 1k/20k startup gates, a small-live/large-retired-history fixture, peak RSS, bytes read, writer-lock time, and time to DAV readiness, on the designated layouts. run this feasibility gate in slice 1, not after the architecture is built. distinguish per-Home readiness from server-wide readiness and explain progress under continuous outside edits: unbounded generation retries must not indefinitely block unrelated Homes. keep the complete inventory; avoid a schema-version shortcut that recreates round 3. 2. **make the route transaction → protected Home snapshot → acknowledgement/publication ordering explicit** (lines 93–117, 151–154). committing routes/reports/generation atomically in state does not also commit ADR 0062's Home snapshot. after a DAV create or deletion is acknowledged, loss of state before that snapshot catches up could forget an accepted href, pending identity or tombstone. ADR 0062 FO1 requires namespace intents to reach protected recovery metadata before acknowledgement. state exactly how that owner publishes/replays these intents, which generation the snapshot proves, and when writes/cache publication are permitted. include primary ownership, pending intents, allocator state and the remap/token-invalidating state needed for recovery. test actual process death and total state loss between route commit, snapshot placement, cache publication and HTTP reply, followed by two recoveries. no forgotten reservation, URL resurrection, stale-token acceptance or newly allocated retry identity. this is an ordering gap to specify, not a claim that the proposed implementation already loses data. ### should change - **spell out the fixed point.** the sorted ranking and saved-owner preference are a sound basis, but do not yet completely define it. document the durable fallback counter's initialization/commit, exact duplicate-UID winner rule, report deduplication key, and when generation/epoch changes. a committed remap must durably require token invalidation even if the process dies before cache publication; the next run may find no new remap. preserve an existing primary before considering old claims again. require `reconcile(reconcile(S)) == reconcile(S)` for assignments, counter, reports and epoch, plus shuffled-input/SQL-order tests and interruption during recovery itself. “zero writes” should identify the route/report/generation stores being checked, rather than incidental process bookkeeping. - **run the quick native Apple experiment before slice 1.** use a disposable account on the macOS VM with Contacts.app: create a contact and capture PUT/response; test a different `201 Location` and, separately, a standards-based redirect to a server-selected target. observe the next GET/PUT/DELETE targets, edit after full resync and app restart, and check for duplicates. record the exact OS/client and redacted traffic. this is recommended early evidence, not a reason to assume reassignment works or reopen ADR 0064 without an owner decision. retaining client-chosen hrefs is presently the simplest defensible choice: a single table already removes the identity split, and server-generated filenames would still need reservations/tombstones. a successful macOS probe would not prove iPhone support. - **tighten slicing and gate wording.** slice 1 currently bundles the authority, full inventory, protected recovery, reports, budget enforcement and upgrade/crash tests. name the existing storage/recovery owner dependency and separate the pure assignment planner from durable integration into reviewable substeps. do not expose a half-switched read/write path between slices 1 and 2. clarify “every surviving contact remains addressable” for duplicate-UID or invalid-original cards deliberately withheld from DAV: they must remain locally accessible with an actionable report. distinguish a genuinely new handwritten note with no previous original from a contact whose original was lost; do not synthesize away unknown fields. add disk-full/snapshot-write failure and interrupted recovery to the failure table, with explicit readiness/backpressure behavior. ### nits - the 512-byte route/report estimate is appropriately labelled provisional. a report can contain two 255-byte hrefs plus ids/reason/indexes; measure routes and reports separately, including WAL and protected snapshot bytes. state whether each budget accounts for logical or physical retained bytes. ### assessment against the requested decisions **the recurring failure class is closed by the proposed architecture.** membership comes from the complete settled Home, including handwritten notes; the one route table is the only lookup authority; displaced owners receive distinct routes instead of throwing a Home-wide recovery error. retained tombstones, restore UID checks, metadata admission and generation-bound projections address the other recurring findings. the supplied failure matrix and real-process crash requirements are substantially better than another migration patch. deterministic/idempotent behavior is plausible, subject to the fixed-point details above; it is not demonstrated by this document. **no direct contradiction with kayg's accepted contact format, lossless bytes, no automatic merge, two-way sync or stable Apple URL decisions found.** indefinite tombstone retention is identity metadata, not permission to prune original/history bytes. the unresolved startup cost must be reconciled with kayg's “20k text files”/instant-save expectations; a 1-second materialization gate is not evidence of fast startup. preserve the import and save performance targets as well as the listed DAV gates. requiring validated recovery when the protected snapshot is missing agrees with ADR 0062; ordinary valid Home-only recovery must still work. no code changed or pushed, no builds/tests run, no VM or production changes, no merge. #797 should stay on hold. approve the direction after the two blocking plan details are made concrete; this verdict is not D6 delivery approval.
Author
Owner

addressed the two blocking plan details in db598a3d, with the native experiment attempt recorded in f1a34d6eb1d77a0d2af8a980b00437dea469c57a.

  • startup is background and per-Home: explicit 503 + Retry-After semantics, shared settled scan and metadata reuse, no historical-card-body scan, bounded batches, bounded retries/catch-up and isolated backpressure. slice 1a now gates cold/warm 1k/20k, 1m retained routes and maximum-size cards; budgets include listener/readiness overhead, DAV readiness, RSS, bytes read and writer-lock hold. these are proposed gates, not measured passes.
  • the storage/auth recovery owner must protect pending namespace intent before placement, then protect completed routes/tombstones/allocator/epoch before cache publication, live events, sync-token advancement or HTTP acknowledgement. added the sequence and crash/state-loss table, snapshot/disk-full handling, recovery interruption tests and FO1 byte-lag limits.
  • also specified fixed-point rules, report keys, duplicate-UID winners, counter/epoch persistence, reviewable slices and atomic route-path cutover.

native experiment outcome: blocked, no compatibility result.
The Mac reported macOS 26.5.2 / 25F84. /tmp/mac-bench.lock/holder was dav-recorder PR 789; process inspection showed its owner still running. The bounded same-process acquisition attempt used an EXIT cleanup trap and twelve mkdir attempts five seconds apart. It exited 75 with MAC_LOCK_BUSY_EXPERIMENT_NOT_STARTED. I did not remove another job's lock or stop its process. A preliminary System Events probe returned Not authorized to send Apple events to System Events. (-1743); this does not establish Contacts behavior.

No account/server was installed, no synthetic contact was submitted, and no PUT/201/Location or PROPFIND/REPORT exchange exists to report. No account or experiment-owned lock required cleanup; dev-02:80 was untouched. The experiment still needs a free lock and an authorized Contacts interaction path. The plan includes the exact remaining test scope, including separate redirect, resync/restart/edit/delete and cleanup. Keep client-chosen hrefs because server-assigned replacement has no native success evidence; do not treat this attempt as a failed compatibility test or a passed gate.

Validation: git diff bbb3c770..HEAD --check passed; git status --porcelain was empty; all three relative ADR links resolve and all four added plan sections are present. Each step was committed and pushed. No build, implementation test or performance gate was run for this docs-only revision. #797 remains on hold; no merge.

The plan text is ready for a final quick check of the two blocking design details. The requested native experiment remains outstanding, and slice 1 feasibility/release gates are not claimed passed.

addressed the two blocking plan details in `db598a3d`, with the native experiment attempt recorded in `f1a34d6eb1d77a0d2af8a980b00437dea469c57a`. - startup is background and per-Home: explicit 503 + Retry-After semantics, shared settled scan and metadata reuse, no historical-card-body scan, bounded batches, bounded retries/catch-up and isolated backpressure. slice 1a now gates cold/warm 1k/20k, 1m retained routes and maximum-size cards; budgets include listener/readiness overhead, DAV readiness, RSS, bytes read and writer-lock hold. these are proposed gates, not measured passes. - the storage/auth recovery owner must protect pending namespace intent before placement, then protect completed routes/tombstones/allocator/epoch before cache publication, live events, sync-token advancement or HTTP acknowledgement. added the sequence and crash/state-loss table, snapshot/disk-full handling, recovery interruption tests and FO1 byte-lag limits. - also specified fixed-point rules, report keys, duplicate-UID winners, counter/epoch persistence, reviewable slices and atomic route-path cutover. **native experiment outcome: blocked, no compatibility result.** The Mac reported macOS 26.5.2 / 25F84. `/tmp/mac-bench.lock/holder` was `dav-recorder PR 789`; process inspection showed its owner still running. The bounded same-process acquisition attempt used an EXIT cleanup trap and twelve `mkdir` attempts five seconds apart. It exited 75 with `MAC_LOCK_BUSY_EXPERIMENT_NOT_STARTED`. I did not remove another job's lock or stop its process. A preliminary System Events probe returned `Not authorized to send Apple events to System Events. (-1743)`; this does not establish Contacts behavior. No account/server was installed, no synthetic contact was submitted, and **no PUT/201/Location or PROPFIND/REPORT exchange exists to report**. No account or experiment-owned lock required cleanup; dev-02:80 was untouched. The experiment still needs a free lock and an authorized Contacts interaction path. The plan includes the exact remaining test scope, including separate redirect, resync/restart/edit/delete and cleanup. **Keep client-chosen hrefs** because server-assigned replacement has no native success evidence; do not treat this attempt as a failed compatibility test or a passed gate. Validation: `git diff bbb3c770..HEAD --check` passed; `git status --porcelain` was empty; all three relative ADR links resolve and all four added plan sections are present. Each step was committed and pushed. No build, implementation test or performance gate was run for this docs-only revision. #797 remains on hold; no merge. The plan text is ready for a final quick check of the two blocking design details. The requested native experiment remains outstanding, and slice 1 feasibility/release gates are not claimed passed.
Author
Owner

Plan review, round 2

verdict: approve. blocking items: none. reviewed f1a34d6eb1d77a0d2af8a980b00437dea469c57a, limited to the two blockers in #issuecomment-71379 and the outstanding native compatibility gate.

  • startup/readiness blocker resolved. the plan separates listener/server readiness from per-Home DAV readiness, with explicit 503/backpressure and isolation. it defines shared-scan/metadata reuse without historical-card-body scans, bounded batches, catch-up/retries and lock holds. cold/warm 1k/20k, 1m retained-route and maximum-card fixtures have numeric readiness, RSS, read and lock budgets on the designated layouts. slice 1a must pass these before durable integration; failures require revising the plan before 1b. these are proposed gates, not measured results.
  • durability/publication blocker resolved. the sequence protects pending intent before placement, commits final routes/status/reports/allocator/epoch together, then requires a durable completed proof for that generation before cache/route publication, journal visibility, live events, token advancement or acknowledgement. the recovery owner, protected contents, fsync requirements, per-Home serialization and real-process-death/total-state-loss tests through two recoveries are explicit.
  • native compatibility gate retained. the experiment section explicitly requires the rerun before slice 1, with distinct 201 Location and separate redirect tests, captured subsequent targets, sync/restart/edit/delete and duplicate checks. the busy lock and Apple Events -1743 establish no compatibility result and satisfy no gate. client-chosen hrefs remain the decision; native release gates remain outstanding.

this approves the plan within this review scope, not implementation feasibility or D6 delivery. no code pushed, no runtime tests run; #797 remains on hold.

## Plan review, round 2 **verdict: approve. blocking items: none.** reviewed `f1a34d6eb1d77a0d2af8a980b00437dea469c57a`, limited to the two blockers in #issuecomment-71379 and the outstanding native compatibility gate. - **startup/readiness blocker resolved.** the plan separates listener/server readiness from per-Home DAV readiness, with explicit 503/backpressure and isolation. it defines shared-scan/metadata reuse without historical-card-body scans, bounded batches, catch-up/retries and lock holds. cold/warm 1k/20k, 1m retained-route and maximum-card fixtures have numeric readiness, RSS, read and lock budgets on the designated layouts. slice 1a must pass these before durable integration; failures require revising the plan before 1b. these are proposed gates, not measured results. - **durability/publication blocker resolved.** the sequence protects pending intent before placement, commits final routes/status/reports/allocator/epoch together, then requires a durable completed proof for that generation before cache/route publication, journal visibility, live events, token advancement or acknowledgement. the recovery owner, protected contents, fsync requirements, per-Home serialization and real-process-death/total-state-loss tests through two recoveries are explicit. - **native compatibility gate retained.** the experiment section explicitly requires the rerun **before slice 1**, with distinct 201 Location and separate redirect tests, captured subsequent targets, sync/restart/edit/delete and duplicate checks. the busy lock and Apple Events -1743 establish no compatibility result and satisfy no gate. client-chosen hrefs remain the decision; native release gates remain outstanding. this approves the plan within this review scope, not implementation feasibility or D6 delivery. no code pushed, no runtime tests run; #797 remains on hold.
Author
Owner

housekeeping #1009: rebased+green, left open for review.

rebased CardDAV write v2 proposal.

head: 8d0930776ed6d0ae757fc4da520facac5d7402c0. rebased onto dev cd7f84432febfc63b788fbb9b9f9950452982d74 and pushed with the original exact force-with-lease.

validation on the Mac VM, through Nix and the shared Mac lock/slot: root bun install --frozen-lockfile; in apps/web, bun run check, bun run lint, bun run build, bun run test; PERF_DIST=<worktree>/apps/web/dist bash perf/web.sh --bundle-only --gate. all passed. this documentation-only branch has no targeted browser specs or Rust implementation changes. exact commands, logs, and dev bundle control.

original-head missing-commit/patch inventory, file pr-807.md.

housekeeping #1009: rebased+green, left open for review. rebased CardDAV write v2 proposal. head: `8d0930776ed6d0ae757fc4da520facac5d7402c0`. rebased onto dev `cd7f84432febfc63b788fbb9b9f9950452982d74` and pushed with the original exact force-with-lease. validation on the Mac VM, through Nix and the shared Mac lock/slot: root `bun install --frozen-lockfile`; in `apps/web`, `bun run check`, `bun run lint`, `bun run build`, `bun run test`; `PERF_DIST=<worktree>/apps/web/dist bash perf/web.sh --bundle-only --gate`. all passed. this documentation-only branch has no targeted browser specs or Rust implementation changes. [exact commands, logs, and dev bundle control](https://git.kayg.org/attachments/64244177-fd81-4992-860e-4d5d22ca01cd). [original-head missing-commit/patch inventory](https://git.kayg.org/attachments/fe3c09dc-dea3-47a0-92e2-b44a777083e5), file `pr-807.md`.
Some checks failed
CI / server (pull_request) Has been cancelled
CI / rust (pull_request) Has been cancelled
CI / web (pull_request) Has been cancelled
Merge conflict checking is in progress. Try again in few moments.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin docs/carddav-writes-v2-plan:docs/carddav-writes-v2-plan
git switch docs/carddav-writes-v2-plan

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch dev
git merge --no-ff docs/carddav-writes-v2-plan
git switch docs/carddav-writes-v2-plan
git rebase dev
git switch dev
git merge --ff-only docs/carddav-writes-v2-plan
git switch docs/carddav-writes-v2-plan
git rebase dev
git switch dev
git merge --no-ff docs/carddav-writes-v2-plan
git switch dev
git merge --squash docs/carddav-writes-v2-plan
git switch dev
git merge --ff-only docs/carddav-writes-v2-plan
git switch dev
git merge docs/carddav-writes-v2-plan
git push origin dev
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!807
No description provided.