kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 191 Pull requests 61 Projects Releases Packages Wiki Activity Actions 20

WIP: DAV recorder v3 — measured Rust size stop (682 changed lines) #822

Draft
kayg wants to merge 1 commit from feat/dav-recorder-v3 into dev
pull from: feat/dav-recorder-v3
merge into: kayg:dev
Conversation 3 Commits 1 Files changed 1 +13
kayg commented 2026-11-01 02:24:57 +00:00
Owner
Copy link

Measured implementation checkpoint, size stop

Incomplete draft, not ready to merge. Feature implementation stopped after formatted incremental production Rust reached 682 changed lines, crossing the raised approximately 650-line stop condition. No Python exporter or captures were produced. No gates are waived. Head: 0ac548edc2d875506b889fd08b7d8febcaa0b419.

Approved contract: #818 at dfcd2fdf, revision 72227, plan review 72220. #820 stays paused at 58e50394. This continues #822's earlier forecast checkpoint under the newly raised allowance.

Actual size and boundary

Counts use git diff --numstat 58e50394 -- crates/dav-replay/src, excluding unchanged reused code and removal of src/record.rs:

Production Rust adaptation Added Deleted Changed
grammar.rs 274 0 274
schema.rs 125 108 233
wire.rs 71 19 90
main.rs 7 76 83
lib.rs 1 1 2
Total incremental adaptation 478 204 682

Separately: old custom proxy removed, 292 Rust lines; old proxy tests removed, 417 lines. New focused Rust tests: 97 lines. New Python production/tests: 0/0. Entire PR relative to starting 62c25e6d adds 5,253 production Rust lines and 2,090 Rust test lines, including unchanged #820 reuse. Do not interpret those reused lines as incremental adaptation.

The raised budget includes closed headers and URI work. This is an actual formatted diff, not another estimate. No code was compressed and no gate removed to meet a count.

Implemented checkpoint and remaining work

Reused #820 typed comparison, bindings, independent seeds, continuity and mutation suites. Removed the custom proxy and disabled the record command. Closed header names, media/charset values, capability/method words and booleans replace raw string fields; Basic realms become slots and unknown/deferred headers refuse normalization. URI root/path, relative, HTTP and HTTPS origins are distinct, with authority slots. Schema objects use recursive serde deny-unknown. Four focused tests check deferred-header refusal, Basic realm omission, protocol-value differences and nested unknown/raw fallback refusal.

Still incomplete: stock Nix mitmproxy packaging; Python exporter and finalizer; resource/receipt/sentinel/publication boundary; format-3 semantic slot/profile validation and v2 refusal; six-field card/XML narrowing; omitted error bodies; complete transport, dates and origin policy; mounted own-server scenarios; exact Radicale QName reproduction; historical through-proxy/privacy and mutation gates. Format intentionally remains 2 pending that work, so validate is only an incomplete-loader checkpoint, not a format-3 acceptance gate. Existing synthetic fixture builders are test helpers, not an approved capture/export path. No privacy or completion guarantee is claimed for this checkpoint.

Retain the approved costs and failure contract from #818: preflight/hash O(raw bytes), conversion O(B log I), one bounded decoded flow plus bounded slots and contained upstream overhead. Failure → effect → detection → recovery → test remains the plan's full table. This checkpoint does not establish those exporter bounds.

Gate table

| Plan first-PR gate | Result |
| Mounted own DAV → stock mitmdump → contacts-crud export/replay; duplicate UID 403/recovery | NOT RUN / NOT COMPLETE; measured size stop |
| Mounted own DAV → stock mitmdump → contacts-initial-sync export/replay | NOT RUN / NOT COMPLETE; measured size stop |
| R1 P1.1 independently seeded FN / UID swap / UID collapse | NOT COMPLETE; reused independently seeded comparison/tests, no v3 integration gate |
| R1 P1.2 actual advertised href following; root/relative/HTTP/HTTPS controls | NOT RUN / NOT COMPLETE; measured size stop |
| R1 P1.3 / R2 P1 capability headers and unknown offline refusal | NOT RUN / NOT COMPLETE; measured size stop |
| R3 P1 Apple/auth/Digest/cookie/Link capture survival and zero-output refusal | PARTIAL: four focused closed-header tests PASS; stock-capture survival/export gate NOT RUN |
| R3 P1 discovery/401/Radicale QName/CardDAV 403 | NOT RUN / NOT COMPLETE; measured size stop |
| Earlier privacy P1s from #789 | NOT RUN / NOT COMPLETE; measured size stop |
| R1 P2 media / length / ETag | NOT RUN / NOT COMPLETE; measured size stop |
| R2 P2 timestamps / charset / three date grammars | NOT RUN / NOT COMPLETE; measured size stop |
| R3 P2 cookie refusal / strict boolean / Basic state | NOT RUN / NOT COMPLETE; measured size stop |
| R3 continuity sync / ETag / 412 / header multiplicity | NOT RUN / NOT COMPLETE; measured size stop |
| Carried P3 cleanup reporting | NOT RUN / NOT COMPLETE; measured size stop |
| Completion sentinel / receipt / prefix / failure matrix | NOT RUN / NOT COMPLETE; measured size stop |
| Pre-reader framing / bounded reads / gzip / child limits | NOT RUN / NOT COMPLETE; measured size stop |
| Strict recursive v3 loader / slots / v2 refusal / determinism | PARTIAL: recursive serde unknown-field rejection added; format remains 2, semantic slot/profile loader NOT IMPLEMENTED |
| Persisted-file and stdout/stderr sentinel matrix | NOT RUN / NOT COMPLETE; measured size stop |
| Publication / I/O / ordering | NOT RUN / NOT COMPLETE; measured size stop |
| dev-01 x86_64 Nix import / FlowWriter–FlowReader / mitmdump smoke | NOT RUN / NOT COMPLETE; measured size stop |
| dev-02 aarch64 Nix import / FlowWriter–FlowReader / mitmdump smoke | NOT RUN / NOT COMPLETE; measured size stop |
| Linux fmt / clippy -D warnings / crate tests / Python tests | PARTIAL: dev-01 fmt and clippy PASS; crate suite FAIL; Python NOT RUN |
| Required-scenario inventory check | PENDING (exit 3); required real captures absent |
| Real Apple capture | PENDING; Mac GUI ready, capture is a later job |
| Calendar/Reminders and real iPhone | PENDING; separate follow-ups |

Linux verification

All commands ran through netbird ssh claude@calternal-dev-01-compute02.vpn.kayg.org, in ~/Developer/Worktrees/dav-recorder-v3-impl, inside the prescribed temporary Nix overlay. No build/test/install ran on code-minipc01. dev-01 had 315 GiB free before work (threshold 50 GiB); dev-02 read-only disk inspection showed 74 GiB free. No dev-02 build was started after the size stop.

nix develop -c bash -c 'cargo fmt --all --check'
# PASS on final head
nix develop -c bash -c '~/bin/slot cargo clippy -j4 -p calternal-dav-replay --locked -- -D warnings'
# PASS on final head
nix develop -c bash -c '~/bin/slot cargo test -j4 -p calternal-dav-replay --locked --test closed_headers'
# PASS, 4/4 on final head
nix develop -c bash -c '~/bin/slot cargo test -j4 -p calternal-dav-replay --locked --no-fail-fast'
# FAIL at 114237b9: six targets failed; no full-suite pass claimed
nix develop -c bash -c '~/bin/slot cargo run -j4 -p calternal-dav-replay --locked -- check crates/dav-replay/fixtures'
# PENDING, exit 3

Full-suite failures: fixtures, replay, review_r2_mutations, review_regressions and schema retain v2 expectations for unknown headers/query/userinfo/calendar media; they need adaptation to the narrow v3 contract. The sixth target, closed_headers, had a test header named unknown colliding with fixed diagnostic text unknown header; corrected to a distinct sentinel and rerun 4/4. These results do not establish every historical P1 gate. Logs remain private on dev-01: ~/dav-recorder-v3-rust.log, ~/dav-recorder-v3-tests.log, ~/dav-recorder-v3-final.log.

No raw capture, receipt, fixture, account or proxy CA was created. No merge. Real Apple capture stays PENDING, with GUI readiness recorded separately from capture evidence.

Cleanup: removed the owned dev-01 worktree target/ (849 MiB); absence verified. Small logs and the worktree remain for resumption. No owned job is still running.

# Measured implementation checkpoint, size stop **Incomplete draft, not ready to merge.** Feature implementation stopped after formatted incremental production Rust reached **682 changed lines**, crossing the raised approximately 650-line stop condition. No Python exporter or captures were produced. No gates are waived. Head: `0ac548edc2d875506b889fd08b7d8febcaa0b419`. Approved contract: [#818](https://git.kayg.org/kayg/calternal/pulls/818) at dfcd2fdf, [revision 72227](https://git.kayg.org/kayg/calternal/pulls/818#issuecomment-72227), [plan review 72220](https://git.kayg.org/kayg/calternal/pulls/818#issuecomment-72220). #820 stays paused at 58e50394. This continues #822's earlier forecast checkpoint under the newly raised allowance. ## Actual size and boundary Counts use `git diff --numstat 58e50394 -- crates/dav-replay/src`, excluding unchanged reused code and removal of `src/record.rs`: | Production Rust adaptation | Added | Deleted | Changed | |---|---:|---:|---:| | grammar.rs | 274 | 0 | 274 | | schema.rs | 125 | 108 | 233 | | wire.rs | 71 | 19 | 90 | | main.rs | 7 | 76 | 83 | | lib.rs | 1 | 1 | 2 | | Total incremental adaptation | 478 | 204 | **682** | Separately: old custom proxy removed, 292 Rust lines; old proxy tests removed, 417 lines. New focused Rust tests: **97 lines**. New Python production/tests: **0/0**. Entire PR relative to starting 62c25e6d adds **5,253 production Rust lines and 2,090 Rust test lines**, including unchanged #820 reuse. Do not interpret those reused lines as incremental adaptation. The raised budget includes closed headers and URI work. This is an actual formatted diff, not another estimate. No code was compressed and no gate removed to meet a count. ## Implemented checkpoint and remaining work Reused #820 typed comparison, bindings, independent seeds, continuity and mutation suites. Removed the custom proxy and disabled the `record` command. Closed header names, media/charset values, capability/method words and booleans replace raw string fields; Basic realms become slots and unknown/deferred headers refuse normalization. URI root/path, relative, HTTP and HTTPS origins are distinct, with authority slots. Schema objects use recursive serde deny-unknown. Four focused tests check deferred-header refusal, Basic realm omission, protocol-value differences and nested unknown/raw fallback refusal. **Still incomplete:** stock Nix mitmproxy packaging; Python exporter and finalizer; resource/receipt/sentinel/publication boundary; format-3 semantic slot/profile validation and v2 refusal; six-field card/XML narrowing; omitted error bodies; complete transport, dates and origin policy; mounted own-server scenarios; exact Radicale QName reproduction; historical through-proxy/privacy and mutation gates. Format intentionally remains **2** pending that work, so `validate` is only an incomplete-loader checkpoint, not a format-3 acceptance gate. Existing synthetic fixture builders are test helpers, not an approved capture/export path. No privacy or completion guarantee is claimed for this checkpoint. Retain the approved costs and failure contract from #818: preflight/hash O(raw bytes), conversion O(B log I), one bounded decoded flow plus bounded slots and contained upstream overhead. Failure → effect → detection → recovery → test remains the plan's full table. This checkpoint does not establish those exporter bounds. ## Gate table | Plan first-PR gate | Result | | Mounted own DAV → stock mitmdump → contacts-crud export/replay; duplicate UID 403/recovery | NOT RUN / NOT COMPLETE; measured size stop | | Mounted own DAV → stock mitmdump → contacts-initial-sync export/replay | NOT RUN / NOT COMPLETE; measured size stop | | R1 P1.1 independently seeded FN / UID swap / UID collapse | NOT COMPLETE; reused independently seeded comparison/tests, no v3 integration gate | | R1 P1.2 actual advertised href following; root/relative/HTTP/HTTPS controls | NOT RUN / NOT COMPLETE; measured size stop | | R1 P1.3 / R2 P1 capability headers and unknown offline refusal | NOT RUN / NOT COMPLETE; measured size stop | | R3 P1 Apple/auth/Digest/cookie/Link capture survival and zero-output refusal | PARTIAL: four focused closed-header tests PASS; stock-capture survival/export gate NOT RUN | | R3 P1 discovery/401/Radicale QName/CardDAV 403 | NOT RUN / NOT COMPLETE; measured size stop | | Earlier privacy P1s from #789 | NOT RUN / NOT COMPLETE; measured size stop | | R1 P2 media / length / ETag | NOT RUN / NOT COMPLETE; measured size stop | | R2 P2 timestamps / charset / three date grammars | NOT RUN / NOT COMPLETE; measured size stop | | R3 P2 cookie refusal / strict boolean / Basic state | NOT RUN / NOT COMPLETE; measured size stop | | R3 continuity sync / ETag / 412 / header multiplicity | NOT RUN / NOT COMPLETE; measured size stop | | Carried P3 cleanup reporting | NOT RUN / NOT COMPLETE; measured size stop | | Completion sentinel / receipt / prefix / failure matrix | NOT RUN / NOT COMPLETE; measured size stop | | Pre-reader framing / bounded reads / gzip / child limits | NOT RUN / NOT COMPLETE; measured size stop | | Strict recursive v3 loader / slots / v2 refusal / determinism | PARTIAL: recursive serde unknown-field rejection added; format remains 2, semantic slot/profile loader NOT IMPLEMENTED | | Persisted-file and stdout/stderr sentinel matrix | NOT RUN / NOT COMPLETE; measured size stop | | Publication / I/O / ordering | NOT RUN / NOT COMPLETE; measured size stop | | dev-01 x86_64 Nix import / FlowWriter–FlowReader / mitmdump smoke | NOT RUN / NOT COMPLETE; measured size stop | | dev-02 aarch64 Nix import / FlowWriter–FlowReader / mitmdump smoke | NOT RUN / NOT COMPLETE; measured size stop | | Linux fmt / clippy -D warnings / crate tests / Python tests | PARTIAL: dev-01 fmt and clippy PASS; crate suite FAIL; Python NOT RUN | | Required-scenario inventory check | PENDING (exit 3); required real captures absent | | Real Apple capture | PENDING; Mac GUI ready, capture is a later job | | Calendar/Reminders and real iPhone | PENDING; separate follow-ups | ## Linux verification All commands ran through `netbird ssh claude@calternal-dev-01-compute02.vpn.kayg.org`, in `~/Developer/Worktrees/dav-recorder-v3-impl`, inside the prescribed temporary Nix overlay. No build/test/install ran on code-minipc01. dev-01 had 315 GiB free before work (threshold 50 GiB); dev-02 read-only disk inspection showed 74 GiB free. No dev-02 build was started after the size stop. ``` nix develop -c bash -c 'cargo fmt --all --check' # PASS on final head nix develop -c bash -c '~/bin/slot cargo clippy -j4 -p calternal-dav-replay --locked -- -D warnings' # PASS on final head nix develop -c bash -c '~/bin/slot cargo test -j4 -p calternal-dav-replay --locked --test closed_headers' # PASS, 4/4 on final head nix develop -c bash -c '~/bin/slot cargo test -j4 -p calternal-dav-replay --locked --no-fail-fast' # FAIL at 114237b9: six targets failed; no full-suite pass claimed nix develop -c bash -c '~/bin/slot cargo run -j4 -p calternal-dav-replay --locked -- check crates/dav-replay/fixtures' # PENDING, exit 3 ``` Full-suite failures: fixtures, replay, review_r2_mutations, review_regressions and schema retain v2 expectations for unknown headers/query/userinfo/calendar media; they need adaptation to the narrow v3 contract. The sixth target, closed_headers, had a test header named `unknown` colliding with fixed diagnostic text `unknown header`; corrected to a distinct sentinel and rerun 4/4. These results do not establish every historical P1 gate. Logs remain private on dev-01: `~/dav-recorder-v3-rust.log`, `~/dav-recorder-v3-tests.log`, `~/dav-recorder-v3-final.log`. No raw capture, receipt, fixture, account or proxy CA was created. No merge. Real Apple capture stays PENDING, with GUI readiness recorded separately from capture evidence. Cleanup: removed the owned dev-01 worktree `target/` (849 MiB); absence verified. Small logs and the worktree remain for resumption. No owned job is still running.
docs(dav-replay): record v3 implementation size checkpoint
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
62c25e6d77
kayg changed title from WIP: DAV recorder v3 — stopped for Rust adaptation size to WIP: DAV recorder v3 — measured Rust size stop (682 changed lines) 2026-11-01 16:56:55 +00:00
kayg commented 2026-11-01 16:56:56 +00:00
Author
Owner
Copy link

Implementation checkpoint pushed at 0ac548edc2d875506b889fd08b7d8febcaa0b419. Stopped under the raised size rule after rustfmt measured 682 incremental production Rust changed lines against #820 at 58e50394: 478 added + 204 deleted, excluding unchanged reuse and removal of the old 292-line proxy. New Rust tests: 97 lines. New Python production/tests: 0/0.

Reused typed replay/comparison and focused tests; disabled old record and removed custom proxy; added closed header/media/capability/boolean forms, slotted Basic realm, URI-origin forms and recursive serde unknown-field rejection. This remains incomplete: format is still 2, semantic format-3 loader and Python export/finalization are not implemented. No fixture or raw capture was produced.

Dev-01 Nix verification: fmt PASS; clippy -D warnings PASS; focused closed-header tests 4/4 PASS. Full crate suite FAIL at 114237b9 in six targets; the new diagnostic-test naming collision is corrected and rerun, while v2 expectation adaptations remain unfinished. Inventory check PENDING, exit 3. The PR description contains the exact commands and full gate table. Both-architecture Nix/mitmdump and mounted own-DAV scenarios are NOT RUN, not passed. Real Apple capture stays PENDING; Mac GUI readiness is acknowledged, capture is a later job.

All edits were on code-minipc01 in the owned worktree; all builds/tests ran on dev-01 through netbird ssh/Nix/slot. Commits pushed after each step; no merge. Deleted the owned 849 MiB target directory and verified absence. No owned job remains running.

PR #822, head 0ac548edc2, gates partial/failed/not run (Apple PENDING), actual lines Rust adaptation +478/-204=682; new Rust tests 97; Python 0.

Implementation checkpoint pushed at `0ac548edc2d875506b889fd08b7d8febcaa0b419`. Stopped under the raised size rule after rustfmt measured **682 incremental production Rust changed lines** against #820 at 58e50394: **478 added + 204 deleted**, excluding unchanged reuse and removal of the old 292-line proxy. New Rust tests: **97 lines**. New Python production/tests: **0/0**. Reused typed replay/comparison and focused tests; disabled old record and removed custom proxy; added closed header/media/capability/boolean forms, slotted Basic realm, URI-origin forms and recursive serde unknown-field rejection. This remains incomplete: format is still 2, semantic format-3 loader and Python export/finalization are not implemented. No fixture or raw capture was produced. Dev-01 Nix verification: fmt PASS; clippy `-D warnings` PASS; focused closed-header tests **4/4 PASS**. Full crate suite **FAIL** at 114237b9 in six targets; the new diagnostic-test naming collision is corrected and rerun, while v2 expectation adaptations remain unfinished. Inventory check **PENDING, exit 3**. The PR description contains the exact commands and full gate table. Both-architecture Nix/mitmdump and mounted own-DAV scenarios are **NOT RUN**, not passed. Real Apple capture stays **PENDING**; Mac GUI readiness is acknowledged, capture is a later job. All edits were on code-minipc01 in the owned worktree; all builds/tests ran on dev-01 through netbird ssh/Nix/slot. Commits pushed after each step; no merge. Deleted the owned 849 MiB target directory and verified absence. No owned job remains running. PR https://git.kayg.org/kayg/calternal/pulls/822, head 0ac548edc2d875506b889fd08b7d8febcaa0b419, gates partial/failed/not run (Apple PENDING), actual lines Rust adaptation +478/-204=682; new Rust tests 97; Python 0.
kayg commented 2026-11-01 16:57:41 +00:00
Author
Owner
Copy link

Paused (orchestrator, 2026-11-01)

This PR stops here at 0ac548ed, and I'm pausing it on purpose:

  • The recorder exists to prove our CardDAV server matches what Apple Contacts expects. The server doesn't serve DAV yet: prod answers 405 on /dav/, because #782 added only app-password auth. So there's nothing to record or replay against for a while.
  • The first-PR estimate has already been exceeded twice: the Rust adaptation alone is 682 changed lines, against 100–180 planned and 550 allowed. The full crate suite fails, and the Python exporter hasn't started.

Resume when: the CardDAV server slice (#807 writes plan, plus the read path) is on dev. At that point, redo the size estimate against the real server surface before writing code. Plan v3 (#818) stays approved as the design. The checkpoint branch stays as it is.

## Paused (orchestrator, 2026-11-01) This PR stops here at `0ac548ed`, and I'm pausing it on purpose: - The recorder exists to prove our CardDAV server matches what Apple Contacts expects. **The server doesn't serve DAV yet**: prod answers 405 on `/dav/`, because #782 added only app-password auth. So there's nothing to record or replay against for a while. - The first-PR estimate has already been exceeded twice: the Rust adaptation alone is 682 changed lines, against 100–180 planned and 550 allowed. The full crate suite fails, and the Python exporter hasn't started. **Resume when:** the CardDAV server slice (#807 writes plan, plus the read path) is on dev. At that point, redo the size estimate against the real server surface before writing code. Plan v3 (#818) stays approved as the design. The checkpoint branch stays as it is.
kayg commented 2026-11-07 18:21:40 +00:00
Author
Owner
Copy link

Housekeeping #1009 status: audited head 0ac548edc2d875506b889fd08b7d8febcaa0b419 against dev cd7f84432feb. git cherry origin/dev <head> reports 5 unmatched commits and 0 equivalent patches. Separate stable patch-id indexing and key-file checks do not establish full landing. Absent from dev: crates/dav-replay/Cargo.toml, crates/dav-replay/README.md, crates/dav-replay/fixtures/required-scenarios.json. Left unchanged as WIP, per kayg’s instruction; no rebase, close, tests, or merge. The #1009 audit records the unmatched commit list.

Housekeeping #1009 status: audited head `0ac548edc2d875506b889fd08b7d8febcaa0b419` against dev `cd7f84432feb`. `git cherry origin/dev <head>` reports 5 unmatched commits and 0 equivalent patches. Separate stable patch-id indexing and key-file checks do not establish full landing. Absent from dev: `crates/dav-replay/Cargo.toml`, `crates/dav-replay/README.md`, `crates/dav-replay/fixtures/required-scenarios.json`. Left unchanged as WIP, per kayg’s instruction; no rebase, close, tests, or merge. The #1009 audit records the unmatched commit list.
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
This pull request is marked as a work in progress.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/dav-recorder-v3:feat/dav-recorder-v3
git switch feat/dav-recorder-v3

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch dev
git merge --no-ff feat/dav-recorder-v3
git switch feat/dav-recorder-v3
git rebase dev
git switch dev
git merge --ff-only feat/dav-recorder-v3
git switch feat/dav-recorder-v3
git rebase dev
git switch dev
git merge --no-ff feat/dav-recorder-v3
git switch dev
git merge --squash feat/dav-recorder-v3
git switch dev
git merge --ff-only feat/dav-recorder-v3
git switch dev
git merge feat/dav-recorder-v3
git push origin dev
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!822
No description provided.