J102: give each restarted recovery its own safety wait

kayg commented

2026-11-02 11:28:13 +00:00

Owner

Owner approved 2026-11-02; D25/D113; finding 8 on #302. Start: fix-recovery-restart from origin/dev 16bf736c7. Plan: inspect request persistence and API-key clock (15 min); failing tests for expired/pending/cancelled restart and exact 24h/48h boundaries (30 min); minimal fix, notifications and docs (30 min); focused tests and verify fast on VM, baseline evidence for dev failures, commit/push/report (60 min). Estimate 2h15m; stop/report at 4h30m. Pending attempts return existing request without resetting its wait; expired/cancelled requests are terminal and fresh attempts start new deadlines. Both VM queues checked: compute01 12 GB below floor, dev-02 32 GB with free slot; use dev-02. No deployment or dependencies.

Owner approved 2026-11-02; D25/D113; finding 8 on #302. Start: fix-recovery-restart from origin/dev 16bf736c7. Plan: inspect request persistence and API-key clock (15 min); failing tests for expired/pending/cancelled restart and exact 24h/48h boundaries (30 min); minimal fix, notifications and docs (30 min); focused tests and verify fast on VM, baseline evidence for dev failures, commit/push/report (60 min). Estimate 2h15m; stop/report at 4h30m. Pending attempts return existing request without resetting its wait; expired/cancelled requests are terminal and fresh attempts start new deadlines. Both VM queues checked: compute01 12 GB below floor, dev-02 32 GB with free slot; use dev-02. No deployment or dependencies.

kayg commented

2026-11-02 11:30:18 +00:00

Author

Owner

Start: branch fix-recovery-restart; base 16bf736c7. Root cause: production verifier forces resume mode, whose ON CONFLICT overwrites expires but preserves ready even on expired rows; expired rows also suppress the new start audit event. Existing audit test explicitly expects unsafe readiness at 48h. Using dev-02, target ~/worktrees/j102/target. Failing tests exercise both HTTP entry points, pending expiry preservation, cancellation, boundaries, notifications and persisted clock rollback.

Start: branch fix-recovery-restart; base 16bf736c7. Root cause: production verifier forces resume mode, whose ON CONFLICT overwrites expires but preserves ready even on expired rows; expired rows also suppress the new start audit event. Existing audit test explicitly expects unsafe readiness at 48h. Using dev-02, target ~/worktrees/j102/target. Failing tests exercise both HTTP entry points, pending expiry preservation, cancellation, boundaries, notifications and persisted clock rollback.

kayg commented

2026-11-02 11:31:52 +00:00

Author

Owner

Start: fix-recovery-restart, base 16bf736c7. Root cause: production verifier forces resume mode, whose ON CONFLICT overwrites expiry but preserves ready even when expired. Audit start is also suppressed for expired rows. Existing test audit_recovery_after_page_discard_can_resume_at_original_deadline explicitly expects the unsafe 48h behavior and must change. Tests now cover both HTTP entry points, pending expiry preservation, cancellation, notifications and persisted clock rollback. Red run on dev-02; target ~/worktrees/j102/target.

Start: fix-recovery-restart, base 16bf736c7. Root cause: production verifier forces resume mode, whose ON CONFLICT overwrites expiry but preserves ready even when expired. Audit start is also suppressed for expired rows. Existing test audit_recovery_after_page_discard_can_resume_at_original_deadline explicitly expects the unsafe 48h behavior and must change. Tests now cover both HTTP entry points, pending expiry preservation, cancellation, notifications and persisted clock rollback. Red run on dev-02; target ~/worktrees/j102/target.

kayg commented

2026-11-02 11:32:49 +00:00

Author

Owner

Red evidence (dev-02, ~/j102-red.log): 4 tests, 1 passed/3 failed, RED_EXIT=101. Expiry restart ready_at=1800086400 instead of 1800259200; pending retry extended expires from 1800172800 to 1800172801; rollback delay=86399 instead of 86400. Cancellation regression passes already. Fix rotates bearer only while pending, replaces both deadlines at expiry equality, records each fresh start, preserves commit-coupled notifications, reuses API-key persisted high-water time. Recovery suite now running.

kayg commented

2026-11-02 11:34:25 +00:00

Author

Owner

Green recovery suite (dev-02, /j102-green.log): test result: ok. 35 passed; 0 failed; 3 ignored; 0 measured; 444 filtered out; GREEN_EXIT=0. Includes both entry-point regressions and existing 16-device notification/socket isolation tests. Full scripts/verify fast started on dev-02 (/j102-gate.log). No new dependency or migration; creation uses the existing persisted API-key high-water mark.

Green recovery suite (dev-02, ~/j102-green.log): test result: ok. 35 passed; 0 failed; 3 ignored; 0 measured; 444 filtered out; GREEN_EXIT=0. Includes both entry-point regressions and existing 16-device notification/socket isolation tests. Full scripts/verify fast started on dev-02 (~/j102-gate.log). No new dependency or migration; creation uses the existing persisted API-key high-water mark.

kayg commented

2026-11-02 11:51:45 +00:00

Author

Owner

Validation blocker: dev-02 filled its shared disk during the workspace gate (No space left on device). Recovery-focused suite was already 35 passed / 0 failed. Stopping this job validation and preserving logs; removed only J102 target/debug/incremental (8.2 GB), freeing 8.1 GB. Will reclaim the remaining own target and retry with incremental/debug symbols disabled after checking both VMs. No other job targets touched.

kayg commented

2026-11-02 11:52:50 +00:00

Author

Owner

Crash-safety checkpoint pushed: fix-recovery-restart at the WIP head (recovery suite green; full gate incomplete due shared disk exhaustion). Reclaimed only this job target. Retry uses CARGO_INCREMENTAL=0, CARGO_PROFILE_DEV_DEBUG=0 and CARGO_PROFILE_TEST_DEBUG=0 to keep debug artifacts bounded; static release flags remain the repository defaults. Final report will include the actual gate result, baseline failure evidence and size deltas.

kayg commented

2026-11-02 12:07:37 +00:00

Author

Owner

Full workspace tests found two additional assertions tied to deliberately changed behavior: reviewer_disk_restart_and_concurrent_resume expects old readiness after 72h and only one start audit; reviewer_server_clock_rollback_retains_deadline_and_reports_more_than_day expects 90000s after rollback instead of the persisted high-water duration of 86400s. Updating these to fresh readiness/two starts and the persisted duration. No production change needed. Capture this test row, then restart the full gate with updated tests and include that file in the base comparison snapshot.

kayg commented

2026-11-02 12:10:55 +00:00

Author

Owner

Checkpoint 2840d00c7 includes the two updated legacy expectations. Superseded dev-02 test row: 712 passed / 11 failed / 56 ignored; 2 are those obsolete expectations, 9 are bootstrap/enrollment failures pending baseline confirmation. Compute01 now has a free slot and 210 GB, while dev-02 has 4 jobs queued, so cancelled our queued retry and removed our arm target before moving validation to compute01. One target remains per job: ~/worktrees/j102/target on compute01. Final validation snapshot includes reviewer_tests.rs; baseline server tests explicitly enable remote to reproduce the workspace feature set.

Checkpoint 2840d00c7 includes the two updated legacy expectations. Superseded dev-02 test row: 712 passed / 11 failed / 56 ignored; 2 are those obsolete expectations, 9 are bootstrap/enrollment failures pending baseline confirmation. Compute01 now has a free slot and 210 GB, while dev-02 has 4 jobs queued, so cancelled our queued retry and removed our arm target before moving validation to compute01. One target remains per job: ~/worktrees/j102/target on compute01. Final validation snapshot includes reviewer_tests.rs; baseline server tests explicitly enable remote to reproduce the workspace feature set.

kayg commented

2026-11-02 12:21:38 +00:00

Author

Owner

Contract clarification required by the authorized restart/time policy: contracts/keywrap.md currently says a resume preserves the deadline even beyond expiry and reports a 25h remainder after clock rollback. I will update that prose to distinguish pending resumes from fresh post-expiry attempts, describe the shared persisted server time, and count each new attempt in the start audit. No wire fields, client wait bounds, cryptography or signed-in exemption change. Expired rows may remain visible as historical notice until cancellation/replacement; they grant no capability authority.

kayg commented

2026-11-02 12:35:37 +00:00

Author

Owner

Progress: latest pushed checkpoint 42f0bd423 includes the contract clarification. Compute01 final gate has passed fmt, VT fmt, CLI compile-out, clippy, SSH helper and fixture-path checks; workspace tests are compiling. Still one target (~worktrees/j102/target), compact debug profiles, release flags unchanged. How to test: (1) cargo test --locked -p infraternal-server --lib recovery_restart; (2) cargo test --locked -p infraternal-server --lib reviewer_disk_restart_and_concurrent_resume; (3) cargo test --locked -p infraternal-server --lib reviewer_server_clock_rollback_retains_deadline_and_uses_persisted_time. Run on a build VM. Full gate/baseline/static evidence still pending.

Progress: latest pushed checkpoint 42f0bd423 includes the contract clarification. Compute01 final gate has passed fmt, VT fmt, CLI compile-out, clippy, SSH helper and fixture-path checks; workspace tests are compiling. Still one target (~worktrees/j102/target), compact debug profiles, release flags unchanged. How to test: (1) cargo test --locked -p infraternal-server --lib recovery_restart; (2) cargo test --locked -p infraternal-server --lib reviewer_disk_restart_and_concurrent_resume; (3) cargo test --locked -p infraternal-server --lib reviewer_server_clock_rollback_retains_deadline_and_uses_persisted_time. Run on a build VM. Full gate/baseline/static evidence still pending.

kayg commented

2026-11-02 12:52:04 +00:00

Author

Owner

Compute01 gate progress: all four new restart regressions and both updated disk/clock tests PASS; full server row 714 passed / 9 failed / 56 ignored. Failures are the same nine bootstrap/enrollment names captured on dev-02, with no auth failure. fmt, CLI compile-out, clippy, helper/fixture checks, SSH inode fixture, shim process-per-user, cargo-deny and gate architecture PASS. Both static release builds are running, then standalone TS (if Rust remains red), unmodified-base remote-feature server tests, and baseline static size comparison. Head 42f0bd423 is pushed; no uncommitted work. Evidence remains ~/j102-*.log on compute01, target ~/worktrees/j102/target.

Compute01 gate progress: all four new restart regressions and both updated disk/clock tests PASS; full server row 714 passed / 9 failed / 56 ignored. Failures are the same nine bootstrap/enrollment names captured on dev-02, with no auth failure. fmt, CLI compile-out, clippy, helper/fixture checks, SSH inode fixture, shim process-per-user, cargo-deny and gate architecture PASS. Both static release builds are running, then standalone TS (if Rust remains red), unmodified-base remote-feature server tests, and baseline static size comparison. Head 42f0bd423 is pushed; no uncommitted work. Evidence remains ~/j102-*.log on compute01, target ~/worktrees/j102/target.

kayg commented

2026-11-02 13:23:24 +00:00

Author

Owner

Full gate result recorded: all Rust rows PASS except workspace test (the nine bootstrap/enrollment failures); both static builds PASS; standalone TypeScript TS_EXIT=0. Fixed sizes: x86_64 22418064 B, aarch64 20450160 B. Baseline tracked tree is clean at 16bf736c7c... (saved /j102-base-tree.log). Baseline command initially omitted the CARGO_TARGET_DIR export normally supplied by scripts/verify, so four notification fixture tests failed on env lookup. Ran that exact baseline test binary with CARGO_TARGET_DIR exported: all four PASS, BASE_NOTIFICATION_EXIT=0 (/j102-base-notifications.log). No repository code changed for that harness correction. Main baseline server run and baseline static builds still active.

Full gate result recorded: all Rust rows PASS except workspace test (the nine bootstrap/enrollment failures); both static builds PASS; standalone TypeScript TS_EXIT=0. Fixed sizes: x86_64 22418064 B, aarch64 20450160 B. Baseline tracked tree is clean at 16bf736c7c... (saved ~/j102-base-tree.log). Baseline command initially omitted the CARGO_TARGET_DIR export normally supplied by scripts/verify, so four notification fixture tests failed on env lookup. Ran that exact baseline test binary with CARGO_TARGET_DIR exported: all four PASS, BASE_NOTIFICATION_EXIT=0 (~/j102-base-notifications.log). No repository code changed for that harness correction. Main baseline server run and baseline static builds still active.

kayg commented

2026-11-02 13:39:23 +00:00

Author

Owner

J102 finish. Branch fix-recovery-restart, head 4bb4b5aef8c7272d482f397e0462c6ecbe978563, pushed from base 16bf736c7cc6b474474d6b148077b516b0893cf1.

Expired and cancelled attempts cannot carry their old readiness into a new attempt. New attempts get fresh 24h/48h deadlines using statement time and the existing persisted server time high-water mark; D113's same-account signed-in exemption remains. Pending start/resume reauthenticates and rotates the bearer for the existing request, preserving BOTH ready and expiry. Each new attempt records its start and notifies the signed-in devices after commit. Updated docs/user/keys.md and contracts/keywrap.md.

Test-first evidence: the expired-restart, pending-expiry and rollback regressions failed before the fix (3 failed / 1 passed); cancellation already worked. All four new HTTP regressions plus both disk/concurrency and clock reviewer tests pass in the full workspace run. Full server result: 714 passed, 9 failed, 56 ignored. All nine failures were reproduced individually on the unmodified base with the proper target-directory environment. The baseline full run also exposed six harness-environment errors; all six passed on corrected individual reruns. No auth failure remains in the gate.

scripts/verify fast ran on compute01 through build-slot with compact debug artifacts (CARGO_INCREMENTAL=0, dev/test debug=0); static release settings were unchanged. The Rust failure stops the dispatcher before TypeScript, so its complete TypeScript function was run separately and passed. Final documentation/tree checks: repository artifact gate PASS (7 changed paths), fmt PASS, clean worktree at the pushed head.

Verbatim gate summary, baseline failures and size evidence:

J102 verification evidence
base 16bf736c7cc6b474474d6b148077b516b0893cf1
All six targeted restart/clock regressions passed in the full workspace run.
Gate summary (verbatim):
+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| test                      | FAIL (101)   |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+
VERIFY_EXIT=1
Standalone TypeScript:
JavaScript transfer, critical: identity=1000711 gzip=311020/327680 brotli=259212/273408 bytes
JavaScript transfer, deferred: identity=176946 gzip=61265/61440 brotli=53949/55296 bytes
JavaScript transfer, onDemand: identity=275012 gzip=109420/20480 brotli=97405/18432 bytes
JavaScript transfer, onDemand largest chunk chunk-nerpr9yd.js: gzip=15452/20480 brotli=13807/18432 bytes (class total gzip=109420 brotli=97405, not limited)
TS_EXIT=0
Pre-existing failures, each reproduced individually on base with the required target-directory environment:
bootstrap_runtime::remote_backend::tests::already_added_breakit::the_typed_address_decides_which_host_is_named_when_two_match
bootstrap_runtime::remote_backend::tests::breakit2_concurrent_same_destination_issues_only_one_invitation
bootstrap_runtime::remote_backend::tests::cancelled_invitation_after_commit_retires_its_host_record
bootstrap_runtime::remote_backend::tests::cold_ca_refuses_before_invitation_or_upload
bootstrap_runtime::remote_backend::tests::failed_upload_retires_only_the_new_host
bootstrap_runtime::remote_backend::tests::readding_a_live_host_reports_already_added_before_issuing_a_certificate
bootstrap_runtime::remote_backend::tests::retirement_contention::cancelled_invitation_retires_after_sqlite_busy_timeout
bootstrap_runtime::remote_backend::tests::review_cleanup::review_cancelled_committed_invitation_survives_temporary_writer_contention
remote_http::tests::enrollment_http_is_bound_single_use_and_preserves_token_while_locked
Baseline full run had six additional fixture-environment failures; all six passed when CARGO_TARGET_DIR was exported.
Static sizes:
x86_64 FIXED_SIZE=22418064
aarch64 FIXED_SIZE=20450160
x86_64 BASE_SIZE=22413328
aarch64 BASE_SIZE=20450288
x86_64 delta=+4736 bytes
aarch64 delta=-128 bytes

Evidence is preserved on compute01 in ~/j102-*.log and ~/j102-report.txt; control-host copy: /home/kayg/.local/state/infraternal-orchestrator/evidence/j102/. The initial red/green recovery logs from dev-02 are included. Every started run has finished or was explicitly stopped. Cleanup: no owned orphan processes; both VM targets removed. No owner decision is needed for this slice; the nine proven baseline bootstrap/enrollment failures remain outside its scope. Issue left open for review/integration.

**J102 finish.** Branch `fix-recovery-restart`, head `4bb4b5aef8c7272d482f397e0462c6ecbe978563`, pushed from base `16bf736c7cc6b474474d6b148077b516b0893cf1`. Expired and cancelled attempts cannot carry their old readiness into a new attempt. New attempts get fresh 24h/48h deadlines using statement time and the existing persisted server time high-water mark; D113's same-account signed-in exemption remains. Pending start/resume reauthenticates and rotates the bearer for the existing request, preserving BOTH ready and expiry. Each new attempt records its start and notifies the signed-in devices after commit. Updated `docs/user/keys.md` and `contracts/keywrap.md`. Test-first evidence: the expired-restart, pending-expiry and rollback regressions failed before the fix (3 failed / 1 passed); cancellation already worked. All four new HTTP regressions plus both disk/concurrency and clock reviewer tests pass in the full workspace run. Full server result: 714 passed, 9 failed, 56 ignored. All nine failures were reproduced individually on the unmodified base with the proper target-directory environment. The baseline full run also exposed six harness-environment errors; all six passed on corrected individual reruns. No auth failure remains in the gate. `scripts/verify fast` ran on compute01 through build-slot with compact debug artifacts (`CARGO_INCREMENTAL=0`, dev/test debug=0); static release settings were unchanged. The Rust failure stops the dispatcher before TypeScript, so its complete TypeScript function was run separately and passed. Final documentation/tree checks: repository artifact gate PASS (7 changed paths), fmt PASS, clean worktree at the pushed head. Verbatim gate summary, baseline failures and size evidence: ```text J102 verification evidence base 16bf736c7cc6b474474d6b148077b516b0893cf1 All six targeted restart/clock regressions passed in the full workspace run. Gate summary (verbatim): +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | test | FAIL (101) | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | static builds | PASS | +---------------------------+--------------+ VERIFY_EXIT=1 Standalone TypeScript: JavaScript transfer, critical: identity=1000711 gzip=311020/327680 brotli=259212/273408 bytes JavaScript transfer, deferred: identity=176946 gzip=61265/61440 brotli=53949/55296 bytes JavaScript transfer, onDemand: identity=275012 gzip=109420/20480 brotli=97405/18432 bytes JavaScript transfer, onDemand largest chunk chunk-nerpr9yd.js: gzip=15452/20480 brotli=13807/18432 bytes (class total gzip=109420 brotli=97405, not limited) TS_EXIT=0 Pre-existing failures, each reproduced individually on base with the required target-directory environment: bootstrap_runtime::remote_backend::tests::already_added_breakit::the_typed_address_decides_which_host_is_named_when_two_match bootstrap_runtime::remote_backend::tests::breakit2_concurrent_same_destination_issues_only_one_invitation bootstrap_runtime::remote_backend::tests::cancelled_invitation_after_commit_retires_its_host_record bootstrap_runtime::remote_backend::tests::cold_ca_refuses_before_invitation_or_upload bootstrap_runtime::remote_backend::tests::failed_upload_retires_only_the_new_host bootstrap_runtime::remote_backend::tests::readding_a_live_host_reports_already_added_before_issuing_a_certificate bootstrap_runtime::remote_backend::tests::retirement_contention::cancelled_invitation_retires_after_sqlite_busy_timeout bootstrap_runtime::remote_backend::tests::review_cleanup::review_cancelled_committed_invitation_survives_temporary_writer_contention remote_http::tests::enrollment_http_is_bound_single_use_and_preserves_token_while_locked Baseline full run had six additional fixture-environment failures; all six passed when CARGO_TARGET_DIR was exported. Static sizes: x86_64 FIXED_SIZE=22418064 aarch64 FIXED_SIZE=20450160 x86_64 BASE_SIZE=22413328 aarch64 BASE_SIZE=20450288 x86_64 delta=+4736 bytes aarch64 delta=-128 bytes ``` Evidence is preserved on compute01 in `~/j102-*.log` and `~/j102-report.txt`; control-host copy: `/home/kayg/.local/state/infraternal-orchestrator/evidence/j102/`. The initial red/green recovery logs from dev-02 are included. Every started run has finished or was explicitly stopped. Cleanup: no owned orphan processes; both VM targets removed. No owner decision is needed for this slice; the nine proven baseline bootstrap/enrollment failures remain outside its scope. Issue left open for review/integration.

kayg commented

2026-11-02 13:44:43 +00:00

Author

Owner

Review start: read-only review of fix-recovery-restart at 4bb4b5aef8 against base 16bf736c7c. I am tracing recovery state, bearer rotation, high-water time, notifications, and production migration risk.

Review start: read-only review of fix-recovery-restart at 4bb4b5aef8c7272d482f397e0462c6ecbe978563 against base 16bf736c7cc6b474474d6b148077b516b0893cf1. I am tracing recovery state, bearer rotation, high-water time, notifications, and production migration risk.

kayg commented

2026-11-02 13:53:23 +00:00

Author

Owner

REQUEST CHANGES

P1 — A forward clock step can bypass D25's 24-hour wait. auth/http.rs:194-209 returns raw SystemTime, and auth/http.rs:324-326 passes it to recovery_ready. auth/recovery.rs:193-205 accepts the bearer when ready <= now < expires; completion rechecks against MAX(now, unixepoch()) in auth/webauthn.rs:143-150 and auth/recovery.rs:240-249. The persisted high-water mark is consulted for start/resume at auth/recovery.rs:94-96, but not for capability use or completion. Concrete scenario: start an unauthenticated attempt at time t, wait one real hour, then step the server clock forward to t+24h. The still-valid original bearer can obtain a registration challenge and complete within its five-minute ceremony lifetime, before 24 real hours have elapsed.
P2 — The completion state notification is not delivered to signed-in device subscribers. Recovery completion deletes all account sessions (auth/recovery.rs:256-258) and then commits with revocation plus recovery events (auth/webauthn.rs:216-219; auth/account_events.rs:46-53). The WebSocket recovery-event branch requires live session authority before sending (websocket.rs:716-720, 1090-1100); the completion revocation observes the deleted session (websocket.rs:729-744). Therefore an active signed-in device cannot receive the auth.recovery completion snapshot; it gets session teardown without an explicit completion notice.

No other findings. Restart/cancel/concurrent mutations serialize in the writer transaction; bearer rotation replaces the stored hash atomically, so the old bearer fails the later hash check. Pending retries preserve both deadlines, the D113 same-account live-session exemption remains, and expired/cancelled attempts get a fresh wait. No migration was added, and I found no recovery bearer in logs or argv.

Read-only review; no code changed and no gate was run. The implementation's gate and base-failure evidence is in the existing finish comment 75003. Head 4bb4b5aef8c7272d482f397e0462c6ecbe978563, base 16bf736c7cc6b474474d6b148077b516b0893cf1.

**REQUEST CHANGES** 1. **P1 — A forward clock step can bypass D25's 24-hour wait.** `auth/http.rs:194-209` returns raw `SystemTime`, and `auth/http.rs:324-326` passes it to `recovery_ready`. `auth/recovery.rs:193-205` accepts the bearer when `ready <= now < expires`; completion rechecks against `MAX(now, unixepoch())` in `auth/webauthn.rs:143-150` and `auth/recovery.rs:240-249`. The persisted high-water mark is consulted for start/resume at `auth/recovery.rs:94-96`, but not for capability use or completion. Concrete scenario: start an unauthenticated attempt at time t, wait one real hour, then step the server clock forward to t+24h. The still-valid original bearer can obtain a registration challenge and complete within its five-minute ceremony lifetime, before 24 real hours have elapsed. 2. **P2 — The completion state notification is not delivered to signed-in device subscribers.** Recovery completion deletes all account sessions (`auth/recovery.rs:256-258`) and then commits with revocation plus recovery events (`auth/webauthn.rs:216-219`; `auth/account_events.rs:46-53`). The WebSocket recovery-event branch requires live session authority before sending (`websocket.rs:716-720, 1090-1100`); the completion revocation observes the deleted session (`websocket.rs:729-744`). Therefore an active signed-in device cannot receive the `auth.recovery` completion snapshot; it gets session teardown without an explicit completion notice. No other findings. Restart/cancel/concurrent mutations serialize in the writer transaction; bearer rotation replaces the stored hash atomically, so the old bearer fails the later hash check. Pending retries preserve both deadlines, the D113 same-account live-session exemption remains, and expired/cancelled attempts get a fresh wait. No migration was added, and I found no recovery bearer in logs or argv. Read-only review; no code changed and no gate was run. The implementation's gate and base-failure evidence is in the existing finish comment 75003. Head `4bb4b5aef8c7272d482f397e0462c6ecbe978563`, base `16bf736c7cc6b474474d6b148077b516b0893cf1`.

kayg commented

2026-11-02 13:58:06 +00:00

Author

Owner

J102 round 2 start: fix-recovery-restart at 4bb4b5aef8. Verified both findings in comment 75039. Plan: (1) prove a forward-jump regression, replace recovery wall-time authority with persisted monotonic elapsed accounting shared by start/resume/status/ready/complete, test rollback and restart conservation, gate and commit; (2) prove completion notification loss with a signed-in socket, deliver an explicit completion teardown notice, gate and commit. Estimate: 3 hours. No other changes. Both VM queues checked; dev-02 is less loaded. Target: ~/worktrees/j102/target, compact debug artifacts. Only the nine previously proven inherited failures are acceptable.

J102 round 2 start: fix-recovery-restart at 4bb4b5aef8c7272d482f397e0462c6ecbe978563. Verified both findings in comment 75039. Plan: (1) prove a forward-jump regression, replace recovery wall-time authority with persisted monotonic elapsed accounting shared by start/resume/status/ready/complete, test rollback and restart conservation, gate and commit; (2) prove completion notification loss with a signed-in socket, deliver an explicit completion teardown notice, gate and commit. Estimate: 3 hours. No other changes. Both VM queues checked; dev-02 is less loaded. Target: ~/worktrees/j102/target, compact debug artifacts. Only the nine previously proven inherited failures are acceptable.

kayg commented

2026-11-02 14:01:03 +00:00

Author

Owner

P1 design/contract clarification before implementation: use a recovery-only persisted logical clock advanced solely by monotonic elapsed process uptime. Every authority/status path shares it. Wall time is used only for session/ceremony policy, audit timestamps and estimated display dates. Process restart loads the recorded clock and credits no unrecorded downtime; a backward wall step adds zero waiting time, while each restart can lose at most the remaining 24h wait (plus downtime). No timer/background infrastructure. Existing wall-based pending rows must conservatively begin a full elapsed wait on upgrade because their prior real elapsed duration cannot be proven. No dependency added. The browser already counts delay_seconds using performance.now().

kayg commented

2026-11-02 14:17:41 +00:00

Author

Owner

Red evidence on compute01: ~/j102-r2-red.log has P1_RED_EXIT=101, four failures: forward step releases envelope, valid completion bypasses elapsed wait, resumed delay is 0 instead of 82800, and restart reports 0 instead of 82800. Corrected socket fixture to use the server-returned ready time; ~/j102-r2-p2-red.log now fails at the intended notice assertion: None instead of Some("recovery completed"), P2_RED_EXIT=101. Implementing P1 first; P2 regression retained separately until its own commit. Dev-02 queued run was stopped before compilation; its target removed. Only compute01 ~/worktrees/j102/target remains.

kayg commented

2026-11-02 14:46:55 +00:00

Author

Owner

Crash-safety checkpoint pushed at 802288d48 (P1 only; P2 still separate). Clock authority now records nanoseconds to avoid rounding a subsecond start early, and migration ends unprovable legacy wall-based requests rather than reviving expired bearers. First iterate log ~/j102-r2-p1-iterate.log was red: two local clippy style findings (corrected) and latency_fixture_accepts_the_unsolicited_startup_inventory timed out before the server test row. Current snapshot is rerunning at /j102-r2-p1-iterate2.log; final gate must have no failures outside the nine accepted inherited names. P2 browser red evidence: unknown completion reason became generic terminal-ended text, and no terminal window meant the sign-out handler received no notice (TEXT_RED_EXIT=1, CLIENT_RED_EXIT=1). After wiring the existing sign-out flow, isolated actual-transport tests are 10 passed / 0 failed (/j102-r2-p2-client-green.log). Server P2 red is None instead of the explicit recovery-completed close reason. One target remains on compute01: ~/worktrees/j102/target.

Crash-safety checkpoint pushed at 802288d48 (P1 only; P2 still separate). Clock authority now records nanoseconds to avoid rounding a subsecond start early, and migration ends unprovable legacy wall-based requests rather than reviving expired bearers. First iterate log ~/j102-r2-p1-iterate.log was red: two local clippy style findings (corrected) and latency_fixture_accepts_the_unsolicited_startup_inventory timed out before the server test row. Current snapshot is rerunning at ~/j102-r2-p1-iterate2.log; final gate must have no failures outside the nine accepted inherited names. P2 browser red evidence: unknown completion reason became generic terminal-ended text, and no terminal window meant the sign-out handler received no notice (TEXT_RED_EXIT=1, CLIENT_RED_EXIT=1). After wiring the existing sign-out flow, isolated actual-transport tests are 10 passed / 0 failed (~/j102-r2-p2-client-green.log). Server P2 red is None instead of the explicit recovery-completed close reason. One target remains on compute01: ~/worktrees/j102/target.

kayg commented

2026-11-02 15:16:08 +00:00

Author

Owner

J102 round 2 checkpoint: 17e46c9c4 on fix-recovery-restart saves both findings separately as WIP commits. P1 targeted elapsed-clock/reviewer/migration tests and clippy are queued on compute01, log ~/j102-r2-p1-targets.log, single target /home/claude/worktrees/j102/target. P2 TypeScript check and 116 connection/close-reason tests pass on dev-02 (~/j102-r2-p2-ts.log). The socket and browser regressions failed before their fixes. I found a commit/publication race during P2 inspection and added a deterministic barrier regression; its implementation and Rust checks are next. Final commits will replace these WIP messages after verification, followed by the full fast gate. No merge or deployment.

J102 round 2 checkpoint: `17e46c9c4` on `fix-recovery-restart` saves both findings separately as WIP commits. P1 targeted elapsed-clock/reviewer/migration tests and clippy are queued on compute01, log `~/j102-r2-p1-targets.log`, single target `/home/claude/worktrees/j102/target`. P2 TypeScript check and 116 connection/close-reason tests pass on dev-02 (`~/j102-r2-p2-ts.log`). The socket and browser regressions failed before their fixes. I found a commit/publication race during P2 inspection and added a deterministic barrier regression; its implementation and Rust checks are next. Final commits will replace these WIP messages after verification, followed by the full fast gate. No merge or deployment.

kayg commented

2026-11-02 15:37:06 +00:00

Author

Owner

J102 round 2: final two-commit head 81cf331ffba9a1d14bc6df75d983e7b9be260699 is pushed. P1 commit e21376b8a preserves recovery waits using recorded monotonic nanoseconds; P2 commit 81cf331ff carries the completion notice in the flushed close frame and signs out the browser. P1: 16 targeted elapsed/reviewer/migration tests and clippy pass. P2: completion/publication barrier test failed first (exit 101), then the eight completion tests and all 17 account-event tests pass; clippy passes. Browser: typecheck and 116 connection/close-reason tests pass. Full scripts/verify fast now runs on this exact head on compute01, log ~/j102-r2-fast.log; full TypeScript gate follows separately if Rust's inherited failures stop the dispatcher. Single target remains /home/claude/worktrees/j102/target. No merge or deployment.

J102 round 2: final two-commit head `81cf331ffba9a1d14bc6df75d983e7b9be260699` is pushed. P1 commit `e21376b8a` preserves recovery waits using recorded monotonic nanoseconds; P2 commit `81cf331ff` carries the completion notice in the flushed close frame and signs out the browser. P1: 16 targeted elapsed/reviewer/migration tests and clippy pass. P2: completion/publication barrier test failed first (exit 101), then the eight completion tests and all 17 account-event tests pass; clippy passes. Browser: typecheck and 116 connection/close-reason tests pass. Full `scripts/verify fast` now runs on this exact head on compute01, log `~/j102-r2-fast.log`; full TypeScript gate follows separately if Rust's inherited failures stop the dispatcher. Single target remains `/home/claude/worktrees/j102/target`. No merge or deployment.

kayg commented

2026-11-02 16:06:44 +00:00

Author

Owner

J102 round 2 gate checkpoint at pushed head 81cf331ffba9a1d14bc6df75d983e7b9be260699: server suite is 723 passed / 9 failed / 56 ignored. Exact failing-name comparison against the saved unmodified-dev reproductions is 9/9, with zero additional failures. All recovery regressions, including the signed-in socket completion notice and publication barrier, pass in the full run. Other workspace crates, cargo-deny and architecture checks pass. The x86_64 stripped static build/key-memory probe passed: 22,424,752 bytes, +6,688 versus round-1 head (22,418,064), +11,424 versus the original dev base (22,413,328). aarch64 static compilation is running, then the full TypeScript gate and production browser notice captures follow. Logs: ~/j102-r2-fast.log, ~/j102-r2-ts-full.log; one target /home/claude/worktrees/j102/target. No source changes since the pushed head.

J102 round 2 gate checkpoint at pushed head `81cf331ffba9a1d14bc6df75d983e7b9be260699`: server suite is 723 passed / 9 failed / 56 ignored. Exact failing-name comparison against the saved unmodified-dev reproductions is 9/9, with zero additional failures. All recovery regressions, including the signed-in socket completion notice and publication barrier, pass in the full run. Other workspace crates, cargo-deny and architecture checks pass. The x86_64 stripped static build/key-memory probe passed: 22,424,752 bytes, +6,688 versus round-1 head (22,418,064), +11,424 versus the original dev base (22,413,328). aarch64 static compilation is running, then the full TypeScript gate and production browser notice captures follow. Logs: `~/j102-r2-fast.log`, `~/j102-r2-ts-full.log`; one target `/home/claude/worktrees/j102/target`. No source changes since the pushed head.

kayg commented

2026-11-02 16:21:12 +00:00

Author

Owner

J102 round 2 complete on fix-recovery-restart, head 81cf331ffba9a1d14bc6df75d983e7b9be260699. Both findings from comment 75039 are fixed, one atomic commit each:

P1: e21376b8a20527ef921ef6ce13b8fde5602bb62d. Recovery authority uses persisted monotonic nanoseconds for start/resume, status, ready and complete. Wall-clock steps add zero credit or wait. Restarts retain committed progress only; each restart can add up to 24 hours to the wait and lose up to 48 hours of lifetime progress, plus downtime. Migration 0037 ends legacy wall-based requests because their elapsed age cannot be proven; credentials remain intact. Pending retries keep their existing wait and reissue the bearer. Docs and the contract are updated.
P2: 81cf331ffba9a1d14bc6df75d983e7b9be260699. Completion is delivered as the flushed WebSocket close frame (1008, recovery completed), then the browser shows “Recovery completed. This device has been signed out.” on the existing sign-in screen. Atomic credential replacement is retained. A committed completion generation survives coalescing, and a commit/publication barrier closes the deletion-before-publication race.

Failing-first evidence: the original clock tests failed before P1 (P1_RED_EXIT=101); the signed-in socket test failed before P2 (P2_RED_EXIT=101), as did its text/transport tests. The deterministic commit/publication barrier test also failed without the barrier (BARRIER_RED_EXIT=101). Green: all five elapsed-clock regressions, reviewer/migration tests, completion/socket tests and 17 account-event tests pass; clippy passes. The full server suite is 723 passed, 9 failed, 56 ignored. The nine names exactly match the previously reproduced unmodified-dev failures, with no additional failures.

scripts/verify fast on compute01 at the exact head, verbatim summary:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| test                      | FAIL (101)   |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+
VERIFY_EXIT=1

The Rust failure stops the dispatcher before TypeScript, so the complete ts_verify_fast function was run separately on the same head. Typecheck, all 656 tests, production build, compressed transfer budgets and protocol oracle passed:

TS_EXIT=0

The only inherited failures (exact names):

inherited dev failure match: 9/9; additional failures: 0
bootstrap_runtime::remote_backend::tests::already_added_breakit::the_typed_address_decides_which_host_is_named_when_two_match
bootstrap_runtime::remote_backend::tests::breakit2_concurrent_same_destination_issues_only_one_invitation
bootstrap_runtime::remote_backend::tests::cancelled_invitation_after_commit_retires_its_host_record
bootstrap_runtime::remote_backend::tests::cold_ca_refuses_before_invitation_or_upload
bootstrap_runtime::remote_backend::tests::failed_upload_retires_only_the_new_host
bootstrap_runtime::remote_backend::tests::readding_a_live_host_reports_already_added_before_issuing_a_certificate
bootstrap_runtime::remote_backend::tests::retirement_contention::cancelled_invitation_retires_after_sqlite_busy_timeout
bootstrap_runtime::remote_backend::tests::review_cleanup::review_cancelled_committed_invitation_survives_temporary_writer_contention
remote_http::tests::enrollment_http_is_bound_single_use_and_preserves_token_while_locked

Stripped static sizes, unchanged release profile:

Architecture	Final bytes	Delta vs round-1 head	Delta vs original dev base
x86_64	22,424,752	+6,688	+11,424
aarch64	20,457,488	+7,328	+7,200

Production browser capture: six checks passed at 1440×900, 393×852 and 375×667, in ink and paper, DPR 2. Each receives the real teardown reason through the transport, sees the completion notice and sign-in button, has no old windows/sidebar, and remains signed out after reload. Notice/button centers differ by at most 0.5 px. All six screenshots were inspected at DPR 2; attached for independent visual review. The capture uses public fixture identity and mocked HTTP/WebSocket authority; the separate Rust socket test exercises real session revocation. The first browser launch lacked libnspr4.so; rerunning with the VM's packaged Chromium passed without repository changes.

How to test: keep an owner device signed in, complete recovery from another device, and observe the completion notice on the old device's sign-in screen. For clock steps/restarts and exact 24/48-hour boundaries, use the deterministic elapsed-clock regressions rather than waiting real days.

Raw logs and capture harness/screenshots are attached and saved under /home/kayg/.local/state/infraternal-orchestrator/evidence/j102/. Both VM targets are absent and no own build/test/browser processes remain. Issue remains open; no merge or deployment.

J102 round 2 complete on `fix-recovery-restart`, head `81cf331ffba9a1d14bc6df75d983e7b9be260699`. Both findings from comment 75039 are fixed, one atomic commit each: - P1: `e21376b8a20527ef921ef6ce13b8fde5602bb62d`. Recovery authority uses persisted monotonic nanoseconds for start/resume, status, ready and complete. Wall-clock steps add zero credit or wait. Restarts retain committed progress only; each restart can add up to 24 hours to the wait and lose up to 48 hours of lifetime progress, plus downtime. Migration 0037 ends legacy wall-based requests because their elapsed age cannot be proven; credentials remain intact. Pending retries keep their existing wait and reissue the bearer. Docs and the contract are updated. - P2: `81cf331ffba9a1d14bc6df75d983e7b9be260699`. Completion is delivered as the flushed WebSocket close frame (1008, `recovery completed`), then the browser shows “Recovery completed. This device has been signed out.” on the existing sign-in screen. Atomic credential replacement is retained. A committed completion generation survives coalescing, and a commit/publication barrier closes the deletion-before-publication race. Failing-first evidence: the original clock tests failed before P1 (`P1_RED_EXIT=101`); the signed-in socket test failed before P2 (`P2_RED_EXIT=101`), as did its text/transport tests. The deterministic commit/publication barrier test also failed without the barrier (`BARRIER_RED_EXIT=101`). Green: all five elapsed-clock regressions, reviewer/migration tests, completion/socket tests and 17 account-event tests pass; clippy passes. The full server suite is 723 passed, 9 failed, 56 ignored. The nine names exactly match the previously reproduced unmodified-dev failures, with no additional failures. `scripts/verify fast` on compute01 at the exact head, verbatim summary: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | test | FAIL (101) | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | static builds | PASS | +---------------------------+--------------+ VERIFY_EXIT=1 ``` The Rust failure stops the dispatcher before TypeScript, so the complete `ts_verify_fast` function was run separately on the same head. Typecheck, all 656 tests, production build, compressed transfer budgets and protocol oracle passed: ``` TS_EXIT=0 ``` The only inherited failures (exact names): ``` inherited dev failure match: 9/9; additional failures: 0 bootstrap_runtime::remote_backend::tests::already_added_breakit::the_typed_address_decides_which_host_is_named_when_two_match bootstrap_runtime::remote_backend::tests::breakit2_concurrent_same_destination_issues_only_one_invitation bootstrap_runtime::remote_backend::tests::cancelled_invitation_after_commit_retires_its_host_record bootstrap_runtime::remote_backend::tests::cold_ca_refuses_before_invitation_or_upload bootstrap_runtime::remote_backend::tests::failed_upload_retires_only_the_new_host bootstrap_runtime::remote_backend::tests::readding_a_live_host_reports_already_added_before_issuing_a_certificate bootstrap_runtime::remote_backend::tests::retirement_contention::cancelled_invitation_retires_after_sqlite_busy_timeout bootstrap_runtime::remote_backend::tests::review_cleanup::review_cancelled_committed_invitation_survives_temporary_writer_contention remote_http::tests::enrollment_http_is_bound_single_use_and_preserves_token_while_locked ``` Stripped static sizes, unchanged release profile: | Architecture | Final bytes | Delta vs round-1 head | Delta vs original dev base | | --- | ---: | ---: | ---: | | x86_64 | 22,424,752 | +6,688 | +11,424 | | aarch64 | 20,457,488 | +7,328 | +7,200 | Production browser capture: six checks passed at 1440×900, 393×852 and 375×667, in ink and paper, DPR 2. Each receives the real teardown reason through the transport, sees the completion notice and sign-in button, has no old windows/sidebar, and remains signed out after reload. Notice/button centers differ by at most 0.5 px. All six screenshots were inspected at DPR 2; attached for independent visual review. The capture uses public fixture identity and mocked HTTP/WebSocket authority; the separate Rust socket test exercises real session revocation. The first browser launch lacked `libnspr4.so`; rerunning with the VM's packaged Chromium passed without repository changes. How to test: keep an owner device signed in, complete recovery from another device, and observe the completion notice on the old device's sign-in screen. For clock steps/restarts and exact 24/48-hour boundaries, use the deterministic elapsed-clock regressions rather than waiting real days. Raw logs and capture harness/screenshots are attached and saved under `/home/kayg/.local/state/infraternal-orchestrator/evidence/j102/`. Both VM targets are absent and no own build/test/browser processes remain. Issue remains open; no merge or deployment.

j102-r2-fast.log

547 KiB

j102-r2-ts-full.log

86 KiB

j102-r2-red.log

17 KiB

j102-r2-p2-red.log

2.5 KiB

j102-r2-targets-final.log

13 KiB

j102-r2-browser.log

671 B

j102-r2-browser-harness.txt

4.2 KiB

recovery-completed-ink-1440.png

92 KiB

recovery-completed-ink-375.png

76 KiB

recovery-completed-ink-393.png

78 KiB

recovery-completed-paper-1440.png

82 KiB

recovery-completed-paper-375.png

65 KiB

recovery-completed-paper-393.png

66 KiB

kayg commented

2026-11-02 16:26:34 +00:00

Author

Owner

J102 round 3 started: mechanical rebase only of fix-recovery-restart from 81cf331ffba9a1d14bc6df75d983e7b9be260699 onto requested dev 99668d8abc8dd55bba038c2c48a04e444c33e324. Plan: rebase without behavior changes, inspect any resolutions, run the complete scripts/verify fast and require every row green, then push and report verbatim output. Estimate: 90 minutes including cold static builds. Both queues checked: compute01 has two free slots; dev-02 is full with a queue. Gate will use compute01 with one target /home/claude/worktrees/j102/target and compact dev/test debug information. J111's parallel review remains on the prior head; no logic changes without owner findings.

J102 round 3 started: mechanical rebase only of `fix-recovery-restart` from `81cf331ffba9a1d14bc6df75d983e7b9be260699` onto requested dev `99668d8abc8dd55bba038c2c48a04e444c33e324`. Plan: rebase without behavior changes, inspect any resolutions, run the complete `scripts/verify fast` and require every row green, then push and report verbatim output. Estimate: 90 minutes including cold static builds. Both queues checked: compute01 has two free slots; dev-02 is full with a queue. Gate will use compute01 with one target `/home/claude/worktrees/j102/target` and compact dev/test debug information. J111's parallel review remains on the prior head; no logic changes without owner findings.

kayg commented

2026-11-02 16:29:59 +00:00

Author

Owner

Mechanical rebase completed at 716183781bb46df735686c1b8299d52c0e04c5e2 on requested dev 99668d8abc8dd55bba038c2c48a04e444c33e324. One conflict, entirely in account_events' cancelled-commit test: retained dev's 60-second bound and database-read-before-notification wait, then kept J102's completion-generation assertion. git range-diff reports the first two commits unchanged and only this test-context adaptation in the third; no production logic or behavior changed. Full scripts/verify fast is running on compute01, exact head verified, log ~/j102-r3-fast.log, sole target /home/claude/worktrees/j102/target. This round requires a fully green result.

Mechanical rebase completed at `716183781bb46df735686c1b8299d52c0e04c5e2` on requested dev `99668d8abc8dd55bba038c2c48a04e444c33e324`. One conflict, entirely in `account_events`' cancelled-commit test: retained dev's 60-second bound and database-read-before-notification wait, then kept J102's completion-generation assertion. `git range-diff` reports the first two commits unchanged and only this test-context adaptation in the third; no production logic or behavior changed. Full `scripts/verify fast` is running on compute01, exact head verified, log `~/j102-r3-fast.log`, sole target `/home/claude/worktrees/j102/target`. This round requires a fully green result.

kayg commented

2026-11-02 16:37:50 +00:00

Author

Owner

J111 review start: read-only review of fix-recovery-restart at 81cf331ffba9a1d14bc6df75d983e7b9be260699 against base 16bf736c7cc6b474474d6b148077b516b0893cf1. I am tracing recovery elapsed-time authority and displayed state, completion notice ordering, bearer rotation, D113, secret handling, and migration compatibility. No code changes; no gate run.

J111 review start: read-only review of `fix-recovery-restart` at `81cf331ffba9a1d14bc6df75d983e7b9be260699` against base `16bf736c7cc6b474474d6b148077b516b0893cf1`. I am tracing recovery elapsed-time authority and displayed state, completion notice ordering, bearer rotation, D113, secret handling, and migration compatibility. No code changes; no gate run.

kayg commented

2026-11-02 16:40:19 +00:00

Author

Owner

J111 finding — P3 (status accuracy; no early-completion bypass): apps/tracer/src/recovery.ts:170 sets a page-local readyClock from the initial delay_seconds; :108-116 then updates the displayed countdown from performance.now() without querying the server. The server reloads only its persisted elapsed epoch on restart (crates/infraternal-server/src/auth/recovery.rs:14-20) and persists progress at recovery/status calls (:57-88); the restart regression intentionally discards unrecorded progress (crates/infraternal-server/src/auth/recovery_restart_tests.rs:247-270). Concrete scenario: an unauthenticated attempt starts, the page stays foreground with no API calls for one hour, then the server restarts. At 24 hours after the original start, the browser can show “Ready now” while the server has credited only 23 hours since restart and rejects recovery_ready (recovery.rs:274-280). Server-side completion remains blocked until 24 hours of credited elapsed time, but the displayed state is not aligned with the authoritative elapsed clock across restart, contrary to the requested status check.

J111 finding — **P3 (status accuracy; no early-completion bypass):** `apps/tracer/src/recovery.ts:170` sets a page-local `readyClock` from the initial `delay_seconds`; `:108-116` then updates the displayed countdown from `performance.now()` without querying the server. The server reloads only its persisted elapsed epoch on restart (`crates/infraternal-server/src/auth/recovery.rs:14-20`) and persists progress at recovery/status calls (`:57-88`); the restart regression intentionally discards unrecorded progress (`crates/infraternal-server/src/auth/recovery_restart_tests.rs:247-270`). Concrete scenario: an unauthenticated attempt starts, the page stays foreground with no API calls for one hour, then the server restarts. At 24 hours after the original start, the browser can show “Ready now” while the server has credited only 23 hours since restart and rejects `recovery_ready` (`recovery.rs:274-280`). Server-side completion remains blocked until 24 hours of credited elapsed time, but the displayed state is not aligned with the authoritative elapsed clock across restart, contrary to the requested status check.

kayg commented

2026-11-02 16:41:25 +00:00

Author

Owner

REQUEST CHANGES

P1 — Closed for server-side authority. Clock is a persisted epoch plus Instant::elapsed() (crates/infraternal-server/src/auth/recovery.rs:6-28); recovery_time samples it after taking SQLite's writer lock and persists MAX(elapsed,candidate) (:50-89). Start/resume (:173-210), ready (:274-280), completion (:328-343) and signed-in status (sessions.rs:63-67 → pending_recovery, recovery.rs:248-265) all derive recovery remaining time from that function. Concrete forward-step case: start at t, wait one real hour, set wall time to t+24h, then call ready or complete; elapsed remains about one hour, so the bearer is denied. A challenge issued earlier is also rechecked at completion. Cancel removes the row; a new attempt gets new deadlines, while pending resume preserves both deadlines and rotates the bearer in the same transaction (:181-210). D113's same-account live-session exemption remains at :188-200; the 24-hour lower bound is for the unauthenticated path. A backward wall step itself adds zero wait. A server restart can discard at most the unpersisted portion of the active 24-hour wait per restart, plus the actual downtime; repeated restarts therefore have no finite cumulative extension bound. This is fail-closed and acceptable for preventing early recovery, with an availability cost.
P2 — Closed. Recovery completion deletes old sessions and credentials in the completion transaction (recovery.rs:345-360). The commit path holds the publication barrier through commit and publishes a completion generation after commit (auth/account_events.rs:61-73). The socket waits for that generation and flushes close code 1008 with reason recovery completed before dropping transport (websocket.rs:991-1039); the client maps it to “Recovery completed. This device has been signed out.” and invokes sign-out (connection.ts:521-523, main.ts:615-616). There is no post-notice authority window: the old session is already deleted, and inbound frames are guarded by while_authorized (websocket.rs:881-898). The socket regression asserts the notice and old-session denial (auth/socket_tests.rs:1027-1089).
P3 — Open: the recovery-page countdown can get ahead of server status after restart. apps/tracer/src/recovery.ts:170 sets a local deadline from the initial server delay_seconds; :108-116 advances the visible “Ready now” state with performance.now() and makes no server request. The server reloads only persisted elapsed progress on restart (recovery.rs:14-20) and persists progress on recovery/status calls (:57-88). Concrete scenario: start an unauthenticated request, leave the page foreground with no further API calls for one hour, then restart the server. At 24 hours after the original start, the page can say “Ready now” while the server has credited only 23 hours since restart and rejects recovery_ready (`recovery_restart_tests.rs:247-280). This does not bypass the wait, but does not meet the requested requirement that displayed status use the same elapsed clock.

Other checks: bearer rotation stays atomic (recovery.rs:202-210, checked by the stored hash at :274-280); D25/D113 are unchanged in DECISIONS.md; I found no recovery words, tokens or keys added to runtime logs or command arguments. Migration 0037 creates the clock and deletes legacy wall-based request rows (migrations/0037_recovery_elapsed.sql:3-10); the existing-database migration test verifies the user recovery verifier remains unchanged and the clock row is created (auth/migration_data_tests.rs:188-217). It is safe for existing DB data, with the intentional behavior that an in-flight legacy recovery is invalidated and must be started again with a fresh wait.

Read-only review; no code changed and no gate rerun. Gate is not applicable to this review; the exact-head implementation gate report is already in #309 comment 75280. Reviewed fix-recovery-restart at 81cf331ffba9a1d14bc6df75d983e7b9be260699 against base 16bf736c7cc6b474474d6b148077b516b0893cf1.

**REQUEST CHANGES** 1. **P1 — Closed for server-side authority.** `Clock` is a persisted epoch plus `Instant::elapsed()` (`crates/infraternal-server/src/auth/recovery.rs:6-28`); `recovery_time` samples it after taking SQLite's writer lock and persists `MAX(elapsed,candidate)` (`:50-89`). Start/resume (`:173-210`), ready (`:274-280`), completion (`:328-343`) and signed-in status (`sessions.rs:63-67` → `pending_recovery`, `recovery.rs:248-265`) all derive recovery remaining time from that function. Concrete forward-step case: start at t, wait one real hour, set wall time to t+24h, then call ready or complete; elapsed remains about one hour, so the bearer is denied. A challenge issued earlier is also rechecked at completion. Cancel removes the row; a new attempt gets new deadlines, while pending resume preserves both deadlines and rotates the bearer in the same transaction (`:181-210`). D113's same-account live-session exemption remains at `:188-200`; the 24-hour lower bound is for the unauthenticated path. A backward wall step itself adds **zero** wait. A server restart can discard at most the unpersisted portion of the active 24-hour wait per restart, plus the actual downtime; repeated restarts therefore have no finite cumulative extension bound. This is fail-closed and acceptable for preventing early recovery, with an availability cost. 2. **P2 — Closed.** Recovery completion deletes old sessions and credentials in the completion transaction (`recovery.rs:345-360`). The commit path holds the publication barrier through commit and publishes a completion generation after commit (`auth/account_events.rs:61-73`). The socket waits for that generation and flushes close code 1008 with reason `recovery completed` before dropping transport (`websocket.rs:991-1039`); the client maps it to “Recovery completed. This device has been signed out.” and invokes sign-out (`connection.ts:521-523`, `main.ts:615-616`). There is no post-notice authority window: the old session is already deleted, and inbound frames are guarded by `while_authorized` (`websocket.rs:881-898`). The socket regression asserts the notice and old-session denial (`auth/socket_tests.rs:1027-1089`). 3. **P3 — Open: the recovery-page countdown can get ahead of server status after restart.** `apps/tracer/src/recovery.ts:170` sets a local deadline from the initial server `delay_seconds`; `:108-116` advances the visible “Ready now” state with `performance.now()` and makes no server request. The server reloads only persisted elapsed progress on restart (`recovery.rs:14-20`) and persists progress on recovery/status calls (`:57-88`). Concrete scenario: start an unauthenticated request, leave the page foreground with no further API calls for one hour, then restart the server. At 24 hours after the original start, the page can say “Ready now” while the server has credited only 23 hours since restart and rejects `recovery_ready` (`recovery_restart_tests.rs:247-280). This does not bypass the wait, but does not meet the requested requirement that displayed status use the same elapsed clock. **Other checks:** bearer rotation stays atomic (`recovery.rs:202-210`, checked by the stored hash at `:274-280`); D25/D113 are unchanged in `DECISIONS.md`; I found no recovery words, tokens or keys added to runtime logs or command arguments. Migration 0037 creates the clock and deletes legacy wall-based request rows (`migrations/0037_recovery_elapsed.sql:3-10`); the existing-database migration test verifies the user recovery verifier remains unchanged and the clock row is created (`auth/migration_data_tests.rs:188-217`). It is safe for existing DB data, with the intentional behavior that an in-flight legacy recovery is invalidated and must be started again with a fresh wait. Read-only review; no code changed and no gate rerun. Gate is not applicable to this review; the exact-head implementation gate report is already in #309 comment 75280. Reviewed `fix-recovery-restart` at `81cf331ffba9a1d14bc6df75d983e7b9be260699` against base `16bf736c7cc6b474474d6b148077b516b0893cf1`.

kayg referenced this issue

2026-11-02 16:45:56 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 16:46:08 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 16:48:35 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg commented

2026-11-02 16:49:54 +00:00

Author

Owner

Owner scope update received: J111 closed both prior fixes (#309 comment 75349). Two additional test-first commits are authorized: (a) token-bound recovery-status plus page refresh on visibility, countdown zero and every 60 seconds; the page never promotes itself to Ready now. A status read must not restart expired/cancelled recovery or rotate its bearer. (b) measured monotonic counter checkpoints every 60 seconds and on graceful shutdown; no wall/downtime credit. Migration remains 0037 as explicitly coordinated with J61b/J60b. The superseded mechanical-only gate was stopped (RUN_EXIT=143); its single warm target is retained. Updated estimate: 3 hours for the two fixes and a fully green complete gate. No other logic changes.

kayg referenced this issue

2026-11-02 16:49:59 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 16:53:44 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg commented

2026-11-02 17:01:08 +00:00

Author

Owner

Crash-recovery checkpoint: rebase plus pending regression tests and additive status contract pushed as marked WIP. Production logic is still the reviewed rebased code. Red runs are queued behind the VM build slots; source and logs are /home/claude/worktrees/j102, ~/j102-red-a.log on compute01. Single retained target: /home/claude/worktrees/j102/target. Both VM queues were checked. The WIP will be split into the two requested atomic commits after red/green verification; migration 0037 is unchanged.

Crash-recovery checkpoint: rebase plus pending regression tests and additive status contract pushed as marked WIP. Production logic is still the reviewed rebased code. Red runs are queued behind the VM build slots; source and logs are `/home/claude/worktrees/j102`, `~/j102-red-a.log` on compute01. Single retained target: `/home/claude/worktrees/j102/target`. Both VM queues were checked. The WIP will be split into the two requested atomic commits after red/green verification; migration 0037 is unchanged.

kayg referenced this issue

2026-11-02 17:25:54 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg commented

2026-11-02 17:27:08 +00:00

Author

Owner

Failing-first evidence confirmed and saved: j102-r3-red-confirmed.log under the control-host evidence directory. The browser enabled Continue at local zero without any status response; the minute-checkpoint regression observed persisted elapsed 0 instead of 60 seconds. The token-bound server-status regression first returned 400 instead of 200, and its implementation is now in the queued verification slice. Client duration/readiness validation: 14 passed, 0 failed.

Both requested fixes are crash-checkpointed as marked WIP. The countdown slice is staged independently and queued for production-build browser checks plus scripts/verify fast (~/j102-a-green.log, compute01). The checkpoint slice is implemented locally but unverified; it adds a minute timer only while a request is active, shared measured-counter persistence, and SIGINT/SIGTERM flushes. Final history will replace WIP with the two atomic commits. Migration remains 0037. Single target remains /home/claude/worktrees/j102/target.

Failing-first evidence confirmed and saved: `j102-r3-red-confirmed.log` under the control-host evidence directory. The browser enabled Continue at local zero without any status response; the minute-checkpoint regression observed persisted elapsed 0 instead of 60 seconds. The token-bound server-status regression first returned 400 instead of 200, and its implementation is now in the queued verification slice. Client duration/readiness validation: 14 passed, 0 failed. Both requested fixes are crash-checkpointed as marked WIP. The countdown slice is staged independently and queued for production-build browser checks plus `scripts/verify fast` (`~/j102-a-green.log`, compute01). The checkpoint slice is implemented locally but unverified; it adds a minute timer only while a request is active, shared measured-counter persistence, and SIGINT/SIGTERM flushes. Final history will replace WIP with the two atomic commits. Migration remains 0037. Single target remains `/home/claude/worktrees/j102/target`.

kayg referenced this issue

2026-11-02 17:46:04 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg commented

2026-11-02 17:58:40 +00:00

Author

Owner

Countdown slice targeted verification is green: 10 Rust recovery regressions, 14 client duration/readiness validations, typecheck, and all 16 production-build recovery browser tests. The new browser test confirms local zero does not enable continuation, minute polling and visibility reads use the same token, and a failed refresh clears previously confirmed readiness. The 2× screenshot is saved under the control-host evidence directory as j102-r3-server-status-unavailable-2x.png and alignment is asserted within 0.5 px.

The first full fast gate has passed workspace and shim tests and is building the static releases (~/j102-a-green.log, compute01). On success, the driver automatically applies ~/j102-b-source.tar, runs the checkpoint regressions, then a second full fast gate. This avoids another queue wait while preserving a separately tested source tree per commit. Latest exact source is pushed as marked WIP; final history will contain the two requested atomic commits. No migration changes; 0037 stays ours. One target only: /home/claude/worktrees/j102/target.

Countdown slice targeted verification is green: 10 Rust recovery regressions, 14 client duration/readiness validations, typecheck, and all 16 production-build recovery browser tests. The new browser test confirms local zero does not enable continuation, minute polling and visibility reads use the same token, and a failed refresh clears previously confirmed readiness. The 2× screenshot is saved under the control-host evidence directory as `j102-r3-server-status-unavailable-2x.png` and alignment is asserted within 0.5 px. The first full fast gate has passed workspace and shim tests and is building the static releases (`~/j102-a-green.log`, compute01). On success, the driver automatically applies `~/j102-b-source.tar`, runs the checkpoint regressions, then a second full fast gate. This avoids another queue wait while preserving a separately tested source tree per commit. Latest exact source is pushed as marked WIP; final history will contain the two requested atomic commits. No migration changes; 0037 stays ours. One target only: `/home/claude/worktrees/j102/target`.

kayg referenced this issue

2026-11-02 18:05:14 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 18:15:31 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg commented

2026-11-02 18:24:03 +00:00

Author

Owner

Countdown fix is now atomic commit 52ad241aee36ce970f5ea509660bc7e8e34b0fbd: its complete fast gate passed, all Rust rows PASS and TypeScript zero failures. The branch is pushed with that parent plus a clearly marked checkpoint WIP while final verification continues.

Verification finding: extracting the second slice preserved timestamps older than the first build, so Cargo initially reused the previous 10-test binary. Source timestamps were refreshed, but that intermediate full gate had already fingerprinted its inputs. It is explicitly unqualified evidence. The same driver will now rebuild/run all 12 recovery regressions, then run a fresh complete scripts/verify fast (marked QUALIFIED_GATE_START and QUALIFIED_RUN_DONE in ~/j102-a-green.log). Final reporting will use only that fresh gate. No production changes were needed for this harness correction.

Countdown fix is now atomic commit `52ad241aee36ce970f5ea509660bc7e8e34b0fbd`: its complete fast gate passed, all Rust rows PASS and TypeScript zero failures. The branch is pushed with that parent plus a clearly marked checkpoint WIP while final verification continues. Verification finding: extracting the second slice preserved timestamps older than the first build, so Cargo initially reused the previous 10-test binary. Source timestamps were refreshed, but that intermediate full gate had already fingerprinted its inputs. It is explicitly unqualified evidence. The same driver will now rebuild/run all 12 recovery regressions, then run a fresh complete `scripts/verify fast` (marked `QUALIFIED_GATE_START` and `QUALIFIED_RUN_DONE` in `~/j102-a-green.log`). Final reporting will use only that fresh gate. No production changes were needed for this harness correction.

kayg referenced this issue

2026-11-02 18:24:32 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg commented

2026-11-02 18:37:45 +00:00

Author

Owner

Intermediate (unqualified) run ended with one failure: crates/infraternal/tests/cli_envelope.rs::status_exits_5_when_something_that_should_run_is_down, line 271, expected exit 5 but got 0 for status --json. This is not being waived as an inherited failure. All other Rust gate rows, including both rebuilt static images, passed. The CLI status sources are unchanged by J102. I am rechecking that exact test in isolation before the final full gate; no out-of-scope CLI fix is being made.

Fresh driver ~/j102-final.sh is queued on compute01 after checking both VM queues. It refreshes source timestamps before any Cargo invocation, runs the 12 recovery regressions (must show the two new checkpoint tests), rechecks the CLI failure, then runs the complete scripts/verify fast. Log: ~/j102-final.log. The prior combined log is retained as j102-r3-a-green-and-unqualified-b.log in the control-host evidence directory. Final reporting will quote only a fully green fresh run. Source checksums match between control worktree and VM.

Intermediate (unqualified) run ended with one failure: `crates/infraternal/tests/cli_envelope.rs::status_exits_5_when_something_that_should_run_is_down`, line 271, expected exit 5 but got 0 for `status --json`. This is not being waived as an inherited failure. All other Rust gate rows, including both rebuilt static images, passed. The CLI status sources are unchanged by J102. I am rechecking that exact test in isolation before the final full gate; no out-of-scope CLI fix is being made. Fresh driver `~/j102-final.sh` is queued on compute01 after checking both VM queues. It refreshes source timestamps before any Cargo invocation, runs the 12 recovery regressions (must show the two new checkpoint tests), rechecks the CLI failure, then runs the complete `scripts/verify fast`. Log: `~/j102-final.log`. The prior combined log is retained as `j102-r3-a-green-and-unqualified-b.log` in the control-host evidence directory. Final reporting will quote only a fully green fresh run. Source checksums match between control worktree and VM.

kayg commented

2026-11-02 18:45:58 +00:00

Author

Owner

Fresh checkpoint regressions passed: 12 passed, 0 failed, including idle minute persistence and graceful flush/repeated restart loss. The isolated CLI exit-code recheck also passed without any CLI source changes.

The fresh full gate then exposed two denied expect() calls in the new worker ownership code and test-only callers still unpacking the old four-field router tuple. I stopped that run. Fixed both expects using the repository's existing PoisonError::into_inner pattern and mechanically updated all three external test callers to ignore the retained fifth Auth field. No recovery behavior changed for these integration corrections.

Latest exact source pushed as marked WIP above the atomic countdown commit 52ad241ae. Driver ~/j102-final2.sh starts with workspace clippy across all targets, then the 12 regressions and full scripts/verify fast; log ~/j102-final2.log, compute01. Both queues checked before launch. Prior stopped run retained as ~/j102-final-stopped-clippy.log. No gate failure is being waived.

Fresh checkpoint regressions passed: **12 passed, 0 failed**, including idle minute persistence and graceful flush/repeated restart loss. The isolated CLI exit-code recheck also passed without any CLI source changes. The fresh full gate then exposed two denied `expect()` calls in the new worker ownership code and test-only callers still unpacking the old four-field router tuple. I stopped that run. Fixed both expects using the repository's existing `PoisonError::into_inner` pattern and mechanically updated all three external test callers to ignore the retained fifth Auth field. No recovery behavior changed for these integration corrections. Latest exact source pushed as marked WIP above the atomic countdown commit `52ad241ae`. Driver `~/j102-final2.sh` starts with workspace clippy across all targets, then the 12 regressions and full `scripts/verify fast`; log `~/j102-final2.log`, compute01. Both queues checked before launch. Prior stopped run retained as `~/j102-final-stopped-clippy.log`. No gate failure is being waived.

kayg commented

2026-11-02 19:04:20 +00:00

Author

Owner

The next full gate encountered a separate fixture failure: infraternal-agent test login_binary::tests::anonymous_kernel_image_can_be_published_after_unlink, line 484, failed to spawn its private copied executable with OS error 26, ETXTBSY (“Text file busy”). Agent sources are unchanged by J102. I stopped the run rather than spend time on static builds after a known red test row. No failure is waived.

The complete gate is being retried with RUST_TEST_THREADS=1 so external-executable fixtures do not race other Rust tests in the same process. All tests remain enabled. Driver ~/j102-final3.sh first rechecks that exact fixture, then runs scripts/verify fast; log ~/j102-final3.log, compute01. Both VM queues were checked. Existing corrected-source evidence remains: workspace clippy across all targets passed; all 12 recovery regressions passed; CLI exit-code test passed both isolated and in the full run. Exact source is already pushed as WIP 645377958 above atomic countdown parent 52ad241ae.

The next full gate encountered a separate fixture failure: `infraternal-agent` test `login_binary::tests::anonymous_kernel_image_can_be_published_after_unlink`, line 484, failed to spawn its private copied executable with OS error 26, `ETXTBSY` (“Text file busy”). Agent sources are unchanged by J102. I stopped the run rather than spend time on static builds after a known red test row. No failure is waived. The complete gate is being retried with `RUST_TEST_THREADS=1` so external-executable fixtures do not race other Rust tests in the same process. All tests remain enabled. Driver `~/j102-final3.sh` first rechecks that exact fixture, then runs `scripts/verify fast`; log `~/j102-final3.log`, compute01. Both VM queues were checked. Existing corrected-source evidence remains: workspace clippy across all targets passed; all 12 recovery regressions passed; CLI exit-code test passed both isolated and in the full run. Exact source is already pushed as WIP `645377958` above atomic countdown parent `52ad241ae`.

kayg commented

2026-11-02 19:31:23 +00:00

Author

Owner

J102 round 3 progress: control and compute01 source checksums match. The serialized full scripts/verify fast run has passed the recovery/checkpoint/socket regressions and the x86 static build (22,466,288 bytes, +48,032 versus baseline). Arm64 static compilation is running. No gate waiver; waiting for the full green result before replacing the checkpoint WIP with its atomic commit. Log: compute01 ~/j102-final3.log.

J102 round 3 progress: control and compute01 source checksums match. The serialized full `scripts/verify fast` run has passed the recovery/checkpoint/socket regressions and the x86 static build (22,466,288 bytes, +48,032 versus baseline). Arm64 static compilation is running. No gate waiver; waiting for the full green result before replacing the checkpoint WIP with its atomic commit. Log: compute01 `~/j102-final3.log`.

kayg commented

2026-11-02 19:36:04 +00:00

Author

Owner

J102 round 3 complete. Branch fix-recovery-restart, head 77616905a01eebf2917617b49d5aa719c15d7e15, rebased onto requested dev 99668d8ab.

Separate atomic commits:

52ad241ae: recovery page uses server-confirmed readiness, refreshing on visibility, zero and every 60 seconds. Local countdown cannot enable recovery.
77616905a: measured monotonic recovery progress is checkpointed every 60 seconds while active and flushed on SIGINT/SIGTERM. No idle timer when no request is active. Under successful saves, abrupt restart loses at most about a minute of measured uptime plus downtime; storage/scheduler delays remain fail-closed. No unmeasured time is credited.

Both changes have failing-first evidence. The full gate includes all 12 recovery regressions and the completion-before-sign-out socket regression. Additional validation: 14 client validation tests, typecheck with 0 errors/warnings, 16 production-build recovery browser tests. Screenshot attached for independent visual review, including alignment assertion within 0.5px. Control/VM source checksums match. Migration remains 0037_recovery_elapsed.sql; no migration renumbering.

The final full scripts/verify fast is entirely green on compute01 with RUST_TEST_THREADS=1, no test filters and no inherited-failure waiver. Earlier stopped attempts and their findings remain documented above. Verbatim Rust gate and test summaries follow; the complete unabridged gate output is attached.

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| test                      | PASS         |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+


 3 pass
 0 fail
 3 expect() calls
Ran 3 tests across 1 file. [622.00ms]


 2 pass
 0 fail
 14 expect() calls
Ran 2 tests across 1 file. [243.00ms]


 2 pass
 0 fail
 121 expect() calls
Ran 2 tests across 1 file. [808.00ms]


 496 pass
 0 fail
 22637 expect() calls
Ran 496 tests across 80 files. [19.16s]


 107 pass
 0 fail
 417 expect() calls
Ran 107 tests across 15 files. [406.00ms]


 17 pass
 0 fail
 1794 expect() calls
Ran 17 tests across 4 files. [403.00ms]


 39 pass
 0 fail
 59970 expect() calls
Ran 39 tests across 10 files. [6.37s]

FINAL_VERIFY_EXIT=0

Static sizes: x86_64 22,466,288 bytes (+48,032 versus 22,418,256); arm64 20,488,072 bytes (+34,792 versus 20,453,280).

How to test: start recovery, hide/return to the page, and confirm readiness follows server status. Restart during the wait and confirm only saved measured uptime survives. Complete recovery and confirm the signed-in device sees completion before sign-out.

Evidence:

J102 round 3 complete. Branch `fix-recovery-restart`, head `77616905a01eebf2917617b49d5aa719c15d7e15`, rebased onto requested dev `99668d8ab`. Separate atomic commits: - `52ad241ae`: recovery page uses server-confirmed readiness, refreshing on visibility, zero and every 60 seconds. Local countdown cannot enable recovery. - `77616905a`: measured monotonic recovery progress is checkpointed every 60 seconds while active and flushed on SIGINT/SIGTERM. No idle timer when no request is active. Under successful saves, abrupt restart loses at most about a minute of measured uptime plus downtime; storage/scheduler delays remain fail-closed. No unmeasured time is credited. Both changes have failing-first evidence. The full gate includes all 12 recovery regressions and the completion-before-sign-out socket regression. Additional validation: 14 client validation tests, typecheck with 0 errors/warnings, 16 production-build recovery browser tests. Screenshot attached for independent visual review, including alignment assertion within 0.5px. Control/VM source checksums match. Migration remains `0037_recovery_elapsed.sql`; no migration renumbering. The final full `scripts/verify fast` is entirely green on compute01 with `RUST_TEST_THREADS=1`, no test filters and no inherited-failure waiver. Earlier stopped attempts and their findings remain documented above. Verbatim Rust gate and test summaries follow; the complete unabridged gate output is attached. ```text +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | test | PASS | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | static builds | PASS | +---------------------------+--------------+ ``` ```text 3 pass 0 fail 3 expect() calls Ran 3 tests across 1 file. [622.00ms] 2 pass 0 fail 14 expect() calls Ran 2 tests across 1 file. [243.00ms] 2 pass 0 fail 121 expect() calls Ran 2 tests across 1 file. [808.00ms] 496 pass 0 fail 22637 expect() calls Ran 496 tests across 80 files. [19.16s] 107 pass 0 fail 417 expect() calls Ran 107 tests across 15 files. [406.00ms] 17 pass 0 fail 1794 expect() calls Ran 17 tests across 4 files. [403.00ms] 39 pass 0 fail 59970 expect() calls Ran 39 tests across 10 files. [6.37s] FINAL_VERIFY_EXIT=0 ``` Static sizes: x86_64 22,466,288 bytes (+48,032 versus 22,418,256); arm64 20,488,072 bytes (+34,792 versus 20,453,280). How to test: start recovery, hide/return to the page, and confirm readiness follows server status. Restart during the wait and confirm only saved measured uptime survives. Complete recovery and confirm the signed-in device sees completion before sign-out. Evidence: - [j102-r3-verify-fast-verbatim.txt](https://git.kayg.org/attachments/7359273c-c1dd-4801-a142-5405a14ccd7d) - [j102-r3-final-green.txt](https://git.kayg.org/attachments/a58fee75-296d-4654-81c6-a4915a3b9268) - [j102-r3-red-confirmed.txt](https://git.kayg.org/attachments/4f6afec0-33d3-4838-a781-d8f323f287c9) - [j102-r3-range-diff.txt](https://git.kayg.org/attachments/3a20ac76-24b5-447a-a1aa-2709315d4613) - [j102-r3-server-status-unavailable-2x.png](https://git.kayg.org/attachments/47c81b0a-ea9f-4ef3-8aea-92e6cbfb9a60)

j102-r3-verify-fast-verbatim.txt

640 KiB

j102-r3-final-green.txt

642 KiB

j102-r3-red-confirmed.txt

11 KiB

j102-r3-range-diff.txt

982 B

j102-r3-server-status-unavailable-2x.png

122 KiB

kayg referenced this issue

2026-11-02 19:41:23 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 19:41:34 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 19:42:36 +00:00

CLI: use sign-in API keys for remote actions (D311) #303

kayg referenced this issue

2026-11-02 19:46:04 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 20:46:49 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-02 22:43:38 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-03 00:43:50 +00:00

Machine trust: self-renewal and 7-day key rotation #285

kayg referenced this issue

2026-11-03 02:37:12 +00:00

Own context menu and swipe actions on rows (Views tree first) #242

kayg referenced this issue

2026-11-03 02:37:12 +00:00

Mockups: sidebar rename/remove and privileged access consent #215

kayg referenced this issue

2026-11-03 02:37:13 +00:00

Sidebar account pill drifts up from the bottom; rename Mesh hosts by double-tap #259

kayg referenced this issue

2026-11-04 10:22:03 +00:00

Engineering practices proposal #344

kayg commented

2026-11-04 17:04:46 +00:00

Author

Owner

J210 owner-authorized housekeeping: closing this completed scope because its work is merged on dev in 92ee8d5ea. Verified the merge commit is an ancestor of current origin/dev. The implementation reports and evidence remain in this issue.

J210 owner-authorized housekeeping: closing this completed scope because its work is merged on dev in [92ee8d5ea](https://git.kayg.org/kayg/infraternal/commit/92ee8d5ea33aa8c88e6ad7d9a57f0dd757656bb9). Verified the merge commit is an ancestor of current origin/dev. The implementation reports and evidence remain in this issue.

kayg closed this issue

2026-11-04 17:04:46 +00:00

kayg referenced this issue

2026-11-04 17:07:06 +00:00

J210: overnight housekeeping, 2026-11-04 #350

Rows
Columns

J102: give each restarted recovery its own safety wait #309