Flaky: real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness (~1/15 on dev)

kayg commented

2026-11-06 17:20:02 +00:00

Owner

Flaky agent test: real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness fails about 1 in 15 on plain dev.

J314 (#394 comment 82622) ran it on dev 68ee8d9a5: 5 of 5 passed, then 37 of 40 passed. The 3 failures were resource_unavailable at agent_daemon.rs:199, the same error J303 saw. Logs are at compute01 ~claude/j314/flaky.log and fail.*. Find the cause; the rule is no timing guesses (#334).

Flaky agent test: `real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness` fails about 1 in 15 on plain dev. J314 (#394 comment 82622) ran it on dev 68ee8d9a5: 5 of 5 passed, then 37 of 40 passed. The 3 failures were `resource_unavailable` at `agent_daemon.rs:199`, the same error J303 saw. Logs are at compute01 `~claude/j314/flaky.log` and `fail.*`. Find the cause; the rule is no timing guesses (#334).

kayg referenced this issue

2026-11-06 17:20:21 +00:00

Flaky: real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness (about 1 in 15 on dev) #405

kayg commented

2026-11-08 11:45:51 +00:00

Author

Owner

J365 starting installer-suite investigation on fix-install-suite-404, base f6f62a875 (origin/dev). The current issue body concerns a different daemon flake; this work follows the assigned installer scope. Plan: inspect launcher/fixtures, measure full-suite failures and order under gate parallelism, create a deterministic concurrent-fixture red test, fix the evidenced cause, run 20 full suites, then verify quick on dev-02 and fast on compute01. Estimate: 90 minutes excluding slot queues and cold builds. No retries, merge, or deployment. Evidence will be retained under ~/j365 on the VMs.

J365 starting installer-suite investigation on fix-install-suite-404, base f6f62a875 (origin/dev). The current issue body concerns a different daemon flake; this work follows the assigned installer scope. Plan: inspect launcher/fixtures, measure full-suite failures and order under gate parallelism, create a deterministic concurrent-fixture red test, fix the evidenced cause, run 20 full suites, then verify quick on dev-02 and fast on compute01. Estimate: 90 minutes excluding slot queues and cold builds. No retries, merge, or deployment. Evidence will be retained under ~/j365 on the VMs.

kayg commented

2026-11-08 11:53:07 +00:00

Author

Owner

J365 reproduction setup: baseline f6f62a875, compute01 nproc=16, cargo -j4 with default libtest parallelism (16), through the inspected build-slot launcher and Nix. State roots are already fixture-specific (exclusive mkdir with random 12-digit suffix); no shared-lock collision found by inspection. The gate builds recovery-helper before cargo test; an initial focused-command setup omitted that helper and produced exit 127, so those invalid runs are excluded. Corrected baseline logs: compute01 ~/j365/baseline-ready-*.log; helper build ~/j365/helper-build.log. First corrected full suite: 107 passed, 0 failed, 2 ignored, 14.79s; host load average was 32.33.

J365 reproduction setup: baseline f6f62a875, compute01 nproc=16, cargo -j4 with default libtest parallelism (16), through the inspected build-slot launcher and Nix. State roots are already fixture-specific (exclusive mkdir with random 12-digit suffix); no shared-lock collision found by inspection. The gate builds recovery-helper before cargo test; an initial focused-command setup omitted that helper and produced exit 127, so those invalid runs are excluded. Corrected baseline logs: compute01 ~/j365/baseline-ready-*.log; helper build ~/j365/helper-build.log. First corrected full suite: 107 passed, 0 failed, 2 ignored, 14.79s; host load average was 32.33.

kayg commented

2026-11-08 11:56:55 +00:00

Author

Owner

J365 hypothesis update: 8/8 corrected baseline suites currently pass, including 16-thread runs at host load >60 (~/j365/load.log and baseline-ready-*.log). Found a fixture lifecycle mismatch to test: Local::run detaches Input::Transfer through setsid --fork --wait, but runs Input::Negotiated directly in process::execute’s client group. That helper deliberately SIGKILLs the client group before reaping on successful completion, so negotiated fixture leases can be killed although real remote SSH leases are outside that group. This is a hypothesis, not yet the cause of the reported flake. Continuing the unmodified baseline loop before a deterministic concurrent-fixture probe.

kayg commented

2026-11-08 11:58:25 +00:00

Author

Owner

J365 second hypothesis: retire_setup_lease treats matching PID/start ticks plus a missing fd/7 as exit 73. A zombie still has /proc/PID/stat but cannot hold any descriptor, so an expired/killed lease pending reaping can be reported as running. I will test this using an explicit parent/reaper handshake, alongside concurrent isolated fixtures. This may also expose a production stale-holder edge; it is not yet established as the original suite failure. Unmodified sweeps remain green so far (11 completed).

kayg commented

2026-11-08 12:01:58 +00:00

Author

Owner

J365 reproduced on unmodified f6f62a875: baseline-ready run 19 failed, runs 1–18 passed. Failure order: only restart_recovers_partial_stage_retirement, its kind case, at original install.rs:3622 during the check after partial cleanup; exit_status Some(73). Suite result: 106 passed; 1 failed; 2 ignored; finished in 15.01s. Baseline failure rate 1/19 suites (5.26%); 1/2033 active test executions. Full completion order and error: compute01 ~/j365/baseline-ready-19.log; every run and status: ~~/j365/baseline-ready-summary.log; host load at run start 40.70 (~~/j365/load.log). Next: trace which lock refuses, then validate the concurrent deterministic regression before implementing.

J365 reproduced on unmodified f6f62a875: baseline-ready run 19 failed, runs 1–18 passed. Failure order: only restart_recovers_partial_stage_retirement, its kind case, at original install.rs:3622 during the check after partial cleanup; exit_status Some(73). Suite result: 106 passed; 1 failed; 2 ignored; finished in 15.01s. Baseline failure rate 1/19 suites (5.26%); 1/2033 active test executions. Full completion order and error: compute01 ~/j365/baseline-ready-19.log; every run and status: ~/j365/baseline-ready-summary.log; host load at run start 40.70 (~/j365/load.log). Next: trace which lock refuses, then validate the concurrent deterministic regression before implementing.

kayg commented

2026-11-08 12:05:02 +00:00

Author

Owner

J365 trace narrowed the reproduced refusal: a second full sweep failed the same test (both case); the appended source-location diagnostics identify previous_attempt.sh:512, scan_holders -> hidden_holder, for /proc/1055118. This is the process-liveness scan, not direct contention on another fixture’s setup-lock path. Evidence compute01 ~/j365/trace-1.log and ~/j365/refusals.log. Adding a diagnostic at each install_process match to distinguish service scope, terminal scope and installation path before making any change.

kayg commented

2026-11-08 12:11:24 +00:00

Author

Owner

J365 progress checkpoint: no production changes chosen yet. Baseline 1/19 suites failed; first diagnostic sweep also failed at hidden_holder (previous_attempt.sh:512). A further 20 diagnostic suites passed under lower host load. The next sweep tests the long scratch-root override/private /tmp fallback and is queued on compute01 (PID 1656725); dev-02 is also busy with two queued jobs. Source diagnostics are preserved in compute01 ~/j365/instrumented/, alongside logs. An uncommitted zombie-lease hypothesis probe was saved outside the branch at control /home/kayg/.cache/j365/zombie-probe.patch and removed from the worktree because it does not yet explain the observed scan failure. Branch remains f6f62a875; exactly one target, compute01 ~/worktrees/j365/target. Continuing in the foreground.

J365 progress checkpoint: no production changes chosen yet. Baseline 1/19 suites failed; first diagnostic sweep also failed at hidden_holder (previous_attempt.sh:512). A further 20 diagnostic suites passed under lower host load. The next sweep tests the long scratch-root override/private /tmp fallback and is queued on compute01 (PID 1656725); dev-02 is also busy with two queued jobs. Source diagnostics are preserved in compute01 ~/j365/instrumented/, alongside logs. An uncommitted zombie-lease hypothesis probe was saved outside the branch at control /home/kayg/.cache/j365/zombie-probe.patch and removed from the worktree because it does not yet explain the observed scan failure. Branch remains f6f62a875; exactly one target, compute01 ~/worktrees/j365/target. Continuing in the foreground.

kayg commented

2026-11-08 12:21:43 +00:00

Author

Owner

J365 additional fixture finding: Fixture::drop embeds setup_lock.sh alone, but retire_setup_lease -> read_lease_birth calls process_start_ticks from setup_guard.sh. Without that definition the conditional skips lease termination, its flock -w 2 times out, and Drop ignores the failure and deletes the fixture tree. In the first trace, lease PIDs 1034802, 1035994 and 1042024 were still sleeping with fd/7 pointing at their deleted fixture stage directories after the suite finished (~/j365/refusals.log records each timeout; source snapshot ~/j365/instrumented/setup_lock.sh). This confirms a fixture leak, but unique directory inodes mean I have not yet linked it to the original false holder. No fix selected from this finding alone. The next diagnostic run remains queued on compute01.

kayg commented

2026-11-08 12:26:37 +00:00

Author

Owner

J365 ROOT CAUSE CONFIRMED: the first /tmp-fallback sweep reproduced the reported multi-test pattern. Every false refusal names the SAME unrelated PID 1822819 in cgroup 0::/user.slice/user-1500.slice/user@1500.service/app.slice/infraternal-session-3ab159ed12af4cbaaa1006c96d2d489b.scope, while MATCH_SCOPE shows distinct isolated fixture roots (/tmp/is-/i/.local/share/infraternal). scan_holders -> hidden_holder -> install_process treats any same-uid infraternal-session-*.scope as belonging to this install. Thus concurrent host-login tests contaminate independent installer fixtures despite unique state roots. Evidence: compute01 ~/j365/trace-tmp-1.log and ~/j365/refusals.log (MATCH_SCOPE plus exact cgroup). This is also a production attribution bug for a rootless install alongside a root-managed login of the same account. Next: bind scope liveness to installation-owned session evidence, with a deterministic concurrent-fixture red test; retain genuine live-holder refusal. No retries or timeout increases.

J365 ROOT CAUSE CONFIRMED: the first /tmp-fallback sweep reproduced the reported multi-test pattern. Every false refusal names the SAME unrelated PID 1822819 in cgroup 0::/user.slice/user-1500.slice/user@1500.service/app.slice/infraternal-session-3ab159ed12af4cbaaa1006c96d2d489b.scope, while MATCH_SCOPE shows distinct isolated fixture roots (/tmp/is-*/i*/.local/share/infraternal). scan_holders -> hidden_holder -> install_process treats any same-uid infraternal-session-*.scope as belonging to this install. Thus concurrent host-login tests contaminate independent installer fixtures despite unique state roots. Evidence: compute01 ~/j365/trace-tmp-1.log and ~/j365/refusals.log (MATCH_SCOPE plus exact cgroup). This is also a production attribution bug for a rootless install alongside a root-managed login of the same account. Next: bind scope liveness to installation-owned session evidence, with a deterministic concurrent-fixture red test; retain genuine live-holder refusal. No retries or timeout increases.

kayg commented

2026-11-08 12:30:32 +00:00

Author

Owner

J365 matching baseline pattern now complete: ~/j365/trace-tmp-1.log reports 98 passed; 9 failed; 2 ignored, 303.28s, exit 101. Failure order is retained in that log; summary names earlier_binary_cache_is_refreshed_before_owned_recovery, earlier_control_socket_receipt_recovers_before_a_new_setup, known_failed_setup_finishes_without_a_second_confirmation, restart_recovers_partial_stage_retirement, review_restart_recovers_cleanup_after_base_retirement, review_restart_recovers_cleanup_after_owner_retirement, stale_previous_attempt_socket_can_be_retired_without_manual_cleanup, start_fresh_resumes_the_same_attempt_after_owner_confirmation, turning_off_paused_host_provisioning_preserves_earlier_leftovers. The last two waited for the 300s hang guard after the false setup-running result. Combined unmodified/diagnostic evidence: original baseline 1/19 failures; same bug reproduced twice more with source tracing, latest 9/107 active tests failed. Root cause is unrelated same-user session scopes, NOT UUID/path collision. Deterministic concurrent-fixture red test is now building on the free dev-02 slot; compute01 is fully occupied again.

kayg commented

2026-11-08 12:33:16 +00:00

Author

Owner

J365 deterministic RED proven on dev-02: concurrent_fixtures_attribute_hidden_session_scopes_to_their_own_install fails with “another install’s hidden scope blocked this fixture” and exit_status Some(73). Its concurrent first fixture correctly refuses its own scope; the second incorrectly refuses that SAME process despite a separate root and no endpoint there. The process is genuinely non-dumpable (prctl), socket/journal files are real; only the cgroup read is redirected to a controlled per-PID fixture view. Both child and installs are cleaned before asserting, including red runs. Evidence dev-02 ~/j365/red.log, ~/j365/red-status.log: RED_EXIT=101. An initial probe hit sun_path before exercising the bug; it was corrected to bind relative to its directory, matching existing fixtures. Implementing only the scope-to-endpoint binding now.

kayg commented

2026-11-08 12:37:10 +00:00

Author

Owner

J365: deterministic red/green proven. The same concurrent-fixture regression fails on the original classifier with exit 73 for fixture B and passes with endpoint-bound scope ownership. Existing independent ownership-shell oracle also passes (2/2, 81.34s). Evidence: dev-02 ~/j365/red.log, red-status.log, green.log, ownership-green.log, green-status.log. Implementation commit b68edbfff pushed; integrated current origin/dev abad8c6c0 before final validation. No retries or test serialization added. Next: quick on dev-02, then 20 default-parallel install suites and fast on compute01.

J365: deterministic red/green proven. The same concurrent-fixture regression fails on the original classifier with exit 73 for fixture B and passes with endpoint-bound scope ownership. Existing independent ownership-shell oracle also passes (2/2, 81.34s). Evidence: dev-02 ~/j365/red.log, red-status.log, green.log, ownership-green.log, green-status.log. Implementation commit b68edbfff pushed; integrated current origin/dev abad8c6c0 before final validation. No retries or test serialization added. Next: quick on dev-02, then 20 default-parallel install suites and fast on compute01.

kayg commented

2026-11-08 12:39:57 +00:00

Author

Owner

J365 real-environment proof: created a unique terminal scope through the real user systemd manager on dev-02, running a non-dumpable process. Two isolated installer ownership checks ran concurrently using unmodified /proc cgroup reads. Baseline exits [73, 73]; fixed exits [73, 0]. The owning installation remains protected and the unrelated installation reports stale recovery normally. Artifact: dev-02 ~/j365/real-scope-proof.py and real-scope-proof.log (REAL_SCOPE_PROOF=PASS), actual scope 414b3f4f448d49ae98cf6f344505c6ad, pid 3524715. Exact child was reaped and private fixture directory removed. No deployment. Quick first stopped on a missing doc comment in the adjacent existing test; fixed in 05aef6860 and rerunning quick.

J365 real-environment proof: created a unique terminal scope through the real user systemd manager on dev-02, running a non-dumpable process. Two isolated installer ownership checks ran concurrently using unmodified /proc cgroup reads. Baseline exits [73, 73]; fixed exits [73, 0]. The owning installation remains protected and the unrelated installation reports stale recovery normally. Artifact: dev-02 ~/j365/real-scope-proof.py and real-scope-proof.log (REAL_SCOPE_PROOF=PASS), actual scope 414b3f4f448d49ae98cf6f344505c6ad, pid 3524715. Exact child was reaped and private fixture directory removed. No deployment. Quick first stopped on a missing doc comment in the adjacent existing test; fixed in 05aef6860 and rerunning quick.

kayg commented

2026-11-08 12:40:53 +00:00

Author

Owner

J365: verify quick passed on dev-02 at head 05aef6860d (124 changed-package unit tests passed, 6 ignored; workspace clippy passed). Artifacts ~/j365/quick-final.log and quick-final-status.log, QUICK_EXIT=0. The requested 20-suite proof is now foreground-queued on compute01 through build-slot, default 16 test threads, -j4, INFRATERNAL_TEST_STATE_ROOT=/tmp/j365-gate-state-long-override (same private short-root fallback condition that reproduced 9 false refusals). Script ~/j365/after-sweep.sh writes after-N.log, after-summary.log, after-load.log, after-head.log. All existing slots busy; no other J365 slot held.

J365: verify quick passed on dev-02 at head 05aef6860d (124 changed-package unit tests passed, 6 ignored; workspace clippy passed). Artifacts ~/j365/quick-final.log and quick-final-status.log, QUICK_EXIT=0. The requested 20-suite proof is now foreground-queued on compute01 through build-slot, default 16 test threads, -j4, INFRATERNAL_TEST_STATE_ROOT=/tmp/j365-gate-state-long-override (same private short-root fallback condition that reproduced 9 false refusals). Script ~/j365/after-sweep.sh writes after-N.log, after-summary.log, after-load.log, after-head.log. All existing slots busy; no other J365 slot held.

kayg commented

2026-11-08 12:46:14 +00:00

Author

Owner

J365 proof sweep complete at head 05aef6860d: AFTER_SUITES=20 FAILED=0. Each run: 108 passed, 0 failed, 2 ignored. Total 2,160 active executions; default 16 test threads, cargo -j4, concurrent shared gate load; no retries. Baseline unmodified: 1/19 full suites failed (5.26%); instrumented reproduction also captured 9/107 failures and the same unrelated hidden scope attributed to distinct fixture roots. Evidence on compute01: ~/j365/after-summary.log, after-{1..20}.log (completion order), after-load.log, after-head.log; baseline-ready-summary.log, baseline-ready-{1..19}.log; trace-tmp-1.log, refusals.log. Proceeding to requested verify fast on compute01.

J365 proof sweep complete at head 05aef6860da7d171975b9ecb483924571410ed6b: AFTER_SUITES=20 FAILED=0. Each run: 108 passed, 0 failed, 2 ignored. Total 2,160 active executions; default 16 test threads, cargo -j4, concurrent shared gate load; no retries. Baseline unmodified: 1/19 full suites failed (5.26%); instrumented reproduction also captured 9/107 failures and the same unrelated hidden scope attributed to distinct fixture roots. Evidence on compute01: ~/j365/after-summary.log, after-{1..20}.log (completion order), after-load.log, after-head.log; baseline-ready-summary.log, baseline-ready-{1..19}.log; trace-tmp-1.log, refusals.log. Proceeding to requested verify fast on compute01.

kayg commented

2026-11-08 13:27:49 +00:00

Author

Owner

J365 gate progress at unchanged head 05aef6860d: compute01 verify fast has passed workspace clippy, workspace tests, the installer suite (108 passed / 0 failed / 2 ignored, 5.52s inside the gate), ownership-shell oracle, SSH inode-reuse proof, shim process-per-user, dependency/doc checks, all 5 selected privileged SSH tests, release-probe exclusion and fuzz-reporter checks. Static release builds now running. Foreground session remains active; no other J365 slot held. Log ~/j365/fast.log, own target ~/worktrees/j365/target. Dev-02 target already removed; its evidence logs retained.

J365 gate progress at unchanged head 05aef6860da7d171975b9ecb483924571410ed6b: compute01 verify fast has passed workspace clippy, workspace tests, the installer suite (108 passed / 0 failed / 2 ignored, 5.52s inside the gate), ownership-shell oracle, SSH inode-reuse proof, shim process-per-user, dependency/doc checks, all 5 selected privileged SSH tests, release-probe exclusion and fuzz-reporter checks. Static release builds now running. Foreground session remains active; no other J365 slot held. Log ~/j365/fast.log, own target ~/worktrees/j365/target. Dev-02 target already removed; its evidence logs retained.

kayg commented

2026-11-08 13:52:39 +00:00

Author

Owner

J365 gate finding: verify fast exited 1 after ALL 24 Rust rows passed, including workspace tests, privileged integration and both static builds. Frontend docs-build.test.ts:185 fails doc.js stays small: Expected <12000, Received 13408. Proven unchanged on detached, unmodified origin/dev abad8c6c0c through build-slot: same test fails with identical 13408 (16 pass, 1 fail). Task branch restored at 05aef6860d. No frontend files differ from origin/dev. Artifacts compute01 ~/j365/fast.log, fast-status.log (FAST_EXIT=1), baseline-docs.log, baseline-docs-status.log (BASELINE_DOCS_EXIT=1), baseline-docs-head.log. This pre-existing frontend gate defect is outside the SSH ownership fix; no gate retry or limit change.

J365 gate finding: verify fast exited 1 after ALL 24 Rust rows passed, including workspace tests, privileged integration and both static builds. Frontend docs-build.test.ts:185 fails doc.js stays small: Expected <12000, Received 13408. Proven unchanged on detached, unmodified origin/dev abad8c6c0ca8c9eaeea66f247c2318b3691463b9 through build-slot: same test fails with identical 13408 (16 pass, 1 fail). Task branch restored at 05aef6860da7d171975b9ecb483924571410ed6b. No frontend files differ from origin/dev. Artifacts compute01 ~/j365/fast.log, fast-status.log (FAST_EXIT=1), baseline-docs.log, baseline-docs-status.log (BASELINE_DOCS_EXIT=1), baseline-docs-head.log. This pre-existing frontend gate defect is outside the SSH ownership fix; no gate retry or limit change.

kayg commented

2026-11-08 13:53:49 +00:00

Author

Owner

J365 finished implementation on fix-install-suite-404, pushed head 05aef6860da7d171975b9ecb483924571410ed6b. Current dev integrated before final validation. No merge to dev, deployment, or issue closure.

Cause: a production liveness-attribution bug. session_scope_process treated every same-uid terminal scope as this installation, even when its endpoint belonged to a different state root. Tracing attributed the same unrelated scope PID to multiple independent fixture paths. The fix binds a canonical scope session id to an owned endpoint in either supported state layout. It preserves refusal for this installation's own hidden process. No retries, test serialization, timeout increase, or new dependency.

Proof:

Unmodified baseline: 1/19 complete suites failed (5.26%), 107 active tests each. Failure and completion order: compute01 ~/j365/baseline-ready-{1..19}.log, summary baseline-ready-summary.log.
Diagnostic long-root reproduction: 9/107 failed, including the reported stream timeouts. trace-tmp-1.log, trace-tmp-summary.log, refusals.log, diagnostic source snapshots in instrumented/.
Deterministic concurrent two-fixture regression: original code incorrectly refuses fixture B with 73; fixed code passes while fixture A still refuses 73. dev-02 ~/j365/red.log, red-status.log, green.log, green-status.log.
Actual systemd user-scope proof with a non-dumpable child and real /proc cgroups: baseline [73,73], fixed [73,0], exact child reaped. dev-02 ~/j365/real-scope-proof.py, real-scope-proof.log.
Fixed: 20/20 complete suites passed, 0 failed suites / 2,160 active executions. Default 16 test threads, cargo -j4, build-slot foreground, shared gate load, same long-root setting as the nine-failure reproduction. compute01 ~/j365/after-{1..20}.log, after-summary.log, after-load.log, after-head.log, after-sweep.sh.
Existing ownership oracle passed on dev-02 (2/2) and in the final compute01 gate. dev-02 ~/j365/ownership-green.log and compute01 ~/j365/fast.log.
verify quick on dev-02: QUICK_EXIT=0, 124 changed-package unit tests passed, 6 ignored. ~/j365/quick-final.log, quick-final-status.log.
verify fast on compute01: all 24 Rust rows passed, including install (108 passed, 0 failed, 2 ignored), privileged integration and static builds. Overall FAST_EXIT=1 on an unrelated existing frontend docs size assertion. Reproduced once on detached unmodified origin/dev abad8c6c0ca8c9eaeea66f247c2318b3691463b9: BASELINE_DOCS_EXIT=1, identical Expected <12000 / Received 13408. ~/j365/baseline-docs.log, baseline-docs-status.log, baseline-docs-head.log. No frontend files differ from dev. The gate stopped in that frontend row, so later frontend checks did not run.

Gate output excerpt verbatim (~/j365/fast-summary.log; full output fast.log):

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| migration names           | PASS         |
| migration tooling tests   | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| Traefik component manifest | PASS         |
| release component publication | PASS         |
| test                      | PASS         |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| privileged integration    | PASS         |
| probe release exclusion   | PASS         |
| nightly fuzz reporter     | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

PT_INTERP: absent; size: 28171984 bytes; baseline 22418256; delta +5753728 bytes (report only, D243 amended 2026-11-01)
PT_INTERP: absent; size: 25305312 bytes; baseline 20453280; delta +4852032 bytes (report only, D243 amended 2026-11-01)

FAST_EXIT=1

Frontend failure excerpt (verbatim):
error: doc.js stays small

Expected: < 12000
Received: 13408

      at <anonymous> (/home/claude/worktrees/j365/apps/tracer/docs-build.test.ts:185:49)
(fail) the real tree builds from the real folder, in the sections of D317 [272.23ms]

The size deltas above are against the gate's configured historical baselines, not an attribution of those deltas to this small shell change. Binary hashes are in compute01 ~/j365/static-artifacts.log. Suite wall times are in suite-times.log; differing shared load makes them diagnostic rather than a performance benchmark.

Separate finding retained for follow-up: Fixture::drop omits setup_guard.sh when sourcing setup_lock.sh, so process_start_ticks is undefined and some leases outlive removed private roots. It did not cause the observed cross-install refusals, and is not bundled into this fix. The earlier finding comment records the exact PIDs and deleted fd paths.

Evidence retained outside targets under ~/j365 on both VMs and copied to control /home/kayg/.cache/j365/. Build targets on both VMs removed; no own fixture orphans remain. Ready for the orchestrator's independent review; the pre-existing frontend gate defect needs separate handling.

J365 finished implementation on `fix-install-suite-404`, pushed head `05aef6860da7d171975b9ecb483924571410ed6b`. Current dev integrated before final validation. No merge to dev, deployment, or issue closure. Cause: a production liveness-attribution bug. `session_scope_process` treated every same-uid terminal scope as this installation, even when its endpoint belonged to a different state root. Tracing attributed the same unrelated scope PID to multiple independent fixture paths. The fix binds a canonical scope session id to an owned endpoint in either supported state layout. It preserves refusal for this installation's own hidden process. No retries, test serialization, timeout increase, or new dependency. Proof: - Unmodified baseline: 1/19 complete suites failed (5.26%), 107 active tests each. Failure and completion order: compute01 `~/j365/baseline-ready-{1..19}.log`, summary `baseline-ready-summary.log`. - Diagnostic long-root reproduction: 9/107 failed, including the reported stream timeouts. `trace-tmp-1.log`, `trace-tmp-summary.log`, `refusals.log`, diagnostic source snapshots in `instrumented/`. - Deterministic concurrent two-fixture regression: original code incorrectly refuses fixture B with 73; fixed code passes while fixture A still refuses 73. dev-02 `~/j365/red.log`, `red-status.log`, `green.log`, `green-status.log`. - Actual systemd user-scope proof with a non-dumpable child and real /proc cgroups: baseline [73,73], fixed [73,0], exact child reaped. dev-02 `~/j365/real-scope-proof.py`, `real-scope-proof.log`. - Fixed: 20/20 complete suites passed, 0 failed suites / 2,160 active executions. Default 16 test threads, cargo -j4, build-slot foreground, shared gate load, same long-root setting as the nine-failure reproduction. compute01 `~/j365/after-{1..20}.log`, `after-summary.log`, `after-load.log`, `after-head.log`, `after-sweep.sh`. - Existing ownership oracle passed on dev-02 (2/2) and in the final compute01 gate. dev-02 `~/j365/ownership-green.log` and compute01 `~/j365/fast.log`. - `verify quick` on dev-02: QUICK_EXIT=0, 124 changed-package unit tests passed, 6 ignored. `~/j365/quick-final.log`, `quick-final-status.log`. - `verify fast` on compute01: all 24 Rust rows passed, including install (108 passed, 0 failed, 2 ignored), privileged integration and static builds. Overall FAST_EXIT=1 on an unrelated existing frontend docs size assertion. Reproduced once on detached unmodified origin/dev `abad8c6c0ca8c9eaeea66f247c2318b3691463b9`: BASELINE_DOCS_EXIT=1, identical Expected <12000 / Received 13408. `~/j365/baseline-docs.log`, `baseline-docs-status.log`, `baseline-docs-head.log`. No frontend files differ from dev. The gate stopped in that frontend row, so later frontend checks did not run. Gate output excerpt verbatim (`~/j365/fast-summary.log`; full output `fast.log`): ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | Nix Cargo vendoring | PASS | | fmt | PASS | | migration names | PASS | | migration tooling tests | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | Traefik component manifest | PASS | | release component publication | PASS | | test | PASS | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | privileged integration | PASS | | probe release exclusion | PASS | | nightly fuzz reporter | PASS | | static builds | PASS | +---------------------------+--------------+ PT_INTERP: absent; size: 28171984 bytes; baseline 22418256; delta +5753728 bytes (report only, D243 amended 2026-11-01) PT_INTERP: absent; size: 25305312 bytes; baseline 20453280; delta +4852032 bytes (report only, D243 amended 2026-11-01) FAST_EXIT=1 Frontend failure excerpt (verbatim): error: doc.js stays small Expected: < 12000 Received: 13408 at <anonymous> (/home/claude/worktrees/j365/apps/tracer/docs-build.test.ts:185:49) (fail) the real tree builds from the real folder, in the sections of D317 [272.23ms] ``` The size deltas above are against the gate's configured historical baselines, not an attribution of those deltas to this small shell change. Binary hashes are in compute01 `~/j365/static-artifacts.log`. Suite wall times are in `suite-times.log`; differing shared load makes them diagnostic rather than a performance benchmark. Separate finding retained for follow-up: Fixture::drop omits setup_guard.sh when sourcing setup_lock.sh, so process_start_ticks is undefined and some leases outlive removed private roots. It did not cause the observed cross-install refusals, and is not bundled into this fix. The earlier finding comment records the exact PIDs and deleted fd paths. Evidence retained outside targets under `~/j365` on both VMs and copied to control `/home/kayg/.cache/j365/`. Build targets on both VMs removed; no own fixture orphans remain. Ready for the orchestrator's independent review; the pre-existing frontend gate defect needs separate handling.

kayg commented

2026-11-08 14:22:33 +00:00

Author

Owner

J371 independent correctness review starting. Branch fix-install-suite-404, head 05aef6860da7d171975b9ecb483924571410ed6b, current origin/dev 7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a (author gate base abad8c6c0). Plan: inspect D337/D355/D359 and J349/J342 findings; reproduce deterministic red/green; rerun actual systemd scopes as root on compute01 in a throwaway clone; assess fixture teardown and confirm unchanged docs-size failure. Estimate 60 minutes excluding build-slot queue. Reviewer brief forbids rerunning full gates; will inspect author receipts and run focused proofs only. No merge, push or deploy.

J371 independent correctness review starting. Branch `fix-install-suite-404`, head `05aef6860da7d171975b9ecb483924571410ed6b`, current origin/dev `7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a` (author gate base `abad8c6c0`). Plan: inspect D337/D355/D359 and J349/J342 findings; reproduce deterministic red/green; rerun actual systemd scopes as root on compute01 in a throwaway clone; assess fixture teardown and confirm unchanged docs-size failure. Estimate 60 minutes excluding build-slot queue. Reviewer brief forbids rerunning full gates; will inspect author receipts and run focused proofs only. No merge, push or deploy.

kayg referenced this issue

2026-11-08 14:24:16 +00:00

doc.js over its size budget on dev #428

kayg commented

2026-11-08 14:30:44 +00:00

Author

Owner

J371 finding update: real root systemd-scope proof on compute01 passed with CAP_SYS_PTRACE dropped so the non-dumpable child is genuinely hidden. Current dev 7c8bd2221 returns [73,73]; 05aef6860 returns [73,0]. Evidence /root/j371-evidence/real-scope-proof.{py,log}; scope child reaped and fixture roots removed.

Pre-existing P2 test-fixture cleanup defect confirmed independently: sourcing setup_lock.sh alone reproduces process_start_ticks: not found, retire exits 73, and the exact live lease still holds fd/7. Sourcing the production setup_guard.sh + setup_lock.sh bundle retires that same lease, exits 0, and it is reaped. Artifact compute01 ~claude/j371-evidence/lease-proof.{py,log}. The production SETUP_LOCK constant concatenates both files, so this omission is in Fixture::drop, not the product bundle. No code changed.

Focused docs test independently rerun on detached unmodified current dev 7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a: 16 pass / 1 fail, Expected <12000 / Received 13408. Artifact /root/j371-evidence/docs-dev.log, head receipt docs-head.log. These inputs have no diff from the review head. Waiting on my focused committed-regression build; no full gate rerun per reviewer rule.

J371 finding update: real root systemd-scope proof on compute01 passed with CAP_SYS_PTRACE dropped so the non-dumpable child is genuinely hidden. Current dev `7c8bd2221` returns [73,73]; `05aef6860` returns [73,0]. Evidence `/root/j371-evidence/real-scope-proof.{py,log}`; scope child reaped and fixture roots removed. Pre-existing P2 test-fixture cleanup defect confirmed independently: sourcing setup_lock.sh alone reproduces `process_start_ticks: not found`, retire exits 73, and the exact live lease still holds fd/7. Sourcing the production setup_guard.sh + setup_lock.sh bundle retires that same lease, exits 0, and it is reaped. Artifact compute01 `~claude/j371-evidence/lease-proof.{py,log}`. The production SETUP_LOCK constant concatenates both files, so this omission is in Fixture::drop, not the product bundle. No code changed. Focused docs test independently rerun on detached unmodified current dev `7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a`: 16 pass / 1 fail, Expected <12000 / Received 13408. Artifact `/root/j371-evidence/docs-dev.log`, head receipt `docs-head.log`. These inputs have no diff from the review head. Waiting on my focused committed-regression build; no full gate rerun per reviewer rule.

kayg commented

2026-11-08 14:50:40 +00:00

Author

Owner

J371 independent correctness review: APPROVE

Reviewed fix-install-suite-404 at 05aef6860da7d171975b9ecb483924571410ed6b against current origin/dev, 7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a. Read the common brief, CLAUDE.md, architecture/process, D337/D355/D359, J365's evidence, and J342/J349's reviews and repair records on #394. No merge, push, deployment, decision change or repository code change.

Findings

P1: none introduced. The terminal-scope classifier requires a canonical 32-character lowercase hex session id and an owned, nonsymlink socket under this install, in either supported layout. A stale socket still counts: it does not require the listener to answer. An unrelated install's same-uid scope no longer suffices. The install-path fallback, service classification, parent/journal flocks, live-socket checks and post-stop holder scan remain intact. The shim binds before spawning its terminal; before binding, its executable path identifies the install. D359 uses scan_holders agent before stopping and scan_holders all afterward, so it does not acquire Replace's service exemption for terminal shims.
P2, pre-existing test defect: Fixture::drop leaks setup leases. tests/install.rs:291 sources setup_lock.sh alone, so read_lease_birth cannot call process_start_ticks; Drop ignores the refusal and removes the private tree. My exact-lease probe reproduced exit 73 plus process_start_ticks: not found, with fd/7 still holding the fixture stage. Supplying the production guard-plus-lock bundle retired the same lease with exit 0; it was reaped. Production's SETUP_LOCK already concatenates both files (src/install.rs:21), so this fixture omission does not hide the same defect in product cleanup. It can contaminate resource observations and should be fixed separately. It does not explain the demonstrated cross-install attribution: that is independently red without a leaked lease. Evidence: compute01 ~claude/j371-evidence/lease-proof.{py,log}.
P3: no new finding. J342's previously recorded limitations of the broader holder scanner (uid filtering, exe/fd rather than cwd/root/mmaps, alias handling) remain unchanged. This patch establishes scope ownership within the supported installation/session lifecycle; it is not a repair of every historical scanner limitation. J342's finished-install deletion guards and J349's sync-before-delete protection are unchanged.

Independent evidence

Committed deterministic regression: built the head's recovery helper and ran only concurrent_fixtures_attribute_hidden_session_scopes_to_their_own_install. Head passes. Replacing only previous_attempt.sh with the exact current origin/dev blob makes the same test fail with the expected “another install's hidden scope blocked this fixture”, exit 73 for fixture B. GREEN_EXIT=0, RED_EXIT=101. All other SSH package source is identical between current dev and head; the transplant isolates the production change. The child readiness handshake and retained stdin keep it live, and both installations and the child are cleaned before assertions. No retries or added sleeps. Artifacts: compute01 ~claude/j371-evidence/{run-regression.sh,green.log,red.log,regression-status.log,dev-previous_attempt.sh}.
Real systemd scopes as uid 0 on compute01: throwaway clone /root/j371-clone, through build-slot, actual /proc cgroup reads and a non-dumpable child in a unique system.slice/infraternal-session-<id>.scope. Dropped CAP_SYS_PTRACE so root genuinely cannot read the child's exe; no substituted cgroup data. Concurrent owning/unrelated install results: dev [73,73], fixed [73,0]. REAL_SCOPE_PROOF=PASS. Exact child reaped and fixture directories removed. Artifacts: /root/j371-evidence/real-scope-proof.log; the executed harness is attached as real-scope-proof-as-run.py. This shell-attribution proof reused the existing API-level-7 recovery helper; the compiled regression separately built the helper at the review head.
D359 / J342 / J349 cross-check: read the pre-stop holders_beside_agent and post-stop live_session_sockets; scan_holders all sequence, stop_agent_unit cgroup check, finished-install validation before deleting whole set-asides, and settle_set_aside before journal/set-aside deletion. None is changed by this branch. An optional nine-test root sample/repeat was cancelled while still queued behind full gates; it did not run and is not claimed as evidence. The completed root-scope proof above did run.
Docs failure independently confirmed on unmodified current dev: detached head receipt 7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a; bun test docs-build.test.ts gives 16 pass / 1 fail, doc.js stays small, Expected <12000 / Received 13408, DOCS_EXIT=1. No docs-build inputs differ from the review head. Artifacts: /root/j371-evidence/{docs-head.log,docs-dev.log}. This agrees with J365's original dev proof at abad8c6c0.
Inspected J365's retained sweep and gate receipts: 1/19 original baseline suites failed; 0/20 fixed suites failed (2,160 active executions). These are J365's measurements, not a fresh sweep by J371. Reviewer brief forbids full gate reruns; I ran only the focused proofs above. J365's gate passed all 24 Rust rows and exited 1 on the independently reproduced docs defect; later frontend rows did not run.

Author gate output, verbatim

From compute01 ~claude/j365/fast-summary.log (author finish comment 86589):

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| migration names           | PASS         |
| migration tooling tests   | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| Traefik component manifest | PASS         |
| release component publication | PASS         |
| test                      | PASS         |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| privileged integration    | PASS         |
| probe release exclusion   | PASS         |
| nightly fuzz reporter     | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

PT_INTERP: absent; size: 28171984 bytes; baseline 22418256; delta +5753728 bytes (report only, D243 amended 2026-11-01)
PT_INTERP: absent; size: 25305312 bytes; baseline 20453280; delta +4852032 bytes (report only, D243 amended 2026-11-01)

FAST_EXIT=1

Frontend failure excerpt (verbatim):
error: doc.js stays small

Expected: < 12000
Received: 13408

      at <anonymous> (/home/claude/worktrees/j365/apps/tracer/docs-build.test.ts:185:49)
(fail) the real tree builds from the real folder, in the sections of D317 [272.23ms]

Cleanup: removed the own build target, the throwaway /root/j371-clone, copied helper binary and root bundle; checked both own transient scope cgroups are absent. The review checkout is clean. No test, build or queue belonging to J371 remains running. Evidence stays under ~claude/j371-evidence, /root/j371-evidence, and control /home/kayg/.cache/codex-scratch/J371/evidence; selected logs/harnesses are attached here.

Verdict: APPROVE for this ownership fix. No new P1/P2 product findings. Keep the pre-existing fixture teardown and frontend-size gate defects tracked separately; this approval does not claim a green full gate.

Head: 05aef6860da7d171975b9ecb483924571410ed6b.

## J371 independent correctness review: APPROVE Reviewed `fix-install-suite-404` at `05aef6860da7d171975b9ecb483924571410ed6b` against current `origin/dev`, `7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a`. Read the common brief, CLAUDE.md, architecture/process, D337/D355/D359, J365's evidence, and J342/J349's reviews and repair records on #394. No merge, push, deployment, decision change or repository code change. ### Findings - **P1: none introduced.** The terminal-scope classifier requires a canonical 32-character lowercase hex session id and an owned, nonsymlink socket under this install, in either supported layout. A stale socket still counts: it does not require the listener to answer. An unrelated install's same-uid scope no longer suffices. The install-path fallback, service classification, parent/journal flocks, live-socket checks and post-stop holder scan remain intact. The shim binds before spawning its terminal; before binding, its executable path identifies the install. D359 uses `scan_holders agent` before stopping and `scan_holders all` afterward, so it does not acquire Replace's `service` exemption for terminal shims. - **P2, pre-existing test defect: Fixture::drop leaks setup leases.** `tests/install.rs:291` sources setup_lock.sh alone, so `read_lease_birth` cannot call `process_start_ticks`; Drop ignores the refusal and removes the private tree. My exact-lease probe reproduced exit 73 plus `process_start_ticks: not found`, with fd/7 still holding the fixture stage. Supplying the production guard-plus-lock bundle retired the *same* lease with exit 0; it was reaped. Production's `SETUP_LOCK` already concatenates both files (`src/install.rs:21`), so this fixture omission does not hide the same defect in product cleanup. It can contaminate resource observations and should be fixed separately. It does not explain the demonstrated cross-install attribution: that is independently red without a leaked lease. Evidence: compute01 `~claude/j371-evidence/lease-proof.{py,log}`. - **P3: no new finding.** J342's previously recorded limitations of the broader holder scanner (uid filtering, exe/fd rather than cwd/root/mmaps, alias handling) remain unchanged. This patch establishes scope ownership within the supported installation/session lifecycle; it is not a repair of every historical scanner limitation. J342's finished-install deletion guards and J349's sync-before-delete protection are unchanged. ### Independent evidence 1. **Committed deterministic regression:** built the head's recovery helper and ran only `concurrent_fixtures_attribute_hidden_session_scopes_to_their_own_install`. Head passes. Replacing only previous_attempt.sh with the exact current origin/dev blob makes the same test fail with the expected “another install's hidden scope blocked this fixture”, exit 73 for fixture B. `GREEN_EXIT=0`, `RED_EXIT=101`. All other SSH package source is identical between current dev and head; the transplant isolates the production change. The child readiness handshake and retained stdin keep it live, and both installations and the child are cleaned before assertions. No retries or added sleeps. Artifacts: compute01 `~claude/j371-evidence/{run-regression.sh,green.log,red.log,regression-status.log,dev-previous_attempt.sh}`. 2. **Real systemd scopes as uid 0 on compute01:** throwaway clone `/root/j371-clone`, through build-slot, actual `/proc` cgroup reads and a non-dumpable child in a unique `system.slice/infraternal-session-<id>.scope`. Dropped CAP_SYS_PTRACE so root genuinely cannot read the child's exe; no substituted cgroup data. Concurrent owning/unrelated install results: dev **[73,73]**, fixed **[73,0]**. `REAL_SCOPE_PROOF=PASS`. Exact child reaped and fixture directories removed. Artifacts: `/root/j371-evidence/real-scope-proof.log`; the executed harness is attached as `real-scope-proof-as-run.py`. This shell-attribution proof reused the existing API-level-7 recovery helper; the compiled regression separately built the helper at the review head. 3. **D359 / J342 / J349 cross-check:** read the pre-stop `holders_beside_agent` and post-stop `live_session_sockets; scan_holders all` sequence, `stop_agent_unit` cgroup check, finished-install validation before deleting whole set-asides, and `settle_set_aside` before journal/set-aside deletion. None is changed by this branch. An optional nine-test root sample/repeat was cancelled while still queued behind full gates; it did not run and is not claimed as evidence. The completed root-scope proof above did run. 4. **Docs failure independently confirmed on unmodified current dev:** detached head receipt `7c8bd2221a2cfabd1cbf2f839edc3a1aeb3bfb7a`; `bun test docs-build.test.ts` gives **16 pass / 1 fail**, `doc.js stays small`, **Expected <12000 / Received 13408**, `DOCS_EXIT=1`. No docs-build inputs differ from the review head. Artifacts: `/root/j371-evidence/{docs-head.log,docs-dev.log}`. This agrees with J365's original dev proof at `abad8c6c0`. 5. Inspected J365's retained sweep and gate receipts: **1/19** original baseline suites failed; **0/20** fixed suites failed (2,160 active executions). These are J365's measurements, not a fresh sweep by J371. Reviewer brief forbids full gate reruns; I ran only the focused proofs above. J365's gate passed all 24 Rust rows and exited 1 on the independently reproduced docs defect; later frontend rows did not run. ### Author gate output, verbatim From compute01 `~claude/j365/fast-summary.log` (author finish comment 86589): ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | Nix Cargo vendoring | PASS | | fmt | PASS | | migration names | PASS | | migration tooling tests | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | Traefik component manifest | PASS | | release component publication | PASS | | test | PASS | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | privileged integration | PASS | | probe release exclusion | PASS | | nightly fuzz reporter | PASS | | static builds | PASS | +---------------------------+--------------+ PT_INTERP: absent; size: 28171984 bytes; baseline 22418256; delta +5753728 bytes (report only, D243 amended 2026-11-01) PT_INTERP: absent; size: 25305312 bytes; baseline 20453280; delta +4852032 bytes (report only, D243 amended 2026-11-01) FAST_EXIT=1 Frontend failure excerpt (verbatim): error: doc.js stays small Expected: < 12000 Received: 13408 at <anonymous> (/home/claude/worktrees/j365/apps/tracer/docs-build.test.ts:185:49) (fail) the real tree builds from the real folder, in the sections of D317 [272.23ms] ``` Cleanup: removed the own build target, the throwaway `/root/j371-clone`, copied helper binary and root bundle; checked both own transient scope cgroups are absent. The review checkout is clean. No test, build or queue belonging to J371 remains running. Evidence stays under `~claude/j371-evidence`, `/root/j371-evidence`, and control `/home/kayg/.cache/codex-scratch/J371/evidence`; selected logs/harnesses are attached here. **Verdict: APPROVE** for this ownership fix. No new P1/P2 product findings. Keep the pre-existing fixture teardown and frontend-size gate defects tracked separately; this approval does not claim a green full gate. Head: `05aef6860da7d171975b9ecb483924571410ed6b`.

J371-review-evidence.zip

29 KiB

kayg referenced this issue

2026-11-08 16:50:06 +00:00

J376: diagnose remaining install-suite failures after #404 #432

kayg referenced this issue

2026-11-08 16:52:19 +00:00

J376: diagnose remaining install-suite failures after #404 #432

kayg referenced this issue

2026-11-08 16:54:09 +00:00

J376: diagnose remaining install-suite failures after #404 #432

kayg referenced this issue

2026-11-08 16:59:54 +00:00

dev: core-screen #rtt and platform-consistency specs fail #430

kayg commented

2026-11-08 17:07:08 +00:00

Author

Owner

J376 follow-up is tracked on #432, branch fix-install-suite-red, base current dev b31c2da2d. The supplied J372 nine-failure fast log predates this merge (merge base 7c8bd2221 and old source lines/test count). Its later standalone failure is different: missing recovery-helper yields Upload/127; reproduced on a clean target and removed by building only that helper on unchanged b31c2da2d, including a fresh HOME/runtime control. All nine historical cases pass on current dev with the helper. No evidence yet that the reported failures indicate a shipped #404 regression. One distinct checked-reader rollback/73 occurred in the control full suite; tracing it on #432 before making any production-impact claim. Evidence compute01 ~/j376-evidence/.

J376 follow-up is tracked on #432, branch fix-install-suite-red, base current dev b31c2da2d. The supplied J372 nine-failure fast log predates this merge (merge base 7c8bd2221 and old source lines/test count). Its later standalone failure is different: missing recovery-helper yields Upload/127; reproduced on a clean target and removed by building only that helper on unchanged b31c2da2d, including a fresh HOME/runtime control. All nine historical cases pass on current dev with the helper. No evidence yet that the reported failures indicate a shipped #404 regression. One distinct checked-reader rollback/73 occurred in the control full suite; tracing it on #432 before making any production-impact claim. Evidence compute01 ~/j376-evidence/.

kayg referenced this issue

2026-11-08 17:30:49 +00:00

D368: confirm and close a finished install’s own terminals during Take over (#404) #433

kayg commented

2026-11-08 18:24:17 +00:00

Author

Owner

J376 final evidence: the confirmed current-dev failure is test-only missing recovery-helper on focused cargo test --test install, fixed on fix-install-suite-red at 046c88931c. Shipped Add host code is unchanged. J372's nine fast-log lock failures predate #404 rather than proving a post-merge regression. Three consecutive compute01 install suites passed 108/0/2 each; quick and full fast pass, FAST_EXIT=0, including all privileged SSH tests. #421 did not reproduce.

One separate unchanged-baseline buffering-reader rollback SetupRunning/73 observation remains unexplained and has not recurred in the recorded diagnostics or validation; its production impact is not established. Full findings, exact gate output and attached red/green receipts: #432 (comment) . No production modifications, merge, or deployment. J376 target removed.

J376 final evidence: the confirmed current-dev failure is test-only missing recovery-helper on focused cargo test --test install, fixed on fix-install-suite-red at 046c88931c65dacaf01e538793ae30170c533d03. Shipped Add host code is unchanged. J372's nine fast-log lock failures predate #404 rather than proving a post-merge regression. Three consecutive compute01 install suites passed 108/0/2 each; quick and full fast pass, FAST_EXIT=0, including all privileged SSH tests. #421 did not reproduce. One separate unchanged-baseline buffering-reader rollback SetupRunning/73 observation remains unexplained and has not recurred in the recorded diagnostics or validation; its production impact is not established. Full findings, exact gate output and attached red/green receipts: https://git.kayg.org/kayg/infraternal/issues/432#issuecomment-86899 . No production modifications, merge, or deployment. J376 target removed.

kayg referenced this issue

2026-11-08 18:39:03 +00:00

dev: core-screen #rtt and platform-consistency specs fail #430

kayg referenced this issue

2026-11-09 00:35:38 +00:00

D368: confirm and close a finished install’s own terminals during Take over (#404) #433

Rows
Columns

Flaky: real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness (~1/15 on dev) #404

J371 independent correctness review: APPROVE

Findings

Independent evidence

Author gate output, verbatim