Static release builds differ by build environment

kayg commented

2026-10-19 02:36:14 +00:00

Owner

At commit 3b2183eaa, a static build from night-release matched both exact size caps, while the same Rust inputs built by pill-flicker were 32,768 bytes larger on x86_64 and 49,152 bytes larger on AArch64. The latter artifacts contained a Cargo registry source path (see #227's final build comparison).

Identify the exact environment paths or settings that change the output across different CARGO_HOME, target directory, and checkout paths. Make both static builds independent of those locations, verify whether sccache keys or compiler metadata retain any paths, and add a gate that builds both environments and requires byte-identical stripped binaries at the existing exact caps. Related prior wrapper work: #229 R5.

At commit 3b2183eaa, a static build from night-release matched both exact size caps, while the same Rust inputs built by pill-flicker were 32,768 bytes larger on x86_64 and 49,152 bytes larger on AArch64. The latter artifacts contained a Cargo registry source path (see #227's final build comparison). Identify the exact environment paths or settings that change the output across different CARGO_HOME, target directory, and checkout paths. Make both static builds independent of those locations, verify whether sccache keys or compiler metadata retain any paths, and add a gate that builds both environments and requires byte-identical stripped binaries at the existing exact caps. Related prior wrapper work: #229 R5.

base-vs-branch-x86-section-diff.txt

1.3 KiB

report.txt

562 B

environment.txt

338 B

x86_64-unknown-linux-musl.log

9.6 KiB

aarch64-unknown-linux-musl.log

8.3 KiB

x86-baseline-diff.txt

1.7 KiB

x86_64-unknown-linux-musl.log

10 KiB

historical-fast-log.md

208 KiB

x86-cold-diff.txt

1.7 KiB

old-wrapper-paths.txt

737 B

cold-cold-comparison.txt

3.5 KiB

environment.txt

494 B

environment.txt

763 B

base-a-toolchain.txt

515 B

base-b-toolchain.txt

628 B

test-musl-cc.log

320 B

test-rustc-paths.log

86 B

test-build-metadata.log

326 B

bun-run-check.log

36 B

x86-cap-finding.txt

613 B

branch-static-size-finding.txt

824 B

branch-static-x86_64-unknown-linux-musl.log

9.7 KiB

branch-static-aarch64-unknown-linux-musl.log

8.5 KiB

base-final-static-sizes.txt

984 B

corrected-flags-size-check.txt

939 B

cold-base-vs-branch-x86.txt

1 KiB

x86-old-order-no-rustup.txt

761 B

branch-x86-old-map-order.log

8.5 KiB

x86-rustup-only.txt

598 B

branch-x86-old-order-with-rustup.log

8.5 KiB

x86-sorted-order-no-rustup.txt

690 B

branch-x86-sorted-order-no-rustup.log

8.5 KiB

x86-registry-before-cargo.txt

934 B

branch-x86-registry-before-cargo.log

8.5 KiB

iterate-runtime-helper-missing.txt

532 B

iterate-symlink-state-root.txt

602 B

iterate-symlink-state-root.txt

602 B

iterate-physical-root-clean.log

160 KiB

iterate-short-root.log

188 KiB

plugin-lock-isolated-remote.log

985 B

sccache-rust-args.json

181 B

sccache-native-x86-args.json

220 B

ssh-state-root-path-failure.txt

3.3 KiB

recovery-support-short-root.log

1.2 KiB

capture-observer-regression.log

214 B

report.txt

3.2 KiB

environment.txt

915 B

same-filesystem-test-root-failure.txt

4.1 KiB

fixture-root-restart-tests.log

18 KiB

fixture-root-same-mount-tests.log

6.7 KiB

fixture-root-targeted-host-tests.log

10 KiB

agent-daemon-short-root.log

3.4 KiB

verify-fast-final.log

164 KiB

verify-fast-final-rerun.log

396 KiB

report.txt

1.6 KiB

environment.txt

916 B

sccache-rust-args.json

1.2 KiB

sccache-native-x86-args.json

1.6 KiB

sccache-native-aarch64-args.json

1.6 KiB

base-final-static-sizes.txt

984 B

verify-fast-final-rerun.log

396 KiB

report.txt

1.6 KiB

base-final-static-sizes.txt

984 B

kayg commented

2026-10-19 02:36:20 +00:00

Author

Owner

Start: branch static-path-remap, base origin/dev 6f7cde60a430590a20f0b9384fc423df465bde7f. The worktree is clean; the remote has no static-path-remap branch. Prior discrepancy is recorded on #227; related wrapper work is #229 R5. Checking exact environment-dependent metadata at 3b2183eaa before changing the current gate.

Start: branch `static-path-remap`, base `origin/dev` `6f7cde60a430590a20f0b9384fc423df465bde7f`. The worktree is clean; the remote has no `static-path-remap` branch. Prior discrepancy is recorded on #227; related wrapper work is #229 R5. Checking exact environment-dependent metadata at 3b2183eaa before changing the current gate.

kayg commented

2026-10-19 02:37:26 +00:00

Author

Owner

Prior #227 comparison: 18ee290 rebuilt the same Rust/Cargo inputs at 3b2183eaa and measured x86_64 21,332,784 B versus 21,300,016 B (+32,768), and AArch64 19,549,528 B versus 19,500,376 B (+49,152). The artifact metadata check found the Cargo registry path in both binaries. At that base, .cargo/rustc.sh already remapped checkout, CARGO_HOME, target root, and each registry source root, so the remaining leak must be identified beyond the existing mappings. No cap changes are justified by this discrepancy.

Prior #227 comparison: `18ee290` rebuilt the same Rust/Cargo inputs at `3b2183eaa` and measured x86_64 21,332,784 B versus 21,300,016 B (+32,768), and AArch64 19,549,528 B versus 19,500,376 B (+49,152). The artifact metadata check found the Cargo registry path in both binaries. At that base, `.cargo/rustc.sh` already remapped checkout, CARGO_HOME, target root, and each registry source root, so the remaining leak must be identified beyond the existing mappings. No cap changes are justified by this discrepancy.

kayg commented

2026-10-19 02:47:07 +00:00

Author

Owner

Native-wrapper regression is red before the fix: CARGO_TARGET_DIR=$HOME/.cache/codex-scratch/static-path-remap/evidence python3 scripts/test-musl-cc.py fails at line 32 for x86_64 because .cargo/musl-cc.sh emits only checkout and target -ffile-prefix-map flags. The new expected flags also cover CARGO_HOME and each registry source root, including paths with spaces.

Native-wrapper regression is red before the fix: `CARGO_TARGET_DIR=$HOME/.cache/codex-scratch/static-path-remap/evidence python3 scripts/test-musl-cc.py` fails at line 32 for x86_64 because `.cargo/musl-cc.sh` emits only checkout and target `-ffile-prefix-map` flags. The new expected flags also cover CARGO_HOME and each registry source root, including paths with spaces.

kayg commented

2026-10-19 02:55:15 +00:00

Author

Owner

Rust/cache finding: before the wrapper fix, build-metadata.test.ts failed at line 51 because .cargo/rustc.sh emitted no RUSTUP_HOME mapping. At base 3b2183eaa, scripts/static-cargo passed no target rustflags to Cargo, so paths appended by the Rust compiler script were not visible to RUSTC_WRAPPER=sccache. After the shared remap helper and cache-visible target flags, the test passes 2/2 with 41 assertions; test-rustc-paths.py also passes.

Rust/cache finding: before the wrapper fix, `build-metadata.test.ts` failed at line 51 because `.cargo/rustc.sh` emitted no RUSTUP_HOME mapping. At base `3b2183eaa`, `scripts/static-cargo` passed no target rustflags to Cargo, so paths appended by the Rust compiler script were not visible to `RUSTC_WRAPPER=sccache`. After the shared remap helper and cache-visible target flags, the test passes 2/2 with 41 assertions; `test-rustc-paths.py` also passes.

kayg commented

2026-10-19 03:11:43 +00:00

Author

Owner

Baseline A completed at 3b2183eaac: x86_64 21,300,016 B / cap 21,300,016 B; AArch64 19,500,376 B / cap 19,500,376 B. Environment and full build logs attached: report, environment, x86 log, AArch64 log. Baseline B is compiling with separate checkout, Cargo, target and Rustup roots.

Baseline A completed at 3b2183eaac8a9d8cc52a922dfbbbaecf895fd854: x86_64 21,300,016 B / cap 21,300,016 B; AArch64 19,500,376 B / cap 19,500,376 B. Environment and full build logs attached: [report](https://git.kayg.org/attachments/2c71d23c-6fe4-4d6f-b1e1-6b0285bc2d2a), [environment](https://git.kayg.org/attachments/22fda370-8881-4544-8783-b95f13c905d9), [x86 log](https://git.kayg.org/attachments/3421d8d4-2c0c-403b-b677-d69b9d4652b8), [AArch64 log](https://git.kayg.org/attachments/5410ecc4-6a43-470e-aa4c-b3347cfd7ed5). Baseline B is compiling with separate checkout, Cargo, target and Rustup roots.

kayg commented

2026-10-19 03:20:23 +00:00

Author

Owner

Baseline B x86_64 (same 3b2183eaac, alternate checkout/CARGO_HOME/target/RUSTUP_HOME) is byte-identical to A: 21,300,016 B, SHA-256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606. ELF .text/.rodata/.data sizes match, and neither artifact contains either checkout, Cargo, target or Rustup root. The shared sccache directory may have supplied cached objects, so I am checking B again with an isolated cache before concluding. binary and section comparison, build log.

Baseline B x86_64 (same 3b2183eaac8a9d8cc52a922dfbbbaecf895fd854, alternate checkout/CARGO_HOME/target/RUSTUP_HOME) is byte-identical to A: 21,300,016 B, SHA-256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606. ELF .text/.rodata/.data sizes match, and neither artifact contains either checkout, Cargo, target or Rustup root. The shared sccache directory may have supplied cached objects, so I am checking B again with an isolated cache before concluding. [binary and section comparison](https://git.kayg.org/attachments/b7f139a9-a3e2-45f9-b2b8-35e10d4e51bb), [build log](https://git.kayg.org/attachments/d01495ae-ddad-4258-96e1-78e4e8c49110).

kayg commented

2026-10-19 03:51:35 +00:00

Author

Owner

Finding: #227's gate log reports x86_64 +32,768 B and AArch64 +49,152 B over the then caps, and both static metadata checks failed because the binary contained the resolved Cargo registryRoot path.

At 3b2183ea, a forced compilation through the existing sccache service with alternate checkout, CARGO_HOME, CARGO_TARGET_DIR, and RUSTUP_HOME produced the same x86_64 binary as baseline A: 21,300,016 B, SHA-256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606. Relevant ELF section sizes match; none of the alternate checkout/Cargo/target/Rustup roots or the effective cache root occurs in the bytes. AArch64 comparison is still running. This does not reproduce the historical drift, so the exact old environment/cache state remains unknown; SCCACHE_DIR did not isolate a second cache because the existing shared server reports its configured cache root.

Receipts: historical #227 gate log, x86_64 cold diff.

Finding: #227's gate log reports x86_64 +32,768 B and AArch64 +49,152 B over the then caps, and both static metadata checks failed because the binary contained the resolved Cargo `registryRoot` path. At 3b2183ea, a forced compilation through the existing sccache service with alternate checkout, CARGO_HOME, CARGO_TARGET_DIR, and RUSTUP_HOME produced the same x86_64 binary as baseline A: 21,300,016 B, SHA-256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606. Relevant ELF section sizes match; none of the alternate checkout/Cargo/target/Rustup roots or the effective cache root occurs in the bytes. AArch64 comparison is still running. This does not reproduce the historical drift, so the exact old environment/cache state remains unknown; `SCCACHE_DIR` did not isolate a second cache because the existing shared server reports its configured cache root. Receipts: [historical #227 gate log](https://git.kayg.org/attachments/5d2146a6-e3c7-411d-90e7-d63daef000da), [x86_64 cold diff](https://git.kayg.org/attachments/62756a8c-d825-4546-9ef1-79d82d661e36).

kayg commented

2026-10-19 04:07:45 +00:00

Author

Owner

Finding from the 3b2183eaa wrapper source: .cargo/musl-cc.sh passed prefix maps for the checkout and target only, leaving Cargo registry source paths unmapped in native C compiles. Rust maps were appended inside .cargo/rustc.sh, after the sccache wrapper had received the compile invocation. This matches #227's registryRoot metadata failure and leaves both native and Rust cache keys without the final maps. The exact historical CARGO_HOME and sccache server settings are absent from the saved #227 log.

The implementation now passes Rust maps in Cargo target rustflags and native maps through CFLAGS before cc-rs applies RUSTC_WRAPPER. The reproducibility gate captures those pre-cache arguments for Rust and both native target wrappers.

Evidence: 3b wrapper excerpts, historical #227 gate log.

Finding from the 3b2183eaa wrapper source: `.cargo/musl-cc.sh` passed prefix maps for the checkout and target only, leaving Cargo registry source paths unmapped in native C compiles. Rust maps were appended inside `.cargo/rustc.sh`, after the sccache wrapper had received the compile invocation. This matches #227's `registryRoot` metadata failure and leaves both native and Rust cache keys without the final maps. The exact historical CARGO_HOME and sccache server settings are absent from the saved #227 log. The implementation now passes Rust maps in Cargo target rustflags and native maps through CFLAGS before cc-rs applies RUSTC_WRAPPER. The reproducibility gate captures those pre-cache arguments for Rust and both native target wrappers. Evidence: [3b wrapper excerpts](https://git.kayg.org/attachments/a65d9f83-bf03-4663-8cd6-9b02e77b09cb), [historical #227 gate log](https://git.kayg.org/attachments/5d2146a6-e3c7-411d-90e7-d63daef000da).

kayg commented

2026-10-19 04:17:07 +00:00

Author

Owner

Cold control completed at 3b2183eaac. A used checkout /home/kayg/.cache/codex-scratch/static-path-remap/checkout-a, CARGO_HOME /home/kayg/.cargo, target /home/kayg/.cache/codex-scratch/static-path-remap/target-a, and RUSTUP_HOME /home/kayg/.rustup. B used a deliberately longer checkout, alternate copied registry, target-b, and alternate copied Rustup home. Both builds set SCCACHE_RECACHE=1; B's different SCCACHE_DIR setting did not move the shared service from /home/kayg/.cache/sccache-infraternal. Both selected rustc 1.99.0 commit b940084d7eb6a299eb4bfeb8e34901bc051e7ac4 and Cargo 1.99.0.

The outputs are byte-identical for both targets and equal the historical caps: x86_64 21,300,016 B (SHA-256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606); AArch64 19,500,376 B (SHA-256 0e300cec45579a7949567ffb26e014c5be7f4027add438095dddc66f54675577). Relevant .text, .rodata, .data, .comment, and .note.infraternal sections match, and neither binary contains any tested checkout/Cargo/target/Rustup/cache path. The expanded metadata review also passes on B for both architectures (2 tests, 18 expectations).

The historical +32,768/+49,152 B drift is not reproduced. #227's saved gate log does not include the historical CARGO_HOME or active sccache state. The source-level gaps recorded above remain consistent with the registry-root failure; the new gate checks pre-cache Rust and native compiler arguments under distinct roots.

Evidence: cold A/B comparison, A environment, B environment, A toolchain, B toolchain.

Cold control completed at 3b2183eaac8a9d8cc52a922dfbbbaecf895fd854. A used checkout `/home/kayg/.cache/codex-scratch/static-path-remap/checkout-a`, CARGO_HOME `/home/kayg/.cargo`, target `/home/kayg/.cache/codex-scratch/static-path-remap/target-a`, and RUSTUP_HOME `/home/kayg/.rustup`. B used a deliberately longer checkout, alternate copied registry, `target-b`, and alternate copied Rustup home. Both builds set `SCCACHE_RECACHE=1`; B's different `SCCACHE_DIR` setting did not move the shared service from `/home/kayg/.cache/sccache-infraternal`. Both selected rustc 1.99.0 commit `b940084d7eb6a299eb4bfeb8e34901bc051e7ac4` and Cargo 1.99.0. The outputs are byte-identical for both targets and equal the historical caps: x86_64 21,300,016 B (SHA-256 `236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606`); AArch64 19,500,376 B (SHA-256 `0e300cec45579a7949567ffb26e014c5be7f4027add438095dddc66f54675577`). Relevant `.text`, `.rodata`, `.data`, `.comment`, and `.note.infraternal` sections match, and neither binary contains any tested checkout/Cargo/target/Rustup/cache path. The expanded metadata review also passes on B for both architectures (2 tests, 18 expectations). The historical +32,768/+49,152 B drift is not reproduced. #227's saved gate log does not include the historical CARGO_HOME or active sccache state. The source-level gaps recorded above remain consistent with the registry-root failure; the new gate checks pre-cache Rust and native compiler arguments under distinct roots. Evidence: [cold A/B comparison](https://git.kayg.org/attachments/1e568014-74e6-4138-9ee3-a8217f41e83e), [A environment](https://git.kayg.org/attachments/00ddd8f0-e848-452c-b6ab-80b3e9521c8a), [B environment](https://git.kayg.org/attachments/b8913dee-aa6b-4f67-b118-2ea344334032), [A toolchain](https://git.kayg.org/attachments/0ec1590b-3609-4e26-9718-c0d15c8cbd6b), [B toolchain](https://git.kayg.org/attachments/7b784ae1-7ae1-4a54-8f59-bf107613bfb9).

kayg commented

2026-10-19 04:17:33 +00:00

Author

Owner

Base update: origin/dev advanced from 6f7cde60a430590a20f0b9384fc423df465bde7f to f29713863e5da84f4e7a9dd7d2702557b4287755 (D307, DECISIONS.md only). 6f7cde60 is an ancestor of the current base, and no origin/static-path-remap branch exists. I will merge origin/dev before the first feature commit; the current uncommitted changes do not touch DECISIONS.md.

Base update: `origin/dev` advanced from `6f7cde60a430590a20f0b9384fc423df465bde7f` to `f29713863e5da84f4e7a9dd7d2702557b4287755` (D307, DECISIONS.md only). `6f7cde60` is an ancestor of the current base, and no `origin/static-path-remap` branch exists. I will merge `origin/dev` before the first feature commit; the current uncommitted changes do not touch DECISIONS.md.

kayg commented

2026-10-19 04:17:44 +00:00

Author

Owner

Base merge completed as a fast-forward to f29713863e5da84f4e7a9dd7d2702557b4287755; no conflicts. The only incoming file was DECISIONS.md. The branch worktree is now based on current origin/dev.

Base merge completed as a fast-forward to `f29713863e5da84f4e7a9dd7d2702557b4287755`; no conflicts. The only incoming file was DECISIONS.md. The branch worktree is now based on current `origin/dev`.

kayg commented

2026-10-19 04:20:05 +00:00

Author

Owner

Implementation commit 73946d79bee23105fad9151ca913b9226f1943e6 is on current base f29713863e5da84f4e7a9dd7d2702557b4287755. Rust remaps now enter Cargo target flags before RUSTC_WRAPPER; native maps enter target CFLAGS before cc-rs wraps the native compiler. The C wrapper avoids duplicating maps already supplied by the cache-visible flags.

Checks passed: build metadata tests 2/2 (105 expectations), native wrapper checks for both targets, two-root Rust artifact equality, tracer bun run check, shell syntax and git diff --check. Receipt links: metadata tests, native wrapper, Rust roots, tracer typecheck. Next: build both static targets on this branch.

Implementation commit `73946d79bee23105fad9151ca913b9226f1943e6` is on current base `f29713863e5da84f4e7a9dd7d2702557b4287755`. Rust remaps now enter Cargo target flags before RUSTC_WRAPPER; native maps enter target CFLAGS before cc-rs wraps the native compiler. The C wrapper avoids duplicating maps already supplied by the cache-visible flags. Checks passed: build metadata tests 2/2 (105 expectations), native wrapper checks for both targets, two-root Rust artifact equality, tracer `bun run check`, shell syntax and `git diff --check`. Receipt links: [metadata tests](https://git.kayg.org/attachments/49ab3451-ad0a-4777-b126-484c1230a9d1), [native wrapper](https://git.kayg.org/attachments/fea6b96e-470f-4329-b1d3-50b2e2609017), [Rust roots](https://git.kayg.org/attachments/452bfef2-7faf-4008-9c56-6006e277c89a), [tracer typecheck](https://git.kayg.org/attachments/ae100b0f-bc90-4a1c-a9fb-b7616cae391e). Next: build both static targets on this branch.

kayg commented

2026-10-19 04:35:53 +00:00

Author

Owner

Finding: the fresh branch build is above the documented cap. Commit 73946d79 produced a 21,275,024-byte x86_64 artifact at /home/kayg/.cache/codex-scratch/static-path-remap/target-a/x86_64-unknown-linux-musl/release/infraternal; cap 21,225,872, delta +49,152 bytes. SHA-256 6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8. Receipt: x86-cap-finding.txt. I will build current origin/dev from another target root to establish whether this is already present at the base.

Finding: the fresh branch build is above the documented cap. Commit 73946d79 produced a 21,275,024-byte x86_64 artifact at `/home/kayg/.cache/codex-scratch/static-path-remap/target-a/x86_64-unknown-linux-musl/release/infraternal`; cap 21,225,872, delta +49,152 bytes. SHA-256 6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8. Receipt: [x86-cap-finding.txt](https://git.kayg.org/attachments/52238aff-f87d-4440-a268-f121cf2d7f4e). I will build current origin/dev from another target root to establish whether this is already present at the base.

kayg commented

2026-10-19 04:48:05 +00:00

Author

Owner

The branch builds are complete. Both stripped static images exceed the exact caps, while the expanded metadata review passes (2 tests, 18 expectations; no tested environment roots found). x86_64: 21,275,024 B vs 21,225,872 B (+49,152), SHA-256 6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8. AArch64: 19,480,936 B vs 19,427,688 B (+53,248), SHA-256 af8f053248bbf9c198953f81f542447c358bce0c62365e0f2a2ad25ec54ae97d. Receipts: sizes, x86 build, AArch64 build. The clean f297 base comparison is queued behind shared build slots; it will determine whether the oversize predates this branch.

The branch builds are complete. Both stripped static images exceed the exact caps, while the expanded metadata review passes (2 tests, 18 expectations; no tested environment roots found). x86_64: 21,275,024 B vs 21,225,872 B (+49,152), SHA-256 `6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8`. AArch64: 19,480,936 B vs 19,427,688 B (+53,248), SHA-256 `af8f053248bbf9c198953f81f542447c358bce0c62365e0f2a2ad25ec54ae97d`. Receipts: [sizes](https://git.kayg.org/attachments/0f50a8f4-68a4-4c5a-b4df-274d97d5f6fa), [x86 build](https://git.kayg.org/attachments/30da2334-7d1c-43ec-8465-f8f0adb44e1b), [AArch64 build](https://git.kayg.org/attachments/1543669d-17a6-467f-9a12-4ede9767e8f5). The clean f297 base comparison is queued behind shared build slots; it will determine whether the oversize predates this branch.

kayg commented

2026-10-19 05:06:54 +00:00

Author

Owner

The clean f297 base x86_64 build matches the 21,225,872-byte cap (SHA-256 f8afe9bed532d7cf274a72aa9c7f89d1f4aa6d6ca1f8c73b05a7938c2e08da73); branch 73946d79 is +49,152 bytes (SHA-256 6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8). ELF diff: .text identical at 0xf5bda8; .rodata grows from 0x1b7570 to 0x1c3570 (+49,152); .data.rel.ro and .data sizes match. Static metadata review passed for both branch targets, with no known environment path strings. This points to changed compiler inputs/cache results, not a literal path leak. The base build was not forced through sccache recache. Receipt: base-vs-branch section diff.

The clean f297 base x86_64 build matches the 21,225,872-byte cap (SHA-256 `f8afe9bed532d7cf274a72aa9c7f89d1f4aa6d6ca1f8c73b05a7938c2e08da73`); branch 73946d79 is +49,152 bytes (SHA-256 `6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8`). ELF diff: `.text` identical at 0xf5bda8; `.rodata` grows from 0x1b7570 to 0x1c3570 (+49,152); `.data.rel.ro` and `.data` sizes match. Static metadata review passed for both branch targets, with no known environment path strings. This points to changed compiler inputs/cache results, not a literal path leak. The base build was not forced through sccache recache. Receipt: [base-vs-branch section diff](https://git.kayg.org/attachments/1e12bd29-a1ea-4f7c-a240-5373bb42fbe9).

kayg commented

2026-10-19 05:11:08 +00:00

Author

Owner

Correction to the previous base-size comment: after the build runner exited, the persisted x86_64 artifact is 21,225,864 bytes (8 bytes below cap), SHA-256 4be02b932fd3078eb185f44fa46171c06763e568aa0b6974eb02b7092b95bbe7; the earlier 21,225,872 stat was provisional while the runner was settling. AArch64 is 19,427,688 bytes, exactly at cap, SHA-256 a556b4242f93bb55eb40597069a195bea062d179880ddeffd9fc695b2ba1e908. Both base static builds completed successfully; the persisted section sizes are in the final base receipt. No cause is claimed for the 8-byte x86 difference.

Correction to the previous base-size comment: after the build runner exited, the persisted x86_64 artifact is 21,225,864 bytes (8 bytes below cap), SHA-256 `4be02b932fd3078eb185f44fa46171c06763e568aa0b6974eb02b7092b95bbe7`; the earlier 21,225,872 stat was provisional while the runner was settling. AArch64 is 19,427,688 bytes, exactly at cap, SHA-256 `a556b4242f93bb55eb40597069a195bea062d179880ddeffd9fc695b2ba1e908`. Both base static builds completed successfully; the persisted section sizes are in [the final base receipt](https://git.kayg.org/attachments/2cbc1dd7-99ec-47ae-8e93-178b35fef2ab). No cause is claimed for the 8-byte x86 difference.

kayg commented

2026-10-19 05:40:26 +00:00

Author

Owner

Follow-up: both corrected branch targets were rebuilt cold with SCCACHE_RECACHE=1; removing the duplicate +crt-static flag did not change their bytes. x86_64 remains 21,275,024 B (+49,152), SHA-256 6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8; AArch64 remains 19,480,936 B (+53,248), SHA-256 af8f053248bbf9c198953f81f542447c358bce0c62365e0f2a2ad25ec54ae97d. x86 .text is unchanged and .rodata remains +49,152. Receipt: corrected-flags size check. Next I’m force-recompiling the old f297 recipe to separate cache hits from compiler-output differences.

Follow-up: both corrected branch targets were rebuilt cold with `SCCACHE_RECACHE=1`; removing the duplicate `+crt-static` flag did not change their bytes. x86_64 remains 21,275,024 B (+49,152), SHA-256 `6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8`; AArch64 remains 19,480,936 B (+53,248), SHA-256 `af8f053248bbf9c198953f81f542447c358bce0c62365e0f2a2ad25ec54ae97d`. x86 `.text` is unchanged and `.rodata` remains +49,152. Receipt: [corrected-flags size check](https://git.kayg.org/attachments/85670f38-68af-4b27-a6ad-6f75fbf8e195). Next I’m force-recompiling the old f297 recipe to separate cache hits from compiler-output differences.

kayg commented

2026-10-19 05:54:42 +00:00

Author

Owner

Finding: forcing a cold x86_64 build of the old f297 recipe still produces the exact 21,225,872-byte cap (SHA-256 f8afe9bed532d7cf274a72aa9c7f89d1f4aa6d6ca1f8c73b05a7938c2e08da73). The cold branch remains +49,152 B (SHA-256 6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8); .text is equal and .rodata is +49,152 B. This rules out stale cache hits as the cause and points to changed compiler remap inputs. Receipt: cold x86 comparison. I’m testing the old Rust mapping order and removing the newly added Rustup remap while keeping maps in the sccache-visible command.

Finding: forcing a cold x86_64 build of the old f297 recipe still produces the exact 21,225,872-byte cap (SHA-256 `f8afe9bed532d7cf274a72aa9c7f89d1f4aa6d6ca1f8c73b05a7938c2e08da73`). The cold branch remains +49,152 B (SHA-256 `6155a25f48c255ddf4805aea71a2cb881b08693e6ff16ae37a96a544837fa3d8`); `.text` is equal and `.rodata` is +49,152 B. This rules out stale cache hits as the cause and points to changed compiler remap inputs. Receipt: [cold x86 comparison](https://git.kayg.org/attachments/a0a76ff6-502a-43db-9fa1-193975d3aef4). I’m testing the old Rust mapping order and removing the newly added Rustup remap while keeping maps in the sccache-visible command.

kayg commented

2026-10-19 06:27:43 +00:00

Author

Owner

Cold x86 build with the prior Rust remap order and no Rustup remap is exactly at the 21,225,872-byte cap. .text, .rodata, .data.rel.ro and .data match the f297 baseline section sizes; scanned checkout, Cargo, Rustup and target roots are absent. This run changes mapping order and Rustup mapping together, so I am isolating those two inputs next. Receipts: build report, build log.

Cold x86 build with the prior Rust remap order and no Rustup remap is exactly at the 21,225,872-byte cap. `.text`, `.rodata`, `.data.rel.ro` and `.data` match the f297 baseline section sizes; scanned checkout, Cargo, Rustup and target roots are absent. This run changes mapping order and Rustup mapping together, so I am isolating those two inputs next. Receipts: [build report](https://git.kayg.org/attachments/bac82f90-0459-437a-8322-ab959e1e107d), [build log](https://git.kayg.org/attachments/6a9c8402-dbd1-4307-86e7-9b299399e336).

kayg commented

2026-10-19 06:39:39 +00:00

Author

Owner

The one-variable test adding only --remap-path-prefix=$RUSTUP_HOME=/rustup to the prior ordering produced the same bytes as the no-Rustup build: 21,225,872 bytes, SHA-256 b965dc1a…f19f, with matching section sizes. Rustup remapping does not explain the previous +49,152-byte result. Receipts: report, build log. I am testing mapping order alone next.

The one-variable test adding only `--remap-path-prefix=$RUSTUP_HOME=/rustup` to the prior ordering produced the same bytes as the no-Rustup build: 21,225,872 bytes, SHA-256 `b965dc1a…f19f`, with matching section sizes. Rustup remapping does not explain the previous +49,152-byte result. Receipts: [report](https://git.kayg.org/attachments/9dc9eb17-0edc-4036-8184-533b460b10b0), [build log](https://git.kayg.org/attachments/ca1e15e5-aa7e-4ac3-b58b-1b9365bc9585). I am testing mapping order alone next.

kayg commented

2026-10-19 06:55:16 +00:00

Author

Owner

The longest-first Rust mapping order alone recreates the over-cap image: 21,275,024 bytes (+49,152). .text is unchanged; .rodata is 0x1c3570 instead of 0x1b7570 (+0xc000). Rustup is omitted, and scanned build roots are absent. The order change reorders one nested mapping pair: Cargo registry source roots and their enclosing CARGO_HOME. I am testing that pair directly next. Receipts: report, build log.

The longest-first Rust mapping order alone recreates the over-cap image: 21,275,024 bytes (+49,152). `.text` is unchanged; `.rodata` is `0x1c3570` instead of `0x1b7570` (+`0xc000`). Rustup is omitted, and scanned build roots are absent. The order change reorders one nested mapping pair: Cargo registry source roots and their enclosing CARGO_HOME. I am testing that pair directly next. Receipts: [report](https://git.kayg.org/attachments/633fa76b-9650-4ee1-a4e9-cdd2ec33235d), [build log](https://git.kayg.org/attachments/4a469ce6-1abe-4c9a-96ac-43c0d1548a92).

kayg commented

2026-10-19 07:03:55 +00:00

Author

Owner

The minimal overlap test confirms the cause. Moving only the registry source remap ahead of its enclosing CARGO_HOME mapping reproduces the same over-cap bytes and hash as the fully sorted order: 21,275,024 bytes (+49,152), with .rodata +0xc000. The cap-sized output uses canonical /cargo/<crate>/... source paths; the reordered variant retains /cargo/registry/src/<index>/<crate>/.... The registry mapping must follow the broader Cargo-home mapping. Receipts: report, build log.

The minimal overlap test confirms the cause. Moving only the registry source remap ahead of its enclosing CARGO_HOME mapping reproduces the same over-cap bytes and hash as the fully sorted order: 21,275,024 bytes (+49,152), with `.rodata` +`0xc000`. The cap-sized output uses canonical `/cargo/<crate>/...` source paths; the reordered variant retains `/cargo/registry/src/<index>/<crate>/...`. The registry mapping must follow the broader Cargo-home mapping. Receipts: [report](https://git.kayg.org/attachments/ca3123d3-ce71-4a46-8cb2-8843ee8abb53), [build log](https://git.kayg.org/attachments/c5cdab8f-a0bd-4f5d-9348-ff3121562c4d).

kayg commented

2026-10-19 07:09:37 +00:00

Author

Owner

sccache 0.18 documentation describes SCCACHE_BASEDIRS as path stripping for cache-key computation. Its Rust key support is still under upstream PR #2813 as of 2026-10-19, so this change does not depend on that option. The reproducibility gate captures the compiler arguments at the sccache boundary, verifies each remap is present before forwarding, and checks that no checkout/Cargo/Rustup/target root appears in either output. Sources: sccache configuration, Rust key support PR.

sccache 0.18 documentation describes `SCCACHE_BASEDIRS` as path stripping for cache-key computation. Its Rust key support is still under upstream PR #2813 as of 2026-10-19, so this change does not depend on that option. The reproducibility gate captures the compiler arguments at the sccache boundary, verifies each remap is present before forwarding, and checks that no checkout/Cargo/Rustup/target root appears in either output. Sources: [sccache configuration](https://github.com/mozilla/sccache/blob/f94ee004e262d5d8b4155598c194a73b77cad075/docs/Configuration.md), [Rust key support PR](https://github.com/mozilla/sccache/pull/2813).

kayg commented

2026-10-19 07:11:05 +00:00

Author

Owner

Rust wrapper step committed as 84e15e05c. Focused check output:

Rust remapping: differently named checkout roots produce identical stripped artifacts

The shared helper preserves the pinned compiler and the existing sccache chain. Refs: #246.

Rust wrapper step committed as `84e15e05c`. Focused check output: ``` Rust remapping: differently named checkout roots produce identical stripped artifacts ``` The shared helper preserves the pinned compiler and the existing sccache chain. Refs: #246.

kayg commented

2026-10-19 07:11:45 +00:00

Author

Owner

Native wrapper step committed as be6b59306. Focused check output:

x86_64-unknown-linux-musl: exact define removed; build directories remapped; explicit paths preserved; target override refused
aarch64-unknown-linux-musl: exact define removed; build directories remapped; explicit paths preserved; target override refused
x86_64 OpenSSL source: AVX-512 GCM dispatch removed idempotently

2 pass
0 fail
18 expect() calls

The native wrapper preserves pre-supplied maps so static-cargo can make them cache-visible without duplication. Refs: #246.

Native wrapper step committed as `be6b59306`. Focused check output: ``` x86_64-unknown-linux-musl: exact define removed; build directories remapped; explicit paths preserved; target override refused aarch64-unknown-linux-musl: exact define removed; build directories remapped; explicit paths preserved; target override refused x86_64 OpenSSL source: AVX-512 GCM dispatch removed idempotently 2 pass 0 fail 18 expect() calls ``` The native wrapper preserves pre-supplied maps so static-cargo can make them cache-visible without duplication. Refs: #246.

kayg commented

2026-10-19 07:12:41 +00:00

Author

Owner

Static Cargo integration committed as 4eec0d83a. The focused metadata/remap test passed:

2 pass
0 fail
121 expect() calls

The test asserts registry source mappings follow CARGO_HOME and Rustup mapping reaches the cache wrapper. Experimental x86 runs established the order constraint: old order + Rustup map is 21,225,872 bytes (cap); moving registry roots before CARGO_HOME is 21,275,024 bytes (+49,152). Receipts are in the preceding issue comments. Refs: #246.

Static Cargo integration committed as `4eec0d83a`. The focused metadata/remap test passed: ``` 2 pass 0 fail 121 expect() calls ``` The test asserts registry source mappings follow CARGO_HOME and Rustup mapping reaches the cache wrapper. Experimental x86 runs established the order constraint: old order + Rustup map is 21,225,872 bytes (cap); moving registry roots before CARGO_HOME is 21,275,024 bytes (+49,152). Receipts are in the preceding issue comments. Refs: #246.

kayg commented

2026-10-19 07:13:19 +00:00

Author

Owner

Cross-root static reproducibility gate committed as ad9a337ac. Static sources pass syntax checks:

python3 -m py_compile scripts/test-static-reproducibility.py
bash -n scripts/verify.d/rust.sh
git diff --check

The gate now rebuilds both targets under distinct checkout, Cargo registry, Rustup and target roots, captures compiler args at the cache boundary, asserts registry override order, compares full bytes, and scans the alternate image for all roots. Full execution is scheduled in the final scripts/verify fast. Refs: #246.

Cross-root static reproducibility gate committed as `ad9a337ac`. Static sources pass syntax checks: ``` python3 -m py_compile scripts/test-static-reproducibility.py bash -n scripts/verify.d/rust.sh git diff --check ``` The gate now rebuilds both targets under distinct checkout, Cargo registry, Rustup and target roots, captures compiler args at the cache boundary, asserts registry override order, compares full bytes, and scans the alternate image for all roots. Full execution is scheduled in the final `scripts/verify fast`. Refs: #246.

kayg commented

2026-10-19 07:32:26 +00:00

Author

Owner

The first scripts/verify iterate invocation at ad9a337ac exited 1 in SSH previous_attempt_shell because the gate environment omitted INFRATERNAL_TEST_RUNTIME_HELPER; the test reported KeyError: 'INFRATERNAL_TEST_RUNTIME_HELPER'. Earlier checks, including clippy and the main Rust suites, passed. The helper binary is present in this job's target directory; I am rerunning iterate with it set and a short scratch state root. Receipt.

The first `scripts/verify iterate` invocation at `ad9a337ac` exited 1 in SSH `previous_attempt_shell` because the gate environment omitted `INFRATERNAL_TEST_RUNTIME_HELPER`; the test reported `KeyError: 'INFRATERNAL_TEST_RUNTIME_HELPER'`. Earlier checks, including clippy and the main Rust suites, passed. The helper binary is present in this job's target directory; I am rerunning iterate with it set and a short scratch state root. [Receipt](https://git.kayg.org/attachments/0c42357e-557a-4c9c-a696-a069801791bd).

kayg commented

2026-10-19 07:38:21 +00:00

Author

Owner

The second scripts/verify iterate attempt at ad9a337ac exited 1 because the configured state root was a symlink; restart tests rejected the symlink parent and one assertion observed the alias path. Receipt: iterate-symlink-state-root.txt. I am rerunning with the physical scratch root and the recovery helper configured.

The second `scripts/verify iterate` attempt at `ad9a337ac` exited 1 because the configured state root was a symlink; restart tests rejected the symlink parent and one assertion observed the alias path. Receipt: [iterate-symlink-state-root.txt](https://git.kayg.org/attachments/d0936ad1-ea49-49e0-9094-bfb8a020ff69). I am rerunning with the physical scratch root and the recovery helper configured.

kayg commented

2026-10-19 07:49:58 +00:00

Author

Owner

scripts/verify iterate at ad9a337ac reached Rust tests with the physical job state root, and restart tests passed. agent_daemon failed before its oracle ran because the physical state path made its AF_UNIX socket path 110 bytes (limit checked by the fixture). Full receipt: iterate-physical-root-clean.log. I will rerun with a shorter physical scratch directory (/home/kayg/.cache/codex-scratch/spr) to satisfy both path validation and symlink checks.

`scripts/verify iterate` at `ad9a337ac` reached Rust tests with the physical job state root, and restart tests passed. `agent_daemon` failed before its oracle ran because the physical state path made its AF_UNIX socket path 110 bytes (limit checked by the fixture). Full receipt: [iterate-physical-root-clean.log](https://git.kayg.org/attachments/084e017a-0528-4bad-93ae-f7b08c1756d1). I will rerun with a shorter physical scratch directory (`/home/kayg/.cache/codex-scratch/spr`) to satisfy both path validation and symlink checks.

kayg commented

2026-10-19 07:56:54 +00:00

Author

Owner

scripts/verify iterate at ad9a337ac reached the agent crate; all main CLI/restart suites passed, but plugin_control::tests::retained_runtime_registry_keeps_lifetime_ownership_after_listener_abort failed at crates/infraternal-agent/src/plugin_control.rs:1040 (probe.try_lock().is_ok()). The test process finished 143 passed, 1 failed, 20 ignored. vt_credit passed in 19.78 s. Full receipt: iterate-short-root.log. I am rerunning that test alone to determine whether it reproduces outside the concurrent crate suite.

`scripts/verify iterate` at `ad9a337ac` reached the agent crate; all main CLI/restart suites passed, but `plugin_control::tests::retained_runtime_registry_keeps_lifetime_ownership_after_listener_abort` failed at `crates/infraternal-agent/src/plugin_control.rs:1040` (`probe.try_lock().is_ok()`). The test process finished 143 passed, 1 failed, 20 ignored. `vt_credit` passed in 19.78 s. Full receipt: [iterate-short-root.log](https://git.kayg.org/attachments/b3af0dbe-50d6-4845-b346-396f7ddfd8bf). I am rerunning that test alone to determine whether it reproduces outside the concurrent crate suite.

kayg commented

2026-10-19 08:04:27 +00:00

Author

Owner

The isolated rerun used --features remote and passed: 1 passed, 0 failed, 162 filtered, 0.00 s. Receipt: plugin-lock-isolated-remote.log. I am checking the agent crate suite with the same target and feature set to see whether the failure depends on concurrent tests.

The isolated rerun used `--features remote` and passed: 1 passed, 0 failed, 162 filtered, 0.00 s. Receipt: [plugin-lock-isolated-remote.log](https://git.kayg.org/attachments/68fb46b9-f23b-402d-8400-7f40cd803b88). I am checking the agent crate suite with the same target and feature set to see whether the failure depends on concurrent tests.

kayg commented

2026-10-19 08:24:38 +00:00

Author

Owner

Restarted from clean branch static-path-remap at ad9a337acfac83118b852c8d98ec43a79959e769. Current base is origin/dev e2f1c646e (the branch is four commits ahead and 28 behind); no remote static-path-remap branch exists. Prior issue evidence isolates the +49,152-byte regression to placing registry remaps before CARGO_HOME. I am merging current dev, then will run the focused reproducibility gate and final verify.

Restarted from clean branch `static-path-remap` at `ad9a337acfac83118b852c8d98ec43a79959e769`. Current base is `origin/dev` `e2f1c646e` (the branch is four commits ahead and 28 behind); no remote `static-path-remap` branch exists. Prior issue evidence isolates the +49,152-byte regression to placing registry remaps before CARGO_HOME. I am merging current dev, then will run the focused reproducibility gate and final verify.

kayg commented

2026-10-19 08:25:24 +00:00

Author

Owner

Merged current origin/dev e2f1c646e0d7d0b06ae742cd9aed6cdb7943f592 into static-path-remap; merge commit f44567bbae4d9438ce6e2e29bd9849993b0710e0. Merge was clean. Existing four implementation commits are preserved; proceeding to branch-head verification and the cross-root static builds.

Merged current `origin/dev` `e2f1c646e0d7d0b06ae742cd9aed6cdb7943f592` into `static-path-remap`; merge commit `f44567bbae4d9438ce6e2e29bd9849993b0710e0`. Merge was clean. Existing four implementation commits are preserved; proceeding to branch-head verification and the cross-root static builds.

kayg commented

2026-10-19 08:50:14 +00:00

Author

Owner

Finding in the new cross-root gate: the observer's first-capture rule records compiler capability probes before compilation. The Rust capture has only -vV (2 arguments); the x86 native capture has -dM -E -x (6 arguments). Neither includes the remap flags the gate expects, so it would report false missing-remap failures despite the actual compile invocations being remapped. I will change capture selection to retain the first invocation carrying a path-remap flag, then rerun the final gate. Evidence: Rust probe capture, native probe capture.

Finding in the new cross-root gate: the observer's first-capture rule records compiler capability probes before compilation. The Rust capture has only `-vV` (2 arguments); the x86 native capture has `-dM -E -x` (6 arguments). Neither includes the remap flags the gate expects, so it would report false missing-remap failures despite the actual compile invocations being remapped. I will change capture selection to retain the first invocation carrying a path-remap flag, then rerun the final gate. Evidence: [Rust probe capture](https://git.kayg.org/attachments/9612cad0-d9b4-4404-93d5-87b9cb5e8650), [native probe capture](https://git.kayg.org/attachments/946580b9-6fd4-410d-828e-9eee9357cd1c).

kayg commented

2026-10-19 09:08:23 +00:00

Author

Owner

Finding from scripts/verify fast at f44567b: the workspace test row failed in infraternal-plugin-ssh --test recovery_support because three tests could not bind their Unix-domain sockets (AF_UNIX path too long). Their generated path starts at /home/kayg/.cache/codex-scratch/spr; this is a test-state-root length issue, not a build failure. I will rerun with a short per-job fixture root and then run the final gate. Evidence: SSH state-root failure excerpt.

Finding from `scripts/verify fast` at `f44567b`: the workspace test row failed in `infraternal-plugin-ssh --test recovery_support` because three tests could not bind their Unix-domain sockets (`AF_UNIX path too long`). Their generated path starts at `/home/kayg/.cache/codex-scratch/spr`; this is a test-state-root length issue, not a build failure. I will rerun with a short per-job fixture root and then run the final gate. Evidence: [SSH state-root failure excerpt](https://git.kayg.org/attachments/e07bcb25-3e11-43e9-bdc9-446f7f28c687).

kayg commented

2026-10-19 09:09:08 +00:00

Author

Owner

The path-length issue is resolved by using /tmp/sprm-246 only as the short SSH fixture state root; Rust targets and build worktrees remain under the job scratch directory. Focused rerun: cargo test --locked -p infraternal-plugin-ssh --test recovery_support — 11 passed, 0 failed, 1 ignored. Evidence: short-root SSH recovery test.

The path-length issue is resolved by using `/tmp/sprm-246` only as the short SSH fixture state root; Rust targets and build worktrees remain under the job scratch directory. Focused rerun: `cargo test --locked -p infraternal-plugin-ssh --test recovery_support` — 11 passed, 0 failed, 1 ignored. Evidence: [short-root SSH recovery test](https://git.kayg.org/attachments/4a60f43a-9355-48e8-8157-0438cb067b40).

kayg commented

2026-10-19 09:11:13 +00:00

Author

Owner

Committed aef5efbfa9dcb7e33c76c3f74c2954dfbe1fadef: the sccache observer now skips Rust -vV and native preprocessor probes, capturing the first remapped compile invocation. The focused observer check passed for Rust, native x86_64, and native AArch64; evidence: capture observer check.

The first full cross-root build produced exact, byte-identical images at both caps: x86_64 21,225,872 B (SHA-256 b965dc1a73e92d2306fd08a2b588abf69c080291259f564f9794d5967071f19f) and AArch64 19,427,688 B (SHA-256 75ec2292efeaeb4d452ab6729bd44daeee153e52b873732328c966500b790b6f). The observer's false missing-remap result is corrected in aef5efbfa; the SSH state-root setup failure is resolved with a short fixture root and its focused suite passes 11/0/1.

origin/dev has advanced from e2f1c646e to 149793e5a (D309/D310) during verification. I am merging that current base before the final gate.

Committed `aef5efbfa9dcb7e33c76c3f74c2954dfbe1fadef`: the sccache observer now skips Rust `-vV` and native preprocessor probes, capturing the first remapped compile invocation. The focused observer check passed for Rust, native x86_64, and native AArch64; evidence: [capture observer check](https://git.kayg.org/attachments/17e3ce56-8fd4-4983-b21e-29ecc5f79cce). The first full cross-root build produced exact, byte-identical images at both caps: x86_64 21,225,872 B (SHA-256 `b965dc1a73e92d2306fd08a2b588abf69c080291259f564f9794d5967071f19f`) and AArch64 19,427,688 B (SHA-256 `75ec2292efeaeb4d452ab6729bd44daeee153e52b873732328c966500b790b6f`). The observer's false missing-remap result is corrected in `aef5efbfa`; the SSH state-root setup failure is resolved with a short fixture root and its focused suite passes 11/0/1. `origin/dev` has advanced from `e2f1c646e` to `149793e5a` (D309/D310) during verification. I am merging that current base before the final gate.

kayg commented

2026-10-19 09:12:06 +00:00

Author

Owner

Merged current origin/dev 149793e5abb8aa27e563e1a326252073f6498632 (D309/D310) into static-path-remap; merge commit 709efb4518550403a6946ed69001648a8c240436. The merge was clean and updated only DECISIONS.md. Running the final gate on this current base with the short fixture state root.

Merged current `origin/dev` `149793e5abb8aa27e563e1a326252073f6498632` (D309/D310) into `static-path-remap`; merge commit `709efb4518550403a6946ed69001648a8c240436`. The merge was clean and updated only `DECISIONS.md`. Running the final gate on this current base with the short fixture state root.

kayg commented

2026-10-19 09:37:48 +00:00

Author

Owner

The next full gate cleared the SSH socket-length tests with /tmp/sprm-246, but its workspace test row then failed in six restart tests: the tmpfs fixture root is on another device (CrossesDevices for a hard link) and does not permit executing the fixture image (PermissionDenied). The cross-root static builds still passed. I will rerun the affected tests with a short /proc/<parent>/fd/9/.. alias to this job's scratch root; the final path resolves to a real same-device directory. Evidence: restart fixture failure excerpt.

The next full gate cleared the SSH socket-length tests with `/tmp/sprm-246`, but its workspace test row then failed in six restart tests: the tmpfs fixture root is on another device (`CrossesDevices` for a hard link) and does not permit executing the fixture image (`PermissionDenied`). The cross-root static builds still passed. I will rerun the affected tests with a short `/proc/<parent>/fd/9/..` alias to this job's scratch root; the final path resolves to a real same-device directory. Evidence: [restart fixture failure excerpt](https://git.kayg.org/attachments/322ff6e5-493c-4b7d-8e84-edbef1147774).

kayg commented

2026-10-19 09:54:10 +00:00

Author

Owner

The short /tmp bind root fixes socket length but remains a distinct mount from the test executable even though both report device 2049. The focused restart run had 87 passed, 12 failed, 1 ignored: hard-link cases returned CrossesDevices, and path admission returned PermissionDenied with follow-on timeouts. Receipt: fixture-root-restart-tests.log. I’m making both the test executable and fixture root use paths under one private mount.

The short `/tmp` bind root fixes socket length but remains a distinct mount from the test executable even though both report device 2049. The focused restart run had 87 passed, 12 failed, 1 ignored: hard-link cases returned `CrossesDevices`, and path admission returned `PermissionDenied` with follow-on timeouts. Receipt: [fixture-root-restart-tests.log](https://git.kayg.org/attachments/0893141f-7470-4557-aff0-d1a1172d9588). I’m making both the test executable and fixture root use paths under one private mount.

kayg commented

2026-10-19 10:07:08 +00:00

Author

Owner

The same-mount run executes the test binary from /tmp/target-a, but verified_exec_and_late_refusal_preserve_listener_and_ca still failed before the rest of the serial restart suite completed. That test creates its fixture below $HOME/.cache/codex-scratch/release-pipeline-r4 rather than INFRATERNA_TEST_STATE_ROOT; I stopped the run to keep its fixture activity inside this job’s scratch. Receipt: fixture-root-same-mount-tests.log. I’m isolating the test process HOME under the job scratch and will rerun.

The same-mount run executes the test binary from `/tmp/target-a`, but `verified_exec_and_late_refusal_preserve_listener_and_ca` still failed before the rest of the serial restart suite completed. That test creates its fixture below `$HOME/.cache/codex-scratch/release-pipeline-r4` rather than `INFRATERNA_TEST_STATE_ROOT`; I stopped the run to keep its fixture activity inside this job’s scratch. Receipt: [fixture-root-same-mount-tests.log](https://git.kayg.org/attachments/ad9eb90a-3a39-4a8d-8dc4-cf8741d19f59). I’m isolating the test process HOME under the job scratch and will rerun.

kayg commented

2026-10-19 10:23:02 +00:00

Author

Owner

Focused host tests pass with a per-test-binary Cargo runner: infraternal --bin 99 passed, 0 failed, 1 ignored; SSH recovery_support 11 passed, 0 failed, 1 ignored. The restart fixtures and target binaries share the job scratch filesystem; only the SSH suite uses /tmp for its short Unix socket root. HOME=/home/kayg keeps existing HOME-derived socket fixtures within sun_path, and TMPDIR=/tmp is used only for small transient test fixtures. Receipt: fixture-root-targeted-host-tests.log. I’m running the final full gate with this setup.

Focused host tests pass with a per-test-binary Cargo runner: `infraternal --bin` 99 passed, 0 failed, 1 ignored; SSH `recovery_support` 11 passed, 0 failed, 1 ignored. The restart fixtures and target binaries share the job scratch filesystem; only the SSH suite uses `/tmp` for its short Unix socket root. `HOME=/home/kayg` keeps existing HOME-derived socket fixtures within `sun_path`, and `TMPDIR=/tmp` is used only for small transient test fixtures. Receipt: [fixture-root-targeted-host-tests.log](https://git.kayg.org/attachments/d844465f-447d-412d-8774-d0c206deecb4). I’m running the final full gate with this setup.

kayg commented

2026-10-19 10:27:47 +00:00

Author

Owner

Final fast attempt on 709efb451: both static artifacts passed at their exact caps (x86_64 21,225,872 B; AArch64 19,427,688 B), and the two metadata tests passed. The integration test agent_daemon failed before exercising its behavior because INFRATERNA_TEST_STATE_ROOT=/home/kayg/.cache/codex-scratch/static-path-remap/test-state made its Unix socket path 121 bytes; scripts/agent_fixture.py:55 rejects paths >=108 bytes. Evidence: /home/kayg/.cache/codex-scratch/static-path-remap/evidence/verify-fast-final.log. I will rerun with a short job-unique fixture root, then repeat the full gate.

Final fast attempt on 709efb451: both static artifacts passed at their exact caps (x86_64 21,225,872 B; AArch64 19,427,688 B), and the two metadata tests passed. The integration test `agent_daemon` failed before exercising its behavior because `INFRATERNA_TEST_STATE_ROOT=/home/kayg/.cache/codex-scratch/static-path-remap/test-state` made its Unix socket path 121 bytes; `scripts/agent_fixture.py:55` rejects paths >=108 bytes. Evidence: `/home/kayg/.cache/codex-scratch/static-path-remap/evidence/verify-fast-final.log`. I will rerun with a short job-unique fixture root, then repeat the full gate.

kayg commented

2026-10-19 10:45:19 +00:00

Author

Owner

Harness correction on 709efb451: moved the isolated test state root to /home/kayg/.cache/codex-scratch/static246 (107-byte maximum fixture socket path; below the 108-byte AF_UNIX bound). Focused cargo test --locked -p infraternal --test agent_daemon -- --nocapture: 1 passed, 0 failed; all 24 restart cases passed over 4,710,396 bytes. Receipt: agent-daemon-short-root.log. Rerunning scripts/verify fast now.

Harness correction on 709efb451: moved the isolated test state root to `/home/kayg/.cache/codex-scratch/static246` (107-byte maximum fixture socket path; below the 108-byte AF_UNIX bound). Focused `cargo test --locked -p infraternal --test agent_daemon -- --nocapture`: 1 passed, 0 failed; all 24 restart cases passed over 4,710,396 bytes. Receipt: [agent-daemon-short-root.log](https://git.kayg.org/attachments/4ef5b215-a0ce-4fb8-afb2-185bb2e6e760). Rerunning `scripts/verify fast` now.

kayg commented

2026-10-19 11:30:07 +00:00

Author

Owner

Completed on static-path-remap at 709efb4518550403a6946ed69001648a8c240436 (merge base 149793e5abb8aa27e563e1a326252073f6498632). Rust and native static-build compiler inputs now remap checkout, Cargo home and registry, Rustup, and target paths. The static gate builds both targets under distinct roots, checks remaps at the sccache boundary, and requires byte-identical artifacts.

Cross-root outputs are equal and at their exact caps: x86_64 21,225,872 B, SHA-256 b965dc1a73e92d2306fd08a2b588abf69c080291259f564f9794d5967071f19f; AArch64 19,427,688 B, SHA-256 75ec2292efeaeb4d452ab6729bd44daeee153e52b873732328c966500b790b6f. Compared with the Rust/Cargo-identical f29713863e5da84f4e7a9dd7d2702557b4287755 baseline, current size delta is +8 B x86_64 and 0 B AArch64.

Receipts: final verify log, cross-root report, baseline sizes.

Final scripts/verify fast exited 0. Rust gate output:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| test                      | PASS         |
| cargo-deny                | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

Metadata compiler checks: 2 pass, 0 fail, 121 expect() calls. TypeScript protocol checks: 37 pass, 0 fail, 34463 expect() calls, Ran 37 tests across 10 files. [2.77s].

Branch remains local per the worktree boundary. Refs: #246

Completed on `static-path-remap` at `709efb4518550403a6946ed69001648a8c240436` (merge base `149793e5abb8aa27e563e1a326252073f6498632`). Rust and native static-build compiler inputs now remap checkout, Cargo home and registry, Rustup, and target paths. The static gate builds both targets under distinct roots, checks remaps at the sccache boundary, and requires byte-identical artifacts. Cross-root outputs are equal and at their exact caps: x86_64 21,225,872 B, SHA-256 `b965dc1a73e92d2306fd08a2b588abf69c080291259f564f9794d5967071f19f`; AArch64 19,427,688 B, SHA-256 `75ec2292efeaeb4d452ab6729bd44daeee153e52b873732328c966500b790b6f`. Compared with the Rust/Cargo-identical `f29713863e5da84f4e7a9dd7d2702557b4287755` baseline, current size delta is +8 B x86_64 and 0 B AArch64. Receipts: [final verify log](https://git.kayg.org/attachments/207146e5-e81e-4816-92af-0a0c67b968e2), [cross-root report](https://git.kayg.org/attachments/a079452a-d141-4efb-93e5-adf698275f0a), [baseline sizes](https://git.kayg.org/attachments/831a2f22-fde1-4b73-bde9-dc3a73a0ac12). Final `scripts/verify fast` exited 0. Rust gate output: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | test | PASS | | cargo-deny | PASS | | static builds | PASS | +---------------------------+--------------+ ``` Metadata compiler checks: `2 pass`, `0 fail`, `121 expect() calls`. TypeScript protocol checks: `37 pass`, `0 fail`, `34463 expect() calls`, `Ran 37 tests across 10 files. [2.77s]`. Branch remains local per the worktree boundary. Refs: #246

kayg referenced this issue

2026-10-19 16:19:24 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 17:01:19 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 21:19:45 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 21:20:03 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 21:22:11 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 21:33:37 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 21:59:58 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-19 21:59:59 +00:00

Mesh view: connected nodes shown offline and named by internal id #236

kayg referenced this issue

2026-10-20 01:01:53 +00:00

Mesh view: connected nodes shown offline and named by internal id #236

kayg referenced this issue

2026-10-20 01:01:54 +00:00

Add-host retry is blocked forever by a failed attempt's rollback receipt (production) #230

kayg referenced this issue

2026-10-20 01:01:55 +00:00

Status board: work in flight (kept current) #24

kayg commented

2026-10-28 17:53:26 +00:00

Author

Owner

Closed by the orchestrator during the 2026-10-28 handover (#254), on the owner's request to close finished issues. Evidence: Cross-root reproducible static builds merged to dev (702e671a0, 709efb451 ancestor of dev) with passing final gate. Reopen if anything is still missing.

Closed by the orchestrator during the 2026-10-28 handover (#254), on the owner's request to close finished issues. Evidence: Cross-root reproducible static builds merged to dev (702e671a0, 709efb451 ancestor of dev) with passing final gate. Reopen if anything is still missing.

kayg closed this issue

2026-10-28 17:53:26 +00:00

~~kayg referenced this issue 2026-10-28 17:53:55 +00:00~~

Orchestrator handover 2026-10-28: moving to a new machine #254

kayg referenced this issue

2026-10-28 18:00:45 +00:00

Orchestrator handover 2026-10-28: moving to a new machine #254

kayg referenced this issue

2026-11-03 18:56:49 +00:00

J149: verify quick pre-check and shared sccache on build VMs #329

kayg referenced this issue

2026-11-03 20:40:36 +00:00