D217: in-memory CA handoff on update/restart; deploys stop locking the CA #154
Open
opened 2026-10-10 10:40:47 +00:00 by kayg
·
242 comments
No Branch/Tag specified
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/infraternal#154
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
D217 (owner, 2026-10-10): updates and deploys must hand the unlocked CA key to the new server process in memory, as D168 always said. A plain stop/start counts as a cold start and locks the CA, and the orchestrator's deploy script did exactly that on every deploy today.
Scope: the server's in-place update/restart path, a local authenticated handoff channel, the deploy procedure, and tests.
Starting #154 on ca-handoff, base
068574588f(release-prod-2). Read CLAUDE.md, DECISIONS.md, architecture/process and D168/D217 from origin/dev. #154 has no comments or linked parent. Existing ca_memory.rs handoff is process-only; remote_startup.rs always constructs a cold-locked source. Will implement authenticated inherited-socketpair exec restart, local-owner restart control, reload integration, parser/process tests and deployment procedure. No deployment.Design/contract proposal (D168/D217): bounded v1 frame (magic, issuer DER length, 32-byte Ed25519 seed, issuer DER) transferred on an anonymous Unix socketpair inherited as stdin exclusively by exec. No token, named key socket, secret arguments/environment, or new dependency. The receiver checks SO_PEERCRED UID and PID against its own identity (PID survives exec), validates the frame and retained issuer/key before unlock, and otherwise continues cold-locked. Successful exec erases the old address space even before receiver validation, stronger than retaining old signing material until acknowledgement; failed exec terminates without externalizing material. Serialized seed/frame buffers are zeroized. The separate restart-request socket carries only a binary pathname, restricted to the service UID in a 0700 directory/0600 socket. SIGHUP requests the same exec path with current binary. This is an additive local control contract, not a public network protocol change.
Security finding: std::io::Stdin uses a global buffered reader, which could retain an unzeroized copy of the signing seed after a bounded handoff read. ca_handoff.rs uses direct nix::unistd::read into Zeroizing storage instead, replaces stdin with /dev/null before child startup, and requires canonical exact issuer DER (OpenSSL can accept a DER prefix). Bounds: 32-byte seed, 2048-byte issuer maximum, 2090-byte frame maximum, 2-second socket deadline, one channel per restart rather than one per session/node. Host load during initial builds: 24.92/24.85/18.38 on 8 cores; no load generators started.
Initial gate found a fixture prerequisite, not a product regression: verify-agent-attach-credit.py creates target/ac without creating target itself, and this fresh worktree had no target directory (FileNotFoundError at script line 17). Created an ignored target symlink to this job's scratch target directory, keeping build/test output off /tmp and out of tracked files. Clippy's new module doc-markdown error was corrected; dedicated workspace clippy now passes. Full gate rerun will validate both corrections.
Correction to fixture setup note: target/ is an empty ignored fixture directory, not a symlink (the repository ignores target/ directories but would track a target symlink). CARGO_TARGET_DIR remains /home/kayg/.cache/codex-scratch/ca-handoff/target for every Rust build. No build targets are in the worktree or /tmp.
Handoff module tests: 3 passed, 0 failed, including the real socketpair signer roundtrip, cold-start lock, every frame truncation and every-byte mutation, and 4097 deterministic malformed inputs. Corrected terminal fixture test: Attach without Credit: no Output/Snapshot for 350 ms; control responsive; funded replay 149 bytes in 664.5 ms: PASS. Transfer disables process dumpability before exporting/importing the seed; successful exec replaces all old key references. Initial unscoped network self-tests failed under shared host scheduling (321.68/379.39 ms and 420.98/932.98 ms p50/p99 for nominal 300±30 ms); CPUWeight=10000 scope self-test passes unchanged at 311.85/315.85 ms. Full gate rerun is in progress.
Completed CA handoff module:
5564516981, pushed ca-handoff. Real socketpair/property tests: 3 passed, 0 failed; maximum frame 2090 bytes, one per server. x86 static: 19,896,992 bytes; size ceiling is the only remaining fast-gate failure (D205). Supplemental TS gate passes (36 protocol tests, 0 failures); full logs will also be committed as final evidence.Verbatim scripts/verify fast output:
Registry gate caught a new action declaration error before commit: PluginDescriptor::validate requires action IDs to start with their plugin ID. The proposed server.restart ID invalidated the remote descriptor, causing 13 configuration tests to fail with InvalidConfiguration (e.g. auth/tests.rs:1274 and remote_startup_tests.rs:73). Corrected to remote.server.restart; the CLI spelling remains infraternal server restart. No contract or validator relaxation. Full gate rerun follows.
Restart path hardening (before runtime wiring): an owner-authenticated request alone is insufficient if another local user can replace the selected pathname. The exec adapter will reject symlinks, set-ID executables, group/world-writable files/directories and ownership outside the service UID or root, checking the original path before canonicalization and checking again on the server. This prevents an untrusted symlink from redirecting to even a root-owned program that prints stdin. SIGHUP applies the same executable checks. Staging remains beside the existing trusted binary, as the deployment brief requires.
Production-line baseline measured from an archived, unchanged
068574588fcheckout with the pinned 1.99.0 Rust / 0.15.2 Zig static release profile: x86_64 19,896,992 bytes; aarch64 17,729,832 bytes; both stripped static PIE. First handoff-module commit55645169x86_64 is also 19,896,992 bytes (0-byte size delta; IPC is not wired yet). Thus the old fast-gate size ceiling already fails on the production base, independently of #154. No new dependencies; final live-path deltas will be measured against these binaries.Compatibility choice before CLI wiring: server startup attempts import only through authenticated stdin (kernel UID/PID first, before any read), then replaces stdin with /dev/null. No private handoff argv flag is needed. This avoids rejecting an older executable on rollback merely because it lacks that flag; a compatible normal server that lacks import stays cold-locked, per D168. Warm continuity requires a running predecessor with restart/handoff support; the first rollout from a pre-#154 binary cannot retroactively add its missing IPC endpoint. No deploy is performed in this job.
Completed shared restart action:
de5621cf8a, pushed ca-handoff. Correct remote.server.restart namespace; CLI help and skills share its D202 instructions. x86 static 19,897,280 bytes (+288 bytes over measured production base); no new dependency or runtime task. Supplemental TS checks pass.Verbatim scripts/verify fast output:
Completed lifecycle wiring:
77785aa7, pushed ca-handoff. CLI restart, --binary, same-PID exec and SIGHUP reload; private control endpoint; verified binary paths; failed import remains cold-locked. Help and absent-service --json checked against real CLI. x86 stripped static 19,938,848 bytes (+41,856 vs unchanged production base 19,896,992); no new dependencies. All non-size fast rows PASS, legacy size-only failure accepted under D205. Supplemental TS gate passed in the previous step; TS unchanged.Verbatim scripts/verify fast:
Finding: new real control test failed before the fix (test-private-ancestor-before.log, owner restart deadline 5.00s). The executable path lives under /home/kayg (0700) and .cache (0700), but owned build directories are 0775. An unconditional ancestor group-write ban rejects the running executable despite the inaccessible private ancestor. Updating path validation to walk root-to-leaf, permit directory group writes only below an owned private ancestor, and retain refusal of foreign ownership, symlinks, world-write directories, writable executable files and set-ID files. No signer export occurs before path validation.
CA handoff suite: 7 passed, 0 failed, 1 explicitly ignored shipping benchmark; includes same-PID exec, foreign PID refusal, malformed frames and abrupt receiver stop.
Failing regression output:
Author security review found a lexical path escape: parent components could leave a validated private ancestor. Added a failing regression first (private_path_prefix_cannot_authorize_a_parent_directory_escape); then required rooted executable paths with no ParentDir components before validating ownership and permissions. Regression output below.
Debug shipping proof passed: 25 same-PID replacements (update/rollback/reload), CA remained unlocked; 128 real PTYs at the single-host limit retained exact shim PIDs and reattached with successful new input/output. 10000 retained host records, 50 users. p50 43.261ms, max 73.151ms; new HTTP TCP refusal bracket p50 30.955ms, max 57.493ms (configured 1ms probe). Cold and malformed shipping imports locked, non-owner restart socket access refused. Final measurements will use the stripped release build. Fixture reuses isolated shell profile and obeys odd terminal streams; setup is paced to avoid bursty DB authority reads on this shared host.
Gate finding: the existing auth::tests::reviewer_tests::reviewer_disk_restart_and_concurrent_resume failed with SQLite code 5 (database is locked) during concurrent recovery_start_clock, at reviewer_tests.rs:67. No recovery/SQL code changed in this lane. This is not one of the authorized #118 exceptions, so no baseline waiver is used: rerunning the complete fast gate and requiring its test row to pass. The isolated unchanged test passed in 3.10s; outputs below.
Completed verification/hardening step:
5197e376, pushed ca-handoff. Same-PID/foreign-PID real exec, partial seed import abrupt stop, cold/truncated/wrong-version cases, bounded property corpus and fuzz target; private endpoint/path regressions pass. Shipping release proof: 25 replacements including update/rollback, omitted --binary restart and SIGHUP; 128 live PTYs preserve exact shim PIDs, reattach and produce new output. 10000 retained hosts and 50 users (not claimed as live QUIC connections). CA suite 42 PASS, remote suite 49 PASS; ignored diagnostics listed in attached output.Release metrics and machine/load, verbatim:
Sizes: x86 19938720 bytes (+41728 vs base19986992; -128 from lifecycle commit); ARM 17769176 bytes (+39344 vs base17729832). Both stripped/static, no new dependencies. All non-size fast rows PASS; legacy size-only failure under D205. Supplemental TS checks PASS.
Verbatim scripts/verify fast:
Documentation step
44e2572dpushed on ca-handoff: docs/operations/deploy.md stages verified executables, invokes warm replacement, checks health plus CA state and terminal continuity, and rolls back through the same handoff; includes systemd ExecReload/KillMode snippet and initial cold compatibility transition. Author review and measured evidence summary added. Fast gate: fmt/VT fmt/clippy/test/cargo-deny PASS; legacy size-only FAIL under D205. Documentation adds zero binary bytes (x86 19938720, ARM 17769176).Packaging finding: the repository ignores *.log. Explicitly adding the reviewed evidence transcripts in the next atomic commit so committed links resolve; the source and measurements are unchanged.
Finished #154 on ca-handoff, head
3d10854b45; all six atomic commits pushed. No deployment. Worktree source is clean. Operations procedure: docs/operations/deploy.md; author review and tracked evidence: evidence/ca-handoff/.Implemented anonymous inherited socketpair transfer authenticated by service UID and the preserved PID, bounded/zeroized frame import, exact retained-issuer validation, stdin detachment before children, same-PID CLI restart/default/--binary and SIGHUP ExecReload. Exec destroys the old signer address space before import; failures remain cold-locked. Initial installation from a release without handoff support requires one cold transition, documented.
Shipping stripped release proof: 25 replacements, CA remained unlocked; 128 actual PTYs retain their original shim PIDs, all reattach, resumed input/output verified. Scale: 10000 retained host records, 50 users; these are not claimed as live QUIC connections. Restart p50=30.736ms, max=39.541ms. New HTTP TCP refusal upper bracket p50=23.795ms, max=29.478ms, configured 1ms probe, 212 refusal samples. 8 CPUs, Intel i9-9900K, Linux 6.12.111; local weighted scope CPUWeight=10000; load before=5.93/13.18/15.33, after=6.10/12.97/15.24. Cold/malformed imports locked; foreign PID and non-owner access refused; abrupt receiver stop during seed import followed by locked cold successor. Parser property corpus and fuzz target included.
CA suite 42 PASS (7 ignored diagnostics); remote server suite 49 PASS (10 ignored diagnostics); infraternal-agent remote suite 22 PASS (2 ignored); shipping handoff integration 1 PASS explicitly executed. TypeScript compiler/check/production bundle budgets/protocol tests PASS (34462 assertions). Remote feature-off cargo check PASS; cargo tree -d inspected, no new dependencies.
Both binaries stripped/static, PT_INTERP absent: x86 19938720 bytes (+41728 vs production19896992); ARM 17769176 (+39344 vs production17729832). Final docs/evidence add zero bytes. D205 permits the legacy size-only gate failure; no budget/profile changes or baseline waiver. Existing SQLite concurrent recovery failure was rerun; later complete gates pass its test row. Raw output whitespace is preserved deliberately in the tracked transcripts.
How to test:
Verbatim final scripts/verify fast output (size-only failure):
Starting review round 2 on ca-handoff, prior head
3d10854b45, production base9fba1eb2dd. Read all 19 comments on #154; no linked parent. Clean merge of origin/release-prod-2 produced 7fade3e0; diff now contains only the CA Rust/action/doc/evidence work. Addressing all eleven findings with failing regressions, atomic commits pushed after each, local weighted gates/fuzz/25-replacement scale proof. Owner-approved round-2 contract: root-only warm handoff, opened-fd exec, versioned binary marker, cold unmarked/rootless fallback; D184/D224 signature verification remains a hook. No deployment.P1 reproduced before fix: replacement_never_exports_to_a_general_purpose_binary failed, source.load calls=1 for /usr/bin/bash (p1-before.log). New implementation opens O_RDONLY|O_CLOEXEC|O_NOFOLLOW once, checks inode and parents, and fexecve uses that fd. Warm eligibility requires euid=0, root-owned protected inode/parents and exact ELF INFRATERNAL note type0x49464341/version1. General binaries and rootless replacements use null stdin and explicit cold reply. Six restart tests now pass; separate root invocation confirms /usr/bin/bash never loads the CA source (p1-root-unmarked.log). readelf -n confirms the note; replacement-path rename test confirms the checked inode remains pinned. Signature hook verify_release_fd is explicitly deferred to D184/D224, not advertised as verification. Full gate running; no deployment.
Completed p1, ca-handoff head
a26bd19430, pushed. P1 reproduced before fix: replacement_never_exports_to_a_general_purpose_binary failed, source.load calls=1 for /usr/bin/bash (p1-before.log). New implementation opens O_RDONLY|O_CLOEXEC|O_NOFOLLOW once, checks inode and parents, and fexecve uses that fd. Warm eligibility requires euid=0, root-owned protected inode/parents and exact ELF INFRATERNAL note type0x49464341/version1. General binaries and rootless replacements use null stdin and explicit cold reply. Eight restart tests now pass under both rootless and root execution (root-specific assertions explicitly rerun under /root/.cache/codex-scratch/ca-handoff); separate root invocation confirms /usr/bin/bash never loads the CA source (p1-root-unmarked.log). readelf -n confirms the note; replacement-path rename test confirms the checked inode remains pinned. Signature hook verify_release_fd is explicitly deferred to D184/D224, not advertised as verification. Clippy clean. Full gate permits only the unchanged historical static-size ceiling; no deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Fuzz prerequisite finding: first CPUWeight=30 cargo fuzz build failed fetching already pinned VT transitive dependencies (HttpConnectionClosing in vaxis/zigimg/uucode; fuzz.log). Copied this job's successfully built pinned VT native/download cache (1.2 GiB) into its separate fuzz target and retried; no dependency/version or compiler change. The five-minute execution timer starts only after successful compilation. Production
9fba1eb2has no diff in crates/, Cargo.toml, Cargo.lock or .cargo versus prior measured production base06857458, so the existing x8619896992/ARM17729832 byte size baselines remain applicable. Fast workspace test row completed without a baseline exception; first static build linking under host load ~32/35/33 on 8 CPUs.Startup compatibility finding: SSH prompt authorization at crates/plugins/ssh/src/prompt.rs:49 reads the helper executable through /proc//exe. A real same-UID fork probe with PR_SET_DUMPABLE(false) refuses that lookup with EACCES (round2/ssh-dumpability-probe.log). Checking the real encrypted-key CLI integration before landing global entry hardening; the exact executable/direct-parent authorization must remain intact. CA handoff stays root-only, with first-entry dumpability disabled. No deployment.
Completed p2, ca-handoff head
6214794713, pushed. P2 child inheritance fixed. Real subprocess regression failed before the fix (child fd0 was socket), then passed with /dev/null after POSIX spawn actions. Shim helper and systemctl spawn paths now explicitly use null stdin; SSH helpers already have pipes/null and PTY shells explicitly use their PTY. P1 real-fexecve regression proves cold replacement fd0 is /dev/null; unmarked rollback never invokes the signer source. Evidence p2-before.log/p2-after.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
P3 full gate first attempt: new assertion hit clippy bool_assert_comparison (restart_tests.rs:232), corrected to assert!(!...). Unchanged remote_http_tests.rs:181 also failed certificate admission with Denied under load; this is outside the permitted #118 exceptions and is not waived. Isolated regression and full gate are being rerun with RUST_TEST_THREADS=2 (all tests retained, individual concurrency tests unchanged). Full original output saved as round2/p3-fast-first.log; other fixes remain uncommitted until all required non-size rows pass. No deployment.
Fuzz build finding: nightly-2026-10-01 inherited production fat LTO and failed to link the ASan-instrumented harness with undefined _sancov_gen* references from asan.module_dtor (round2/fuzz-retry.log). No actual fuzz execution has occurred yet. Retrying with CARGO_PROFILE_RELEASE_LTO=false and STRIP=none for this fuzz command only, retaining ASan/coverage and CPUWeight=30; production profiles and gate budgets remain unchanged. No deployment.
Resuming CA handoff round 2 on ca-handoff at
6214794713after disk-full interruption. Read status/diff/log and all #154 comments (no linked parent in issue). P1/P2 and initial production merge are committed; P3 restart-error recovery remains uncommitted with before/after evidence. Production moved again to6e08065b; will merge it, retain atomic fixes, complete remaining findings and gates. Targets remain under $HOME/.cache/codex-scratch/ca-handoff. No deployment.Completed p3, ca-handoff head
37575d887b, pushed. P3 reply/reload failures fixed. Both regressions failed before repair: closed reply propagated EPIPE out of Control.requested, and renaming a staged binary over the running inode made current_binary fail NotFound. Request errors now log and loop; serve keeps its pinned future on listener errors. Deleted current_exe reopens configured argv[0]. The next real control request still succeeds cold. A third regression reproduced2 receiver opens for a default request; resolution now returns only the path and the receiver checks/execs one pinned fd. Evidence p3-before.log/p3-after.log. No deployment. First full gate exposed one corrected bool assertion lint and unchanged enrollment admission Denied under shared load. Isolated unchanged enrollment test passed0.22s; no #118 exception used. Full gate rerun retains every test with RUST_TEST_THREADS=2.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p4, ca-handoff head
96128ba5b4, pushed. P4 startup ordering fixed. Fresh exec regression failed while get_dumpable was true; protection is now the first main action. A real inherited-socket subprocess also failed because missing cold trust left fd0 as socket; receive/detach now precedes cold_start, with private Received material installed only against retained issuer. Evidence p4-before.log/p4-after.log. No deployment. Existing encrypted-key CLI regression reproduced proc helper authentication failure after entry hardening; detach helper stdin before restoring its existing dumpable policy, retaining exact executable/direct-parent checks. Server/CA paths remain protected. Evidence p4-ssh-before.log/p4-ssh-after.log.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
P11 actual cargo fuzz execution completed under local systemd CPUWeight=30: remote_ca_handoff_decode exercised the bounded CA frame and exact ELF note parsers for311 seconds,12051706 executions, no crash/ASan finding, final cov108/ft231, RSS505MiB. Full build/run transcript is round2/fuzz-final.log; environment observation is round2/fuzz-environment.txt. ASan and coverage retained; command-only LTO=false/strip=none workaround; production profile unchanged. No valid CA key corpus saved or logged. Evidence will be committed after remaining repairs.
Fuzz execution environment observed during the active run
timestamp=2026-10-10T20:47:36.658425+00:00
machine=infraternal-dev
kernel=6.12.111+deb13-amd64
arch=x86_64
logical_cpus=8
load_observed=19.62 35.74 37.53 13/3210 3498824
CPUWeight=30; ASan and sanitizer coverage enabled; max_total_time=310 seconds.
Command-only CARGO_PROFILE_RELEASE_LTO=false and STRIP=none work around nightly sanitizer/fat-LTO link failure; production profiles unchanged.
Final execution line, verbatim:
Done 12051706 runs in 311 second(s)
Completed p5, ca-handoff head
38822500fb, pushed. P5 descriptor cleanup fixed. Before repair, injected /dev/null open failure left fd0 open (F_GETFD returned 0). The regression now exercises both failed open and failed dup2 in real isolated subprocesses; each returns import failure and closes fd0 (EBADF). Evidence p5-before.log/p5-after.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p6dep, ca-handoff head
db0e63d9ee, pushed. P6 prerequisite dependency wiring: reused existing infraternal-shim with an optional ca-handoff feature enabled only by remote. No new third-party dependency/version or foundational library. Unsafe page ownership remains at the existing D40 audited shim boundary. Implementation follows separately. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p6module, ca-handoff head
f1bafe84c8, pushed. P6 audited mapping module added separately. Kernel /proc/self/smaps regression failed before mlock/MADV_DONTDUMP, then passed with lo and dd flags. Maximum requested storage4096 bytes, exclusive anonymous page ownership, volatile clearing/unmapping on Drop, fail-closed syscall errors. No unrelated allocator pages are locked. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p6, ca-handoff head
1367413c83, pushed. P6 CA buffers fixed. Real serialized-frame smaps regression failed on the old heap vector, then passed with lo/dd flags. Export seed/frame and receive storage now use LockedBytes; failed mlock or MADV_DONTDUMP refuses the handoff cold. On this4KiB-page host peak2 pages=8KiB per server×1 simultaneous transfer, independent of10000 retained hosts/128PTYS; receive needs1 page. OpenSSL-owned key state/API export vector and kernel socket buffers are remaining limitations; the transient export vector is zeroized immediately. Evidence p6-before.log/p6-after.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
P7 targeted shutdown regressions pass after repair: child cleanup2.03s, runtime bound4.03s. Full gate caught two author-code lint failures: identical waitpid match arms (embedded.rs:1604) and the fixture Child wait missing on its successful manual-reap path (embedded_regressions.rs:725). Combined the arms and made fixture Child::wait unconditional; no lint suppression or runtime budget change. The current gate is retained as failed; a complete corrected gate must pass before P7 commit. No deployment.
Completed p7, ca-handoff head
b0a56a42b4, pushed. P7 shutdown fixed. Two regressions failed before repair: SIGTERM-resistant real child exceeded4s and runtime Drop exceeded8s on blocking waitpid. Child cleanup now escalates inside the blocking task (TERM deadline2s, KILL/reap deadline3s), and runtime.shutdown_timeout bounds overall wait4s. No CPU load generators; fixtures use idle sleep and clean their own process group on failure. Evidence p7-before-0/1.log and p7-after-0/1.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p8, ca-handoff head
cae65fd5ff, pushed. P8 early reload fixed. Isolated fresh-image regression died on SIGHUP before Control::start, then passed with a queued reload. The exec thread blocks SIGHUP before fexecve; first server entry blocks before spawning runtime threads, and Control registers the handler before unblocking. Pending signal is consumed after readiness. A second real child regression first reproduced inherited blocked SIGHUP; POSIX spawn now explicitly resets its child signal mask so the infraternal agent and PTY shells retain normal signal policy. Evidence p8-before.log/p8-after.log and p8-mask-before.log/p8-mask-after.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p9, ca-handoff head
1d7ad48d2b, pushed. P9 startup resilience fixed. A real-process regression with the restart directory path occupied by a regular file failed before the fix; after repair the same server reaches HTTP /healthz200 and logs local control unavailability. Control::start errors no longer abort service startup. Evidence p9-before.log/p9-after.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed p10dep, ca-handoff head
cd0280fd19, pushed. P10 prerequisite: optional direct reuse of existing tracing, enabled only by remote. No new library/version or separate log sink; shared server logging presents human/JSON diagnostics. Audit behavior follows separately. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Audit full gate first attempt: tests PASS, Clippy FAIL (101): nonminimal peer boolean, manual-let-else/single-match-else, and SO_PEERCRED doc backticks in crates/infraternal/src/restart.rs. Fixed all expressions/comments without suppressions; rerunning full gate before commit. Earlier running-log report that Clippy passed was premature. Evidence p10-fast-first.log retained. No deployment.
Completed p10, ca-handoff head
15b0a3e614, pushed. P10 audit fixed. Real subprocess regression failed before instrumentation, then matched exact SO_PEERCRED PID/UID, target /usr/bin/bash and cold outcome. All request branches record accepted warm/cold, refusal or reply failure; SIGHUP records origin with no fabricated peer credentials. Debug formatting escapes path control characters. No key bytes or key lengths are logged. Exec failure separately records cold outcome and error kind; successor startup records its actual retained-issuer installation result for correlation by the unchanged service PID. Evidence p10-before.log/p10-after.log. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Author review found an additional fd edge case after the P5 close-on-error boundary: if fd0 is closed, Opened::open can allocate the checked image on fd0, then stdin redirection overwrites that image; alternatively opening /dev/null on fd0 and dropping its owned File closes it before exec. Two real-fexecve subprocess stages failed before repair (stdin-before.log). Pin the image above all standard descriptors using fd clones, retain the null File through exec, and clear stdin close-on-exec explicitly. No additional target pathname open or unsafe core code. No deployment.
Completed stdin, ca-handoff head
23f382a35e, pushed. Author review found an additional fd edge case after the P5 close-on-error boundary: if fd0 is closed, Opened::open can allocate the checked image on fd0, then stdin redirection overwrites that image; alternatively opening /dev/null on fd0 and dropping its owned File closes it before exec. Two real-fexecve subprocess stages failed before repair (stdin-before.log). Pin the image above all standard descriptors using fd clones, retain the null File through exec, and clear stdin close-on-exec explicitly. No additional target pathname open or unsafe core code. No deployment. The initial Rust successor automatically repaired closed fd0 and masked one case; a native libc probe now independently observes both failures before repair and null stdin afterward. The existing cc toolchain builds it with TMPDIR inside the private job fixture; no dependency was added. Evidence stdin-runtime-masked-before.log, stdin-before.log and stdin-after.log.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Root shipping first attempt failed only in the harness shim_pids lookup: root sessions deliberately use /run/infraternal, ignoring XDG_RUNTIME_DIR (crates/infraternal-agent/src/sessions.rs:175). Rootless shipping passed explicit cold reply/state, same PID, null stdin and EPIPE preservation. Terminated exactly the 128 shims belonging to this job’s private copied executable using pidfds; no other process was signalled. Rerunning root benchmark in a private mount namespace with /run bound to this job’s root scratch runtime directory; the existing production runtime policy is unchanged and all sockets stay inside the job. Evidence shipping-root-first.log and shipping-root-first-server.log; no deployment.
Real shipping checks pass. User mode: explicit cold reply, locked CA after same-PID replacement, /dev/null fd0, deterministic EPIPE preserves the live server. Root, private mount namespace bound to this job’s scratch runtime (PID/network unchanged): 25 warm replacements, 50 users/10000 retained records/128 live PTYs; original shim PIDs survive, all reattach and accept new input. Unmarked root rollback also replies cold and proves locked CA/null fd0. Root p50=26.856 ms/max=36.962 ms (round1 30.736/39.541, observed deltas -3.880/-2.579 ms); refusal brackets p50=21.023/max=26.725 ms, 205 samples. Load before12.92/17.07/25.40, after16.90/17.75/25.53; CPUWeight=10000 locally on infraternal-dev. Different UID/load than round1: delta is observed, not an isolated causal estimate. Evidence shipping-root.log, shipping-rootless.log and server audit logs under job round2 scratch; source crates/infraternal-server/src/ca_handoff_shipping_tests.rs. Full gate running before test commit. No deployment.
Shipping full gate first attempt failed the unchanged crates/infraternal/tests/shim.rs attached_backpressure_and_detached_progress test: producer never stalled at head1515 (minimum1847). Clippy and dependency rows pass; static ceiling remains the known failure. This is outside the #118 waiver list: no waiver or assertion change is used. Rerun the exact test, then require a passing fresh full test row before committing. Real root25/user-mode shipping proofs already passed; evidence shipping-fast-first.log retained.
Exact unchanged shim backpressure rerun passes: 1 passed/0 failed in0.26 s (shipping-shim-rerun.log). Initial full-gate failure retained in shipping-fast-first.log. No #118 waiver, assertion change or production fix: rerunning the full fast gate and requiring every non-size row PASS before committing the shipping tests. Root/user-mode shipping proof remains green; no need to repeat its25 replacements.
Completed shipping, ca-handoff head
1b494d4b58, pushed. Shipping integration completed: load_before=12.08 27.38 31.19 8/3003 1045314Running as unit: run-p1045315-i42945559.scope; invocation ID: f72e25824c6246a9ae8b5aa68610443c
running 1 test
METRIC name=ca_handoff_rootless n=1 restart_cold=true reply_cold=true pid_preserved=true fd0_dev_null=true epipe_keeps_server=true
test ca_handoff::shipping_tests::shipping_restart_reload_rollback_and_cold_start ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 310 filtered out; finished in 0.54s
load_after=12.08 27.38 31.19 10/3018 1045390
Private mount namespace: /run bind-mounted from /root/.cache/codex-scratch/ca-handoff/r; PID/network namespaces unchanged.
load_before=12.92 17.07 25.40 5/2863 1073164
Running as unit: run-p1073165-i42973409.scope; invocation ID: e2a3496aa7e149c9bbb0d0b347aa4405
running 1 test
METRIC name=ca_handoff_unmarked_rollback root_owned=true restart_cold=true reply_cold=true pid_preserved=true fd0_dev_null=true epipe_keeps_server=true
METRIC name=ca_handoff_restart n=25 retained_nodes=10000 users=50 real_pty_sessions=128 reattached_sessions=128 p50_ms=26.856 max_ms=36.962 refused_window_bracket_p50_ms=21.023 refused_window_bracket_max_ms=26.725 refused_samples=205 probe_interval_ms=1 cold_start_locked=true invalid_frame_locked=true non_owner_refused=true resumed_pty_input=true
test ca_handoff::shipping_tests::shipping_restart_reload_rollback_and_cold_start ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 310 filtered out; finished in 6.30s
load_after=16.90 17.75 25.53 13/2912 1075091
CPUWeight=10000 locally; release binary, protected root install, private mount namespace runtime paths, 50 users/10000 retained records/128 real PTYs. Root fixed runtime-path harness failure and cleanup are recorded separately; no production runtime policy changed. No deployment.
Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Deployment-doc gate first test attempt hit the unchanged real_cli_askpass_refuses_unregistered_parent_and_cleans_helper fixture at crates/infraternal/tests/ssh_askpass.rs:46: exec Text file busy (errno26), before helper code runs. Encrypted synthetic-key helper test passed in the same suite. Exact compiled-fixture rerun: Running as unit: run-p1300676-i43200915.scope; invocation ID: 788737b1aa2345c9a153ae7b01837f8a
running 1 test
test real_cli_askpass_refuses_unregistered_parent_and_cleans_helper ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.01s
No #118 waiver, assertion, permission or production code change. Retain docs-fast-first.log after this gate finishes, then require a fresh full test row PASS. ARM compilation is independently progressing and holds the release build directory lock. No deployment.
Fresh deployment-doc gate failed two unchanged mesh fixtures at crates/infraternal-server/src/mesh_api_tests.rs:52 with PoolTimedOut during pool setup: claude_review_scoped_grant_cannot_target_node_in_unscoped_tag and concurrent_tag_edits_serialize_revisions_and_unique_names. Server result278 passed/2 failed/31 ignored; load29.22/34.68/28.02. Outside #118: no waiver, assertion or production source change. Retain docs-fast-second.log, rerun exact cases, then serialize the test harness (RUST_TEST_THREADS=1) for the remaining full gates to reduce between-fixture contention. Internal concurrent-request scenarios remain unchanged. No deployment.
The new job-local fixture root was initially too long: real-binary agent fixture preflight rejected AF_UNIX upstream path116 bytes before running the daemon. Shortened verification-only INFRATERNAL_TEST_STATE_ROOT to /home/kayg/.cache/codex-scratch/ca-handoff/s; scripts/agent_fixture.py preflight accepts the path within the unchanged108-byte kernel bound. No production code/test assertion changed; docs-fast-third.log retained. The next full gate remains serialized, without a waiver.
Documentation gate fourth attempt: existing auth socket scale continuation assertion failed (279 passed/1 failed/31 ignored). Exact comparison on detached origin/dev
f91517019apassed in106.24s at load36.09/32.93/29.57. This does not establish a #118 exception; rerunning full gate. Outputs archived in evidence/ca-handoff/round2/docs-fast-fourth.log and auth-socket-baseline.log.Completed docs, ca-handoff head
0c51226234, pushed. Updated docs/operations/deploy.md for root-only marked warm replacement, explicit cold user-mode/unmarked rollback, pinned executable/null descriptor lifetime, external release verification until D184/D224 integration, restart error isolation, early signals, bounded shutdown and protected-buffer residual copies. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Completed evidence, ca-handoff head
1da307870c, pushed. Archived all failing-first fixes, real root and user-mode shipping proofs, 25 replacements p50/max 26.856/36.962 ms (observed change -3.880/-2.579 ms), 12051706 fuzz runs in311 seconds, final suites and static growth +77904/+66992 bytes. The unchanged size ceiling remains; no budget/profile change or #118 exception. Independent review remains required. No deployment.Verbatim scripts/verify fast output (pre-existing static size ceiling unchanged):
Finished review fixes #154, branch ca-handoff, head
1da307870c(pushed). Root-only compatible opened-fd handoff, cold user-mode/unmarked rollback, lifecycle/startup/error handling, protected buffers and audit fixes are covered by failing-first regressions and real shipping tests. Additional closed-stdin exec edge fixed. No deployment; issue remains open.METRIC name=ca_handoff_restart n=25 retained_nodes=10000 users=50 real_pty_sessions=128 reattached_sessions=128 p50_ms=26.856 max_ms=36.962 refused_window_bracket_p50_ms=21.023 refused_window_bracket_max_ms=26.725 refused_samples=205 probe_interval_ms=1 cold_start_locked=true invalid_frame_locked=true non_owner_refused=true resumed_pty_input=true
User mode and unmarked root rollback confirm explicit cold reply, locked CA, same PID and /dev/null stdin. Both real processes preserve service on reply EPIPE. Root scale: 50 users, 10000 retained records, 128 real PTYs; all shim PIDs/reattach/input retained. Benchmark locally CPUWeight=10000; load and baseline comparison in evidence/ca-handoff/README.md. Fuzz: 12051706 runs in311 seconds, CPUWeight=30, no sanitizer finding. ARM/TS/final suites/remote-disabled build all pass. Final static bytes: {'x86_64-unknown-linux-musl': 19974896, 'aarch64-unknown-linux-musl': 17796824}. Only the unchanged pre-existing x86 static-size ceiling fails; no #118 waiver, budget/profile change or private corpus. Release-signature hook deliberately deferred to D184/D224.
Evidence: evidence/ca-handoff/README.md and evidence/ca-handoff/round2; deployment procedure docs/operations/deploy.md. Own author checklist completed; independent review still required. Own scratch targets/fixtures will be removed after evidence archival.
Verbatim final scripts/verify fast output:
Cleanup complete at pushed head
1da307870c. Removed only this job’s user/root scratch targets, fixtures, exact failed SSH fixture and detached baseline checkout. No root shipping process remains. Worktree clean; origin/ca-handoff matches HEAD. All evidence remains committed in evidence/ca-handoff/round2. No deployment; issue remains open.Starting independent verification on review-ca-handoff, detached target
1da307870c; merge base with local dev2eaaaa36a2(multiple merge bases). Read #154 discussion including previous round; no linked parent. Reviewing target behavior and earlier fixes, with failing tests committed locally only; no push or deployment. Main decisions, contracts and full fast gate included.Independent review completed two reproductions, committed locally on review-ca-handoff (
beac3e41and8fc382eb). CA-R1 P2: server startup blocks SIGHUP before runtime workers; process::execute leaves it blocked in SSH helpers (real child reports 1 instead of 0; regression reviewer_ssh_helper_does_not_inherit_server_reload_mask fails in 0.04s). CA-R2 P2: restart_binary::Opened::open opens a FIFO before checking regular-file metadata; reviewer_fifo_replacement_is_refused_without_waiting_for_a_writer fails at its 3s subprocess watchdog (3.12s). Exact Rust test bodies exercised production code; normal Cargo runs are queued behind compilation. No implementation fixes or push.Initial head gate hit only fixture setup at this point: review fixture socket path112 bytes; shortening the review-owned state root before a full rerun. No waiver. Static build/fuzz and independent root scale proof continue.
CA-R2 additional real-server evidence: target-head debug binary returns HTTP200 before the owner sends a raw control request naming a FIFO with no writer. A new /healthz connection then times out at1s while the service PID remains alive. Controller polls Opened::open synchronously in the serving future. Own fixture process/FIFO/data cleaned. Test commit8fc382eb remains an intentional P2 failure; no implementation change. CA suite independently passed45/0failed/7ignored in45.93s.
Independent root shipping proof PASS on the exact 19,974,896-byte stripped static x86 target. 25 replacements; 50 users, 10,000 retained node records, 128 real PTYs. Every shim PID survives, every PTY reattaches and accepts input; CA unlock preserved. p50 67.958 ms/max 111.892 ms; refusal brackets p50 49.441/max 99.931 ms, 436 samples. CPUWeight=10000 locally; load before 36.47/33.80/29.18, after 34.07/33.38/29.09. Different load window from author: observed slowdown, not an isolated code regression. Private mount namespace maps only this review scratch tree to root/runtime paths. First run stopped at PTY setup call20 before replacement; unchanged rerun passes, both logs retained.
Normal Cargo runs confirm both committed P2 tests fail (SSH 0.20s, FIFO 3.12s). Corrected scripts/verify fast is running against restored target tracked sources (git diff
1da30787was empty before execution); reviewer tests restored after its test binaries compiled. No assertion changes/waiver/push/deployment.Starting round 3 on ca-handoff at
1da307870c. Read all 61 comments on #154; no linked parent. Cherry-picking unchanged regression commitsbeac3e41and8fc382eb, then fixing CA-R1 child signal inheritance and CA-R2 nonblocking replacement validation (D217/D206). Each logical fix is gated and pushed separately. No deployment.Pushed unchanged failing-first review tests as
a1ee82a4(CA-R1) ande8d4c260(CA-R2). Existing audited infraternal-shim dependency added to SSH inb9a072a9; locked Cargo metadata passes, zero new library choices. Production child-launch audit: SSH execute/master/key helper, embedded POSIX spawn, shim shell, session launcher, systemctl installation and remote uninstall helpers. Embedded mask and PTY mask already reset, but ignored dispositions also need defaults. Native builds are in job-local scratch; regression traces remain files only.CA-R1 reproduced locally with unchanged reviewer test: 0 passed/1 failed in 0.13s (signal-before.log); the real helper retains the blocked reload mask. Shared audited reset helper test passes for both Command/pre-exec and POSIX spawn: ignored reload disposition restored to default, child reload unblocked, parent mask unchanged (helper-test.log, 0.01s). Helper clippy passes. CA-R2 initial build stopped because make was absent; retry uses documented nix shell gnumake/perl, with validator still unchanged. Logs under /home/kayg/.cache/codex-scratch/ca-handoff; no signal/process traces printed.
CA-R2 reproduced with unchanged reviewer subprocess regression: 0 passed/1 failed in 3.06s at the 3-second watchdog (fifo-before-retry.log). Validator now requests O_NONBLOCK|O_NOFOLLOW|O_CLOEXEC, fstats and refuses non-regular inodes before any read, then clears only O_NONBLOCK. Exact-inode pinning and close-on-exec remain intact. CA-R1 SSH process suite passes 5/0 in0.28s; extended SSH launch suites pass (14 ordinary tests; five explicit ignored diagnostics). Spawn diagnostic: 7 alternating batches of128 real children, baseline/reset median462.582/509.044ms, +362.988us/child, +46.462ms/128 launches; no retained per-child storage or idle task. This bounded startup cost restores required signal correctness. Full fast gate running locally CPUWeight=10000; no budget/profile/test relaxation.
Completed crates/plugins/ssh/src/process.rs at
6de850b774, pushed ca-handoff. CA-R1: runtime workers inherit the blocked server reload mask. Apply the shared audited child reset before execute_inner spawns a helper. Reject unblocking the server thread or resetting only the reload signal: both leave inherited child policy inconsistent. The unchanged reviewer test and all five process tests pass; extended SSH launcher suites and real authentication pass. D206 requires preserving and fixing the reviewer P2 test forward.Completed crates/plugins/ssh/src/credentials.rs at
f48861f149, pushed ca-handoff. Apply the shared child policy to the direct temporary key-helper spawn found by the CA-R1 launch audit. This path bypasses bounded execute, so fixing execute alone leaves inherited masks and ignored dispositions behind. Reject a duplicate reset implementation. Credential tests and real memory-only key authentication pass; no key lifetime or transport change. D206 requires completing the P2 launch audit.Completed crates/plugins/ssh/src/master.rs at
b347340856, pushed ca-handoff. Apply the audited reset to the direct connection-helper spawn found by the CA-R1 audit. Reject relying on the bounded-command launcher because the master bypasses it. The real SSH single-connection oracle passes, including connection reuse, temporary-helper retirement and refusal to reconnect with retired one-time credentials. D206 requires fixing this P2 finding forward.Feature-isolation check caught an author wiring error before the embedded adapter commit: standalone infraternal-server default features omit remote, so its optional direct shim dependency was unavailable (E0433 in embedded-after.log). The embedded infraternal agent already depends on shim unconditionally. Make that existing shared process boundary a nonoptional direct server dependency; keep the ca-handoff feature remote-only. No new engine/version or runtime task. Retrying standalone embedded tests and requiring remote-disabled checks; complete fast gate will rerun after this manifest change.
Completed step at
2f119822, pushed ca-handoff. Shared audited Command/POSIX spawn signal-reset adapters and PTY reuse. Real ignored-disposition/parent-mask fixture passes; helper clippy passes. Imported reviewer regressions are retained unchanged.Completed step at
15808847, pushed ca-handoff. Separate real subprocess tests and seven-round 128-child diagnostic. Both spawn adapters preserve parent policy; diagnostic measured +362.988us/child, +46.462ms/128 launches (infraternal-dev, CPUWeight=10000). Imported reviewer regressions are retained unchanged.Completed crates/infraternal-server/Cargo.toml at
752ab1be61, pushed ca-handoff. The embedded launcher always exists, including without remote. Make its existing transitive shim boundary a direct unconditional dependency, while keeping ca-handoff buffers remote-feature-only. Reject gating child signal correctness on the remote plugin or duplicating syscall code in core. Standalone embedded suite passes 15/0 (one ignored diagnostic); no engine/version was added. D206 requires a build-removable repair.Completed crates/infraternal-server/src/embedded.rs at
880a225c1a, pushed ca-handoff. Reuse the audited POSIX spawn adapter for embedded infraternal agents, restoring default dispositions as well as the already-cleared mask. Reject retaining a separate mask-only policy, since ignored dispositions survive exec. Standalone embedded suite passes 15/0 with one diagnostic ignored; existing real stdin and reload-mask subprocess tests remain intact. D206 requires completing the CA-R1 launch audit.Completed crates/infraternal-agent/src/sessions.rs at
2fa48cc442, pushed ca-handoff. Apply the audited child reset before the shell closes the trusted channel and launches an init helper or detached shim. Reject relying on the eventual PTY shell reset: intermediary helpers must also receive normal signals. Session suite passes 6/0, one ignored diagnostic; explicit null input and fd3 closure remain unchanged. D206 requires auditing all CA-R1 child launch paths.Completed crates/infraternal/src/install.rs at
ba87fc8f35, pushed ca-handoff. Apply the shared default child policy to both systemctl spawns found by the CA-R1 audit. Reject assuming service-management helpers cannot inherit startup masks, or copying reset code. CLI unit suite passes 36/0 including installation dry-run behavior; helper subprocess tests cover the shared policy. D206 requires fixing the full P2 finding forward.Completed crates/infraternal/src/restart_binary.rs at
9a0637d41d, pushed ca-handoff. CA-R2: read-only open of a FIFO without a writer stalls validation outside the request deadline. Open with NONBLOCK, NOFOLLOW and CLOEXEC, fstat and refuse nonregular/untrusted inodes before reading, then clear only NONBLOCK on the same inode. Reject pathname prechecks because substitutions race the subsequent open, and reject an async timeout because it cannot interrupt a blocked open. The unchanged reviewer regression fails before in 3.06s; all 16 restart tests pass after in6.27s, including pinned exec and closed-stdin cases. D206 requires keeping the P2 reviewer regression intact.Completed crates/plugins/remote/src/uninstall.rs at
a71b3315e9, pushed ca-handoff. Apply the shared audited child policy to the independent uninstall launcher and its systemctl helper, found by the CA-R1 launch audit. Reject relying on SSH execute or embedded spawn because these are direct Command paths. Explicit private-plan/null input and component shutdown ownership remain unchanged. Remote component suite with agent enabled passes 12/0 in6.67s; helper subprocess tests cover the default signal policy. D217 preserves reload ordering in the parent; D206 requires fixing this P2 audit forward.Completed round-3 author review/documentation at
99b263a5e5, pushed ca-handoff. Both reviewer patches unchanged; real before/after kernel tests, real SSH, rootless shipping and remote-disabled build pass. Final fast: fmt/VT fmt/clippy/test/cargo-deny PASS, known static-size-only FAIL. Static sizes x86 19976240/ARM17798168, +1344 bytes each vs1da30787. Final128-child diagnostic +570.173us/child (+72.982ms total), CPUWeight10000 locally, load before8.87/13.04/17.37 after9.92/13.19/17.40. Evidence links and author checklist verified; raw transcripts follow separately.Finished ca-handoff round 3 at
2e4526a85a, pushed after all 16 atomic commits. CA-R1 and CA-R2 fixed; both original reviewer patches remain byte-for-byte unchanged. All ordinary workspace/earlier CA tests pass; no #118 baseline exception. Real SSH authentication/connection reuse, rootless shipping and remote-disabled build pass. TypeScript: 144 unit/tracer and 36 protocol tests pass, production checks pass.Static x86=19976240 bytes, ARM=17798168 bytes; each +1344 vs
1da30787, stripped/static with PT_INTERP absent. No ceiling/profile/version change; fast exits 1 solely at the known static-size ceiling. Local CPUWeight=10000 diagnostic, 128 real child launches ×7 alternating batches: baseline/reset medians 204.024/277.006ms, +570.173us/child (+72.982ms/128). Machine infraternal-dev, 8 CPUs; load before 8.87/13.04/17.37, after 9.92/13.19/17.40. No new idle task; total child RSS was not measured.Evidence and author checklist: evidence/ca-handoff/round3/README.md plus raw transcripts. Source unchanged by docs/archive. Own scratch targets/fixtures and generated Python cache removed; no compiled fixture process remains. No deployment; issue remains open.
Verbatim scripts/verify fast summary:
Starting independent delta review on review-ca-handoff at
2e4526a8; reviewed base1da30787, merge base with dev2eaaaa36. Prior review commits beac3e41/8fc382eb are already cherry-picked unchanged as a1ee82a4/e8d4c260. Will rerun both, inspect only the repair delta and run a fresh weighted fast gate. No push/deployment. #154 has no linked parent.Independent delta evidence committed locally at
b515acc1on review-ca-handoff. Original patches compare byte-identical; CA-R1 now passes1/0 in0.04s, CA-R2 passes1/0 in0.03s. Both spawn adapters/parent mask test passes1/0 in0.01s. Local weighted128-child diagnostic: baseline/reset medians145.456/200.210ms, observed +427.764us/child (+54.754ms/batch), load17.84/19.43/17.60 ->17.86/19.41/17.60 on infraternal-dev. No own build overlap; shared tests/load remain. No new confirmed finding. Evidence evidence/review-ca-handoff/delta/. Fresh full head gate still running, with shortened AF_UNIX fixture root. No push/deployment.Finished independent delta review #154: MERGE (no P0/P1), verdict merge. Target2e4526a8; reviewed1da30787..2e4526a8 only. New findings P0=0/P1=0/P2=0/P3=0. Earlier CA-R1/CA-R2 P2 tests unchanged and now PASS (0.04s/0.03s); both also pass the full gate. Real child adapters, real-sshd auth/reuse and optional-features-disabled build PASS. Local128-child diagnostic observed +427.764us/child (+54.754ms/batch), load17.84/19.43/17.60 ->17.86/19.41/17.60. Fresh x86 stripped static19,976,240 bytes (+1,344 vs earlier reviewed head), PT_INTERP absent. ARM not independently rebuilt in this bounded delta review. No #118 waiver.
Local evidence commits
b515acc1andd80692bceaon review-ca-handoff; no implementation changes, push or deployment. Evidence evidence/review-ca-handoff/delta/README.md; appended durable review finding status.Verbatim scripts/verify fast output on target sources (exit1, solely known D205 size ceiling; Rust rows otherwise pass):
Starting production-batch verification on prod-batch, base/head
ebcc73a0ec. Read repository contracts, merge message and #154 history (82 comments; no linked parent), continuing the reviewed round-3 fixes. Auditing every production child launch, then fresh weighted fast/CA/SB-01/DS-01..04/auth/TLS/production tracer gates. TRACER_TEST_PORT=5799; job-local scratch targets, no deployment. Exact origin/tls-session-prod base is5d02b8d6; recorded x86 artifact 19991488 bytes will be independently rebuilt for the byte delta.Launch audit finding: crates/infraternal-shim/src/pty.rs:192,200 double-forks without the audited reset_child_signals policy. sessions.rs:435/436 already uses prepare_child for detection, systemd-run and detached shim launch; PTY spawn uses the shared reset, embedded.rs uses prepare_spawn, install/SSH/uninstall adapters are prepared. Retention/ended-record paths create no subprocess. Same-PID fexecve in restart.rs intentionally preserves queued reload (D217), rather than launching a child. Adding standalone blocked+ignored-SIGHUP detach regression before repair; parent mask must remain unchanged. Initial tracer check lacked installed dependencies; frozen-lockfile install/rerun underway. No new dependencies or budget changes.
Production tracer verification completed at
ebcc73a0plus uncommitted backend regression fixture (frontend unchanged). Frozen install, bun run check, bun test src, production build, D188 transfer and DS-02 independent review test PASS. Playwright at5799: dead-session (4), detached-output (1), DS-04 retry/input (2), 7 passed in1.8m. Browser plugin unavailable; repository Playwright used. Real production screenshots captured by dead-session spec; no author visual-quality judgment. Raw evidence logs/tracer-retry.log in job scratch.(pass) palette rows are always titled > a titled row passes through untouched [0.38ms]
(pass) stagesOf > an auth failure before sign-in fails detect and says it in plain words, not the server's [0.22ms]
(pass) stagesOf > an unsupported host fails the system check, after sign-in [0.09ms]
(pass) stagesOf > an install failure fails the install stage and keeps the earlier ones done [0.06ms]
(pass) guideFor > installation failures never claim that incomplete cleanup succeeded [0.16ms]
(pass) guideFor > each failure kind names its fix [0.16ms]
(pass) plain words > an unknown or missing failure gets the generic sentence, never the server's text [0.07ms]
(pass) install diagnostics name the failed step and public host error [0.09ms]
142 pass
0 fail
10901 expect() calls
onDemand chunk-9a9ye4ws.js: identity=3891 gzip=1736 brotli=1521 bytes
critical chunk-28rbcdd5.js: identity=39 gzip=39 brotli=39 bytes
onDemand chunk-zg6n7gqh.js: identity=13351 gzip=5014 brotli=4482 bytes
deferred chunk-e72ce8mc.js: identity=41401 gzip=15945 brotli=14012 bytes
onDemand chunk-acdpthvq.js: identity=2420 gzip=1198 brotli=1081 bytes
onDemand chunk-rfb9fjvk.js: identity=7662 gzip=3395 brotli=2978 bytes
critical chunk-90h7pn57.js: identity=41568 gzip=14175 brotli=12726 bytes
critical chunk-ec72wxvj.js: identity=9810 gzip=4018 brotli=3459 bytes
critical chunk-bwppvaha.js: identity=703957 gzip=203950 brotli=165155 bytes
critical faulty-terminal.worker.js: identity=1086 gzip=562 brotli=482 bytes
critical backdrop.worker.js: identity=243 gzip=212 brotli=158 bytes
onDemand chunk-hygxtb89.js: identity=5823 gzip=2201 brotli=1945 bytes
critical chunk-edkge1nx.js: identity=21235 gzip=6855 brotli=6121 bytes
onDemand chunk-es98hjwk.js: identity=2897 gzip=1207 brotli=1037 bytes
critical chunk-4h8jzww1.js: identity=5443 gzip=2606 brotli=2384 bytes
deferred chunk-qr00vzsb.js: identity=102280 gzip=34768 brotli=30649 bytes
critical chunk-958cgt7z.js: identity=8377 gzip=3575 brotli=3186 bytes
onDemand chunk-e0fs94mx.js: identity=1026 gzip=622 brotli=497 bytes
critical chunk-axbkkqp1.js: identity=90 gzip=90 brotli=78 bytes
onDemand chunk-gb6bdgd8.js: identity=2188 gzip=1048 brotli=878 bytes
onDemand chunk-zn341cf6.js: identity=552 gzip=406 brotli=306 bytes
critical chunk-0yh8rwvm.js: identity=11771 gzip=4749 brotli=4222 bytes
onDemand chunk-c8aj9mpg.js: identity=2108 gzip=1037 brotli=869 bytes
onDemand chunk-fj8r81sy.js: identity=2824 gzip=1385 brotli=1192 bytes
onDemand chunk-p653n5h0.js: identity=31889 gzip=12265 brotli=11011 bytes
onDemand chunk-fgacwwp5.js: identity=6773 gzip=3175 brotli=2886 bytes
critical chunk-f2r0hhqc.js: identity=128 gzip=109 brotli=111 bytes
critical chunk-9xd3g65e.js: identity=39 gzip=39 brotli=39 bytes
onDemand chunk-etaeeaeh.js: identity=540 gzip=353 brotli=313 bytes
critical chunk-ww6108r2.js: identity=2943 gzip=1315 brotli=1134 bytes
onDemand chunk-zwt60rqz.js: identity=5969 gzip=2675 brotli=2430 bytes
onDemand chunk-k4jpemns.js: identity=47806 gzip=15001 brotli=13326 bytes
onDemand chunk-4b1wpzgp.js: identity=1170 gzip=707 brotli=588 bytes
onDemand chunk-dx2h6rry.js: identity=4542 gzip=2002 brotli=1785 bytes
critical chunk-9v9mmkw2.js: identity=199 gzip=160 brotli=139 bytes
critical main.js: identity=108947 gzip=39701 brotli=34474 bytes
onDemand chunk-swe72sn1.js: identity=11529 gzip=4652 brotli=4250 bytes
critical chunk-61xwpgvs.js: identity=49 gzip=49 brotli=49 bytes
onDemand chunk-saa8dhvk.js: identity=1616 gzip=865 brotli=739 bytes
onDemand chunk-jh1z1rdv.js: identity=1603 gzip=736 brotli=628 bytes
critical chunk-vnmaej5z.js: identity=1914 gzip=840 brotli=714 bytes
onDemand chunk-702kdt64.js: identity=4640 gzip=1931 brotli=1701 bytes
critical chunk-ey1gkh07.js: identity=1104 gzip=668 brotli=606 bytes
onDemand chunk-9jb4avcs.js: identity=5359 gzip=2580 brotli=2299 bytes
critical chunk-qvzpx6av.js: identity=17525 gzip=6117 brotli=5445 bytes
critical chunk-arbccdz8.js: identity=590 gzip=326 brotli=303 bytes
critical chunk-59me0bpc.js: identity=386 gzip=238 brotli=197 bytes
JavaScript transfer, critical: identity=937443 gzip=290393/307200 brotli=241221/256000 bytes
JavaScript transfer, deferred: identity=143681 gzip=50713/51200 brotli=44661/45056 bytes
JavaScript transfer, onDemand: identity=168178 gzip=66191/20480 brotli=58742/18432 bytes
JavaScript transfer, onDemand largest chunk chunk-k4jpemns.js: gzip=15001/20480 brotli=13326/18432 bytes (class total gzip=66191 brotli=58742, not limited)
fonts/monaspace-neon/italic.latin.woff2: identity=44324 gzip=44324 brotli=44324 bytes
fonts/monaspace-neon/roman.latin.woff2: identity=40016 gzip=40016 brotli=40016 bytes
fonts/jetbrains-mono/italic.latin.woff2: identity=36316 gzip=36316 brotli=36316 bytes
fonts/jetbrains-mono/roman.latin.woff2: identity=34068 gzip=34068 brotli=34068 bytes
fonts/source-code-pro/italic.latin.woff2: identity=23612 gzip=23612 brotli=23612 bytes
fonts/source-code-pro/roman.latin.woff2: identity=22872 gzip=22872 brotli=22872 bytes
fonts/geist-mono/italic.latin.woff2: identity=21380 gzip=21380 brotli=21380 bytes
fonts/geist-mono/roman.latin.woff2: identity=19984 gzip=19984 brotli=19981 bytes
fonts/ibm-plex-mono/italic.latin.woff2: identity=22296 gzip=22296 brotli=22296 bytes
fonts/ibm-plex-mono/roman.latin.woff2: identity=19048 gzip=19048 brotli=19047 bytes
fonts/cascadia-code/italic.latin.woff2: identity=44404 gzip=44404 brotli=44404 bytes
fonts/cascadia-code/roman.latin.woff2: identity=40604 gzip=40604 brotli=40604 bytes
fonts/maple-mono-nf/italic.latin.woff2: identity=53444 gzip=53444 brotli=53444 bytes
fonts/maple-mono-nf/roman.latin.woff2: identity=46432 gzip=46432 brotli=46432 bytes
fonts/nerd-symbols.woff2: identity=223748 gzip=223748 brotli=223748 bytes
fonts/fira-code/roman.latin.woff2: identity=36756 gzip=36756 brotli=36756 bytes
fonts/commit-mono/italic-400.latin.woff2: identity=18716 gzip=18716 brotli=18716 bytes
fonts/commit-mono/italic-700.latin.woff2: identity=18352 gzip=18352 brotli=18352 bytes
fonts/commit-mono/roman-400.latin.woff2: identity=17484 gzip=17484 brotli=17484 bytes
fonts/commit-mono/roman-700.latin.woff2: identity=17260 gzip=17260 brotli=17260 bytes
Font transfer: monaspace-neon=84340/88064 jetbrains-mono=70384/75776 source-code-pro=46484/49152 geist-mono=41364/45056 ibm-plex-mono=41344/45056 cascadia-code=85008/88064 maple-mono-nf=99876/106496 (critical 46432/51200) symbols=223748/235520 fira-code=36756/40960 commit-mono=71812/75776 bytes
deferred_brotli_bytes=44661 budget_bytes=45056
(pass) dead-session production canvas fits the deferred Brotli budget [5.30ms]
1 pass
0 fail
1 expect() calls
accepted_input_bytes: 0,
accepted_input_bytes: 0,
7 passed (1.8m)
Confirmed double-fork regression before repair at
ebcc73a0: detached_child_restores_default_signals_without_changing_parent, 0 passed/1 failed. Exact actual signal_mask_empty=false child_sighup=None, expected signal_mask_empty=true child_sighup=Some(1), crates/infraternal-shim/tests/detach.rs:45. Standalone probe inherits ignored SIGHUP via a shell and blocks it before detach; the original caller mask is checked separately. Runtime148.27s includes nested Cargo build waiting on the protocol oracle; no signal watchdog fired. Preliminary fast run interrupted during Clippy (exit130) to prioritize this required failing-before proof; not claimed as a completed gate. Full final gate will rerun. Standard supplemental TS fast now PASS: 36 protocol tests/34462 assertions, production budgets unchanged. Logs under job scratch.Double-fork fix now passes both standalone detach tests (2/0,2.13s), including all seven closed-stdio masks and original caller-mask preservation. Existing reset_child_signals is reused at each raw fork; no duplicated policy/new dependency/timer. Local CPUWeight10000 benchmark, same GNU debug profile before/after,7 alternating batches of200 real detach launches,no own compile overlap. Running as unit: run-p153205-i46245282.scope; invocation ID: b63e85eace41495f99065b1bad37613f
load_before=22.72 25.53 24.54 4/2949 153207
{"round": 0, "label": "before", "launches": 200, "batch_ms": 371.994879}
{"round": 0, "label": "after", "launches": 200, "batch_ms": 490.346872}
{"round": 1, "label": "after", "launches": 200, "batch_ms": 447.016183}
{"round": 1, "label": "before", "launches": 200, "batch_ms": 450.118447}
{"round": 2, "label": "before", "launches": 200, "batch_ms": 407.638212}
{"round": 2, "label": "after", "launches": 200, "batch_ms": 554.874078}
{"round": 3, "label": "after", "launches": 200, "batch_ms": 447.193228}
{"round": 3, "label": "before", "launches": 200, "batch_ms": 489.794495}
{"round": 4, "label": "before", "launches": 200, "batch_ms": 450.729554}
{"round": 4, "label": "after", "launches": 200, "batch_ms": 479.080426}
{"round": 5, "label": "after", "launches": 200, "batch_ms": 502.731566}
{"round": 5, "label": "before", "launches": 200, "batch_ms": 443.986792}
{"round": 6, "label": "before", "launches": 200, "batch_ms": 446.576097}
{"round": 6, "label": "after", "launches": 200, "batch_ms": 499.230224}
METRIC name=double_fork_signal_reset launches=200 rounds=7 before_batch_p50_ms=446.576 after_batch_p50_ms=490.347 observed_delta_per_launch_us=218.854 observed_delta_at_200_launches_ms=43.771 profile=GNU-debug persistent_reset_state_bytes=0
load_after=20.07 24.87 24.34 6/2955 161806
Bounded launch-only cost is justified by default child signal correctness (D13/D217). Persistent reset state is0 by implementation; total child RSS was not measured. Full final fast gate now running; not yet committed.
Confirmed production merge interaction (#151 + #154), not a load waiver: local_session_counts_and_memory_refusals_follow_real_lifecycle failed in full gate and isolated retry (0/1,4.15s), verify-session-bounds.py:82, expected(1,1),actual(0,0). Registry Live::alive at crates/infraternal-agent/src/lifecycle.rs:21 follows /proc//exe, but CA entry protection makes the shim non-dumpable. Owned real-shim diagnostic: socket_exists=True,state=S,exe_exists=False,exe_stat_errno=13(EACCES),exe_lstat_is_symlink=True.
Additive fix choice before implementation: inspect executable-link metadata without following the ptrace-protected link; retain the bounded kernel birth/state check (already excludes Z/X and recycled PIDs), authenticated rendezvous and no idle timer. Never restore dumpability or remove lifetime validation. Add a disposable non-dumpable subprocess regression first.
Interrupted failed fast run during static compilation (exit130); no completed-green summary claimed. Preserved raw logs. Temporarily parked the separately reproduced double-fork patch so registry and fork fixes can be gated/pushed as distinct logical commits. No auth/secret/protocol-policy changes; no deployment.
Added disposable non-dumpable-process regression before repair: nondumpable_process_keeps_its_authenticated_lifetime,0 passed/1 failed in0.02s,count0 instead of1; process reaped before assertion. Implemented executable-link lstat/type check, retaining birth tick, dead-state exclusion and rendezvous checks. All four lifecycle unit tests now PASS, including PID-reuse and proc-inode eviction. Child dumpability stays disabled; privileged test runners may follow the link, so the new test asserts lifetime under either ptrace policy. Complete fast gate running on this registry-only delta; separately saved double-fork patch remains parked. Logs registry-before-workspace.log and registry-after.log. The first package-only regression build was interrupted to reuse the already-built full-workspace feature set; no test failure waived.
Owner requirement (2026-10-11): after the owner unlocks the CA on the batch deploy, verify the update path before relying on it:
Plan: an independent audit job (defensive verification with failing tests) on the deployed code, plus live checks on monitor01 during a real update to the first signed edge build.
SB-01 fixture compatibility finding: corrected registry gate passes all8 agent_review tests, then attached_window_recovers_final_output_and_exit_after_shim_link_detaches fails before its data oracle at verify-session-exit-credit.py:50,PermissionError13 on the protected shim FD directory. verify-session-retention.py uses the same ptrace-gated observation. Existing non-dumpable protection is retained.
Proposed shared test-only observer: count the listener/accepted endpoints for this fixture’s unique private rendezvous path in /proc//net/unix. Namespace entries for other paths are excluded. This preserves the exact kernel-backed detach count,5s watchdog,unfunded window,FINAL_OUTPUT/status7 and retention assertions; no fixed sleep substitutes for actual detach. Prototype read succeeds on the real protected shim and observes listener+accepted control=2. Imported Rust SB-01 test remains unchanged. Both fixtures will reuse one helper. No production/protocol change or privilege escalation.
Shared dumpability-safe Unix-socket observer implemented in scripts/agent_fixture.py and reused by both retention fixtures. It selects only the unique fixture rendezvous in the owned PID network namespace. Unchanged Rust SB-01 now PASS1/0 in0.98s (real final output/status7 after actual detach); retention PASS1/0 in1.92s (bound,unfunded notification,restart record,retirement handshake race,sibling/control). No deadline or product assertion relaxed; no dumpability change. Broader server suite is running directly from the current compiled test binary while the failed gate finishes its cold static build, to expose any other merge compatibility failures. A fresh complete fast gate is still required before commit. Logs sb01-after.log and retention-after.log.
Protected-shim compatibility fix passed the full workspace tests (including SB-01, retention, 5 DS-03 native cases, CA handoff and auth/TLS); static gate is still waiting on the exact TLS baseline build. Separate real-browser auth: 4 passkey scenarios + 2 recovery scenarios PASS.
Scale: real static shipping binary 20,064,880 bytes, SHA256 b887c4eace727fb3b98cf9570f528c1d1a4a2f80f2d7915cc7cbe7a0e6e54318, recovered 200 simultaneous real shims, admitted 201, after reap 200. 101 facts reads: p50 3.407 ms, maximum 12.664 ms. infraternal-dev/8 CPUs/CPUWeight=10000, local RTT=0; load 14.95/20.68/25.35 ->16.80/20.96/25.41, overlapping own TLS baseline compilation. Timing is a loaded-host diagnostic, not a causal delta. No memory claim: facts-only avoids protected smaps. Evidence: evidence/prod-batch/protected-scale.log (staged after code commits).
Exact TLS baseline attempt hit a Cargo cache reuse problem: git-archive source mtimes predate the merged build, so its reused binary build-script executable emitted the CA handoff linker-script flag into the TLS checkout, which has no handoff-note.ld. Link failed; no baseline artifact was accepted. Refreshed only the temporary checkout's file mtimes to force rebuilding its own workspace sources/build scripts; retry queued. Failed output will be preserved in evidence/prod-batch/baseline-cache-failure.log. Production tests all pass; current static build is proceeding separately. No source or release-profile workaround.
Completed protected-shim compatibility step:
b67d034556, pushed prod-batch. Session registry now lstat-checks /proc/PID/exe without weakening dumpability, birth ticks or PID-reuse checks. SB-01/retention share exact-private-path kernel socket observation. Regression: 0/1 before, 4/0 lifecycle after; full workspace tests PASS. At 200 simultaneous shims: recovered200/admitted201/reaped200, 101 facts reads p50 3.407ms/max12.664ms (loaded-host conditions recorded separately). No extra session state/timers/dependencies.Exact TLS baseline fresh build: 19,991,456 bytes. This step: 20,064,880 bytes, +73,424. Static PIE, no PT_INTERP. Fixed build-cache flags; the only gate exception is the unchanged D205 size ceiling. scripts/verify fast output, verbatim:
How to test: run scripts/verify fast with an isolated Cargo target and TRACER_TEST_PORT=5799. Observe protected process remains counted, then disappears after reap. SB-01 and retention must detach uncredited output and preserve final status.
Double-fork fix gate hit one server test failure: auth::tests::socket_tests::claude_review_visible_liveness::claude_review_watch_expires_host_made_visible_by_membership_edit, line49, initial event type "error" vs "snapshot" (290 PASS/1 FAIL/32 ignored). Isolated same merged binary retry PASS in9.28s. No timeout or production auth change; no #118 baseline waiver claimed. Fresh full gate running before commit. Failed transcript: evidence/prod-batch/fast-detach.log; retry: liveness-retry.log.
Static build succeeds: 20,064,944 bytes, no PT_INTERP; +64 vs protected-shim step and +73,488 vs fresh exact TLS baseline19,991,456. Unchanged D205 ceiling16,404,816 exceeded; this is distinct from the test failure.
Full rerun passed the previously failing liveness test, then failed auth::tests::socket_tests::scale::review_mesh_headroom_snapshot_does_not_close_shared_socket: 10,000 nodes/30,000 grants, snapshot64,595 bytes, liveness resync64,602 bytes, then unexpected websocket close. Load at its measurement9.36/10.52/13.87. Full server290PASS/1FAIL/32ignored. No source timeout changes or exception accepted yet.
Preparing detached dev baseline5bc58323ab14a99613df16c1dbe254e99bda3d28 in this job's baseline directory and a separate warmed dev-target, then compare both binaries in the same load window under CPUWeight10000 (#118). Both failing test source files are byte-identical to origin/dev. Current detach binary separately PASS2/0 in2.62s (signal regression plus all seven closed-stdio cases). Evidence: fast-final-clean.log, detach-current.log; baseline comparison pending.
Detached dev baseline5bc58323ab14a99613df16c1dbe254e99bda3d28 and merged binary tested together under one CPUWeight10000 scope/default300ms RTT. Both unchanged socket cases PASS on both trees. Liveness:9.54s each. Headroom:dev77.75s/merged79.50s, both assembled10,000 nodes/30,000 grants across88 pages, max frame64,612 bytes, liveness resync then shared-socket Pong. Load21.22/17.12/15.96 ->24.03/20.18/17.21; comparison overlapped non-server compilation. No matching baseline failure, therefore no #118 waiver accepted.
The separate exclude-server workspace run failed terminal.open with resource_unavailable after the initial live session exited. Isolated same binary retry PASS41.44s, all24 abrupt stops and4,710,396 payload bytes exact. No timeout changes or production workaround. Fresh full scripts/verify fast running with no other own build/test activity. Both failed transcripts and retries are preserved in evidence/prod-batch/.
Completed child-signal step:
431ec8a743, pushed prod-batch. Both raw double-fork children call the existing audited reset_child_signals helper; parent reload mask/dispositions preserved. Regression0/1 before,2/0 after, all seven closed-stdio masks preserved. Existing SSH-helper and embedded-child mask regressions PASS. Shared-host200-launch measurement recorded earlier: observed+218.854us/launch, +43.771ms/200, no persistent reset state.Final complete workspace tests PASS, including server291/0/32ignored, shim65/0/13ignored, detach2/0; SB-01, retention, five DS-03 cases, ordinary CA handoff and native/proxy TLS all PASS. Both intermittent socket failures passed this complete run without timeout changes or a baseline waiver. Detached dev/merged comparisons passed both cases; failed attempts and retries are preserved.
Static20,064,944 bytes, no PT_INTERP; fresh exact origin/tls-session-prod19,991,456 =>+73,488 bytes (+64 for this signal fix). Unchanged D205 ceiling16,404,816 is the sole failing row. scripts/verify fast output, verbatim:
How to test: run scripts/verify fast with TRACER_TEST_PORT=5799 and a Cargo target isolated from other checkouts. The real detach signal test must report an empty mask and SIGHUP termination without changing the parent. Existing closed-stdio, credit, session-bounds and TLS tests must still pass.
Finished production batch verification. Head
0c9d3b6158, branch prod-batch, pushed after each atomic commit; worktree clean. No deployment; issues remain open.Commits:
b67d0345protected-shim lifetime/counting plus compatible SB-01/retention observers;431ec8a7shared default signal reset at both raw-fork child branches plus real regression;0c9d3b61audit, author checklist, transcripts, numbers and production screenshots. Both bugs have failing-before tests. Every production child-launch site is covered by prepare_child/prepare_spawn or their shared raw-fork reset; same-PID fexecve intentionally retains reload policy.scripts/verify fast completed on the exact final source code (evidence-only final commit). Output verbatim:
Only D205's known unchanged binary-size ceiling fails: x86 static PIE/no PT_INTERP20,064,944 bytes vs ceiling16,404,816; fresh exact origin/tls-session-prod5d02b8d6 build19,991,456 =>+73,488 bytes, +64 for signal fix. No budget/profile changes. ARM is not reached after the x86 ceiling; no ARM claim.
All workspace tests PASS: server291/0/32existing ignored diagnostics, shim65/0/13ignored, detach2/0; ordinary CA handoff, SB-01, retention, DS-03's five real native cases and auth/native+proxy TLS pass. Prior socket and terminal-open failures are preserved; isolated retries and final full gate pass. Detached dev5bc58323 and merged paired socket tests both pass in the same loaded window, so no #118 waiver was used. Explicit ignored warm shipping-restart/128-PTY benchmark not rerun: harness writes the separate ca-handoff job scratch directory outside this brief's scope; prior reviewed evidence remains, no new warm-shipping claim.
Tracer check PASS; src142tests/10,901 assertions PASS; production build+all D188/font budgets PASS; DS-02 transfer review PASS; 7 Playwright tests PASS (dead-session4, detached-output1, transient retry/input2), TRACER_TEST_PORT5799/default300ms. Supplemental standard TS fast PASS, 36protocol tests/34,462 assertions. Real-browser auth PASS4passkey+2recovery scenarios (Chromium149.0.7827.55).
Final-head frontend budget output verbatim:
Scale: final shipping binarySHA256a265bc649927e8b7654228a30a705fec3a839aa1e4c7f7fbc67a31ece9c27beb; 200 simultaneous real shims recovered,201admitted,200after extra reap. 101facts reads p50 4.706ms/max14.941ms, localRTT0/80x24/infraternal-dev8CPUs/CPUWeight10000, no other own build/test overlap; load14.20/22.66/21.65 ->16.03/22.90/21.73. Fork benchmark:7 alternating 200-launch GNU debug batches, observed+218.854us/launch/+43.771ms per200; shared-host timing limits recorded, no added persistent reset/registry state, no total RSS/PSS claim.
Evidence and required author review: evidence/prod-batch/README.md, reviewer-checklist.md, binary-final.txt, fast-complete.log, final-scale.log, detach-cost.log, screenshots/*.png. Final frontend head budget is recorded above because the evidence-only commit changes build revision metadata. Own temporary targets, artifacts and detached checkouts are being removed after recording this finish; committed evidence is retained.
Cleanup complete at
0c9d3b6158: own prod-batch/pb scratch roots, build targets, binary artifacts and detached baseline checkouts removed. Committed evidence retained. Worktree clean; local HEAD equals origin/prod-batch. Final-head production build and every D188/font budget pass; finish SHA and full gate output are in the preceding comment. No deployment or issue closure.Orchestrator: batch verification finished at
0c9d3b61(shim signal reset in both detached children, protected-shim session counting). Mergedorigin/ui-inbox(machine sections, CA unlock, inbox) →707a4572, pushed.Merged-tree gates:
bun run check: passbun test src: 147 pass, 0 failStatic x86_64 + aarch64 builds running; deploy follows.
Orchestrator: deployed
707a4572to production (backup/var/backups/infraternal/20261011-125449-707a4572). Static x86_64 20,064,944 B, aarch64 17,884,216 B.infraternal-serveractive, NRestarts=0, no warnings since start; the protected shim kept the running session across the restart;server_ca_startup outcome="cold"(CA locked, awaiting the owner's unlock from the new sign-in toast / inbox).release-prod-2now at6320f5b5(merge of the batch; tree identical to the deployed one).Next (owner rule from 2026-10-11): after the owner unlocks, audit (1) the CA stays unlocked with no leak and (2) update signature verification.
Orchestrator: owner reports the CA unlocked (2026-10-11). Live checks on production (pid 157470, started 12:26:13 UTC): core limit 0, fs.suid_dumpable=0, no swap, VmSwap 0, environ has no secret variables (LANG PATH USER INVOCATION_ID JOURNAL_STREAM SYSTEMD_EXEC_PID MEMORY_PRESSURE_*), argv has no secrets, only child is the embedded agent (--fd 3). Finding: the unlock leaves no journal event, so custody cannot be confirmed from logs. Finding (orchestrator ops): deploy-prod.sh restarted the server on web-only deploys at 12:07 and 12:26 UTC, re-locking the CA; fixed (web-only now swaps the directory live, no restart). Audit job
audit-ca-updatelaunched on release-prod-215d15c25for CA custody + update signature verification.Starting post-unlock defensive audit on audit-ca-update, base
15d15c257c(production pin). Read all #154 comments; original scope has no linked parent. Continuing existing handoff findings; only production-code change authorized is an unlock audit event. Pinned tree lacks apps/release-signer and newer DECISIONS entries; will distinguish absent signed-update implementation from tested refusal properties. No production access or deployment.Supplemental signer audit (absent from production pin): local origin/dev
c0239ad6f2, apps/release-signer/src/index.js:86-117. SIG-01 P1: missing VERSION and stable request with edge VERSION both return HTTP200, failing refusal tests. SIG-02 P2: newline flat filename accepted/signed, ambiguous SUMS framing. Real Worker SSHSIG coverage PASS: altered SUMS/other key refused, VERSION hash covered when present. Ephemeral in-memory keys/storage only. Evidence signer-*.txt and reproducible hash-pinned harness in docs/verification/ca-update-audit; README appended. Client must validate signed identity/channel independently. No production change.Proposed additive unlock audit event (authorized brief, D168/#154): instrument CaAdministration::finish only. One server_ca_unlock event per completed parsed authority attempt, including bound requesting user ID when known, current UTC Unix time and outcome. Early foreign/replayed attempts remain unattributed; no cookie hash, ceremony ID, proof, wrapped/unwrapped material or error diagnostics. Cancellation and an install followed by rollback-report failure must retain accurate custody outcomes. Existing bootstrap-to-unlock dispatch calls the same method. Syntax/header refusals stay empty HTTP errors. No new dependency, polling task or schema.
Initial fast gate interrupted after two corrected test-only Clippy lints and replacement fixtures stalled. Evidence: shell umask002; Cargo test executables mode0775; replacement correctly refuses writable candidates. Rerun uses umask022 and owned target executables normalized to remove group/world write; no permission policy/production change. Dedicated audit: five unresolved security-property regressions plus missing unlock event fail as expected; real CA authority23 PASS and handoff7 PASS. Not a #118 load waiver.
CA-01 P2 ca.rs:67-69/ca_memory.rs:174-180: unlocked_signer_requires_persistent_locked_pages FAIL, isolated unlocked process VmLck0kB with dumpability disabled. Persistent OpenSSL key and export/decrypt heap copies not mlocked; no swap/core protections supplied for production mitigate exposure; no leak claimed. Evidence and test results appended to docs/verification/ca-update-audit/README.md; logs evidence/ca-update-audit/before-*.txt. No production fixes except upcoming unlock audit event.
CA-02 P2 ca_administration.rs:301-304: unlock_audit_records_principal_time_outcome_without_material FAIL before, zero events expected3 for success/replay/decrypt refusal. Authorized additive content-free event follows in separate commit; proposal already recorded. Evidence and test results appended to docs/verification/ca-update-audit/README.md; logs evidence/ca-update-audit/before-*.txt. No production fixes except upcoming unlock audit event.
CA-03 P2 ca_http.rs:71,113-125/runtime.rs:819-821: sequential_unlock_starts_require_a_temporal_budget FAIL,31 accepted starts in72ms. Concurrency/deadline/pending caps exist; no temporal per-principal budget. Authenticated-admin availability gap; proposed30/minute regression does not change contract. Evidence and test results appended to docs/verification/ca-update-audit/README.md; logs evidence/ca-update-audit/before-*.txt. No production fixes except upcoming unlock audit event.
UP-01 P1 restart_binary.rs:95-97: unsigned_or_tampered_replacement_is_refused_before_exec FAIL,4/4 crafted unsigned/tampered/truncated/65MiB candidates admitted,0 executed. Real signature hook returnsOk; no release keys/version/channel/downgrade verification. Root/protected-parent warm eligibility remains; external verification only. Evidence and test results appended to docs/verification/ca-update-audit/README.md; logs evidence/ca-update-audit/before-*.txt. No production fixes except upcoming unlock audit event.
UP-02 P2 restart_binary.rs:35,86/restart.rs:303: admitted_inode_cannot_change_in_place_before_execution FAIL, retained fd reads rewritten bytes. Pathname-replacement test PASS. Future verifier must also freeze/check content; no claim of bypassing currently implemented cryptography (UP-01). Evidence and test results appended to docs/verification/ca-update-audit/README.md; logs evidence/ca-update-audit/before-*.txt. No production fixes except upcoming unlock audit event.
UP-03 P1 restart.rs:359-370: unusable_admitted_image_keeps_the_existing_service_running FAIL, empty admitted executable triggers exec refusal after runtime shutdown; existing listener is gone. Supervisor recovery is cold/locked. Violates owner failed-update continuity requirement; early refusal/reply-failure paths keep serving. Evidence and test results appended to docs/verification/ca-update-audit/README.md; logs evidence/ca-update-audit/before-*.txt. No production fixes except upcoming unlock audit event.
Documentation step
2a4d07cc46: committed initial eight findings and coverage matrix in docs/verification/ca-update-audit/README.md, separating production pin from supplemental Worker revision. Initial full fast gate: fmt/VT fmt/clippy/test/deny PASS; known size-only FAIL. x86 static PIE/no PT_INTERP20064976B vs ceiling16404816; server296PASS/35ignored. Failing regressions and raw evidence are being committed in separate logical steps. No production change/deployment.Completed separate test steps:
92c65f03test: reproduce audit results and signer release identity gaps (#154)3f14b5f4test: preserve failing replacement verification and custody expectations (#154)ef315d53test: verify unlocked CA custody and authority boundaries (#154)2a4d07ccdocs: record pinned CA custody and release verification findings (#154)Custody ordinary4PASS; authority24PASS with explicit unresolved ignores; handoff7PASS. Five unresolved production regressions and missing-event regression all FAIL as expected via fresh compiled runner. Baseline100 real unlocks/1000 memberships: finish wall p50=3556903ns,p99=8204650ns,processCPU p50=3805686ns,p99=6921201ns;0 audit events. infraternal-dev/8CPUs/CPUWeight10000, load6.90/14.64/20.30 ->8.20/14.31/20.04; no own build/browser overlap, GNUdebug formatting diagnostic, journal I/O excluded. Initial full fast Rust rows PASS except known static-size ceiling; fresh workspace no-run and Clippy PASS. No production fixes in these commits.
CA-02 red/green: real passkey/SQLite test failed with 0 events before; now passes 4 events (success, replay refusal, wrong wrap refusal, cancellation). Guard logs only server-attributed user/time/outcome and marks custody before rollback await. No authority change. 100 real unlocks at 1000 active administrator records: 100 events/26,800 formatted bytes; finish wall p50 3.557 -> 3.847 ms, p99 8.205 -> 9.963 ms. Shared-host load differs (before 6.90/14.64/20.30; after 18.56/27.23/23.88), so delta is diagnostic, not causal or a release-budget proof. Evidence: docs/verification/ca-update-audit/README.md and evidence/ca-update-audit/unlock-cost-{before,after}.txt. Fresh targeted suites and remote-disabled cargo check pass. Final fast gate reached static build after all other Rust rows passed; separate TypeScript gate passed. Logging commit awaits final gate evidence.
Post-logging fast gate: fmt/VT fmt/clippy/cargo-deny PASS; test row has one failure in auth::tests::socket_tests::review_control::pending_mesh_writers_preserve_authority_pool_capacity (296 PASS/1 FAIL/35 ignored). Pong while pending passed at 17 ms; queued Result was not ok after the fixture's 2-second freshness window. Initial full pre-logging gate passed this test. No assertion/timeout changes. Following #118: detached dev baseline
ded3abd609under job scratch, then same-load-window comparison and full rerun. Static size is known overrun: 20,067,504 vs 16,404,816 bytes; +2,528 bytes vs own pre-logging build 20,064,976. Evidence: evidence/ca-update-audit/fast-final.txt.#118 comparison completed: detached dev
ded3abd6and current audit branch both PASS the exact socket test in the same load window (2.031s vs 2.004s; control pong 1ms vs 2ms; load 24.64/28.41/29.30 ->25.55/28.53/29.33). Baseline did not reproduce the failure, so no #118 waiver is claimed. Full fast gate is rerunning with RUST_TEST_THREADS=1, unchanged assertions/deadlines. Evidence: evidence/ca-update-audit/socket-baseline-comparison.txt and fast-final-serial.txt.Completed authorized unlock logging:
73965a488a. Real event regression FAIL before (0 events), PASS after (4 outcomes incl cancellation); existing authority unchanged. Final server suite 297 PASS/0 FAIL/35 ignored; remote-disabled check and separate TypeScript fast phase PASS (154 tracer/36 protocol). Static PIE/no PT_INTERP 20,067,504 bytes vs 16,404,816 ceiling; +2,528 bytes vs own before build. No waiver or ceiling change. Evidence: docs/verification/ca-update-audit/README.md; evidence/ca-update-audit/fast-final-rebuilt.txt. Verbatim fast output:Finished defensive audit on audit-ca-update, head
9bcd5d7966, pushed only that branch (6 atomic commits). Report/summary/checklist: docs/verification/ca-update-audit/README.md; transcripts: evidence/ca-update-audit/. CA-02 unlock event repaired in73965a48(real regression red before, green after; success/refusal/cancellation, no material). Five unresolved production properties and two supplemental newer Worker findings retain explicit failing tests; signature admission remains a no-op at the production pin. No P0 or observed production leak; no production access/deploy or other production fix.Final scripts/verify fast exit 1 only for known static size: stripped x86 static PIE/no PT_INTERP 20,067,504 bytes vs 16,404,816 ceiling (+2,528 bytes vs own pre-logging build). Server 297 PASS/0 FAIL/35 ignored. Separate TypeScript phase PASS (154 tracer/36 protocol), remote-disabled check PASS. Initial socket failure and baseline/cache evidence retained; no #118 waiver. Verbatim final gate output:
Reproduce: build fresh workspace test binaries with cargo test --locked --workspace --no-run using a dedicated job CARGO_TARGET_DIR and umask 022; run systemd-run --user --scope -p CPUWeight=10000 python3 docs/verification/ca-update-audit/run-audit.py --target --phase after; inspect expected FAIL transcripts and the repaired event PASS. Use unique TRACER_TEST_PORT=5854 and separate baseline targets. Before-state evidence is committed; the before runner applies to
92c65f03. Benchmark: 100 real unlocks/1000 active records, 268 B/event; wall p50 +0.290 ms on unequal shared-host load windows, not causal or budget certification. Only owned baseline/scratch targets removed after preserving evidence; clean worktree. Independent review/owner follow-up remains, issue left open.Starting the authorized CA-01/CA-03 repairs on audit-ca-update, base/head
9bcd5d7966(production audit pin). Read project contracts, audit README and all 121 #154 comments; no parent linked. Reuse OpenSSL secure heap and auth admission; retain cold-start/no-timer custody. Worker SIG-01/SIG-02 changes will use separate signer-hardening from origin/dev. UP-01/02/03 stay deferred with precise implementation notes. No deployment or production access. Evidence: evidence/ca-update-hardening/ and audit README.Worker lane started on signer-hardening from origin/dev
ded3abd609. Read its CLAUDE.md, DECISIONS.md, architecture/process; no Worker test harness existed. SIG-01 reproductions next to the Worker: missing/mismatched VERSION HTTP200 before, no signed identity manifest. Canonical additive format proposed: MANIFEST with LF-only format/version/channel lines followed by exact SHA256SUMS, plus MANIFEST.sig in existing SSHSIG namespace. Keep SHA256SUMS and its signature for existing operator verification; new consumers must require MANIFEST identity/channel and VERSION hashes. Reject unsafe names rather than skip them. Owner deploys manually; this job never deploys.CA-01 implementation detail: extend existing exclusive LockedBytes storage ceiling from 4096 to 32768 for the already-bounded HTTP unlock/bootstrap intake; handoff frame remains independently bounded at 2090 bytes. Stream body chunks directly into that mapping after auth/admission; parsed scoped_key remains Zeroizing and body mapping drops before authority awaits. Add Send only for exclusive ownership transfer, no Sync. This reuses audited mmap/mlock/MADV_DONTDUMP rather than introducing a movable heap copy or another allocator. HTTP peak: 32 KiB x 8 admitted bodies = 256 KiB per server, plus process secure heap. No new parser or protocol.
Fresh before-state regressions reproduced from the dependency-only workspace build (production behavior unchanged): CA-01 unlocked_signer_requires_persistent_locked_pages FAIL, VmLck=0 KiB for one synthetic unlocked signer. CA-03 sequential_unlock_starts_require_a_temporal_budget FAIL,31/31 starts accepted in98ms. Evidence: evidence/ca-update-hardening/before-{unlocked_signer_requires_persistent_locked_pages,sequential_unlock_starts_require_a_temporal_budget}.txt. Tests use real OpenSSL and passkey/SQLite authority; no production state or key inspection. Current gate is still running; no completed gate claim.
Completed CA-01 prerequisite
cadac1dd: optional direct reuse of the already pinned OpenSSL at the D40 audited shim FFI boundary, enabled only with remote ca-handoff. No dependency versions or active production behavior changed. Fresh gate: server297 PASS/0 FAIL/35 ignored; static20067504B vs16404816 ceiling, delta0B vs9bcd5d79. Initial missing-lock-entry gate was retained; fixed-lockfile full rerun passes every non-size row. Evidence: dependency-fast*.txt. Verbatim scripts/verify fast summary:Worker gate setup/base correction: first dev gate failed the unchanged terminal restart fixture at iteration22 with terminal command queue full; second passed that test but failed initial_attach because its ignored target/ fixture parent was absent. No waiver or fixture/code changes. Created only the ignored target/ directory; build output stays in the dedicated scratch target. Shared origin/dev is now50a89572 (production backmerge); signer-hardening was moved from the older
ded3abd6ref to50a89572 before any fix commit, preserving the Worker patch/tests. The upstream Worker is byte-identical across these bases. Read the updated dev contracts; current-dev full gate is running. SIG-01 real Worker tests pass8/8. Evidence: evidence/signer-hardening/sig01-{fast,fast-retry,current-dev-fast,current-dev-tests}.txt. No deployment.CA-01 stronger test found heap initialization alone is insufficient for imported signers. Pinned OpenSSL3.6.3 crypto/ec/ecx_backend.c:64-73 uses OSSL_PARAM_get_octet_string for raw provider import; crypto/params.c:1364 allocates ordinary malloc. Real imported-key secure counter delta=0 despite VmLck1024KiB. Existing DER decoder instead allocates32secure bytes, provider-backed, still32 after an actual Ed25519 signature. Evidence: import-probe.txt and ca01-locked-pages-debug.txt. Repair will reuse that decoder via one fixed48-byte in-memory PKCS#8 buffer in LockedBytes for wrapping unlock and handoff import; no disk/PEM, persistence/frame change or crypto-engine choice. Decoder PKCS#8 transient storage is cleansed on free (crypto/asn1/p8_pkey.c:24-28). Heap-only attempt and subsequent corrected unused-import/doc lints are retained as failed evidence. Final gate must pass all non-size rows.
Resuming audit-ca-update after host resize. Restored orchestrator checkpoint 7cf68310 with git reset --soft HEAD~1; base
cadac1ddb1. Read #154 history (128 comments, no linked parent), contracts and audit README. Reviewing CA-01 patch and existing signer-hardening scratch worktree; will split commits, run targeted gates per commit and full fast on final heads. No deployment.SIG-01 committed
a7c11bfbon signer-hardening from origin/dev lineage50a89572(head at resume25e4e026). VERSION is mandatory and must equal the request with optional final LF; same read validated/hashed. Additive LF canonical MANIFEST includes format/version/derived-channel plus exact sorted SHA256SUMS, with MANIFEST.sig in existing SSHSIG namespace. Existing SHA256SUMS signatures preserved. Real Worker/independent ssh-keygen: 8/8 PASS; missing/mismatch regressions red before. SIG-02 unsafe-name regressions reproduced red; fix and docs follow. No deployment.Completed shim prerequisite
0f689f46: once-only pinned OpenSSL secure heap wrapper behind ca-handoff, 1MiB/32-byte blocks; reuse exclusive LockedBytes for up to32KiB with Send ownership transfer. fmt, feature-enabled shim clippy and kernel locked-page/bounds test PASS (1/1). Broader shim64 PASS/13 ignored, detach2 PASS; mouse probe could not find zig because targeted command omitted gate PATH (stderr evidence), no source failure inferred. Added existing staged Zig to targeted PATH; full gate will run the probe. Replaced only orchestrator WIP7cf68310 via exact-SHA force-with-lease to publish split history; no other branch changed.CA-01 code committed
a1e6f85cand pushed: secure heap initialized before cold generation/inherited signer import; root result0/2 refuses startup, non-root warns. Unlock/handoff imports reuse secure DER decoder via48-byte LockedBytes; existing formats and cold/no-timer custody unchanged. Remote-feature Clippy PASS. Checkpoint compiled regression already showed VmLck0->1024KiB, secure imported-seed delta32bytes (ca01-final-unlocked_signer_requires_persistent_locked_pages.txt); current fresh CA tests pending. Correction: resume package-only test lacked remote and ran0 CA tests; explicit --features remote rerun is building. No passing fresh CA result claimed yet. Full checkpoint ca01-fast-final also retains mesh continuation failure (not a waiver); final full gate follows.SIG-02 committed9e60256d; docs/evidence50be0d21; both pushed signer-hardening. Refuse unsafe/nested filenames with static HTTP400 before publication. Flat1-128ASCII bytes, initial alphanumeric, remainder letters/digits/dot/underscore/hyphen; reject LF/CR/NUL/path separators/whitespace/non-ASCII/leading option markers. 14 unsafe-name tests FAIL before and PASS after,23total Worker tests PASS (node and real ssh-keygen). Owner manual deployment summary: mandatory exact VERSION(409 missing/mismatch), additive MANIFEST+MANIFEST.sig binding version/derived channel/hashes, legacy SHA256SUMS signatures preserved, conservative filename validation(400). No new binding/key rotation; consumers must require manifest plus pinned-key/channel/hash/monotone checks. apps/release-signer/README.md gives exact bytes and verify commands. Full branch-head gate running; never deployed.
Completed CA-01 intake7ef543a5, shared auth helper0c2cf292, CA-03 routes7aebe5af; each pushed audit-ca-update. Final targeted fmt/remote-Clippy PASS, helper1PASS/1diagnostic ignore, realHTTP/intake5PASS, CA-03realregressions2PASS. 31start requests now30accepted/HTTP429 Retry-After60/static body (155ms), before31accepted/98ms. Sameprincipal cookies/routes share budget; finish/status unaffected; helper tests principal/time/capacity boundaries. Remote CA suite56PASS/10ignored including8handoff tests and heap-refusal/import-lock regressions. Head7aebe5af full milestone gate exited at initial latency-proxy self-test: configured300ms observedp50347.05ms/p99347.93ms, before any Rust rows. No waiver or cause claimed; unchanged full gate rerun follows. Evidence ca03-final*.txt, milestone-fast.txt.
CA-01 scale on resized infraternal-dev/12CPUs, CPUWeight10000, load36.98/38.61/31.36, no own build/browser overlap:10000 realEd25519 keys without/with secureheap VmLck0->1024KiB; secureused320512B (32B/key plus512B randomstate),1MiB heap69.4%headroom. RSS before/live keys9964/18408KiB vs11880/18216KiB; noisy-process diagnostics, not whole-server budget or improvement proof. Wrapped/unlocked fixture VmLck1024KiB, RSS11332->18840KiB, secure832B; additional importedkey32B allocation/release asserted. PeakHTTP mappings32KiB*8=256KiB/server; no idle worker. heap-scale.txt and locked-pages-resume.txt.
Finished the authorized repair scope. Production head
b0cf61efca(audit-ca-update); Worker head50be0d219d(signer-hardening). Both pushed; no deployment; issue remains open.Completed evidence/reproducer steps:
3c740108hardening-phase audit harness,d1f4159fWorker cost reproducer,b0cf61efrepair/evidence/checklist/deferred-update documentation. Evidence: docs/verification/ca-update-audit/README.md and evidence/ca-update-hardening/. All11 expected audit outcomes match; CA-01/CA-03 repaired, CA-02 still passes, UP-01/02/03 deliberately still fail. Worker23 tests pass; TypeScript154 tracer/36 protocol pass; remote-disabled build passes.CA sizing: 10000 real keys use320512 secure bytes including512 RNG bytes; persistent VmLck1024KiB, measured before0. One key import/release changes secure use by32B. Before/after diagnostic RSS endpoints18408/18216KiB at10000keys; high host load, no performance-improvement claim. Shared principal admission is30/min across all start routes/cookies; finish/status bypass it. README records4096-principal and12/1002-file Worker diagnostics and limitations.
Owner manual Worker change summary: upload requires matching VERSION (409 otherwise), artifact names restricted to1-128 conservative ASCII characters (400 otherwise), additive MANIFEST + MANIFEST.sig bind version/channel/exact sorted hashes using existing key/namespace. Legacy SHA256SUMS signatures preserved for compatibility; legacy verification alone cannot establish the identity binding. No key/binding rotation required; owner deploys manually. UP notes specify pinned-key/channel/monotone verification, sealed memfd admission/execution, and signed-version self-check before runtime shutdown; implementation remains on release-pipeline.
Final production scripts/verify fast on
b0cf61ef: network self-test PASS (p50306.75ms/p99313.40ms);303 server PASS/0FAIL/35ignored. No timeout/assertion changes or #118 waiver. Static x86_64 stripped PIE, no PT_INTERP:20090448B versus16404816B ceiling, excess3685632B, delta+22944B versus own audit base20067504B. ARM not reached. Verbatim final gate output:Final Worker-branch scripts/verify fast on50be0d21:291 server PASS/0FAIL/32ignored. Static x86_64:20294992B, delta0B versus same dev baseline, existing16404816B ceiling; ARM not reached. Verbatim final gate output:
Author reviewer checklist complete. Both full gates fail only their unchanged static-size ceiling; every other row passes. Independent review and manual Worker deployment remain with the owner.
Starting independent verification review on review-audit-ca-update at target
b0cf61efca; merge base with dev15d15c257c. Read all 136 issue comments; original issue has no linked parent. Continuing recorded CA repairs and deferred UP-01/02/03 evidence, checking production behavior and fresh fast gate. Local commits only, no implementation fixes, push or deployment. Unique TRACER_TEST_PORT=5874; job-local Cargo target and short fixture root.Independent review gate evidence: supplemental TypeScript phase on target
b0cf61effailed only the unchanged apps/tracer/src/canvas/mesh/layout.test.ts:75 timing assertion (1000 nodes, 10 layouts, average114.127ms versus25ms maximum;153PASS/1FAIL). Host load immediately after52.69/44.82/33.09; own cold Rust compilation overlapped. No code/assertion change or load waiver. Retaining ts-head.txt and requiring a fresh weighted full TS pass after own builds stop. Full Rust fast gate still running against unchanged target sources.Review gate setup correction: first fast run stopped its workspace test row at scripts/agent_fixture.py:33 before the real daemon oracle: chosen review state root yielded a117-byte AF_UNIX path. Corrected to review-owned /home/kayg/.cache/codex-scratch/acur/s; preflight passes. No product code/assertion/timeout change and no #118 waiver. Retain fast-head.txt; full unchanged-source gate will rerun after the initial static build. Independently compiled target-head CA/deferred-UP cases are running with the corrected fixture root.
Independent target-head verification
b0cf61efconfirms deferred P1 UP-01 and UP-03 remain: unsigned_or_tampered_replacement_is_refused_before_exec fails exit101 (4/4 unsigned/tampered/truncated/65MiB candidates admitted;0 executed;0.016s), restart_binary.rs:95 hook returnsOk. unusable_admitted_image_keeps_the_existing_service_running fails exit101 (live listener before replacement, refused connection after exec refusal;0.099s), restart.rs replacement shuts down runtime before exec. UP-02 P2 also reproduces retained-fd byte mutation (exit101;0.018s). These are existing deferred findings, not new CA regressions. Target-head CA custody, handoff, passkey authority, protected intake, root-memory-refusal and temporal-boundary controls pass. Durable review findings appended; independent real-ELF test commits and final gate/evidence follow. No implementation changes/push/deployment.Independent UP-01 P1 test committed locally on review-audit-ca-update at
c938abd9f7. reviewer_real_elf_requires_release_authentication uses actual copied ELF executables: unsigned and invalid-signature fixtures2/2 admitted,0 executed;exit101 in0.966s, restart_review_tests.rs:58. Explicit ignored regression preserves deferred state; implementation unchanged. Formatting/workspace test compilation pass. Corrected exact-target fast gate completed:303 server tests PASS; all Rust rows PASS except unchanged static ceiling. Fresh static20090384B/no PT_INTERP; separate TS154/36PASS and feature-disabled cargo check PASS. Isolated10000-key secureuse320512B/VmLck1024KiB;4096-principal admission p50/p9998.811/142.392us;100unlocks/100events/26800public log bytes at load59.66->58.04. No push/deployment.Independent UP-02 P2 test committed locally at
92eef26200on review-audit-ca-update. reviewer_admitted_elf_header_cannot_be_superseded admits an actual ELF, writes/syncs its header through the source inode, then observes the changed header on the retained descriptor. Exit 101 in 1.748 s at restart_review_tests.rs:75; candidate executions = 0. Formatting and workspace test compilation pass. Existing deferred finding remains confirmed; no implementation change, push or deployment.Independent UP-03 P1 test committed locally at
6b376eef55on review-audit-ca-update. reviewer_exec_refusal_preserves_existing_listener runs in a bounded isolated subprocess: corrupts a real ELF before admission, proves its listener is live, invokes actual replacement, then fails because exec refusal removed the listener (restart_review_tests.rs:143). Exit 101 in 0.360 s; unusable candidate cannot execute. Formatting/workspace test compilation pass. All three confirmed deferred findings now have separate independent test commits. No implementation changes, push or deployment. Final reviewer checks/evidence archive follow.Reviewer checks: initial new-test Clippy caught option_map_unwrap_or_else in the private fixture (restart_review_tests.rs:9); corrected to equivalent map_or_else without a lint suppression or production change. Ordinary CLI check initially refused its pre-existing C probe under group-writable umask; rerun with umask 022 passes 38 tests/0 failures/6 explicit ignored regressions in 7.45 s. Corrected workspace all-target Clippy and formatting pass. Failed transcripts retained; no waiver or permission-policy change. Rechecking the three intentional failures before committing this test-only fixture cleanup and final evidence.
Reviewer fixture lint cleanup committed at
0c2e1318e4: equivalent map_or_else root selection, no assertion/permission-policy change. Formatting and corrected all-target workspace Clippy PASS; corrected ordinary CLI suite 38 PASS/0 FAIL/6 explicit ignores. All three explicit review probes still fail in one fresh Cargo invocation (0 PASS/3 FAIL in 2.51 s). Earlier test-only lint/umask attempts retained. Reviewer code changes are cfg(test) only; final static byte check and evidence archive underway. No implementation changes, push or deployment.Resuming independent custody-only verification on review-audit-ca-update, base/head
4a598a1c58. Targetb0cf61efca, merge base15d15c257c. Read all 145 comments on #154; issue has no linked parent. UP-01/02/03 settled/deferred and will not be rerun. Fresh custody tests, locked/release observations, principal isolation, restart/log outcomes and local weighted measurements; no push or deployment. Details remain evidence/review-audit-ca-update/.Starting independent verification on review-signer-hardening at target
50be0d219d; merge base with origin/dev25e4e026f6. #154 has no linked parent. The requested signer-hardening/result.md is absent; the latest Worker report is recorded in #154 comments 123–136. Reviewing the three SIG-01/SIG-02 commits only. Local commits only; no implementation edits, push or deployment.SIG-R1 P2 confirmed at apps/release-signer/src/index.js:84,126-128. The committed Worker regression signer_partial_manifest_publication_can_be_retried injects one SHA256SUMS put failure: the first request returns 500 after MANIFEST.sig persists, with releases/index.json absent; retry returns 409 instead of 200. Failing test committed locally on review-signer-hardening at
5125185221. Implementation unchanged. Full fast gate remains pending.Custody continuation: independent libc observer of actual chunked_intake_is_bounded_before_parsing PASS. Both 32,768-byte locked mappings contain only zeroes immediately before real munmap, including the over-limit refusal path; retained evidence locked-release-observation.txt and locked_release_probe.c. No byte dumps or private material emitted. New authenticated-principal/cold-restart/provider-release tests and fresh fast gate still running; no verdict yet. Branch head remains
4a598a1c.SIG-R2 P2: D226 mismatch at apps/release-signer/src/index.js:40,52. signer_edge_version_is_a_real_utc_timestamp shows edge-20260230T120000Z, edge-20260101T240000Z, and edge-99999999T999999Z all return HTTP 200 and write six R2 objects including releases/channels/edge.json. The invalid far-future identity can interfere with monotone edge admission. Failing test committed locally at
8b4c0e144e; implementation unchanged.SIG-R3 P2 at apps/release-signer/src/index.js:85,131-142. Barrier-controlled signer_concurrent_publications_remain_indexed makes two authorized release requests read the same empty index: both return HTTP 200, but the final releases/index.json has one row (accepted=2, indexed=1). This loses a successfully signed release from D224's append-only index. Failing test committed locally at
c19b396047; implementation unchanged.Custody controls: fresh 59 PASS/0 FAIL/11 diagnostic ignores, including 3 new independent tests (2 principals with 30 starts each; cold source plus real proof/signature; secure seed occurrence 1 -> 0 on last release). Final-source fmt/VT fmt/Clippy pass. Workspace test run with two test threads reports the existing auth socket fresh-delete test failed; isolated unchanged target test passes 1/1 in1.32s. Its source is byte-identical to dev. Failure details and detached dev baseline check pending; no #118 waiver or new custody finding inferred. Evidence: continuation-corrected-fast.txt, socket-fresh-delete-isolated.txt, dev-baseline-*.txt. Branch head still4a598a1c; no push/deploy.
Gate comparison: two-thread full server suite305 PASS/1 FAIL/36 ignored (458.99s). Existing fresh_delete.rs:113 received static "mesh unavailable" instead of the expected fresh-proof refusal. This is an assertion/availability failure, not a #118 timeout; no waiver. Exact unchanged isolated test PASS on target1/1 in1.32s and detached dev1/1; test source identical. Failure and comparison archived in socket-failure-{details,decoded} and socket-fresh-delete-{isolated,dev}. Requiring complete serial fast rerun, unchanged assertions/deadlines. Cold-source review fixture also now decodes public certificate DER as a restarted process would instead of sharing its predecessor's provider reference. No production change or new custody P0/P1 inferred.
Finished independent verification. Target
50be0d219dee0111a44e2c8b82f7735d98c67d4b; merge base25e4e026f6d7b312829653d9f163faba692f4db9; local review branch headc19b3960477720a2be2287f03e3aa38f9c0f5cfe. Findings: P0=0, P1=0, P2=3, P3=0. Verdict: MERGE (no P0/P1) under the owner rule; the P2 failing tests are for forward fixes. Test-only commits:51251852,8b4c0e14,c19b3960. No implementation change, push or deployment. Findings are recorded in/home/kayg/.local/state/codex-jobs/infraternal/review-signer-hardening/findings.md.Worker tests: 23 existing pass; the three independent regressions fail as recorded. Exact-head
scripts/verify fast: exit 1; Rust tests 291 passed, 0 failed, 32 ignored. The network self-test passed. Static x86_64 exceeded the configured size ceiling by 3,890,176 bytes; ARM was not reached. The TypeScript row was not reached after the Rust static-size failure.Verbatim gate summary on
50be0d21:Starting the signer fix-forward on branch
signer-hardening, base50be0d219dee0111a44e2c8b82f7735d98c67d4b. I read the review findings and acceptance-test history in #154; the issue has no linked parent. I’ll cherry-pick the three test commits, fix the Worker withinapps/release-signer/, run its suite and the final fast gate, and will not deploy.Finding from acceptance-test review: the R3 test at
c19b3960counts 200 responses and checksindex.length === accepted, which still passes if one concurrent request is refused. The brief requires both requests to return 200. I’ll retain the test commit unchanged and addaccepted === 2as an explicit assertion.Cherry-picked R1 acceptance test as
0305e719. Red reproduction:node --test --test-name-pattern=signer_partial_manifest_publication_can_be_retried apps/release-signer/test/signer.test.mjsfails at signer.test.mjs:111: retry returned 409, expected 200, after the injected SHA256SUMS write failure left MANIFEST.sig present.Cherry-picked R2 acceptance test as
65d0b8aa. Red reproduction:node --test --test-name-pattern=signer_edge_version_is_a_real_utc_timestamp apps/release-signer/test/signer.test.mjsaccepts all three impossible values (2026-02-30, 24:00:00, and out-of-range date/time), returning 200 and writing six objects each; expected 400 with zero writes.Cherry-picked R3 acceptance test as
816a58c0. Red reproduction: the barrier-controlled requests both return 200, but the final index contains one row (accepted=2, indexed=1; test fails at signer.test.mjs:131). The original test’s equality could also pass on fewer accepted requests, so I’m adding the explicit both-200 assertion noted above.Strengthened the R3 test in
20385481: both concurrent responses must be 200 before index cardinality is checked. The focused red test still reportsaccepted=2, indexed=1, so the assertion preserves the intended acceptance behavior and does not weaken the reviewer regression.Completed independent provider-release module:
fe33439da3, local review-audit-ca-update only. Synthetic seed has1 occurrence in lo/dd pages while a key reference remains,0 after final drop; secure usage +32B -> prior count. Source crates/infraternal-server/src/ca_custody_review_tests.rs; evidence continuation-independent-tests.txt and continuation-serial-fast.txt. Serial fast:306 server PASS/0 FAIL/36 ignored, every non-size row PASS; unchanged static ceiling only. Release binary delta0B and identical target hash. No push/deploy.Added the R1 same-version content check in
a5dabf43: after the injected failure, changed artifact bytes are refused without writes, and restoring the original bytes must allow retry. The focused test still reaches the expected red at the final retry assertion (409, expected 200); the changed-content refusal assertions pass before it.Completed independent HTTP/cold-unlock module
815be93483, local review-audit-ca-update only. Two separately registered principals each accept30 starts and then refuse; second principal cannot reset first. Cold trust is decoded public DER plus encrypted copies; new administrator proof restores an independently verified signature. Source auth/ca_custody_review_tests.rs; final serial fast306 PASS/0 FAIL/36 ignored, all non-size rows PASS, unchanged size-only failure; release delta0B. No push/deploy.Updated the fixture at
af2edb93to model R2 etags and conditionalputresults; only the two admission reads are forced stale, then retries see current storage. The focused test still reproduces the product race (accepted=2, indexed=1). Cloudflare’s Workers API reference documentsput()preconditions, etags, andnullon a failed condition: https://developers.cloudflare.com/r2/api/workers/workers-api-reference/.Completed custody log/measurement wrapper module
16a8690238, local review-audit-ca-update only. Four real outcomes pass; checks actual synthetic seed/wrapping values as raw bytes/base64/hex/JSON arrays plus existing field/session/ceremony absence. Final serial fast306 server PASS/0 FAIL/36 ignored, all non-size rows PASS, unchanged static ceiling; release delta0B. Paired measurements still running without own build/test overlap. Evidence auth/ca_unlock_audit_tests.rs and continuation-serial-fast.txt. No push/deploy.Committed R1 fix
07fe74e2. Release outputs use create-only R2 writes; a retry accepts an existing output only when bytes match, and the index remains the completion marker. The changed-content retry check refuses with no extra writes. Focused test:node --test --test-name-pattern=signer_partial_manifest_publication_can_be_retried apps/release-signer/test/signer.test.mjs— 1 passed, 0 failed (0.33 s).node --check apps/release-signer/src/index.jsandgit diff --checkpass.Committed R2 fix
3e4541da.validVersionchecks D226 edge timestamps using Gregorian month lengths and leap years, rejecting invalid year/day/time values in the request handler beforesignReleasecan write. Focused results: impossible timestamp regression 1/1 PASS; valid edge signed-identity regression 1/1 PASS;node --checkandgit diff --checkpass.Completed independent locked-intake release observer
f15ab9cbc0, local review-audit-ca-update only. Real normal/overflow mappings2/2 entirely zero immediately before munmap;32768B each. C observer/build/observation committed with no dependency or production change. Full serial fast every non-size row PASS (306 server tests); unchanged D205 size ceiling, release delta0B/identical target hash. Evidence locked_release_probe.c and locked-release-observation.txt. No push/deploy.Committed R3 fix
c1a5b537. Index publication now uses R2 etag preconditions (etagDoesNotMatch: "*"for create,etagMatchesfor replace) and retries from the latest index after a conflict. The minute limit is checked at admission, allowing both concurrent requests admitted from the same snapshot to finish.node --test apps/release-signer/test/signer.test.mjs: 26 passed, 0 failed;node --checkandgit diff --checkpass.Extended the R2 regression inside the same test case in
3dcdc72a: valid leap day 2024-02-29 is accepted; non-leap century day, second 60, and other impossible values are rejected before writes. Focused timestamp test: 1 passed, 0 failed.Strengthened the R3 regression again in
95af79be: it now asserts both successful responses and the exact two distinct versions (1.2.3,1.2.4) in the index. Focused concurrency test: 1 passed, 0 failed.Strengthened the R3 fixture in
28ca4c9b: both append attempts now snapshot the same old etag, exactly one conditional write must lose, and the loser must retry. Focused test passes 1/1 and verifies both distinct versions are retained after both responses return 200.P2 custody diagnostic finding RV-CA-01: zero unlock-latency regression is not confirmed. Five alternating before(cadac1dd)/protected(b0cf61ef) real authority processes per mode,1000 memberships/private copies and100 unlocks each, GNU debug/CPUWeight10000/no own build/test overlap. All5 paired p50 differences positive:0.309-0.737ms, median+0.609ms; medians of per-run p50s4.577->5.314ms, CPU p50s4.744->5.201ms. No release/whole-HTTP certification inferred. RSS medians after fixture35,628->36,292KiB (+664KiB, within1MiB locked heap); no idle worker added. Failing requested nonregression check committed immediately at
1f20bc0c57; verify_unlock_nonregression.py and paired-*.txt/paired-summary.json. P0/P1 new custody findings remain0. No production fix/push/deploy.Completed paired measurement reproducer
9848807cf2, local review-audit-ca-update only. Ten weighted real-engine processes pass (five alternating pairs,1000 memberships/100 unlocks each), raw public logs committed; initial parser setup failure retained. RV-CA-01 diagnostic nonregression check fails as separately committed/reported, median paired p50+0.609ms; no new P0/P1. Fresh scale10,000 keys:320512 secureB, drop512B, VmLck0->1024KiB;4,096-principal helper p50/p99 88536/156157ns. Final serial fast all non-size rows PASS, unchanged size-only failure; supplemental TS PASS. No push/deploy.Finished custody-only independent verification: MERGE (no new P0/P1 in the custody changes). Head
62c852f805, local review-audit-ca-update; targetb0cf61ef, merge base15d15c25. All requested custody controls tested; UP-01/02/03 remain settled/deferred, not rerun. No push/deploy; issue remains open.Evidence/report: docs/verification/review-audit-ca-update/README.md and evidence/review-audit-ca-update/. 59 focused custody PASS; final server306 PASS/0 FAIL/36 ignored. Synthetic seed occurrence1->0 after last key reference, exact32 secureB;2 full32768B intake mappings zero before munmap;2 authenticated principals30 starts each cannot share/reset; outcomes no synthetic seed/wrapping bytes; public-DER cold source requires new real proof and restores verified signing. VmLck0->1024KiB;10,000 keys320512B secure/drop512B;4,096-principal helper88,536/156,157ns p50/p99. Transient copies are not all locked.
P2 RV-CA-01: GNU debug nonregression check FAIL, committed immediately; all5 pairs positive, median paired finish-p50+0.609ms. Per-run p50 medians4.577->5.314ms; RSS35,628->36,292KiB (+664KiB within1MiB locked heap). Loaded-host diagnostics, no release/whole-HTTP certification. The scoped no-new-P0/P1 rule permits MERGE; no zero-regression claim. Source/action performance follow-up has complete evidence.
Final serial scripts/verify fast: every non-size row PASS, unchanged static ceiling-only FAIL (20,090,384B vs16,404,816B; delta0B/identical target hash). Supplemental TS154/36 PASS. Earlier non-timeout socket availability assertion retained; isolated target/dev PASS, no #118 waiver, complete serial rerun PASS. Owned scratch baseline/targets/probes removed.
Verbatim final gate output:
Orchestrator: CA custody verification attempt 2 MERGE (review-audit-ca-update @
62c852f8, pushed): no new P0/P1; VmLck 0 → 1,024 KiB; P2 unlock p50 +0.61 ms (debug). audit-ca-update joins production batch 3. Signer Worker: review MERGE with 3 P2 (retry after partial publish, timestamp validation, concurrent index) — fix-forward job signer-fixes running before owner redeploy.R3 review found one avoidable R2 read per sign: both requests read the index at admission and then read it again before their conditional append. The concurrency regression now counts reads while forcing both stale writes; the red run reports 5 reads where 3 are sufficient (two admission reads plus one conflict retry). I will reuse the admission etag and retain the conditional retry.
Finished the #154 fix-forward on branch signer-hardening at
2f49c526cf. Local commit messages now include the relevant decision references; worktree is clean.R1: output objects are create-only and retries accept only byte-identical partial outputs; changed bytes at the same version remain refused. R2: D226 edge timestamps are Gregorian UTC values validated before signing or writes. R3: conditional index append uses the admission etag, merges and retries after a conflict, and the regression forces one stale-etag conflict while both responses remain 200 and both versions stay indexed. The forced test path uses 3 index reads (two admissions, one retry).
Worker test output (verbatim summary):
1..26
tests 26
suites 0
pass 26
fail 0
cancelled 0
skipped 0
todo 0
duration_ms 1064.002315
Bundle dry-run: before 10.63 KiB upload / 3.58 KiB gzip; after 13.01 KiB / 4.40 KiB.
Final scripts/verify fast gate did not complete. It passed fmt, VT native fmt, CLI plugin compile-out, clippy, and its 24-test binary, then stalled in the unchanged Rust restart suite. Gate output excerpt (verbatim):
==> test
test restart::tests::closed_reply_and_missing_reload_target_do_not_end_restart_control ...
The same standalone restart test previously timed out at its existing 5-second deadline. I interrupted the final gate after the test remained stalled for over 60 seconds (runner exit status 130). This branch changes only apps/release-signer/{README.md,src/index.js,test/signer.test.mjs}; no Rust files changed. No deploy or push performed.
Correction to the gate note above: the standalone 5-second failure was restart::tests::restart_requests_use_the_servers_binary_and_reject_untrusted_paths; the final gate stalled at restart::tests::closed_reply_and_missing_reload_target_do_not_end_restart_control. A separate 20-second isolated run of the latter also timed out. These are distinct unchanged Rust tests.
Starting independent delta review on
review-signer-hardeningat2f49c526cfc3acb749396fd8ff906efdedc714da; review base50be0d219dee0111a44e2c8b82f7735d98c67d4b, merge base withdev25e4e026f6d7b312829653d9f163faba692f4db9. Read all 179 #154 comments; no linked parent. Prior SIG-R1/R2/R3 tests are carried forward in the target with updated R2-etag fixtures. The specified signer-hardening result.md is absent; used signer-fixes/result.md and the final #154 report. Reviewing only the requested signer delta; no push or deployment.Worker suite on review-signer-hardening at
2f49c526cf: 26 passed, 0 failed, 0 skipped (525.9 ms). Carried SIG-R1, SIG-R2 and SIG-R3 cases all pass; the forced stale-etag case records 3 index reads and 1 conditional conflict. No new failing test found in this step.Performance evidence on #154: the forced stale-etag regression at pre-optimization commit
520f5769fails with 5 index reads; the target2f49c526passes with 3 reads and one conditional conflict. The delta reuses the admission snapshot and removes one avoidable read per signing request in this two-request race. No new finding from the read-count change.Finished independent delta review at target
2f49c526cfc3acb749396fd8ff906efdedc714da, review base50be0d219dee0111a44e2c8b82f7735d98c67d4b. Verdict: MERGE (no P0/P1). New findings: P0 0, P1 0, P2 0, P3 0. Earlier SIG-R1, SIG-R2 and SIG-R3 all PASS in the target Worker suite (26 passed, 0 failed). The stale-etag scenario uses 3 index reads at the target vs 5 at pre-optimization520f5769.scripts/verify fastwas run on the target but did not complete. Formatting, VT formatting, CLI compile-out, Clippy, and the CLI compile-out test (24/24) passed. The Rust test suite reported six unchanged restart-test failures and two unchanged restart tests still running after over 60 seconds. I interrupted the stalled run; no final gate summary or TypeScript row was emitted. Full captured log:/home/kayg/.cache/codex-scratch/signer-hardening-review/verify-fast.txt.Captured output (verbatim):
Local review branch
review-signer-hardeningis clean at the target; no new failing test or code change was needed. No push or deployment.Orchestrator: signer-hardening delta review MERGE (no findings) at
2f49c526. Ready for the owner to redeploy the Worker from branch signer-hardening (deployment steps in apps/release-signer/README.md). The full Rust gate stalls in restart::tests, unrelated to the JS-only Worker; tracked as #174.Orchestrator: production deploy of release-prod-3b (
8cff05cd, add-host fixes + CA custody) at 02:51 UTC failed its health check and rolled back automatically. The static release server refused startup as root: "OpenSSL secure heap could not be locked and excluded from dumps; check the server memlock limit" with LimitMEMLOCK=8 MiB. Production is healthy on the previous binary and web (healthz 200); the restart re-locked the CA. The custody review only measured a GNU debug build. Job ca-static-heap (Sol) is reproducing on the perf VM with the static build and will add a static-build regression. Tonight ships release-prod-3 (add-host only) instead.Starting static key-memory diagnosis on audit-ca-update at
b0cf61efca. #154 history continues prior GNU-only custody measurements; no linked parent. Will diagnose pinned musl/Zig OpenSSL return values and root systemd 8MiB locking on the explicitly named perf VM. No deployment. Foundational build-option/dependency changes will be reported as an owner question before repair.Pinned OpenSSL static build source evidence for #154.
openssl-src=300.6.1+3.6.3
Production batch
8cff05cdand audit-ca-updateb0cf61efhave identical provider versions.631: if target.contains("musl") {
632: // Hack around openssl/openssl#7207 for now
633: configure.arg("-DOPENSSL_NO_SECURE_MEMORY");
634: }
635: }
mem_sec.c:109-127: CRYPTO_secure_malloc_init returns 0 unconditionally when OPENSSL_NO_SECURE_MEMORY is defined.
The actual generated x86 Configure command contains -DOPENSSL_NO_SECURE_MEMORY; no-secure-memory is not needed as a Configure feature name.
The 2018 historical upstream issue #7207 concerned a missing linux/mman.h header.
Candidate compilation without the macro using checksum-pinned Zig 0.15.2 succeeded (enabled-compile-x86.txt).
Owner build-option question for static key memory, #154.
Source cause confirmed: pinned openssl-src 300.6.1+3.6.3/src/lib.rs:631-634 adds -DOPENSSL_NO_SECURE_MEMORY to every musl Configure command. crypto/mem_sec.c:109-127 consequently returns 0 without attempting mlock or madvise. Production
8cff05cdand branchb0cf61efpin identical provider versions. Increasing memlock or changing systemd sandboxing cannot enable code removed by compilation.The brief explicitly requires an owner decision before a different OpenSSL build option. Candidate: remove the musl-only disable define while keeping the existing pinned OpenSSL and Zig, then add a self-check of the production static executable that requires return 1, resident locked pages and dump exclusion under 8MiB. Alternatives: patch openssl-src itself (explicit but fork upkeep), narrowly remove the define in the pinned compiler wrapper (small but compiler-policy coupling), or add a separately built OpenSSL (duplicates build ownership; not recommended). No replacement crypto engine is proposed.
Measurements are attached in the final evidence report. No build-policy repair or dependency change is applied until the owner selects how the existing provider is built. Root fail-closed remains unchanged. The new runtime regression will intentionally fail on today's static binary; it must land with the selected repair, rather than weaken startup enforcement.
Root/systemd measurements on infraternal-perf-test (Linux 6.12.69+deb13-amd64), shared lock held, load 0.00/0.00/0.00 -> 0.08/0.02/0.01:
Owner question: should this repository narrowly remove the upstream musl disable define in its pinned musl compiler wrapper, or maintain an explicit openssl-src patch? Both enable the existing secure heap; choose the build ownership before the production change. Recommendation: narrow wrapper handling of the exact upstream define for the two pinned Linux/musl targets, documented and covered by the final executable self-check. Reject increasing memlock: unlimited still returns 0. Reject replacing the allocator: the existing provider protects real pages once compiled in.
Blocked for owner build-option choice as expressly required by this brief; repair/regression not claimed complete. Evidence: evidence/ca-static-heap/root-systemd.txt, probe-build-x86.txt, heap_probe.c and build_probe.py. Fast gate was invoked before handoff; its final state is recorded in the finish note.
Completed diagnostic-source step
77f94ef4on audit-ca-update, pushed. Real static provider baseline returns0/VmLck0 under root systemd at8MiB, infinity and default; unshipped enabled-mem_sec candidate returns1/VmLck1024KiB, all10000 allocations secure/resident/lo+dd, usage320000->0B. Provider diagnostic size+8352B; not a whole-server repair delta. All six expected root outcomes pass; C Wall/Wextra/Werror, shell/Python/fmt/diff checks pass. Fast gate invoked then interrupted in Clippy at the owner build-option boundary; no complete gate claim. Production repair and executable regression pending owner choice already posted on #154.Investigation handoff (repair blocked for the owner build-option choice explicitly required by the brief): audit-ca-update head
45828e165e, both atomic commits pushed (77f94ef4diagnostic,45828e16 report/evidence). No production build policy/dependency/decision changed; root fail-closed preserved. Issue remains open.Cause: openssl-src 300.6.1+3.6.3 src/lib.rs:631-634 adds OPENSSL_NO_SECURE_MEMORY to all musl targets; mem_sec.c returns0 without locking syscalls. Root/systemd baseline init0/VmLck0 at8MiB, infinity and default. Unshipped candidate init1/VmLck1024KiB;10000 synthetic allocations resident/secure/lo+dd, usage320000->0B. Diagnostic binary+8352B; whole-server repair delta not measured. Owner question/options already posted. Report: docs/verification/ca-static-heap/README.md; raw evidence: evidence/ca-static-heap/.
Checks: strict C compilation and all six expected real root outcomes PASS; bash syntax/Python compilation/fmt/diff PASS. Full scripts/verify fast was invoked in CPUWeight10000 scope, network/formatting completed; interrupted during Clippy (exit130) at the required owner boundary. Full-server and aarch64 builds also interrupted; no complete gate or static-server repair pass claimed. Worktree source unchanged after diagnostic gate; final commit only records docs/evidence. Cleanup removes only this job's uploaded diagnostics and scratch target.
Verbatim captured scripts/verify fast output (incomplete, exit130):
Completed/pushed lifecycle recovery
552fbaa2: real probe before shutdown, pinned current-image recovery plus independent CA frame, current-code fallback if both execs refuse. Targeted restart gate 25 passed/0 failed/0 unresolved ignores; optimized measurement is explicitly ignored. Root container real fexec/anonymous socketpair proof passes successful replacement and forced late kernel refusal: PID preserved, listener serves, CA remains unlocked. Rootless proof remains intentionally cold. Receipt root-ca-recovery-with-loader.txt. Probe closed-stdin regression reproduced and fixed by reserving all null descriptors above stdio.Completed/pushed audited direct child policy
a8eb2443, infraternal-agent launch repair8c7572fb, embedded spawn repair49ec3b70. Real fd3 test red before/green after; signal/parent mask controls pass. Real baseline anonymous-image terminal.open fails resource_unavailable (anonymous-agent-before-short.txt). Session/embedded suites and all-target clippy pass. Shipping rebuild and final-head full fast gate are running; native runtime gate reuses the integrated dev baseline's recorded three-crate opt-level=1 profile, unchanged release/dependency profiles and old size ceilings. All benchmarking stays local; no deployment or publication.Starting D244 implementation on audit-ca-update, base
239da1f488(earlier diagnostic commits are present with rewritten SHAs). Read repository contract, architecture/process, origin/dev D244 (63dabc80) and all #154 history; no linked parent. Owner selected exact compiler-wrapper argument removal, unchanged OpenSSL source/version. Add a real static release key-memory check under 8MiB, then root/systemd proof on the explicitly authorized perf VM with lock, before/after startup and sizes. No deployment.D244 regression before repair: scripts/test-musl-cc.py exits 1 because the exact OPENSSL_NO_SECURE_MEMORY argument still reaches the compiler (evidence/ca-static-heap/d244-wrapper-before.txt). The test also requires preserving all other arguments, including the distinct =1 spelling, for both pinned targets and retaining target-override refusal. Existing static size ceilings remain unchanged; the gate will accumulate size failures while still testing both provider builds. The hidden diagnostic requires initialize()==1 even for non-root and creates/imports no keys. Perf VM access/lock probe succeeds; baseline build is underway on infraternal-dev, observed load 47.73/47.97/43.64, no load generators.
D244 validation scope: baseline release binaries are built from 239da1f4's production Rust and unchanged wrapper; the new diagnostic is parked until both baselines finish. Server baseline is expected to refuse as root, so before/after startup reports refusal time versus successful /healthz readiness, not a causal successful-start latency delta. Root proof uses the named perf VM, flock and systemd LimitMEMLOCK=8388608; it reads only public counters and mapping flags, never key bytes or addresses. Static release test executable independently exercises the existing real passkey/encrypted-copy HTTP authority flow and the 10,000-key provider test. The passkey test is an in-process HTTP router proof, distinguished from the standalone production server startup check. Its fixture now follows TMPDIR instead of another job's fixed remote-agents scratch path. No OpenSSL version/source/dependency or security policy changes.
Completed history reconciliation
331fb34b, pushed audit-ca-update as a fast-forward from published45828e16. Local job base239da1f4 had rebased the audit onto newer UI content; normal merge preserves both histories with zero indexed/tree changes (git diff239da1f4..331fb34b empty). Formatting and unchanged-tree check pass; no crates touched. Rejected force-push. Both original static baselines continue from the unchanged production source; implementation remains separate.Original-source x86_64 musl release baseline completed: 20,090,384 bytes (
239da1f4production source, unchanged wrapper/OpenSSL). ARM baseline still builds. The original x86 binary is being exercised on the named perf VM as root under systemd LimitMEMLOCK=8388608 before repair; expected startup refusal will be retained in d244-root-before.txt. Existing x86 size ceiling16,404,816 is already exceeded by3,685,568 bytes before this job's repair; it is not raised.Real original release x86 server refusal reproduced on infraternal-perf-test as root, holding /var/lock/perf-bench.lock under systemd LimitMEMLOCK=8388608: exit1, state refused, key-memory initialization unavailable before serving. Public output retained in evidence/ca-static-heap/d244-root-before.txt. This is the actual 20,090,384-byte stripped musl server binary, not the prior C diagnostic or a GNU test substitute. Repair not yet applied while the unchanged ARM baseline finishes.
Completed/pushed fixture isolation
cd70ca5896. Existing CA passkey fixture now follows TMPDIR plus a unique name, rather than another job's fixed scratch path; no production code changes. cargo fmt --all --check and remote server all-target Clippy PASS (2m34s); real passkey HTTP unlock test1PASS/0FAIL (1.76s), evidence d244-fixture-{test,clippy}.txt. Original x86 baseline20090384B/root8MiB refusal already recorded; original ARM baseline still compiles before wrapper removal.Completed/pushed hidden diagnostic2615f909 and real CLI integration test (following commit, see branch log). Existing initializer only: success requires return1; root policy unchanged; no key creation/import. fmt/all-target CLI Clippy PASS; real CLI1PASS/0FAIL in0.09s (8MiB ready, ordinary-user zero-limit refused, JSON and hidden help).
Both original release baselines completed without the diagnostic (its unique message is absent): x86_64=20,090,384B; aarch64=17,906,336B. Thus both before/after size comparisons cover the full repair, not just the wrapper. ARM cannot run on this host (x86_64/no qemu-aarch64); no ARM runtime result claimed. New diagnostic against the still-disabled x86 provider is building before wrapper repair.
Static release diagnostic red reproduced after adding the diagnostic: hidden __key-memory-check under prlimit8388608 returns exit1/state refused with the still-disabled provider (d244-diagnostic-before.txt). Its GNU CLI limit/JSON/help test passes1/0. D244 wrapper repair now removes exactly -DOPENSSL_NO_SECURE_MEMORY, citing D244/#154; no provider source/version or other flags changed. Wrapper regression now PASS for both targets, preserving =1 spelling/other flags and target-override refusal. Only release openssl-sys outputs are cleaned for both musl targets to avoid stale compiled policy; rebuilding real releases. Atomic production wrapper commit awaits static green evidence. Diagnostic tests committed/pushed at
ed6aa196.Completed/pushed wrapper repair
c04784b9, exact-policy regressionac60ac3f, and runtime static gate1f82bd17. Fresh x86 real release under8MiB returns1; runtime gate PASS. Both compiler target-policy cases PASS; ARM runtime explicitly untested on this x86 host. No OpenSSL source/version/other options or size caps changed.Root perf-VM proof (flock, systemd CPUWeight10000/LimitMEMLOCK8388608) PASS:21starts health200; VmLck1024KiB; actual lo+dd mappings1024KiB; zero-limit root server still refuses with IPC_LOCK removed. x86 size20090384->20098960B (+8576).31alternating diagnostic pairs p50 before/after2.684/2.563ms (no causal improvement claim). Baseline server refuses p503.815ms; repaired readiness p5057.143/max153.004ms, incomparable as healthy startup delta. VM4CPUs, load0.01/0.06/0.06 before/after. d244-root-startup.txt. ARM repair build and static-release real CA/passkey/10000-key tests continue. Workspace test compilation and CLI feature-off check pass.
Owner deployed the release signer Worker from signer-hardening (
2f49c526) on 2026-10-12. Smoke check from the build host: GET /pubkey 200 with the ssh-ed25519 public key; POST /sign without a token 401; GET / 404. These checks do not distinguish the new code from the old; the first signed edge release proves it (MANIFEST + MANIFEST.sig present, index row kept).Completed/pushed root proof reproducer
d747b97f. Both repaired releases built/stripped/static/PT_INTERP absent: x8620098960B (+8576), ARM17914528B (+8192) versus pristine reference images; ARM runtime explicitly untested on x86 host.Root/static release real-engine proofs under8MiB/shared perf lock PASS all3selected cases: imported signer1024KiB locked/832secureB, +32B import/release assertion; real signed-passkey/encrypted-copy HTTP authority unlock;10000live keys320512secureB/drop512B, VmLck1024KiB. VM4CPUs, scale load0.05/0.04/0.01->0.13/0.05/0.01; no load generators. Receipts d244-root-{startup,ca-tests}.txt. No standalone shipping-CLI/browser unlock claim: authority router and production CLI startup are distinct measured paths. Final head fast gate follows evidence/report commit; old ceilings unchanged.
First final-head fast run (
87cb4bac) failed one infraternal-agent test at daemon_gone_tests.rs:98: UnixListener::bind reports path shorter than SUN_LEN. Cause is my job TMPDIR setup: its UUID/session socket pathname is157bytes vs105bytes with normal /tmp (kernel limit108 including NUL). Same unchanged exact test passes with short normal fixture TMPDIR. No product/test assertion/timeout change and no #118 waiver. Cargo targets/worktrees remain job-owned under HOME; only small existing test fixtures use their normal temporary path. Fresh complete head gate is rerunning. Static heap gate PASS; both unchanged size ceilings FAIL as measured; all other first-run rows PASS. Failure, exact retry and fresh head output will be quoted/retained on #154.Complete short-TMPDIR gate on D244 source reported302serverPASS/1FAIL/35ignored: existing ca_bootstrap::tests::generated_remote_leaf_is_verifiable_after_cold_restart returned false at ca_bootstrap.rs:340. The same unchanged exact test immediately passes1/0 in0.01s. ca_bootstrap.rs is unchanged versus45828e16; musl-only compiler policy cannot change the GNU test provider. Cause not established; no timing/source/assertion change, no load waiver, no claim that isolated pass fixes the full-run failure. Fresh complete fast gate runs on final documentation head018f0298. Static key-memory gate PASS; both old size caps remain FAIL. Failure and isolated output retained in own scratch and will be included in finish evidence.
Finished D244 on audit-ca-update, head018f0298fc05b4e449f5d93567ba72d1def1d789, all10atomic/history/evidence steps pushed. Worktree clean; no deployment/issue closure. Exact musl compiler disable argument removed; provider version/sources/other flags and root refusal unchanged. Hidden key-free release diagnostic and native8MiB runtime gate landed.
Real root/perf-VM proof:21starts health200, VmLck1024KiB and actual lo+dd1024KiB; zero-memlock root still refused. Static-release imported signer/passkey/10000-key proofs3PASS;10000keys320512secureB/drop512B, one1MiB heap/server. Startup readiness p5057.143/max153.004ms; before server refused (not a healthy latency baseline).31alternating diagnostic p502.684->2.563ms, no causal improvement claim. VM4CPUs, startup load0.01/0.06/0.06.
Full shipping size x8620090384->20098960B (+8576), ARM17906336->17914528B (+8192); stripped/static/PT_INTERP absent. ARM runtime explicitly untested on x86 host. Old ceilings remain unchanged and exceeded on both before images.
Final branch-head scripts/verify fast exit1 solely for existing size caps. fmt/VT fmt/Clippy/test/deny PASS;303serverPASS/0FAIL/35ignored; new x86 key heap checkPASS. Supplemental TS157/36PASS, production transfer/font budgetsPASS. Earlier long-TMPDIR setup failure157vs105byte socket fixed by normal small-fixture path, no source/assertion change. One earlier unchanged bootstrap verification assertion failed; isolated1/0 and final complete gatePASS, cause unestablished, no waiver. Prior failure output is preserved below. No benchmarks/load generators ran outside the authorized VM; no secrets inspected/emitted.
Report: docs/verification/ca-static-heap/README.md. Root/source/build receipts: evidence/ca-static-heap/d244-*.txt; reproducers run-d244-root.sh/run_d244_root.py. Author checklist complete; independent review remains orchestrator work. Owned VM uploads removed; own local scratch cleanup follows. Full final gate output is quoted verbatim in numbered continuation comments.
Verbatim final Rust summary:
Final head018f0298 scripts/verify fast, verbatim output part1/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 scripts/verify fast, verbatim output part2/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 scripts/verify fast, verbatim output part3/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 scripts/verify fast, verbatim output part4/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 scripts/verify fast, verbatim output part5/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 scripts/verify fast, verbatim output part6/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 scripts/verify fast, verbatim output part7/7 (exit1: old size caps only). Concatenate code-block contents in order.
Final head018f0298 supplemental TS phase (exit0), verbatim output:
Retained earlier unchanged bootstrap full-run failure (not the final-head outcome; no causal explanation/waiver). Its exact isolated retry1PASS/0FAIL and final full303serverPASS/0FAIL. ca_bootstrap.rs unchanged versus45828e16. Failure excerpt, verbatim:
Cleanup complete at018f0298fc05b4e449f5d93567ba72d1def1d789, pushed audit-ca-update. Only owned local scratch targets/fixtures/uploads and owned named-VM files removed; shared VM directory retained. Worktree clean and remote matches HEAD. Final branch-head fast output preserved verbatim in7numbered comments plus supplemental TS output; only unchanged static-size ceilings fail. Bootstrap transient cause remains unestablished but exact retry and final complete gate pass; no source/assertion/timeout change or waiver. Report/evidence committed; no deployment or issue closure. Initial forced-delete cleanup was rejected automatically; non-forced owned-directory deletion succeeded.
Final cleanup verification: non-forced deletion finished, both owned local scratch roots absent, worktree clean at018f0298fc05b4e449f5d93567ba72d1def1d789. Earlier cleanup comment was posted while the large deletion was still completing; absence is verified now. All final-head gate/TS output is preserved on #154; committed source/report/root evidence retained. No deployment or issue closure.
Starting independent delta review on review-audit-ca-update at
018f0298fc; prior review baseb0cf61ef, target/dev merge base93b00f5ab9. Read #154 history (217 comments), no linked parent; continuing custody-only scope and settled/deferred UP-01/02/03. Preserved previous review as review-audit-ca-update-prior-018f. Reviewing only delta excluding dev merges, replaying prior custody regressions. No push/deploy/perf VM; no build-host timing evidence.Completed independent D244 compiler-policy verification at local review commit
a29a41b6. Both x86_64/aarch64 policy cases PASS, retaining =1 and whitespace arguments and refusing foreign target overrides; shell syntax/diff PASS. Evidence evidence/review-audit-ca-update/delta-018f/. Six code/test/gate files differ fromb0cf61efafter excluding dev merges; no dependency/contract delta. Target-head full fast gate still compiling. No push/deployment/perf VM or build-host performance claims.Scope clarification: current delta brief explicitly requests replaying earlier failing tests and stating whether each passes. Will replay UP-01/02/03 as well as custody controls; older custody-only deferral will be reported separately, not treated as a fix. Current verdict rule requires HOLD if confirmed P0/P1 remain. Historical RV-CA-01 data/check will be retained but cannot establish current latency: perf VM is expressly prohibited and shared-host timing is not evidence. No unchanged-code re-audit beyond requested prior regressions.
Replayed historical RV-CA-01 evidence/test in local commit
eda5f3cd(cherry-pick of1f20bc0c). The preserved five-pair nonregression assertion still FAILS against its original data; this is not a new timing measurement or a verification of current release latency. No current performance claim is made because this review forbids the perf VM. Full target-head Cargo gate remains untouched by these evidence-only files.Restored original independent UP regressions as bf798631/5be04ff8/181b71a1 and fixture lint correction
dab9edfc; custody release/principal/cold-signing/log assertions as 0ce6cffe/48fcb4e8/a13b47b4. All are test-only cherry-picks; production delta remains exactly018f0298. Native cold build still in progress. Since source tests were restored during dependency compilation, the running fast gate is now target production plus reviewer tests, not a pristine-target-only test-count claim. Explicit deferred-regression execution follows compilation. No push/deployment.Restored original full-mapping release observer as027afa9e (cherry-pick
f15ab9cb). Observer builds with -Wall -Wextra -Werror; replayed Rust sources pass formatting. Runtime observation awaits native test binary. Native VT build has completed; full gate continues through Clippy. Prior artifacts remain labelled historical; fresh replay output will be stored in delta-018f. No push/deployment.Gate setup correction: first owned gate interrupted (exit143) after Clippy PASS while native test executables were compiling; inherited umask002 did not match prior documented022 executable fixtures. Retained verify-first-umask002.txt. Owned executable artifacts made non-group-writable; replay runs now use022 and short fixture root. First corrected unweighted retry exited1 at network RTT self-test before Rust rows (verify-umask022-unweighted.txt), no waiver. Unchanged full gate restarted under CPUWeight10000, serial tests, port4384. No product/assertion/deadline changes or performance claims.
UP-01 P1 earlier finding revalidated at target018f0298: 2/2 real ELF fixtures admitted without valid release authentication;0 executed. Existing independent test restart_binary::review_tests::reviewer_real_elf_requires_release_authentication still FAILS (exit101). Source crates/infraternal/src/restart_binary.rs:94; test crates/infraternal/src/restart_review_tests.rs; fresh evidence evidence/review-audit-ca-update/delta-018f/reviewer_real_elf_requires_release_authentication.txt, local commit
99b5290f. No new delta defect or implementation fix. Current brief explicitly requires prior-finding replay and HOLD if P0/P1 remain; earlier custody-only deferral is reported separately. No push/deployment.UP-02 P2 earlier finding revalidated at target018f0298: durable in-place write changes the retained ELF header. Existing independent test restart_binary::review_tests::reviewer_admitted_elf_header_cannot_be_superseded still FAILS (exit101). Source crates/infraternal/src/restart_binary.rs:82; test crates/infraternal/src/restart_review_tests.rs; fresh evidence evidence/review-audit-ca-update/delta-018f/reviewer_admitted_elf_header_cannot_be_superseded.txt, local commit
ed1593fb. No new delta defect or implementation fix. Current brief explicitly requires prior-finding replay and HOLD if P0/P1 remain; earlier custody-only deferral is reported separately. No push/deployment.UP-03 P1 earlier finding revalidated at target018f0298: existing listener is available before replacement and unavailable after actual exec refusal. Existing independent test restart_binary::review_tests::reviewer_exec_refusal_preserves_existing_listener still FAILS (exit101). Source crates/infraternal/src/restart.rs:378; test crates/infraternal/src/restart_review_tests.rs; fresh evidence evidence/review-audit-ca-update/delta-018f/reviewer_exec_refusal_preserves_existing_listener.txt, local commit
331e0a26. No new delta defect or implementation fix. Current brief explicitly requires prior-finding replay and HOLD if P0/P1 remain; earlier custody-only deferral is reported separately. No push/deployment.Fresh independent custody controls PASS, local evidence commit above (see branch log):3independent tests (two principals30starts each; cold public trust needs new real administrator proof and verifies restored signature; provider seed clears only on final reference), strengthened four-outcome audit test1PASS, full-intake observer1PASS with2entire mappings cleared before munmap. Evidence delta-018f/{custody-replay,audit-log-replay,intake-observer-replay}.txt. These are functional GNU tests, not new performance/static/root claims. Full fast gate retained a fixture-parent setup failure before server tests; empty ignored target/ now exists, unchanged retry and complete rerun required. Static builds continue. UP-01/03 earlierP1 and UP-02P2 remain confirmed; no new delta finding.
Custody evidence commit is
a99c2f41. After creating the ignored fixture parent, unchanged exact initial_attach test PASS1/0; no waiver/assertion change. Final pristine-target source gate started: temporarily restored four reviewer import/log-test files from018f0298 so native test set and source are exactly the target, with unused reviewer modules retained on disk and committed in HEAD. Will restore review imports from HEAD after gate. Final log verify-final.txt; umask022, serial tests, CPUWeight10000, short owned fixture root, port4384. Existing first gate still compiles static targets; no production changes, push/deployment or VM use.Pristine-target fast gate native test row FAIL: existing local_session_counts_and_memory_refusals_follow_real_lifecycle, scripts/verify-session-bounds.py:63, dead shim exceeded one-second retirement bound. Full log verify-final.txt; own static compilation overlaps. This case is outside listed#118 exceptions: no waiver, no causal claim, no deadline/assertion change. Require unchanged retry and fresh complete gate after own builds stop. Static shipping/provider compilation continues; earlier UP-01/03P1 remain independently confirmed.
Fresh actual x86_64 musl shipping CLI D244 PASS:8MiB diagnostic ready/exit0;zero memlock refused/exit1;both JSON records fixed/no stderr;public help omits hidden command. Image20098960B, stripped/static; matches author after size and+8576B vs earlier20090384B reference. Evidence static-x86-check.json, new local commit (branch log; SHA follows in finish). No root/whole-server/ARM/runtime/performance remeasurement claim. ARM compilation continues. Current fast attempts retain fixture-parent failure then unchanged retirement-deadline failure; neither waived. Final native gate requires rerun once own static builds stop.
x86 receipt commit
c7119ea8; ARM receipt now committed separately. Both fresh shipping sizes match author:20098960B/17914528B; static, stripped,PT_INTERP absent. Independent hashes retained; x86 hash differs from author receipt, so root receipt is not claimed as proof of this independently rebuilt image. Strict x868MiB/zero-limit checks PASS; ARM native runtime not checked. Feature-disabled cargo check PASS. Both earlier full fast attempts finish with testFAIL101 (documented fixture parent, then unchanged retirement deadline) and old sizeFAIL1; no waiver. Own compilation stopped; unchanged retirement retry and fresh pristine-target complete gate are now running without own build overlap. No push/deployment/perf VM.Finished independent delta review: HOLD (P0/P1 present); verdict: do not merge under the current brief's rule. Target
018f0298; prior reviewedb0cf61ef; local review branch review-audit-ca-update, headaa7a10710d. New delta findings: P0 0 / P1 0 / P2 0 / P3 0. Earlier findings: P0 0 / P1 2 / P2 2 / P3 0; one P2 is historical and not remeasured.UP-01 and UP-03 still FAIL, each exit 101: two real ELF fixtures admitted without valid release authentication (0 executed), and actual exec refusal removes the prior listener. UP-02 still FAILS after a durable write changes the admitted ELF header. Tests and fresh failures committed separately (bf798631/5be04ff8/181b71a1; evidence 99b5290f/ed1593fb/331e0a26). The earlier custody-only deferral is recorded separately; the current brief explicitly requests prior-finding replay and prohibits merge with P0/P1.
RV-CA-01's original-data nonregression check still FAILS; current performance was not remeasured because this brief prohibits the perf VM and host timings are not performance evidence. Independent custody controls PASS (
a99c2f41): principal isolation, new administrator proof after cold reconstruction with verified signing, final-reference seed clearing, four outcomes excluding private encodings, and full intake clearing before release.D244 shipping checks PASS on the fresh x86 musl CLI under 8 MiB; zero memlock is refused; help omits the diagnostic (
c7119ea8). Fresh stripped/static images: x86 20,098,960 B / ARM 17,914,528 B; no PT_INTERP; match author sizes (+8,576 / +8,192 against prior references, not rebuilt here). Independent artifact hashes are retained; the x86 hash differs from the author receipt. No root whole-server or native ARM runtime claim. ARM receipt1c23f342. Both-target wrapper policy, feature-off check and reviewer formatting PASS.Final complete scripts/verify fast on pristine target source: 303 server PASS / 0 FAIL / 35 ignored; every non-size Rust row PASS, unchanged static ceilings only FAIL. Earlier setup and retirement failures plus unchanged passing retries are retained; no deadline/assertion/implementation/ceiling change or #118 waiver. Reviewer imports restored afterward. Gate archive
5f45b228; report/checklist docs/verification/review-audit-ca-update/delta-018f.md; evidence evidence/review-audit-ca-update/delta-018f/.Verbatim final gate summary:
Full final target-source gate output follows verbatim in numbered parts. No implementation fix, push, deployment, perf VM, credential investigation or issue closure. Owned scratch cleanup follows evidence archival.
Final pristine target
018f0298scripts/verify fast, verbatim output part 1/6 (exit 1: unchanged size ceilings only). Framework timings are not performance evidence.Final pristine target
018f0298scripts/verify fast, verbatim output part 2/6 (exit 1: unchanged size ceilings only). Framework timings are not performance evidence.Final pristine target
018f0298scripts/verify fast, verbatim output part 3/6 (exit 1: unchanged size ceilings only). Framework timings are not performance evidence.Final pristine target
018f0298scripts/verify fast, verbatim output part 4/6 (exit 1: unchanged size ceilings only). Framework timings are not performance evidence.Final pristine target
018f0298scripts/verify fast, verbatim output part 5/6 (exit 1: unchanged size ceilings only). Framework timings are not performance evidence.Final pristine target
018f0298scripts/verify fast, verbatim output part 6/6 (exit 1: unchanged size ceilings only). Framework timings are not performance evidence.Cleanup verified: owned audit-ca-update-review scratch root is absent; review worktree is clean at
aa7a10710d. Durable tests/report/evidence are committed locally, findings.md appended, and full final pristine-target gate output is preserved in six numbered #154 comments. Verdict remains HOLD: two earlier P1 failures remain, no new delta finding. No push, deployment, perf VM or issue closure.Orchestrator decision on the D244 delta review (review-audit-ca-update-d244 @
aa7a1071, HOLD): the HOLD rests only on UP-01/UP-03 (and UP-02/RV-CA-01 P2), which are pre-existing in production and deliberately deferred to release-pipeline round 4 (implemented, under review). The D244 delta itself has no new findings and the static key-memory checks pass. CA custody therefore ships with the next binary deploy; UP-01..03 ship with release-pipeline round 4 after its review.Orchestrator: DEPLOYED to production at 12:08 UTC (17:38 IST): release-prod-5 @
5bf3f42e= live4d1b1950+ add-host rounds 2-3 (#169, reviewed MERGE) + start fresh (server + screen; review SF-1 fixed in5bf3f42e) + CA key memory on static builds (#154, D244). sha256 prefixes x86 2f2b4a5a / arm 7cc6e2cd match on host; healthz 200, public 200, 0 errors in journal; VmLck 1024 kB (CA key memory locked under the 8 MiB limit). Restart re-locked the CA: owner unlock needed. Owner test: add infraternal-dev (leftovers untouched) -> expect "This host has leftovers from an earlier setup" -> Start fresh.kayg referenced this issue2026-10-28 17:53:56 +00:00