Add-host bootstrap fails on a NetBird-SSH-only host; failure step not logged #169
Open
opened 2026-10-11 16:39:23 +00:00 by kayg
·
260 comments
No Branch/Tag specified
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/infraternal#169
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Owner, production 2026-10-11 ~18:37 CEST: add-host of 10.69.69.125 (infraternal-dev) failed: "The remote bootstrap command failed. Remote command returned a failure status; raw output was withheld."
Findings so far (orchestrator):
sudo -n trueprobes as kayg at 18:37:33 and 18:37:36 succeeded (journal), then a later step returned non-zero.To do: reproduce against a NetBird-SSH-only target (stdin upload via
cat >,set -euC, sudo -n sh -c, systemctl), fix, and add the step logging. Base: release-prod-2.Root cause found (orchestrator, 2026-10-11): the bootstrap took the rootless path and its precondition refused the existing
~/.local/share/infraternal(mode 775, created 2026-10-06 by a local dev tracer-server, no.bootstrap-owner) with exit 71 in the Upload step. Behaviour is correct (never adopt foreign data); the defects are: (1) the user sees a generic "remote command failed"; exit 71/72 must map to plain messages naming the folder and the fix; (2) the server logs nothing — log step + exit status (no raw output); (3) why rootless whensudo -n truesucceeded twice — check against D229. On infraternal-dev the folder was renamed to ~/.local/share/infraternal.dev-2026-10-06 so the owner can retry.Second failure found (orchestrator, 18:41 retry): install and join succeeded; the terminal canary failed after 10 s ("The host connected, but its terminal did not answer"). Cause:
canary()(crates/infraternal-server/src/bootstrap_remote.rs:280) typesprintf marker; exitinto the user's interactive login shell. On infraternal-dev the zsh startup files launch zellij (zellij --new-session-with-layout welcome), so the typed command went into zellij and the marker never appeared. Any user whose rc starts tmux/zellij/screen, a menu or a slow prompt will hit this. Also: the rollback left the canary session running (orphaninfraternal-session-702fa8d0….scopewith zsh → zellij, stopped by hand).Fixes for #169: (1) the readiness probe must not depend on the user's interactive shell: open the canary session with a fixed non-interactive command (e.g.
/bin/sh -cprinting the marker) or accept the shim Hello + first output as ready; (2) rollback must end any session it opened; (3) step + exit status logging (above).Owner request linked: sessions should detect auto-started tmux/zellij and use infraternal's session manager instead — tracked as its own issue (see the newest issue referencing #169). The readiness-check fix here stays independent of that.
Tracked as #170.
Starting #169 on addhost-fixes at base
93b00f5ab9(release-prod-2). Read issue body and all four comments; no parent is linked. Production DECISIONS.md ends at D112; inspected dev decisions for D139 and D229. D139 explicitly supersedes automatic sudo promotion: add-host root needs consent, so rootless without consent is intended. D229 specifies server init as root, not add-host implicit escalation.Contract proposal before implementation: add optional readiness=true to trusted terminal.open. It selects only a fixed non-interactive /bin/sh readiness command (no arbitrary executable or argv), under the same enrolled uid and terminal.open plugin admission. Default terminals keep login shells. Private canary sessions are explicitly ended on all exits, including timeout/cancellation, before install rollback.
Mode finding: no automatic root-selection bug. D139 (dev DECISIONS.md lines 907–913) supersedes the automatic parts of D18/D120: passwordless sudo is never permission. contracts/ssh-bootstrap.md admission requires mode=root AND root_consent=true, otherwise defaults rootless. crates/plugins/ssh/src/model.rs Target::validate refuses root without consent; apps/tracer/src/canvas/addhost/wizard.ts:542 sends root only after its consent page. D229 specifies root for server init, a separate operation. Preserve add-host consent. Existing regression: tests/model.rs root_consent_is_required_before_ssh_and_uid_zero_cannot_be_rootless.
Failing test before readiness fix: canary-before.log records both actor-routing regressions failing because terminal.open input readiness was Null, expected true (2 failed, 0 passed). Added fixed readiness=true leg-B option under unchanged uid/plugin admission, propagated through shim launch. Fixed /bin/sh reads one newline then emits a static marker; no arbitrary command override. Real-engine fixture isolates startup profiles within worktree/job scratch, without changing accounts or user profiles.
Diagnostics regressions before fixes: diagnostics-before.log has every_failed_step_names_the_step_and_a_next_action failing (Connecting to the host failed, no fix). ui-before.log has the production-build browser assertion failing specifically at .ah-details pre: server supplied failed_step=start, but Details omits it. Readiness real-engine evidence: four isolated startup profiles (tmux, available zellij, non-returning menu, bash rc) all returned confirmed PTY marker without running rc; 1236.5–1268.3 ms with default 300 ms RTT. Machine infraternal-dev, 8 cores, load 37.48/38.35/31.72, CPUWeight=10000 local scope; log readiness-real.log.
Readiness fix committed: 8dcbe2e8e5398955e55690b92f3b2c28f243fd5c. Two failing actor regressions now pass. Real leg-B daemon + PTY proof bypasses tmux, available zellij, a non-returning menu and bash rc (readiness-real.log): 1264.2, 1268.3, 1236.5, 1237.6 ms; default 300 ms RTT; local CPUWeight=10000 scope; infraternal-dev load 37.48/38.35/31.72. cargo fmt --all --check and clippy all targets for infraternal/server/agent/shim pass. Shim unit gate: test result: ok. 64 passed; 0 failed; 13 ignored; 0 measured; 0 filtered out; finished in 76.11s. No foundational dependency change. Next: failure/cancellation cleanup, then diagnostics.
Cleanup regression before fix: cleanup-before.log records failed_attachment_finishes_the_private_command failing because no cleanup input reaches the session after its first attachment is refused (1 failed). Fix uses the existing terminal protocol: the attempt records its private session before attachment, and cleanup sends only a newline through a second writer, waiting for the fixed command to exit. Backend cleanup runs before SSH rollback stops the service, including cancellation/logout. A private 10-second absolute shim ceiling covers lost replies/transports; ordinary user PTYs have no new timer. Private probes leave no durable exit record that would prevent removal of a fresh bootstrap directory.
Cleanup fix committed: 5ba7492ad649e049b1b851338712a7b314aa4eaf. Existing/new bootstrap tests: 28 passed, 0 failed, 1 ignored; shim: 64 passed, 0 failed, 13 ignored; scheduler/event metadata: 7 passed, 0 failed, 1 ignored. fmt and touched-crate all-target clippy pass. Real PTY scopes are inactive with 0 probe artifacts: explicit completion 343.3 ms, abandoned connection 9913.6 ms (10-second absolute ceiling). Cancellation stores the session before attachment and cleans it before service rollback. Evidence: cleanup-before.log, cleanup-real-before.log, cleanup-real-after.log, cleanup-bootstrap-tests.log.
Diagnostics contract proposal before implementation: Failure adds optional public exit_status from the authenticated command process, without exposing stdout/stderr. Preserve trusted failed_step and classify bootstrap's own 71 (foreign/non-private existing data) and 72 (lock or unfinished rollback) by selected root/rootless path; other command failures name the boundary and next check. One backend failure-record callback per completed attempt emits only host UUID (or unassigned before issuance), request-local target id, step, exit status and static mode. It runs before cleanup warnings can obscure the original phase. Both configured remote and unavailable-runtime server adapters use the same logger. UI uses its existing Details disclosure for the trusted step label.
Fix 3 complete: 24cc3a2e231f17aa08a290dbb4541aac4d43d8fc. Numeric exit status, 50 step/mode/status mappings, actionable 71/72 paths, exactly one content-free server failure line, failed step in existing dialog details. Failing mapping/filesystem/browser regressions preserved. SSH targeted: 49 passed; bootstrap: 29 passed, 1 ignored. fmt/clippy pass. Tracer check, 150 src tests, production transfer budgets pass. Playwright port 4891: 27 passed (7.1m; overall timeout 120s for host load, individual assertions unchanged). Failure size 120 bytes × 1024 retained events × 16 concurrent batches = 1966080 bytes, unchanged. Screenshot: evidence/addhost-169/failure-details.png for independent visual review. Root selection unchanged.
Item 4 recorded and protected: 45169c73929e2404209c8bcd521e169cbfaa2bb8. No selection bug: D139 requires explicit root consent despite passwordless sudo; D229 applies to server init. New mode regression: 5 model tests passed. fmt and touched-crate all-target clippy pass. Author checklist: evidence/addhost-169/reviewer-checklist.md. Full scripts/verify fast is running on this head, including both static targets.
Full gate environment finding: shell umask=002 produced test executable target/debug/deps/infraternal-695a777d772bb491 mode 0775. Strict restart admission rejects group-writable images; five restart tests failed and two waited indefinitely for a refused default receiver. Stopped only that gate test process. Normalized 124 executable artifacts in this job scratch target to remove group/other write; no repository or shared-parent permissions changed. rootless_even_marked regression now passes (1/1, 0.49s). Will rerun the full gate with umask 022, retaining strict security checks. This is not a #118 baseline exemption.
Gate scheduling finding: an unscoped retry stopped at the initial RTT self-check (p50=330.39 ms, p99=512.97 ms, configured 300 ms, load around 31). Restarted the exact full gate with umask 022 under systemd-run --user --scope -p CPUWeight=10000; no deadline or assertion changes. RTT self-check now passes, and previously refused restart tests pass. Static cold builds remain in progress.
Base correction: initial worktree branch addhost-fixes started at
93b00f5a(recorded in start comment), but that commit contains dev-only additions beyond production release-prod-2=1c08c4b73961c4081a88a17f56ba5073dea9be68. I missed the ancestry mismatch at start. Full Rust tests passed (654/0/66), but dev-derived x86 static binary is 20093648 bytes vs production gate max16404816. Rebase the four unpublished logical fixes onto exact production1c08c4b7; repeat targeted per-commit gates and full final gate. No push/deploy and no budget change. Existing SHAs/evidence above describe the initial base, superseded by corrected production commits to follow.Production fix 1:
b45ec594. Parent is production1c08c4b7, not dev. Kept production admission and terminal lifetime; no dev-only features imported. Bootstrap25 passed/1 ignored; shim62 passed/12 ignored; fmt and all-target touched clippy pass. Real daemon/shim PTYs bypass tmux/zellij/menu/bash startup in1202.2–1308.0ms at300ms RTT, on infraternal-dev8cores load28.20/19.49/21.75. New production-* logs supersede earlier gates. Cleanup is being ported to production exit acknowledgment: only private probes retire after exit delivery; normal user sessions stay unchanged.Production fix 2:
95f27ef5. Same-attempt session tracking and cleanup before rollback; cancellation/lost reply covered. Production-specific real failing case preserved in production-cleanup-real-before.log: private shim awaited ordinary exit acknowledgment. Fix retires only private probes after exit delivery drains and bounds abandonment at10s, preserving normal PTYs. Real scope cleanup332.2ms explicit/9892.2ms abandoned, artifacts0. Bootstrap27 passed/1 ignored; SSH concurrency6 passed/1 ignored + event_metadata1 passed. fmt/clippy pass. Runtime benchmarks in own CPUWeight10000 scope on infraternal-dev8cores; host load during gates reached75.46/58.69/41.96. Corrected-base production-* evidence supersedes initial SHAs. Diagnostics port is validating in own CPUWeight10000 scope.Production fix3:
a33c9f73. Same classified exit71/72/other messages, failed step details and one content-free failure log per attempt. SSH47 passed; bootstrap28 passed/1 ignored; fmt/clippy pass. Tracer check,137 source tests, production build/transfer budgets pass. Playwright27 passed in5.6m on4891. Refreshed production screenshot evidence/addhost-169/failure-details.png for independent visual review. Failure120bytes×1024events×16batches=1966080bytes, unchanged. Production-* logs supersede initial dev-based gates.Production item4:
b800019c. D139/D229 distinction unchanged; model5 passed, fmt/clippy pass. All four unpublished fixes now start at production1c08c4b7, with no unrelated dev code or dependency changes. Final full scripts/verify fast is running onb800019c. Independent milestone x86 build is19931264 bytes versus gate16404816; comparing untouched production base on both targets before any disposition. No gate exception, release-profile change or baseline increase applied.Full gate on
b800019c: fmt/native fmt/clippy/cargo-deny PASS; one auth socket test failed at plugin_lifecycle.rs217 (mesh.list ok=false) during high-load concurrent suite, and static x86 size failed. The exact socket test rerun passes in0.97s with discovery/lazy enable/disable/restart/same-socket Pong proof. No test assertion or deadline changed; no baseline exemption applied. Will rerun all tests with RUST_TEST_THREADS=4 to reduce independent-test contention. Untouched production1c08c4b7 target builds are underway to establish inherited static-size excess. Release profile and dependencies are unchanged.Size-gate repair proposal/rationale before change (docs/PROCESS.md§7): untouched production1c08c4b7 builds to19904800 bytes x86 and17738488 ARM with pinned toolchain and unchanged opt3/LTO/one-codegen/strip/unwind profile. Both already fail the #57 snapshots16142672/13555672; D173 accepted the existing QUIC/provider growth in #78. Refresh the comparison snapshot to these measured, already-shipped production values, retaining the262144-byte allowance exactly (new maxima20166944/18000632). This changes no dependency, feature or release setting. Fresh x86 fix19931264: delta26464, below existing allowance; fresh ARM delta follows when linking finishes. No failure is exempted: restore an accurate production comparison and rerun the full gate. Evidence: production-base-size.log and production-fixed-size.log, own CPUWeight10000 scopes, infraternal-dev8cores, load22:07=35.46/28.37/26.81, no load generators. Also force fresh head builds after baseline-target reuse; x86 __shim help confirms readiness flag.
Gate repair committed:
64d4e213. Untouched production1c08c4b7is 19,904,800 bytes x86 musl and 17,738,488 bytes ARM musl; fresh fixed artifacts are 19,931,264 and 17,764,464 (+26,464/+25,976). Both static ELF checks and CLI stub checks pass. PROCESS section 7 measurement/rationale was posted before updating the stale #57 snapshots; D173 already accepted existing provider growth. Allowance remains 262,144 bytes. No profile, dependency, provider or feature changes. Evidence: evidence/addhost-169/static-size.md and production-{base-size,fixed-size,static-after}.log. bash -n, cargo fmt --all --check and git diff --check pass. Final full gate now running on64d4e213with RUST_TEST_THREADS=4 after the isolated mesh retry passed in 0.97s; no failure exemption claimed.Final gate output, verbatim, part 1/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 2/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 3/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 4/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 5/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 6/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 7/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Final gate output, verbatim, part 8/8. Head
64d4e213b2; scripts/verify fast exit 0. CPUWeight=10000, RUST_TEST_THREADS=4, TRACER_TEST_PORT=4891.Finished and pushed only addhost-fixes at
64d4e213b2, based on production1c08c4b7. Five atomic commits:b45ec594readiness;95f27ef5cleanup;a33c9f73diagnostics/UI;b800019cexplicit-root-consent regression;64d4e213measured gate snapshot repair. No deployment, issue remains open. Worktree clean.Readiness uses an authorized private non-login sh session; real tmux/zellij/menu/bash startup profiles are bypassed. Marker arrives in 1.18–1.65s at the configured 300ms RTT. Explicit cleanup: 332.2ms; abandoned private session: 9892.2ms; inactive scopes and zero retained artifacts in both cases. Tests cover failure after open and cancellation. These are real daemon/shim readiness-boundary tests, not full SSH enrollment on another host.
Failures now classify every install step and statuses 71/72, retain numeric status, show the failed step in dialog Details and log one safe metadata line per attempt. Failure remains 120 bytes (1,966,080 inline bytes at 1024 retained events × 16 batches). Passwordless sudo does not select root: D139 requires explicit consent; D229 concerns server init. Static growth +26,464 bytes x86/+25,976 ARM against unchanged production; 256KiB allowance preserved.
Final scripts/verify fast exit 0. Complete 209546-byte output quoted verbatim in the preceding 8 parts, SHA256 5267ab60256c3c0d3f815f3db591cd4dda488608e1b951f73db3514b2eb37a11. No #118 exemption used; the earlier mesh failure passed both its isolated retry and the final full gate.
Production tracer check/build/transfer budget exit 0; source tests 137 pass, zero failures. Add-host Playwright: 27 passed (5.6m), TRACER_TEST_PORT=4891. Final full gate also passes 145 tracer test cases and the protocol differential/client suite (36 pass, 34462 assertions).
Evidence: evidence/addhost-169/production-*.log, static-size.md and reviewer-checklist.md. Production screenshot evidence/addhost-169/failure-details.png attached for independent visual review; no author visual-quality claim or physical iPhone proof. Reproduce readiness with scripts/verify-addhost-readiness.py --binary ; reproduce wizard with TRACER_TEST_PORT=4891 bun x playwright test test/addhost-wizard.spec.ts --timeout=120000 from apps/tracer.
Starting independent verification review on branch review-addhost-fixes at head
64d4e213b2; merge base with dev is1c08c4b739. Read the issue history and job report; review is testing the five-commit diff against production contracts.Starting break-it lane on addhost-breakit2, base
64d4e213b2(origin/addhost-fixes, production-derived). Read #169 body and all 33 comments; no parent linked. Production DECISIONS ends at D112; read origin/dev D139/D229/D235/D237 as requested. Product code stays unchanged. Disposable local Docker targets only, namespace-scoped netem; no real hosts or host networking edits. Findings/tests/timings/captures go in docs/verification/addhost-breakit2. Docker requires sudo and is available. Prior real readiness tests are boundary proof, not full SSH enrollment.T1 P1: real filesystem regressions breakit2_rollback_preserves_foreign_launcher_new and breakit2_rollback_preserves_foreign_unit_new both fail: unrelated .new files are deleted after install failure/rollback. install.rs254-266 creates replacement suffixes; install.rs394 removes them unconditionally. The unit write was refused by noclobber, but cleanup deleted its foreign predecessor. Evidence docs/verification/addhost-breakit2/transaction-red.log; no product fix.
T2 P2: breakit2_stage_refuses_parent_that_cannot_be_rolled_back fails: mode0775 .local/share is accepted by stage, then fresh rollback rejects the same parent and leaves payload/lock. install.rs146 versus validate_parent449. Existing strict cleanup test is correct; preflight and mutation admission disagree. Real shell/filesystem; transaction-red.log.
T3 P2: breakit2_interrupted_lock_creation_allows_cleanup_and_retry fails after real mkdir creates .bootstrap-lock and fixture exits42 before transaction/kind journaling. Rollback cannot authenticate its own empty lock, leaving retry blocked. install.rs146-155/369-373. Preserve foreign-lock protection; journal acquisition safely. transaction-red.log; total four intended failures in0.32s.
Independent finding F-001 (P2), committed on review-addhost-fixes at
b1950b14d1. The installer runs the uploadedagent joinprocess underset -e; its status 71 is passed to the same mapper used for the installer's directory refusal. The new real-shell fixture fails: 0 passed, 1 failed, 30 filtered;enrollment_process_exit_71_is_not_reported_as_directory_refusalexpected the Join boundary recovery message but got the directory move-aside advice. Log:$HOME/.cache/codex-scratch/addhost-fixes-review/status-71-red.log. This can direct an operator to move an otherwise valid existing data directory after a Join failure.Step 1 committed
f6728ef7: four opt-in red installation regressions, no product fix. Real filesystem ordinary suite30 passed/0 failed/4 ignored in2.38s; explicit red suite0 passed/4 failed in0.32s at intended assertions. cargo fmt --all --check, SSH all-target clippy -D warnings and git diff --check pass. Evidence transaction-{red,direct,clippy}.log in docs/verification/addhost-breakit2. Next: real SQLite unpublished/race boundaries, disposable SSH target timings and production-renderer UX captures.Independent UI verification on exact head
64d4e213: the production tracer Playwright case for the failed-install disclosure passed (1 passed, 1.5m) on unique port 4387 in a temporary detached checkout. The refreshed screenshot after opening Details shows “Failed step: Start the host service” visibly in the disclosure. Browser plugin was unavailable; used the repository Playwright harness.A1 P2: wizard.ts346-350 drops explicit port when a typed target matches a listed host with the same username. Opt-in wizard regression target@lab:2222 expects port2222 but gets undefined (original discovered chip). No product fix. Evidence docs/verification/addhost-breakit2/address-red.log. This is selection proof, not a claim of actual wrong-host traffic.
A2 P3: model.ts191 IPv6 regex accepts [:::] as a chip; server Destination::parse correctly requires a real Ipv6Addr. Opt-in malformed-address regression fails at expected undefined versus accepted target@[:::]. Additional malformed cases are in the test but first failure stops the loop. address-red.log; no product fix.
S1 P1: real SQLite regression proves unpublished authority leak after invitation commit but before address persistence. A trigger refuses remote_host_bootstrap INSERT; attempt fails and in-memory attempts is empty, yet live remote_hosts rows=2 instead of prior1. bootstrap_remote.rs42-49/180-205 guard only retires memory. authority-red.log, intended assertion; no exploit/capability theft claimed.
S2 P1: eight concurrent same-destination batches all issue distinct host identities (expected one winner). Real backend+SQLite, read-only synthetic SSH fixture. bootstrap_remote.rs140-167 SELECT then issue has no atomic destination reservation. authority-red.log:8 vs1, two server regressions fail in0.16s. No claim eight daemons installed; host-local locks do not solve server admission.
H1 P2: fixed non-login terminal readiness does not bypass the SSH login-shell startup preceding CHECK_SCRIPT. Real sshd probe: clean bash/zsh/fish/sh canonical; printing bashrc/zshenv/fish config corrupts record; zshenv exit0 returns zero bytes; exec tmux/screen fail; sleep10 exceeds fixture4s ceiling. Opt-in targets.py --assert-startup-safe fails naming seven profiles; startup-red.log. Production command ceiling30s, not claimed measured here. No profiles on real hosts changed. Changed disposable public host pin is refused status255 (67.658ms), no trust acceptance.
Step2 committed336ad82d: two opt-in address regressions, no product fix. Tracer typecheck passes; ordinary add-host source46 passed/2 skipped/0 failed (3.89s); explicit red tests0 passed/2 failed. Production-build capture harness port4927 passes2 tests in4.1m, desktop1440x900 and phone393x852, eight captures in docs/verification/addhost-breakit2/captures. Browser plugin unavailable; backend scenarios are test doubles; no author visual-quality judgment or real-iPhone proof.
Steps committed:480fba3d server consistency regressions (5 ordinary backend tests pass0.20s; red counts2vs1 durable authority and8vs1 duplicate admissions fail0.16s);
d039a8ddproduction capture harness (2 browser tests pass4.1m,8 desktop/phone captures);377a1b35 disposable Debian/systemd/SSH fixture (imagebuild and real user manager/probe evidence). fmt, Python compile, tracercheck and all-target workspaceclippy pass (cold clippy9m25s under shared load). Native payload, milestone full fast and live timing fixture builds continue; no product fixes or real-host mutations.UX list only for design pass, production-build Chromium captures1440x900/393x852,8PNGs,2 capture tests pass4.1m,zero pageerrors; test-double backend. P2U1 coarse progress lacks uploadbytes/per-stepelapsed; P2U2 stop action cancels wholebatch and no cleanupreceipt after socketclose; P2U3 representative installerror gives generic service advice; P3U4 discoverywait no elapsed/requestreload/deadline (real.ts58 fetch lacks AbortSignal, mockwait5s does not prove infinitewait). Paths/caveats in docs/verification/addhost-breakit2/README.md. No author visual-quality judgment or physicaliPhone/Safari claim.
Milestone fast environment correction: INFRATERNAL_TEST_STATE_ROOT job/state generated115-byte AF_UNIX .upstream path; daemon oracle refused before execution. No product failure/baseline exemption. Use owned short alias $HOME/.cache/codex-scratch/b1692 -> addhost-breakit2/state (same job storage) for final full gate; no assertion/deadline changes. Actual machine reports12 online/affinity CPUs, not brief8; benchmark logs report12.
Step committed:
8d3302d6, ignored real-SSH/live-CA/real transport timing and cancel helper; product code unchanged. fmt + server remote all-target clippy pass, nine ordinary backend tests pass (0.43 s). Static x86_64-musl payload 19,931,264 bytes: local Done=5,444 ms; isolated target-netns 75 ms ingress + 75 ms egress netem Done=13,168 ms. One sample each, debug controller, release payload, load 62.64/47.13/46.13 and 72.70/53.69/48.51; not production p99. Both consume their invitation and retain no pending attempts. Evidence docs/verification/addhost-breakit2/{static-local,static-netem,live-gate,live-clippy}.log.T4 P2 finding: with a 1 MiB tmpfs home, real SSH upload fails and rollback also fails; owned base, .bootstrap-lock/payload, kind, transaction and .bootstrap-owner remain. Server active authority returns to prior=1 and no live unconsumed token. measure.py --case tiny-disk --expect-failure fails specifically at failed add-host left owned installation artifacts, after its native refusal/authority assertions pass. Evidence tiny-disk.log, install.rs staging/rollback receipt creation. No claim that stale data belongs to a foreign host; container and all data are our fixture. Follow-up must preserve ownership safety while allowing cleanup on a full filesystem.
R1 P1 reliability finding: 10 fresh Debian systemd targets, identical static x86_64-musl payload (19,931,264 bytes), actual SSH/HTTPS enrollment/remote transport. 7 reached Done; samples 02,03,08 failed terminal readiness after successful upload/join/start and authenticated facts. Failure cumulative times 4,490/4,754/4,144 ms; successes 3,938–5,487 ms. Logs docs/verification/addhost-breakit2/reliability-{01..10}.log. Every failed run retired authority (active=1 prior=1, no unconsumed live token/pending attempt) and removed owned base/launcher/unit. Existing ignored real end-to-end success-contract test fails on these runs at expected Done vs Failed. No product fix. Source bootstrap_remote.rs:333–389 canary boundary; exact fast failure/root cause not yet assigned. Earlier GNU diagnostics established open=true/output=false/cleanup=true on a failure, but do not prove the static-path cause. These are ten samples under shared load (roughly load 42–52), not a production failure-rate estimate. /tmp noexec also produced one readiness failure then one success; do not attribute this general failure to the mount.
Steps committed
630a299b(isolated timing/cleanup driver),1b2e92a1(shell/privilege real SSH probes),d6e8bdea(Alpine fixture); Python compile/diff checks pass. Alpine/no-systemd real SSH check succeeds but canonical init=other is refused with only generic unsupported-system copy. An opt-in desired-message regression now fails because the refusal omits systemd; evidence alpine-install.log/init-red.log (P3 H2).Completed steps:
60c5458bpreserves the red unsupported-init guidance test; ordinary probe test passes, opt-in fails at missing systemd wording.30ca4eedadds actual batch API retry after a controlled first-install refusal: upload/join are real, rollback is real, retry Done in 4,325 ms; both full uploads resent (981.790/816.319 ms, total 39,862,528 bytes).c0a62c5eadds fixed process/scope receipts: retry and four diagnostic completions have zero running private terminal scopes and one intended daemon.da9a0af5probes a listener exclusively on ::1 and a strict client in the same owned target netns: status 0/canonical record, 253.234 ms. This is not a routed full IPv6 enrollment test. Python compile, fmt, touched-server/SSH clippy and ordinary tests pass.At-scale diagnostic (existing ignored real HTTP/SQLite test): 1000 invitation/redemption pairs, p50 7,466 us, p99 36,150 us, 819 CPU ticks, RSS delta 4,748 KiB; load 27.98/36.84/44.03, measured 12 CPUs. Excludes TLS/QUIC/daemon/renewal and uses a debug controller/in-memory SQLite. enrollment-scale.log. Static binaries pass architecture/size checks: x86 19,931,264 bytes, arm64 17,764,464 bytes, no PT_INTERP.
Gate setup correction: short scratch alias fixed the 115-byte AF_UNIX fixture path. Existing attach-credit harness also assumes relative target/ exists despite CARGO_TARGET_DIR; an ignored target symlink into owned scratch now supplies it. Two failed gate outputs retained; neither is classified as a product finding or load exemption. Full gate rerunning. Coverage limits remain explicit: no owned NetBird-only management plane/ARM runtime, routed IPv6-only full enrollment, wall-clock skew, live removal/re-add or every power-loss/cancel boundary. SELinux enforcing unavailable on this running host. No perf VM, production host or host networking changed.
Harness correction completed on detached job head
64d4e213: created the assumed repository-local target/ parent in the scratch checkout. The previously failing exact harness test now passes:cargo test -p infraternal --test agent_review initial_attach_requires_explicit_credit_before_replay -- --exact— 1 passed, 0 failed, 2.62s. Fullscripts/verify fastrerun is starting.Corrected full exact-head gate completed with fmt, VT native fmt, clippy, cargo-deny and static builds passing. The workspace test row reported 271 passed, 1 failed, 30 ignored in 467.12s:
remote_http::tests::enrollment_http_is_bound_single_use_and_preserves_token_while_lockedreturnedDeniedat remote_http_tests.rs:181 while re-admitting withnowcaptured at line 106. This file and its route are unchanged from the merge base. A workspace-filtered isolated rerun passed 1/1 in 0.09s. I’m rerunning the full gate once to check this timing-sensitive case under gate concurrency.Second corrected full exact-head run: RTT self-test passed (p50 298.99 ms, p99 313.70 ms). Cargo tests had one failure:
transport::remote_deadline_tests::queued_suffix_cannot_complete_an_expired_cancelled_framefailed its 20 ms timeout assertion at unchangedcrates/infraternal-agent/src/transport.rs:320(46 passed, 1 failed, 10 ignored). Workspace-filtered isolated rerun passed 1/1 in 0.17s; the test file is unchanged from the merge base. I’m running one standard full gate with Cargo’s default test-thread count.Gate comparison outcome: original failure was a received watch event with type=error instead of resync (gate-liveness-error.log), not a receive timeout; no #118 waiver used. The exact unchanged liveness test passes on detached dev
8e7467c07cin 9.43 s, and on this branch targeted repeat in 10.53 s. Baseline build began under load 58.18/49.07/44.97; measured after targeted runs 21.42/32.50/39.58. Evidence baseline-liveness.log and branch-liveness-repeat.log. These isolated successes do not establish the cause of the earlier event error. Full fast rerun uses RUST_TEST_THREADS=1, unchanged assertions/deadlines, and the liveness test now passes within that whole run. Remaining TS gate independently passes: 36 protocol tests, 34,462 assertions; compiler/build/transfer budget pass. No product fix.Independent review complete.
Verdict: MERGE (no P0/P1). Counts: P0=0, P1=0, P2=1, P3=0. F-001 is P2 and is committed as a failing regression test on local branch
review-addhost-fixes, headb1950b14d17531fe1c519037a70c24e8323d74cc. The job head reviewed was64d4e213b2e717c2b8e24f85cb4698b625a71fc3. The review branch was not pushed.The standard full gate on the job head was not green. Its one failing test is unchanged from the merge base and passed when rerun alone: 1 passed, 0 failed, 0 ignored, 0 measured, 0 filtered out; 1.21s. Earlier corrected full runs also exposed two unchanged timing-sensitive tests; both passed in isolated workspace-filtered reruns. Their evidence is in the prior comments. Full output log:
/home/kayg/.cache/codex-scratch/addhost-fixes-review/verify-fast-head-standard.log.Exact
scripts/verify fastoutput from the standard run:METRIC name=network_rtt_selftest configured_rtt_ms=300 jitter_ms=30 loss=unsupported transport=tcp-stream n=3 p50_ms=312.29 p99_ms=325.54 result=PASS
+---------------------------+--------------+
| Rust gate | Result |
+---------------------------+--------------+
| fmt | PASS |
| VT native fmt | PASS |
| clippy | PASS |
| test | FAIL (101) |
| cargo-deny | PASS |
| static builds | PASS |
+---------------------------+--------------+
failures:
auth::tests::server_setup_terminal_echo_and_resume_preserve_offsets_once
test result: FAILED. 271 passed; 1 failed; 30 ignored; 0 measured; 0 filtered out; finished in 381.83s
error: test failed, to rerun pass
-p infraternal-server --libCompleted isolated NetBird SSH coverage: commits
3b2a40d1(pinned upstream server fixture) and91ad8d84(driver). Actual network-only sign-in completes with OpenSSH masked. Static payload 19,931,264 bytes; local Check→Done 4,696 ms, configured 150 ms namespace path 11,654 ms (Done event 11,658 ms). Upload 971.019/2908.530 ms. Both ledgers: zero pending attempts/live unconsumed invitations, one intended daemon. Evidence: docs/verification/addhost-breakit2/netbird-{local,netem}.log. Synthetic container/gateway peer policy, JWT disabled: management/WireGuard/JWT/inventory remain uncovered. Full ordinary gate atda9a0af5passed all six Rust rows and TS checks/builds/protocol 36 tests; final head rerun follows evidence commit. No product fixes.Evidence committed as
43f4abaa: docs/verification/addhost-breakit2/README.md, eleven P1–P3 findings, opt-in red tests/results, per-step OpenSSH and isolated NetBird timings, eight production desktop/phone captures, four UX gaps for design review. git diff --check and all report links pass. Logs normalize trailing whitespace only; final branch-head gate is running with RUST_TEST_THREADS=1 (no changed assertions/deadlines). Owned fixture containers/images destroyed. Product code remains unchanged outside cfg(test) hooks.Starting production add-host integration on release-prod-3, base
b4ae9e22. Read #169, #158 and #154 history; no parent linked for #169. Mergeb800019conly (four logical commits); exclude64d4e213. Preserve plugin admission/size validation, ordinary PTY retention/ended records, and CA restart handling. No deployment. Gates run locally, CPUWeight=10000, umask 022; browser port 4933.Conflict resolution: daemon keeps production MAX_CONTROL_JSON, enrolled/effective uid, positive dimensions and 200000-cell bounds, plugins.admit and typed OpenRefusal; only launch gains readiness. The old 128-session cap was existing on the old branch, not introduced by #169; production D214/resource admission remains authoritative. session.rs keeps ordinary exit_deadline, SESSION_OWNER and ended::record; private readiness exits after delivery or 10s without ended data. Poll selects earliest deadline and preserves the nonblocking post-exit drain. main.rs keeps exit_retention_ms/run_with_retention and all D217 early handoff/signal paths, adding readiness only. Size-baseline commit64d4e213 excluded. Imported review/break-it findings remain recorded on #169, outside these four approved commits; no additional fixes silently imported.
Final branch-head scripts/verify fast output, verbatim part 1/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 2/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 3/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 4/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 5/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 6/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 7/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Final branch-head scripts/verify fast output, verbatim part 8/8. Head:
43f4abaacb. Exit 0. RUST_TEST_THREADS=1; TRACER_TEST_PORT=4933; owned HOME scratch target/state; CPUWeight=10000. No changed assertions/deadlines or load exemptions.Finished evidence lane at
43f4abaacb; pushed only addhost-breakit2. Sixteen atomic commits, no product fixes. README: docs/verification/addhost-breakit2/README.md. Eleven findings, including 3/10 supported-target readiness failures, foreign suffix deletion, leaked and duplicate durable authority; opt-in red tests fail at intended assertions. OpenSSH Check→Done local/150 ms: 5.444/13.168 s; isolated real NetBird SSH package: 4.696/11.654 s. Payload 19,931,264 bytes, retry resends all of it. Eight production-build desktop/phone captures and four prioritized UX gaps retained. Final scripts/verify fast exit 0: all six Rust rows PASS, TS checks/build/budget PASS, tracer 147 pass/0 fail/2 skip, protocol 36 pass/0 fail/34,462 assertions; full verbatim output in preceding eight comments. git diff --check passes; worktree clean after temporary target symlink removal. Owned fixture containers/images and detached baseline removed; remaining owned build scratch is being deleted. Remaining gaps explicitly recorded: real mesh management/JWT/inventory, ARM64 runtime, routed IPv6 enrollment, actual removal/re-add, live clock skew, exhaustive cancel/power-loss windows. No production reliability-rate or physical iPhone claim. Issue left open for fix/design lanes.Correction to finish test count: the tracer ran 147 tests: 145 PASS, 2 SKIP, 0 FAIL (not 147 passing). Protocol: 36 PASS, 0 FAIL. The verbatim gate parts above contain the correct counts. Head remains
43f4abaacb; ordinary gate exit 0.Merged tracer: bun run check PASS; bun test src 150 PASS/0 FAIL/10921 assertions in7.43s. Production transfer: critical gzip293452/307200,Brotli243980/256000; deferred gzip48878/51200,Brotli43141/45056; largest on-demand gzip15091/20480,Brotli13392/18432. Browser gate running on4933 (upstream4934). Native cold build under CPUWeight10000; observed host12logical CPUs/load77.31/67.40/52.86. No budget/assertion changes.
Orchestrator: breakit2 (
43f4abaa) found 11 findings: P1 R1 readiness 3/10 fail, T1 rollback deletes foreign .new files, S1 leaked durable authority on unpublished invitation, S2 8 concurrent admissions for one destination; P2 T2/T3/T4/H1/A1; P3 A2/H2; UX U1-U4 for the design pass. These are in code tonight's round-1 fixes do not touch, so round 1 still ships tonight (it fixes the owner-hit zellij/rollback failures). Round 2 (addhost-fixes2, Sol) is chained after the production-line port and will first check each finding onb4ae9e22.Touched-crate all-target Clippy PASS (20m06s). Initial fmt ran before formatting completed; corrected cargo fmt --all --check PASS. I changed the running gate script to shorten its Unix-socket fixture root; Bash then resumed at a changed file offset and misread the next line (ssh: Could not resolve hostname cargo). No host resolved. Stopped only that owned scope; restarting all remaining targeted tests from a new immutable script, with explicit remote features and short job fixture root. Native-run.log retains the failed invocation; no completed test result claimed from it. Browser split case timed out at120s; inspecting and rerunning unchanged assertions, no waiver.
First production browser gate:63PASS/8FAIL/8existing opt-in capture skips in21.9m; all27 add-host wizard cases PASS. Evidence browser.log. Four session fixture assumptions conflict with existing production code: window-card.ts creates a hidden overflow button, narrow split headers use its menu, windows/frame.ts gives touch headers Close only, title taps open details. Separate test adaptation will preserve geometry/layout/manager assertions while driving those existing routes. Other failures (saved-status reconnect count4 vs3, theme long tasks54/71ms, connection readiness, first-paint timeout) will be rerun without relaxed assertions or waivers.
Native fixture failure:daemon_gone_tests.rs uses temp_dir plus a long UUID name, producing SUN_LEN refusal under job TMPDIR;54 other agent tests PASS. Rerun uses normal ephemeral fixture path; all Rust targets remain under job HOME cache.
Fresh real readiness PASS: four isolated startup profiles1238.2–2529.9ms, explicit cleanup358.4ms, abandoned9901.2ms, zero artifacts/scopes; infraternal-dev12CPUs/load75.76/63.27/54.91, CPUWeight10000,300ms RTT. readiness-real.log.
Browser phone finding is product behavior, not only stale fixtures: toolbar/row.ts:53 requires a header Sessions button, while windows/frame.ts:31 deliberately omits it on touch. The same guard prevents mobile mock manager scenes and palette/terminal manager entry. Original phone and swipe browser regressions fail. Follow-up adaptation: extend existing session-hooks.toggleManager to accept an absent anchor and choose the existing window pill once; callers must not refuse the action before that fallback. Reuse existing manager/popover; no new component/style/dependency/timer. Separate from the merge and from stale geometry/split fixture adaptation.
Completed and pushed merge6ec4aabd (only
b800019capproved ancestry;64d4e213 excluded). Targeted fmt/clippy PASS; agent55PASS/11ignored, shim64PASS/13ignored, SSH suites PASS; bootstrap29PASS/1ignored; corrected setup11PASS/0FAIL. Real readiness and cleanup PASS with zero artifacts.Mobile follow-up split by existing module and pushed after each gate:session-hooks
c4043ec7, terminal entry0e470186, canvas rowa716060d, paletteffffc4bb, mock scenes08592ffc. Each runs fmt,bun check and150source tests/10921assertions. Focused original phone/swipe plus corrected geometry/split4PASS/57.7s. Full session replay running.Full workspace Clippy selected another native OpenSSL build hash. Interrupted only that redundant owned scope and reused this job’s already-built exact locked3.6.3 GNU static archives using target-prefixed openssl-sys environment. SHA256 in native-archive-checksums.txt. No dependency/version/profile or musl recipe changes; RUSTC_WRAPPER remains enabled. Initial incomplete fast-merge.log retained; full gate restarted.
Fixture adaptation16b557eb committed/pushed after fmt,weighted bun check and150source tests/10921assertions PASS; original mesh timing failure41.2/25ms retained, no waiver. Full sessions-canvas20PASS/4.1m, including unchanged phone manager/swipe assertions; screenshot evidence/release-prod-3/mobile-manager.png.
Fresh production x86 static20,093,776B, stripped static PIE,no PT_INTERP; unchanged gate16,404,816B remains excluded from the merge repair. Static payload real readiness four startup profiles1216.6–1308.7ms; explicit310.8ms/abandoned9893.3ms cleanup, zero artifacts/scopes;12CPUs/load68.31/51.06/50.78/CPUWeight10000/default300ms RTT. failure_storage.log:120B per Failure,122880B per1024 events,1966080 inline bytes at16batches (excludes dynamic allocations).
Full Rust gate in progress:CLI36PASS including every restart test,no stalls so far; review3/socket control cases PASS,no#118 exception. Browser timing/reconnect cases will be replayed after native work completes.
Browser harness finding: playwright.config.ts always sets INFRATERNAL_CLUSTERS_PREVIEW=1. Preview output measured critical Brotli283037/256000 and deferred52240/45056, while actual product build is243877/256000 and43132/45056. Performance specs label their source production-build; terminal and product dead-session cases do not need preview modules. Add validated TRACER_TEST_PREVIEW=0|1, default1 preserving canvas fixtures; run terminal performance with0 using the actual product bundle. No assertion/deadline/RTT/budget change. Source:config,main.ts335–346,build.ts18. Quiet preview theme passes p991.2ms/zero>50ms tasks; main sign-in p994066.9ms/5000 passes, but later terminal checks and first paint still fail. Raw browser-remaining.log retained. Saved-status extra attach persists and remains under investigation, not waived.
First-paint gate finding: index.html preloads the default font but no initial JS modules. The build splits main.ts and its static imports, so cold access traverses serial module requests at300ms RTT before start() can run. Product and preview first-paint gates remain red; no waiver/budget change. Proposed integration adaptation: derive only main's transitive static imports from the existing Bun metafile and emit modulepreload tags into built HTML. Keep the terminal, deferred canvas, workers and on-demand chunks off that initial preload list; no auth request/order, renderer/engine, runtime timer or JS payload changes. Verify graph isolation/cycles, production transfer budgets and unchanged30-sample first-paint assertions at300ms RTT. D100/D188. This is build scheduling using the existing graph, not a new foundational engine.
Module preload experiment: unchanged30-sample300ms RTT gate remains red (setup upper-bound p991514.8ms,sign-in2056.2ms). The initial HTML also requests eight separate stylesheets; the fixture serves HTTP/1 with no-store, so these compete with main/static imports and font in the browser's connection pool. Next measured adaptation: build one stylesheet from those existing sources in their exact order, preserving their relative URLs and the separate old files for existing clients. No CSS declarations/design/component copy, auth ordering, framework/engine or budget change. Combined initial font/CSS/main/static graph then fits one request wave. Build/source tests plus transfer budget and unchanged first-paint/reconnect gates will decide the result; no green first-paint claim yet.
Code head7bb2801e is pushed.27 add-host browser cases and20 session cases pass. Full workspace gate fast-final.log: fmt/VT fmt/clippy/test/cargo-deny PASS; static FAIL20093776B versus16404816B; no restart stalls. Remaining committed-fixture browser replay5PASS/3FAIL: saved-status/Close attach4vs3; reconnect p991002.2ms/900ms; first-paint setup1566.8ms/sign-in2188.1ms/1000ms (30samples,300ms RTT). Exact-product and preloading/CSS/compression experiments still fail timing gates; all experiment source changes reverted, no budget/assertion waiver. Fresh code-head full gate and remaining browser replay running. Evidence under evidence/release-prod-3. Release readiness remains blocked by these unwaived browser gates; no deployment.
Integration handoff. Head
1f77a449edpushed to release-prod-3; source head7bb2801e, final commit evidence only. Merge6ec4aabd preserves production admission/lifecycle/startup.64d4e213 excluded. No deployment. No restart stalls or waivers. Release remains blocked by five unwaived browser failures; no baseline-causality claim. Evidence: evidence/release-prod-3/README.md and raw logs.Final scripts/verify fast output, verbatim:
PT_INTERP: absent; size: 20093776 bytes (baseline 16142672 + allowance 262144 = max 16404816)
binary exceeds the 16404816-byte size budget: 20093776 bytes
+---------------------------+--------------+
| Rust gate | Result |
+---------------------------+--------------+
| fmt | PASS |
| VT native fmt | PASS |
| clippy | PASS |
| test | PASS |
| cargo-deny | PASS |
| static builds | FAIL (1) |
+---------------------------+--------------+
Targeted clippy, verbatim:
Finished
devprofile [unoptimized + debuginfo] target(s) in 20m 06snative-corrected.log results, verbatim:
test result: ok. 55 passed; 0 failed; 11 ignored; 0 measured; 0 filtered out; finished in 5.22s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 11 filtered out; finished in 0.00s
test result: ok. 11 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.55s
test result: ok. 6 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.24s
test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.72s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.37s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 76 filtered out; finished in 0.01s
test result: ok. 64 passed; 0 failed; 13 ignored; 0 measured; 0 filtered out; finished in 82.64s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 15.66s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.11s
remote-server-bootstrap.log results, verbatim:
test result: ok. 29 passed; 0 failed; 1 ignored; 0 measured; 296 filtered out; finished in 17.09s
setup-corrected.log results, verbatim:
test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 315 filtered out; finished in 10.31s
Tracer check exit0; source output, verbatim:
150 pass
0 fail
10921 expect() calls
Ran 150 tests across 20 files. [304.00ms]
Build exit0; transfer output, verbatim:
JavaScript transfer, critical: identity=945381 gzip=293452/307200 brotli=243939/256000 bytes
JavaScript transfer, deferred: identity=138070 gzip=48914/51200 brotli=43191/45056 bytes
JavaScript transfer, onDemand: identity=184107 gzip=72252/20480 brotli=64259/18432 bytes
JavaScript transfer, onDemand largest chunk chunk-p0c4kg94.js: gzip=15090/20480 brotli=13399/18432 bytes (class total gzip=72252 brotli=64259, not limited)
All27 add-host wizard cases PASS in browser.log. Sessions, verbatim:
20 passed (4.1m)
Remaining browser output, verbatim:
5 failed
test/dead-session.spec.ts:82:30 › a retired session carries exit 7 through the same final ended path
test/dead-session.spec.ts:82:30 › a retired session carries exit 0 through the same final ended path
test/terminal-theme.spec.ts:205:1 › every curated theme recolors the initial terminal and its existing ANSI text
test/terminal.spec.ts:5:1 › first run, passkey login, terminal resume, and the latency probe ───
test/terminal.spec.ts:630:1 › access screens render within the local first-paint budget ────────
3 passed (5.7m)
Failures: retired exit0/7 attach3vs2; theme3long tasks over50ms; sign-in5072.4/5000ms and prediction22.9/17ms; first-paint setup3036.0/1000ms, sign-in2194.2/1000ms. Real static readiness1216.6–1308.7ms; cleanup310.8ms explicit/9893.3ms abandoned, zero artifacts/scopes. Experiments reverted, no assertions/budgets changed. Scratch targets/state removed after evidence retention. Issue remains open.
Starting the #169 A/B browser check. BASE is the detached production checkout at
b4ae9e2289(no branch); comparison is release-prod-3 at1f77a449ed. Both worktrees are clean before build. I’ll run the requested specs three times each in BASE/HEAD alternating order, using ports 4941/4942 and recording load before every run.BASE replay 1 complete at
b4ae9e22; pre-run load 43.68/41.39/34.43; 3.9m; 3 passed, 5 failed. Dead-session: retired exit 0 expected attach count 2, got 3; saved-status/final Close expected 3, got 4; the other two cases passed. Theme: switch p50 0.800 ms, p99/max 9.000 ms, 4 long tasks >50 ms ([70, 54, 81, 62] ms), failed. Terminal resume failed waiting 5 s for the test double connection; no latency metric was emitted. Access screen: setup render p99 2004.50/1000 ms, setup FCP 1698.70 ms; sign-in render p99 2340.90/1000 ms, sign-in FCP 2196.00 ms. The other two terminal cases passed. The test rewrote two tracked capture PNGs; restored them to the clean initial BASE state.HEAD replay 1 complete at
1f77a449; pre-run load 35.31/39.12/35.54; 4.4m; 4 passed, 4 failed. Dead-session: retired exit 0 expected attach count 2, got 3; the other three cases passed. Theme failed with switch p50 0.800 ms, p99/max 5.600 ms, and one long task (58 ms). Terminal resume failed waiting 5 s for the test double connection; the status-guidance case passed. Access screen setup render p99 1959.50/1000 ms, setup FCP 1796.00 ms. The sign-in sample navigation timed out at 120 s before its metrics printed. Continuing alternating replay.BASE replay 2 complete at
b4ae9e22; pre-run load 39.08/36.59/35.32; 4.1m; 4 passed, 4 failed. Dead-session: saved-status/final Close expected attach count 3, got 4; the other three cases passed. Theme failed with switch p50 1.200 ms, p99/max 10.500 ms, and one long task (58 ms). Terminal resume failed waiting 5 s for the test double connection; status guidance passed. Access screen setup render p99 2042.90/1000 ms, setup FCP 1804.20 ms. Sign-in sample navigation timed out at 120 s before its metrics printed.HEAD replay 2 complete at
1f77a449; pre-run load 41.57/40.48/37.48; 4.1m; 4 passed, 4 failed. Dead-session: retired exit 7 expected attach count 2, got 3; exit 0 and saved-status/final Close passed. Theme failed with switch p50 1.000 ms, p99/max 5.800 ms, and five long tasks >50 ms ([52, 84, 74, 80, 75] ms). Terminal resume failed waiting 5 s for the test double connection; status guidance passed. Access screen setup render p99 2148.40/1000 ms, setup FCP 1875.70 ms. Sign-in sample navigation timed out at 120 s before its metrics printed.Starting add-host repair round 2 on addhost-fixes2 at base
1f77a449ed(origin/release-prod-3). Read #169 body/history and breakit2 README at43f4abaa; no parent is linked. Will import unchanged acceptance tests, reproduce each finding on productionb4ae9e22, then repair R1, T1, S1/S2, P2/P3, and payload retry reuse in order. UX U1–U4 excluded. Existing release static-size/browser blockers remain unwaived. Evidence under evidence/addhost-fixes2; local CPUWeight=10000 measurements and unique browser port 4957. No deployment.BASE replay 3 complete at
b4ae9e22; pre-run load 55.68/46.48/40.56; 6.0m; 4 passed, 4 failed. Dead-session: retired exit 0 expected attach count 2, got 3; other three cases passed. Theme: p50 0.800 ms, p99/max 6.400 ms, one long task (179 ms). Terminal metrics: returning sign-in usable p99 5099.80/5000 ms (p50 4104.80); resumed reconnect p99 1038.80/900 ms (p50 763.60); predicted glyph p99 18.00/17 ms (p50 4.10); measured RTT 390.50 ms vs configured 300. Access screen: setup render p99 1708.90/1000 ms, setup FCP 1605.40 ms; sign-in render p99 2342.60/1000 ms, sign-in FCP 2191.20 ms. Logs retained in scratch pending final summary.Baseline verification A1/A2: unchanged breakit2 assertions fail on production
b4ae9e22and release-prod-3 base1f77a449. target@lab:2222 resolves with port undefined (expected 2222); target@[:::] is accepted (expected editable/undefined). Both runs 0 pass/2 fail. Logs evidence/addhost-fixes2/{baseline,head}-address-red.log. No address product fix yet; R1 remains first. Baseline live harness needs only diagnostic-adapter/metadata compatibility with its older API; Done/authority assertions remain unchanged.HEAD replay 3 complete at
1f77a449; pre-run load 25.69/30.15/35.08; 4.5m; 4 passed, 4 failed. Dead-session: saved-status/final Close expected attach count 3, got 4; exit 0 and exit 7 passed. Theme: p50 0.700 ms, p99/max 6.700 ms, one long task (54 ms). Terminal resume failed waiting 5 s for the test double connection; status guidance passed. Access screen setup render p99 2318.70/1000 ms and setup FCP 1924.20 ms. Sign-in sample navigation timed out at 120 s before metrics printed.Baseline verification S1/S2: unchanged ignored regressions fail on production
b4ae9e22(0 pass/2 fail, 0.13 s) and release-prod-3 base1f77a449(0 pass/2 fail, 0.22 s). A refused remote_host_bootstrap INSERT leaves 2 live host rows versus expected 1; 8 concurrent same-destination batches receive 8 identities versus expected 1. Logs evidence/addhost-fixes2/{baseline,head}-authority-red.log. Ordinary head bootstrap suite 29 pass/0 fail/4 ignored in 24.38 s. No authority product fix yet; R1 ordering reproduction builds first.Diagnostic for the single HEAD-only-looking exit 7 mismatch: at intermediate source commit
7bb2801e23,dead-session.spec.ts --grep 'a retired session carries exit 7'also failed with attach count 3 vs expected 2. Pre-run load was 77.04/56.78/44.61; one test, 12.4 s. This is not yet evidence of a regression because the machine load was high; checking the same case before the mobile session-manager commits.Completed fixture import
d0d444a4, pushed addhost-fixes2 only. Eight unchanged disposable-target fixture files from breakit2; Debian Docker image build, Python compilation, cargo fmt --all --check and git diff --check pass. Acceptance source tests remain uncommitted pending native/TypeScript gates. No product fix or deployment. Native server controller build completed in 8m09s; ordinary bootstrap 29 pass/0 fail/4 ignored, 24.38 s.The focused exit 7 case also failed at
6ec4aabd(beforec4043ec7and all mobile session-manager commits): expected attach count 2, got 3; pre-run load 55.11/54.52/45.04; 14.4 s. Thedead-session.spec.tstest is unchanged across the comparison range. The only tracer source change in b4ae9e22..6ec4aabd is add-host wizard code, outside this case. This rules out the mobile session-manager commits as the cause; doing one same-case BASE check to classify the mismatch.Baseline verification T1/T2/T3:
b4ae9e22unchanged acceptance install suite 0 pass/4 fail in 0.28 s. Both foreign .new files deleted after rollback; mode0775 parent accepted by stage then refused by cleanup; interrupted mkdir leaves an unauthenticated owned lock and blocks retry. H2: canonical init=other fails missing-systemd assertion (0 pass/1 fail). Logs baseline-install-red.log and baseline-init-red.log. R1 mechanism: new real-Unix-socket ordering regression fails onb4ae9e22and1f77a449with UnexpectedEof while private output/Exited remain queued (baseline 0.04 s, head 0.00 s). Private shim closes two sockets after writing final frames; daemon owner-EOF branch wins before data drain, and private probes have no ended record. Frozen pre-fix static build retained for live comparison. Experimental relay change now drains the existing data socket; not yet validated or committed.R1 additive contract proposal before diagnostic implementation: add trusted failed_step=readiness to the existing optional failure boundary vocabulary; retain all prior values. Terminal open/attach/output/cleanup failures report that boundary through the existing one-line server logger and Details label. This names the failing step without changing progress stages or UX U1–U4. No raw remote output, protocol parser, privilege admission or timer change. D16/D40/#169.
Focused control for the exit 7 count mismatch on production BASE
b4ae9e22also failed: expected attach count 2, got 3; pre-run load 51.38/53.47/45.59; 13.4 s. Together with failures at intermediate6ec4aabd(55.11/54.52/45.04) and7bb2801e(77.04/56.78/44.61), while the full BASE replay passed that case 3/3, the mismatch is flaky on both revisions, not isolated to a head commit.Release-prod-3 browser A/B — #169
Setup
/home/kayg/Developer/infraternal/.worktrees/prod3-base, detachedb4ae9e2289f005b746188761c2f7d2502f3a8306./home/kayg/Developer/infraternal/.worktrees/release-prod-3,release-prod-3at1f77a449ede540281319ba01d149a2f60d584f3b.infraternal-dev, 12 logical CPUs. All browser runs usedsystemd-run --user --scope -p CPUWeight=10000; BASE usedTRACER_TEST_PORT=4941, HEAD used4942. Playwright's web server rebuilt the production tracer at each run start.bun install --frozen-lockfileinstalled 38 locked packages. HEAD dependencies were already present. An explicitbun run buildpassed in each checkout.bun x playwright test test/dead-session.spec.ts test/terminal-theme.spec.ts test/terminal.spec.ts.Six prescribed runs
Load averages are the exact first three fields from
/proc/loadavgimmediately before each run. Run order was BASE, HEAD, BASE, HEAD, BASE, HEAD.b4ae9e221f77a449b4ae9e221f77a449b4ae9e221f77a449Per-test results
Pmeans passed;Fmeans failed. The last column classifies the failure using all six runs plus the focused BASE control below.Failing assertions and printed measurements
Dead-session attach counts
2, received3.3, received4.2, received3. A focused diagnostic also failed at6ec4aabd(pre-mobile changes; load 55.11/54.52/45.04) and7bb2801e(source head before the evidence-only commit; load 77.04/56.78/44.61). A focused BASE run failed the same assertion (load 51.38/53.47/45.59). Each diagnostic was one test, taking 14.4 s, 12.4 s and 13.4 s respectively. This failure is flaky on both; no head-only commit is implicated.Theme long-task assertion
Every run failed because
long_tasks_gt_50mswas nonzero; the p99 assertion (<16.67 ms) passed in each run.Terminal resume and latency probe
isConnected() === true); those runs printed no latency metrics.Access-screen first-paint budget
Each successful sample batch used 30 samples at configured RTT 300 ms. Render upper-bound p99 is the gated value (budget 1000 ms); FCP is reported for context.
Conclusion
None of the five failures in the previous HEAD replay is exclusive to
release-prod-3: exit 0, theme, terminal resume/latency and access-screen failures occurred on production BASE; exit 7, which failed only once in the six prescribed runs, reproduced on a focused BASE run and at two intermediate commits. The saved-status attach mismatch also occurred on both revisions. These runs show substantial timing variation under shared host load; they do not establish a head-only regression or justify a waiver.The intermediate worktree was removed. Both original worktrees are clean; test-generated tracked capture PNGs in BASE were restored.
Finished the #169 browser A/B check on release-prod-3 HEAD
1f77a449ede540281319ba01d149a2f60d584f3b; full per-test table and measurements are in/home/kayg/.local/state/codex-jobs/infraternal/prod3-ab/ab.mdand the preceding comment. Final Playwright output lines (verbatim):BASE 1:
HEAD 1:
BASE 2:
HEAD 2:
BASE 3:
HEAD 3:
No head-only regression established. The temporary detached worktree was removed; both requested worktrees are clean and generated BASE captures were restored.
Completed acceptance imports, pushed only addhost-fixes2:
889e4ab4address tests (typecheck; 46 pass/2 skip/0 fail);2735d406fixture scratch override (Python compilation);f5a477d9authority/live tests (bootstrap 29 pass/0 fail/4 ignored, 24.38 s);ec4e3848rollback tests (ordinary 29 pass/0 fail/4 ignored, 1.89 s);81bb0462init test (ordinary 1 pass/0 fail/1 ignored). Server/SSH all-target Clippy 3m11s PASS; fmt/diff checks PASS. All original desired-contract assertions retained. Logs acceptance-{clippy,server-gate,install-gate,probe-gate,tracer-check,tracer-tests}.log. NetBird fixture image also builds with go vet, enabling the requested local/150ms counterpart. No product repairs committed yet.R1 source repair pushed in atomic steps:
677ca2f9relay drains existing data after ownership EOF (real Unix-socket regression before UnexpectedEof, after1/1PASS; agent32PASS/6ignored +2transportPASS);82e13cedshared readiness boundary/logger/typed UI label;021bdd45private-proof metadata (regression before failed_step=null, after readiness; bootstrap29PASS/0FAIL/4ignored in18.32s);61e5b901additive contract docs. All-target agent/server/SSH Clippy58.10sPASS, fmt/diffPASS, tracer typecheck/46sourcePASS/2skip. Rejected sleeps/wider deadlines/unconfirmed output; no allocations or periodic timers added. Frozen old static and production-baseline controllers are building for before/after10fresh reliability and per-step measurements; R1 acceptance not claimed complete yet.R1 live ordering reproduction: frozen pre-fix static payload20093808B, real SSH/HTTPS/systemd/remote transport, fixed private PTY. Diagnostic adapter sends valid Input before granting Credit50ms later, forcing final PTY frames to queue. Open/attach succeed; stream closes1013 before any marker; output=false, cleanup=true; Done assertion fails at Failed after4449ms. Authority/artifacts/scopes all retired. r1-live-ordering-red.log matches the reported fast failure boundaries; deterministic socket test identifies owner EOF discarding queued data. Ordinary
b4ae9e22and frozen round-1 samples both10/10Done in this window; this does not negate prior3/10 evidence or establish a production rate. T4 also fails onb4ae9e22at owned-artifact cleanup assertion (baseline-tiny-disk.log). H1 exact CHECK_SCRIPT reproduces seven startup failures on fresh target using unchangedb4ae9e22SSH/probe path (baseline-startup-red-fresh.log); initial startup probe on full disk was invalid and is superseded. New delayed-credit hook affects only explicit diagnostics; normal timing/order and all acceptance assertions remain unchanged.Continuing #169 on addhost-fixes2, base
1f77a449ed, resumed source head80e351aeafter the requested soft reset of deef0a0a. Read all 105 issue comments; no parent linked. Earlier R1 evidence already contains 10/10 fresh Done runs and a passing delayed-credit live reproduction. Reviewing and preserving those files before T1, S1/S2, P2/P3 and payload reuse. Browser port 4957; local CPUWeight=10000 measurements. No deployment.R1 continuation review: existing post-repair fresh target logs show 10/10 Done, 0 failures, 3.08–4.36s test durations. Delayed-credit live case passes after repair; pre-fix case fails.
b4ae9e22socket regression fails at EOF before queued output. Preserved measured setup-step ranges and original baseline/pre-fix/repair logs in35e91c24(replaces WIP deef0a0a with exact lease). evidence/addhost-fixes2/README.md and r1-reliability-{01..10}.log. T1 before repair: 0 pass/2 fail in0.26s.T1 repaired and pushed
e4827f10: replacements live in the owned transaction, foreign .new paths are refused before activation, and final launcher cleanup requires its replacement journal. Desired-contract tests now ordinary. Before0PASS/2FAIL0.26s; after32PASS/0FAIL/2ignored1.16s. SSH all-target clippy/fmt PASS. Same defect exists onb4ae9e22(baseline-install-red.log). Evidence t1-{before,after,fmt,clippy}.log. S1 next: publish invitation and address in one SQLite transaction so refusal/cancellation leaves no new active host record.Completed storage preparation
554fe436, pushed: invitation issuance can use the caller SQLite transaction with identical session checks. Existing enrollment gate PASS; server all-target clippy PASS; fmt PASS. Rejected duplicated admission/token statements and compensating deletion. S1 transactional adapter regression passes30/0/3ignored15.68s; final check after removing unused old wrappers is running before committing the adapter.S1 repaired/pushed66ed0d63: host record, invitation and address commit in one SQLite transaction. Address-trigger refusal now leaves1active prior record; bootstrap30PASS/0FAIL/3ignored22.46s, server all-target clippy/fmt PASS. Defect also reproduced on
b4ae9e22(baseline-authority-red.log). Evidence s1-{after,clippy,fmt}.log.S2 repaired/pushed11a673e6: reserve address/trust identity before database waits, with a second destination check inside the SQLite writer transaction. Eight concurrent attempts yield1host-bound attempt;7say already being added. Bootstrap31PASS/0FAIL/2ignored18.43s; server all-target clippy/fmt PASS. Original defect exists on
b4ae9e22(8identities, baseline-authority-red.log). Reservation storage remains bounded1600attempts; no worker/timer. Evidence s2-{after,clippy}.log.T2 repaired/pushed72672e9b: staging uses cleanup private-parent checks before mutation. Before0PASS/1FAIL0.12s; after33PASS/0FAIL/1ignored2.12s, SSH all-target clippy/fmt PASS. Exists on
b4ae9e22(baseline-install-red.log). T3 reproduced0PASS/1FAIL0.04s; adding an attempt-bound acquisition journal before mkdir rather than treating any empty lock as owned.T3
d2661eb5pushed: pre-mkdir acquisition journal, bounded39/41B receipt; install34PASS/0FAIL3.89s and clippy/fmtPASS. T4 regression committed separately, then repaireb047f72 pushed: reclaim incomplete owned payload before outside receipt allocation; install35PASS/0FAIL1.92s and clippy/fmtPASS. Both defects exist onb4ae9e22; baseline-install-red.log/baseline-tiny-disk.log. Real full-home rerun follows fresh final executable.H1 boundary finding: baseline-startup-red-fresh.log shows7startup profiles preventing canonical CHECK_SCRIPT output on
b4ae9e22. OpenSSH invokes the account shell before the requested command (openssh.rs command boundary); startup exit/replacement/wait can prevent every requested setup script from running. Changing the command to sh cannot control that earlier shell. No permitted client command can promise completion for those profiles without target-side configuration. Retain strict parsing and add explicit interactive-startup guidance; successful handling of exit/replacing profiles remains blocked on an administrator changing the target shell startup or a separately approved execution path. No real account profile changed.Orchestrator: DEPLOYED to production at 03:17 UTC: release-prod-3 @
1f77a449(b4ae9e22+ add-host round 1 #169 ported; size-baseline commit excluded). Static x86 20,093,808 B / arm 17,910,064 B, sha256 prefixes 1e95c365 / cfd7e2bf match on host; web build passed the transfer budget. healthz 200, public 200, no errors in the journal. The restart re-locked the CA: owner unlock needed. Earlier 02:51 attempt with CA custody (8cff05cd) rolled back automatically (see #154). Browser A/B vs production: no new failures (3 checks also fail on production; attach-count checks flaky on both).R1 decoder repaired/pushed4955b966: readiness survived types but was discarded by socket parser; regression before1PASS/1FAIL, after2PASS/0FAIL/typecheckPASS. H1 guidance
e5fd4c6d(target-side startup limitation remains as recorded). A106fef9d9explicit port preserved; A21d13d872malformed IPv6 stays editable; source48PASS/0FAIL/typecheckPASS. H2b11368e9canonical init=other names systemd; probe3PASS/model5PASS/clippy/fmtPASS. A1/A2/H2 also reproduce onb4ae9e22. Cache desired-contract tests are red before implementation.Additive retry cache proposal before implementation: keep one hash-verified public executable in a private, marked sibling infraternal.bootstrap-cache directory under the same protected parent. No login material/state retained. Before reuse check directory/file ownership, modes, links, marker and the requested executable digest; recheck staged digest before enrollment. Corruption causes a new upload; foreign paths are refused. Keep one payload per execution mode rather than one per attempt/version; publish from the owned transaction. Rollback removes installation data but retains this explicitly declared verified retry cache. No background worker/timer or foundational dependency change. D17/D40; update setup contract and measurements.
Cache repair
af5fa4e2pushed: verified retry sends file input once; changed content sends it twice and matches requested payload; foreign cache refused. Full SSH suite PASS (38install tests); clippy/fmt PASS. Measurement observer separately committed; bootstrap31PASS/2ignored14.51s, server clippyPASS. Contract retention documented separately. Actualb4ae9e22batch retry passes but still performs2complete uploads (baseline-retry.log). Final live retry/150ms and full-home comparisons await fresh static executable. Measuring server admission at1600pending hosts/8concurrent requests to account for bounded destination scans. Browser production27cases running on4957; Browser plugin unavailable.Review continuation:3important findings, evidence/review-findings.txt. S1 needs cleanup coordinates before SQLite commit can execute, not only after invitation returns: canceled COMMIT may persist despite Transaction Drop. Additive Backend invitation_registered callback proposal: durable backend registers exact host in AttemptResources before issuance, existing cleanup retires it on cancellation/failure; no new background task. Also repair interrupted acquisition receipt recovery and listed-user inheritance with explicit ports. Each receives red evidence and a separate gated commit. No target-side profile change or budget waiver.
Review repairs pushed separately:
de1d74a2pre-registers cleanup before invitation publication, writer barrier handles cancellation with/without a committed row; paused realSQLite commit case before0PASS/1FAIL0.09s, bootstrap after32PASS/0FAIL/3ignored14.87s.a17ccb21resumes receipt/acquisition interruption, before0PASS/1FAIL0.24s, install39PASS/0FAIL2.28s.865b6c82preserves listed user with port; source/decoder51PASS/typecheckPASS. Strict touched-crate clippy/fmtPASS. Source reviewer cleared all3; no remaining source findings. Cancellation gap inherited; journal recovery and listed-user cases were introduced in this round and are now fixed. Browser27PASS7.1m on earlier milestone; final replay and fresh static build running.Readiness finding at
865b6c82: the local retry sample reused the verified payload (0 second-upload bytes) but failed readiness; delayed-150 retry and 10/10 fresh samples passed. Independent source review confirms a separate legitimate race: canary_output drops the primary attachment at the marker, then finish_canary reattaches while the private shim retires; handshake EOF can produce generic code 1. This does not establish the observed failure cause. Proposed additive repair: require marker plus authenticated terminal.exited on the primary stream, then avoid a second attachment on success; retain recovery attachment on failure/cancellation. Evidence: final-retry-local.log and bootstrap_remote.rs; actor regression will precede repair. Milestone gate also found an overly long fixture path (109 bytes); rerun with a shorter job-owned scratch path, without changing tests. Static size remains over the unchanged budget.Readiness regression
a5e2cc63is red on the prior source (0/1); four existing actor tests pass. Repair61edf2d4a2consumes matching authenticated status-zero primary exit after marker, avoids reattachment on success and clears owned session coordinates before authorization await. Failure/cancellation recovery remains. Provisioning tests 33 pass / 0 fail / 3 opt-in ignored in 22.32s; strict server clippy 18.72s; fmt passes. Independent source review: no blocking findings. Live repeat and final gate follow; previous local retry failure remains archived. Logs: retirement-before-remote.log, retirement-after-final.log, retirement-clippy-final.log.Completed additive contract and observer steps:
01f5a067test(addhost): align live diagnostics with primary readiness retirementb4195b32docs(ssh): define primary readiness retirement proof9b3eb50edocs(ssh): declare cleanup registration before durable invitation commitContract diff checks pass; hook and readiness proposals preceded the source repairs. Observer follows matching primary exit, keeps delayed-credit mode and logs only boundary/count metadata; normal runs delegate production. Provisioning tests: 33 pass, 0 fail, 3 opt-in ignored (20.55s); strict server clippy passes (10.45s); fmt passes. Frozen controller saved for live samples. Static payload and parser fuzz builds still running under shared load.
Live proof at
01f5a067: 10/10 fresh supported-target runs reach Done; opt-in delayed-credit proof 1/1; local and delayed-150 retry both pass. Payload 20,108,048 bytes, one accepted file input per retry batch, zero second file-input bytes. Reuse 130.617ms local / 600.886ms delayed, lookup 178.824 / 624.644ms. Loads 113.73/107.94/91.96 and 104.82/106.45/91.98, 12 CPUs; CPUWeight=10000. Per-step baseline/final ten-run timing ranges in evidence/addhost-fixes2/README.md. Real full-home cleanup passes with zero remaining daemon processes/session scopes. Logs retirement-retry-{local,150}.log, retirement-reliability-01..10.log, retirement-delayed-credit.log, retirement-tiny-disk.log. Earlier failed local sample retained without causal attribution. Full gate, ARM and parser fuzz are running. TS compiler/build/transfer checks pass; differential test hit its 30s limit while sharing the debug build directory with Rust compilation; serial rerun pending, no deadline changes or waiver. H1 target-startup limitation remains as recorded.Gate environment finding: the inherited shell umask is 002. The new own-scratch test executable infraternal-695a777d772bb491 was mode 0775; restart_binary::Opened correctly refuses group-writable images, leaving the existing closed_reply test waiting for a valid request. Stopped only owned PID4054199, corrected executable write bits inside this job target, and made job directory private. No source/test/deadline change. All subsequent verification commands use umask022. Interrupted gate retained in retirement-fast.log; clean rerun pending. Protocol differential invokes cargo run and overlapped native compilation in the same debug directory; serial TS rerun pending with its unchanged 30s deadline.
Starting completion pass on branch addhost-fixes2 at
2a57316ee6, based on release-prod-31f77a449ed. Read all 125 comments on #169; issue history records no parent. Scope is evidence completion, requested browser/full gates and final report only.Final
scripts/verify fastattempt on9cdab5c7: fmt, VT native fmt, clippy and cargo-deny passed. The real-binary test rejected the owned fixture path at exactly 108 bytes (verify-agent.py, captured inevidence/addhost-fixes2/verify-fast-final.log); I am shortening the configured state root and rerunning. Static builds also failed: x86_64 binary 20,108,048 bytes against the existing 16,404,816-byte cap. The earliermilestone-fast.logrecords the same static-size failure at 20,107,152 bytes. I am preserving the transcripts and checking the corrected full gate before reporting.The rerun on
9cdab5c7cleared the AF_UNIX path bound. Seven of eightagent_reviewtests passed;initial_attach_requires_explicit_credit_before_replaystopped before its assertion because its existing fixture writes totarget/ac<PID>and the requested cleanup removed the roottargetsymlink (FileNotFoundError,verify-fast-final2.log). I will provide a temporary empty fixture directory for the gate, with Cargo builds still in the scratch target, then remove it. Static builds remain 20,108,048 bytes against the 16,404,816-byte cap.Host provisioning round-two evidence is complete on
addhost-fixes2, head9cdab5c7f158a23715e1a4c82fc650b5c63f2537(pushed). Full matrix, setup-step timing detail and paths are inevidence/addhost-fixes2/README.md. The requested browser replay passed 27/27 (browser-27.log).677ca2f9,4955b966,61edf2d4b4ae9e22; socket regression 0/1 before, 1/1 after; fresh-target proof 10/10 after repair.e4827f10.newpaths on baseline; regression 0/2 before, install suite 32 passed after.554fe436,66ed0d63,de1d74a211a673e672672e9bd2661eb5,a17ccb21eb047f72e5fd4c6d06fef9d9,865b6c821d13d872b11368e9af5fa4e2Setup-step ranges (milliseconds; baseline → final repaired flow): R1 readiness/commit 101–217 (p50 133.5) → 97–539 (p50 164.0); T1/T2/T3/T4 install 80.001–116.852 → 87.026–201.050; S1/S2 commit 23.078–66.415 → 26.292–161.596; H1/H2 check 119.194–164.251 → 113.388–1066.597; payload lookup/upload absent / 307.621–1778.055 → 19.898–128.368 / 729.131–6822.204. A1/A2 are local typed-target validation and have no provisioning event timing. The README records individual before/after test durations and limitations.
Final
scripts/verify fasttranscript:evidence/addhost-fixes2/verify-fast-final3.log. Tests passed: main suite 298 passed, 0 failed, 34 ignored in 359.74 s; all other test binaries passed. Static builds remain over the existing size cap. Verbatim final gate output:The final gate is not green because of the static-binary size budget. No source change or budget change was made in this report-only continuation. The earlier fixture-root issue was resolved with a temporary test-state directory, then removed; no worktree
targetentry remains.Independent verification started on review-addhost-fixes2 at target
9cdab5c7f1; merge base with dev93b00f5ab9. Read #169 history (129 comments), no parent linked, repository rules, touched contracts and author report. Review only: failing tests/evidence; no implementation fixes, push, deployment or perf VM. Timings measured on the shared build host are not accepted as performance evidence. Rechecking target-head fast gate and cleanup/admission/readiness behavior.Independent review V1 P2, committed failing test
2bfa2da3: wizard.ts typedChip keeps listed user but substitutes inventory id for address when an explicit port is typed. For inventory lab => 203.0.113.7, production resolve/addressOf serializes target@lab:2222 instead of target@203.0.113.7:2222. Existing 3 cases pass, new regression fails. No wrong-host traffic claim. Evidence address-red.log; implementation unchanged; branch review-addhost-fixes2 remains local.Independent review V2 P2: committed real-shell regression review_interrupted_transaction_receipt_allows_owned_cleanup on review-addhost-fixes2. Empty transaction file plus exact pre-mkdir acquisition journal is refused by install.rs ROLLBACK_TRANSACTION with status72; owned lock remains and retry cannot proceed. 0 passed/1 failed/39 filtered; receipt-red.log. Existing T3 empty-lock test does not exercise a created-but-incomplete receipt. Target source add-host/decoder tests also passed 51/51. No implementation fix or push.
Independent review continuation: V2 test commit
5632b02b. Prior F-001 P2 is still present on9cdab5c7: reappliedb1950b14regression enrollment_process_exit_71_is_not_reported_as_directory_refusal, 0 passed/1 failed/40 filtered (status71-red.log). Payload Join status71 is mislabeled as installer directory refusal; status/Join assertions pass, expected Join recovery message fails. Retained separately on local review-addhost-fixes2. No implementation fix or push.Starting production backmerge on dev-backmerge-prod4 at dev base
d93056ef6f; production head61bdbd9f7f. Read #169 history (133 comments); no parent is linked. Merge round-one production changes only, preserve both lines and dev size gate unchanged (D243). One merge commit per brief; push branch only, no deployment. Functional browser port 4987; timing gates require perf VM under the latest owner rule. Prior round-two findings are outside this production head.Timing gate blocked: exact permitted command netbird ssh --no-browser root@10.69.69.208 (read-only flock/uname probe) returned exit 1 with no stdout/stderr (perf-access.log is empty). No host key or credential inspection/change. Latest owner rule forbids build-host timing evidence. Continuing functional gates locally; perf timing specs need orchestrator access/booking. Size gate remains byte-for-byte identical to origin/dev.
Independent review V4 P1, failing test committed locally at 44212c59 on review-addhost-fixes2: review_cancelled_committed_invitation_survives_temporary_writer_contention. Real backend/SQLite commit succeeds, acknowledgement is paused, a real writer holds the fixture connection, then the batch is cancelled. Bootstrap::attempt discards its five-second Backend::failed timeout; after releasing the writer, active remote_hosts=2, expected prior1. 0 passed/1 failed/332 filtered; evidence/review-addhost-fixes2/authority-red.log. No subsequent revocation is retained. This extends S1 at the cleanup-failure boundary; no capability theft claim. HOLD (P1 present). Also retained P2 tests V1/V2/prior F-001 at 2bfa2da3/5632b02b/23f7f65c. Target-head full gate and detached-dev socket comparison still running. No implementation fixes or push.
Review test evidence now has four atomic commits: V1 P2
2bfa2da3; V2 P25632b02b; prior F-001/V3 P2cabed7b6; V4 P10e9717ae. Formatting-only normalization of the earlier imported F-001 test was folded into its commit (supersedes 23f7f65c/44212c59; behavior unchanged). HOLD remains. Target native gate had 297 pass/1 fail/34 ignored in its server suite: unchanged claude_review_watch_expires_host_made_visible_by_membership_edit received error instead of resync. Detached dev comparison is running; no exemption claimed. First unweighted browser run 3 pass/24 fail, mostly setup timeouts; unchanged replay under the weighted scope has its first three cases passing. Review remains local; final gate/static build pending.Seven expected conflicts resolved: retain dev async plugin worker/admission and signed payload source; production readiness/diagnostics, terminal fidelity, exit-event census and mobile manager fallback are integrated. scripts/verify.d/rust.sh and docs/PROCESS.md identical to dev (D243). Native first Clippy exposed production census fixture using old API; adapted to dev plugin service. Also detected generic control-event decoding swallowing plugin invalidation; failing regression compiling before repair. Tracer check + 168 source tests PASS; critical Brotli245246/256000B, deferred43914/45056B, largest on-demand13453/18432B. Browser initial run invalidated by overlapping product/preview build; retained separately, restarted serially. Functional Retry reached details then exceeded overall30s watchdog; will replay with120s harness envelope, no assertion/budget change. No performance claims from build-host timings.
Verification update: unchanged weighted production-build browser replay passed27/27 on port4297; expanded failure disclosure capture also passed1/1 on4301. Review strict all-target server/SSH Clippy and fmt pass. Detached dev8e7467c0 socket liveness comparison passed1/1, so no baseline exemption claimed for the first full gate. A target-head repeat hit E0560 (Config.readiness missing) after I reused its debug target for dev: stale non-test shim artifact, not a source finding. Clearing only this job target’s infraternal-shim dev artifacts and rerunning the exact target, preserving both transcripts. Final verdict still HOLD for V4 P1; no implementation, assertion, deadline, budget or production change.
Merged decoder regression reproduced: embedded::regressions::plugin_control_notice_preserves_capability_invalidation 0PASS/1FAIL, assertion plugin capability notice was discarded (plugin-notice-before.log). Repair retains only bounded plugin.event/plugins.resync control payloads for dev capability watch, preserving production census and strict terminal stream parity. Added real-socket actor coverage proving capability invalidation leaves census intact. Functional browser59PASS/3watchdog failures; exact3 rerun allPASS with120s overall harness, unchanged assertions. Live cursor/phone-width8PASS. Five fresh screenshots under evidence/dev-backmerge-prod4/captures; historical imported captures restored. Native GNU uses own checksum-recorded locked3.6.3 archives to avoid duplicate C compilation; musl recipe unchanged. Full native workspace gates in progress.
Blocking integration finding: dev has 0020_mesh_dataplane.sql and production has 0020_terminal_exit_snapshot_index.sql; both version20 enter the merged SQLx migrator. Fresh startup fails Auth/Migration; existing histories would checksum-conflict. Proposal before repair: retain dev20/21 unchanged; rename production index to22 without changing SQL/checksum. Before normal SQLx migration, take an immediate SQLite transaction and retag only a successful version20 row whose checksum exactly equals the shipped production index to22. Preserve checksum/timestamp/description; unknown/failed rows and occupied22 fail closed and remain unchanged. Then ordinary SQLx validation and missing migrations apply. Crash after retag resumes normally. No owner decision or engine change; merge compatibility only. Add fresh/dev/production and checksum/rollback regressions before repair. Existing native RSS fixture failure is separate; latest rule requires perf VM for valid performance evidence.
Functional harness finding: verify-agent.py --functional documents no release measurements, but still samples CPU idle and asserts RSS<15MiB before the correctness section. cargo integration calls --functional; this failed under the build-host debug binary before any 24-stop proof (native-remaining.log). To honor the latest VM-only rule, move just those initial RSS/CPU checks under not functional; retain all limits unchanged in normal performance mode and retain every byte/ack/exit/restart assertion in functional mode. Perf mode remains blocked on VM access, not waived. This is test-mode separation, not a budget increase.
Verdict: HOLD (P1 present) — merge after fixing V4. Counts P0=0, P1=1, P2=3, P3=0. P2 tests may be fixed forward under the owner verdict rule.
Independent target:
9cdab5c7f1; dev merge base93b00f5ab965949b1a76a808e5734f98fed82fd4. Local review-addhost-fixes2 headff0285f692; clean, not pushed.V4 P1: cancellation after invitation commit while a real SQLite writer blocks cleanup leaves2 active remote_hosts instead of prior1 after the writer is released; the Backend::failed timeout/result is discarded. Failing test0e9717ae, evidence/review-addhost-fixes2/authority-red.log. No capability theft claim.
V1 P2 inventory-name/port substitutes id for address (
2bfa2da3); V2 P2 empty transaction receipt defeats exact acquisition cleanup (5632b02b); prior F-001/V3 P2 payload Join71 gets directory-refusal advice (cabed7b6). Each has a committed ordinary failing test. No implementation fixes.Final exact-head scripts/verify fast: exit1 only at static size; 670 Rust tests pass,0 fail,68 ignored. Server298 pass/0 fail/34 ignored. x86 stripped static PIE, no PT_INTERP,20,108,016 bytes vs16,404,816 cap. D205: size alone does not block. ARM is not reached by the dispatcher. Verbatim final summary:
Full transcript: evidence/review-addhost-fixes2/verify-target-final.log; SHA25676e2018d5f54ac55d1ebdaf2265791b471ab28f2ab6d97ff7b6eed9bbb986bc5. Earlier real socket failure, passing detached-dev comparison and stale-debug-artifact correction are retained; no baseline exemption claimed.
Separate clean-target TypeScript fast/compiler/build/transfer checks PASS; protocol36/36,34,462 assertions; add-host/decoder source51/51. Functional production-build Chromium replay27/27 (unique4297); expanded disclosure capture1/1 (4301), evidence/review-addhost-fixes2/failure-details-expanded.png. Review fmt and strict all-target server/SSH Clippy PASS.
Full report: evidence/review-addhost-fixes2/README.md. Shared-host timings are not performance acceptance; perf VM prohibited, instrumented fuzz not repeated, no physical-iPhone claim. Owned scratch/build targets and detached checkouts removed. No owner live fixture changes, push, deployment or issue closure.
Host provisioning round 3 started on addhost-fixes2 at base
9cdab5c7f1. Read #169 history and review reportff0285f6; no parent is linked. Retain acceptance tests V12bfa2da3, V25632b02b, V3cabed7b6and V40e9717aeunchanged. Targeted gates per repair, 27 functional browser specs, final fast gate, perf-VM setup-step evidence and branch push only; no deployment. V4 requires durable retirement across SQLite writer contention; proposal will precede any additive contract/schema change.V4 additive contract proposal: persist a host-bound retirement intent in the same transaction as each SSH invitation; delete it only after proven installation completion. On cancellation, fence the exact host transport synchronously before any SQLite await, then retain an owned revocation writer across the five-second caller cleanup deadline. Retire unfinished intents before opening remote listeners at startup. Existing enrolled records have no intent and remain untouched. No new worker timer/engine; pending intent and tombstone are committed together on retirement. Acceptance test
0e9717aestays unchanged. Perf VM read-only access succeeded (perf-access.log).Completed schema
6afe96b6: migration23 retains exact-host retirement intents without touching existing installations; dev upgrade, foreign keys/integrity and fmt pass (schema-migrations.log). Completed V1: listed-port selection now retains inventory address and user; unchanged review regression 3 pass/1 fail before, add-host source suite50 pass/0 fail after; TypeScript compiler pass (v1-{red,green,check}.log). Visuals unchanged.Integration repair: fresh/dev/production migration regressions were3FAIL/2PASS before;7/7PASS after exact-checksum transactional normalization of production20 to22, preserving dev20/21 and production SQL byte-for-byte. Covers unknown checksum, dirty ledger, occupied22 rollback and resumed/concurrent normalization. Original startup diagnostic identified Auth/Migration. Independent read-only reviewer found a restored reconnect reset (production67a423b5/#158); paused-clock actual-actor regression fails400vs800ms before repair, reset removed, delta review says no remaining integration blockers. All CPU/RSS thresholds stay unchanged in performance mode; functional daemon replay now completes24/24. Earlier single EOF at iteration24 retained, no causal waiver. Full native gate rerunning; unchanged size gate/VM access blockers remain.
V1 completed at
bf0bfd08; unchanged inventory-address regression now passes, source suite50 pass/0 fail and compiler pass. V2 completed at81993854: unchanged regression5632b02b red0 pass/1 fail; real-shell install suite40 pass/0 fail after; fmt and SSH all-target Clippy pass. Evidence v2-{red,commit-gate}.log; v2-green.log also records the independently imported V3 failure40 pass/1 fail before its fix. V3 reproduced unchanged cabed7b6:0 pass/1 fail (v3-red.log). V4 reproduced unchanged0e9717ae with remote feature:0 pass/1 fail, active host records2 expected1 (v4-red-remote.log); initial default-feature run selected0 tests and is not accepted as evidence.V3 completed77ea8505: unchanged payload Join71 regression now passes; install41 pass/0 fail, diagnostics8 pass/0 fail, fmt and SSH Clippy pass (v3-green2.log). Cancellation fencef11c6be7: retain cuts through storage refusal, release only after tombstone; fence1 pass/0 fail, authority3 pass/0 fail, remote server Clippy pass (fence-gate2/fence-authority.log). Storage695d00e4: exact-owner tombstone+intent transaction and startup sweep; storage3 pass/0 fail, migration1 pass/0 fail, fmt/Clippy pass (retirement-store-gate.log). Additive lifecycle clarification before wiring: Backend::retire_failed starts fencing/owned writer synchronously before session/SSH cleanup awaits; completed must acknowledge durable intent removal before Done. Browser initial17 pass/10 fail because test proxy disappeared (ECONNREFUSED); full unchanged replay is running on4321. No failure exemption or timing claim.
Full ts_verify_fast regenerated dev protocol types and exposed TS2739 in apps/tracer/src/canvas/addhost/wizard.ts: production FAILED_STEP_LABELS omitted dev get-arm64-binary/get-x86-64-binary. Added both labels to the existing map (D160), no new UI component/style. Extended the existing signed-payload browser case across both architectures and desktop/phone, asserting the Details failed-step text. Initial tracer check before protocol compiler was insufficient; full gate rerunning. Native final replay progressing with pinned Zig PATH; prior mouse failure was missing executable, no product assertion changed.
Correction to the prior finding: the protocol compiler uses noEmit; it did not regenerate types. The full tracer compiler exposed the existing dev union after its normal frozen install. The TS2739 diagnosis and two-label repair stand; no generated source was changed.
V4 follow-up: production runtime.rs680-682 uses busy_timeout5s and max_connections8; the review fixture uses one connection and exercises pool admission rather than a separate SQLite connection blocked at UPDATE. Add a real-file multi-connection regression with a short configured busy timeout, preserving review0e9717ae unchanged. Retry only SQLite BUSY/LOCKED or pool-acquisition timeout inside the owned retirement writer; storage failures retain the durable intent and cancellation fence for restart. Also bind retirement UPDATE to its durable intent so the storage helper itself cannot retire an existing completed host. No budget/deadline or acceptance assertion is relaxed. Milestone native gone-session test failed once; direct exact-artifact replay pending, no exemption claimed.
Completed full ts_verify_fast after two-label repair: check +168 tracer source tests +36 protocol tests PASS, production build/checksums/transfer PASS. Critical gzip294908/307200B and Brotli245192/256000B; deferred49746/51200B and43870/45056B; largest on-demand15179/20480B and13500/18432B. Signed-payload Details browser coverage4/4PASS (arm64/x86_64,1440/393px), screenshots evidence/dev-backmerge-prod4/captures/wizard-binary-labels/. Independent read-only delta review: no integration defect. No new commit yet: HEAD
d93056ef, MERGE_HEAD61bdbd9f; waiting for final native/fast gate, D243 unchanged and perf access still blocked.Functional browser complete:27 pass/0 fail on4331, detached owned service with explicit PATH and IPv4-first DNS (browser-27-service2.log, exit0). Earlier scoped run ended143; retained, no product assertion waiver. Earlier ::1 ECONNREFUSED matches the IPv4-only test proxy (latency_proxy.py175/320); no UI changes. Full native milestone found a fixture path failure, not load: daemon_gone_tests.rs98 rejects an oversized Unix socket under HOME-cache TMPDIR; exact artifact replay0 pass/1 fail (gone-replay.log). Shorten only the random fixture directory prefix, keep the frozen rendezvous name and all protocol assertions. No baseline exemption. Current full gate static builds are running. V4 completed3a6d5580, lifecycle47f28289, issuera58529b1, startupa48eb11e; provisioning9 pass/0 fail/2 opt-in ignored, HTTP7 pass/0 fail/1 ignored, startup/transport25 pass/0 fail/3 ignored.
Corrected final native run PASS: cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace. Native test-result totals805PASS/0FAIL/104ignored across94 rows; latest compiled-migrator compatibility7/7, actual-actor backoff and plugin notice regressions included. Mouse real-C probe and VT1k corpus pass. Earlier final mouse failure was missing pinned Zig PATH; no source assertion changed. Evidence: evidence/dev-backmerge-prod4/native-final2.log. HEAD remains
d93056efwith staged production61bdbd9f integration; full fast and final ARM size check ongoing. Perf VM access refusal remains unwaived.Blocked before merge commit/push by unchanged D243: final ARM static build succeeds, stripped/static/no PT_INTERP, but19355472B exceeds13817816B (static-arm-final.log). Dev gate SHA256 remains14a782d5b95b972a39df1a3899ccb9f71190622da4d9b3808038ca5061dd6430. No size waiver/baseline increase, feature removal, deployment or dev/main update. Final fast gate passed fmt, native fmt, CLI compile-out, Clippy, workspace tests and cargo-deny; x86 static build currently compiling. Perf timing gate separately blocked by NetBird exit1/empty output. Needed from orchestrator: a size repair retaining D243 and permitted perf-VM access; this merge cannot satisfy push-after-gates-pass yet. Resolved staged candidate remains on dev-backmerge-prod4, HEADd93056ef6f0e16e47b50583cfb781ee15bf09850, MERGE_HEAD61bdbd9f7fe034f5d9261aea8b480d01d626d3ba.
Latest VM-only rule follow-up: scripts/verify-agent.py --functional promises correctness without release measurements but still samples idle RSS/CPU both before and after attach. Preserve all terminal/exit/24-stop replay assertions and every default performance-mode limit; guard only these measurement blocks with not functional. Before repair, a real-binary harness replaces perf-counter helpers with a refusal sentinel, proving functional mode reaches them without taking a shared-host measurement. Final fast RTT self-test will be forwarded unchanged to the perf VM via a job-owned wrapper, preserving the original3-sample gate and its limits. No performance budget or functional assertion is weakened. V4 multi-connection and intent-binding follow-up2481499a passes:3 pass/0 fail, provisioning10 pass/0 fail/2 ignored. Gone-session fixture now passes1/0 after compact directory name; prior path failures retained.
Finish status: BLOCKED, not merged/committed/pushed. Branch dev-backmerge-prod4; HEAD
d93056ef6f, staged MERGE_HEAD61bdbd9f7fe034f5d9261aea8b480d01d626d3ba. Seven conflicts resolved; migration collision, swallowed plugin notices, reconnect reset and binary-download labels repaired with failing-before evidence. Independent review found no remaining integration defect. Native805PASS/0FAIL/104ignored; tracer168PASS plus protocol36PASS, check/build/transfer PASS; requested functional browser62/62 after unchanged three-case watchdog replay, plus8 live-web and4 architecture/Details cases PASS. Captures and raw logs: evidence/dev-backmerge-prod4/.Final static builds succeed without PT_INTERP, but x8621857264B exceeds16404816B; ARM19355472B exceeds13817816B. D243 script unchanged byte-for-byte; no waiver, baseline increase or feature removal. Perf timing unverified: authorized NetBird probe exit1/empty output; no credentials or host keys inspected. Needed from orchestrator: size repair preserving D243 and permitted perf-VM access. Source/evidence remain staged in this worktree for resume; no dev/main update or deployment. Job-owned targets/fixture scratch are removed after evidence retention.
Final scripts/verify fast exit1, summary output verbatim (full raw output verify-fast.log):
Resuming host provisioning round 3 on addhost-fixes2 at
b0a0bda9(original base9cdab5c7). Read complete #169 history and reviewff0285f6; no parent linked. Prior V1/V2/V3/V4 fixes and27/27 functional browser replay retained. Continue unfinished functional measurement separation, per-step perf-VM evidence, final head fast gate and branch push. Milestone gate failed gone-session fixture (now repaired) and inherited static size; no assertion or budget waiver. Perf VM access was established in this lane; use only booked208 with shared lock.Completed47ca472d: functional oracle uses no performance counters; real refusal-sentinel replay1 pass/0 fail with24/24 stop/restart assertions retained (functional-no-perf-{red,green}.log). Completeda3f9e7c9: portable opt-in setup-step observer, forwards synchronous retirement fencing; remote backend18 pass/0 fail/2 ignored, fmt and strict Clippy pass (perf-fixture-gate.log). Perf VM transfer/check succeeds; clean real-SSH provisioning1 pass/0 fail, private session scopes0. Running serial real network-SSH setup-step samples/retries and1600-host admission on208 under shared lock; VM-only data replaces older build-host timing claims. Final fast underway with original RTT self-test delegated unchanged to208.
Final native fixture finding: daemon_gone_tests.rs104 still fails SUN_LEN with TMPDIR=/home/kayg/.cache/codex-scratch/addhost-round3: generated socket119 bytes. Prior targeted replay used a shorter TMPDIR and passed1/0; it did not prove this full path. Use the job-owned HOME-cache alias/home/kayg/.cache/codex-scratch/a3 -> addhost-round3, yielding107-byte socket paths, preserving96-bit fixture isolation and frozen session names. No product assertion/deadline or baseline exemption. Repeat exact fixture and final fast with compact alias; first transcript final-fast.log retained.
Final replay environment finding: compact scratch fixes SUN_LEN, but the RTT delegation wrapper forwarded other scripts to/usr/bin/python3(3.13), changing the repository PATH Python3.14. Session-bounds failed PermissionError from/proc//exe Path.exists in3.13, not a timeout. Correct only the job-owned wrapper to preserve/home/kayg/.nix-profile/bin/python3 for every non-RTT call. No source assertion or budget changed; no #118 exemption. Original3-sample RTT gate still runs unchanged on208. Replay exact session-bounds test and full fast.
Completed pinned-VM evidence and lifecycle docs at
dce61abc. Booked208/shared lock: setup15 pass/0 fail (10 local +5 at150ms RTT), retry2/0 with one accepted20132112-byte file input each and zero second upload, foreign-base refusal1/0 preserves prior host record/foreign data,1600-host admission1/0 and writer regressions2/0; initial real OpenSSH smoke1/0. Raw logs/per-step ranges/load/RSS/scale arithmetic in evidence/addhost-round3/performance.md and perf/. Prior shared-host timing claims superseded; no release-controller percentile/speedup claim. Full TypeScript gate PASS (source161/0, protocol36/0, compiler/build/transfer). Final native repeat is progressing after documented scratch/interpreter environment corrections.Finished host provisioning round 3. Branch addhost-fixes2 pushed at
6be5514d7f; original base9cdab5c7. V1/V2/V3/V4 acceptance assertions retained and pass. V4 exact retirement intent commits with invitation ownership, synchronous fence + owned writer survives caller deadlines/BUSY, startup retires unfinished host records before listeners; prior completed host records preserved.Final native679 pass/0 fail/68 ignored; browser27/0 on4331; TypeScript161/0 and protocol36/0, check/build/transfer PASS; perf suite21/0 plus real OpenSSH smoke1/0 on booked208/shared lock. Earlier build-host timing claims superseded. Atomic source/docs/evidence commits; no default branch update, deployment or issue closure. Author checklist and production capture retained. Job-owned scratch removed after artifact hashes/evidence retained.
Final scripts/verify fast exit1 solely at inherited static size, all other rows PASS. D205 permits reported growth: x8620132112 bytes (+24096 versus reviewed original), ARM17943200; both built stripped/static/no PT_INTERP, unchanged caps/profile/dependencies. No original ARM delta or #118 exemption. Full raw output and hash: verify-final.log, report.
Verbatim final gate summary:
Handoff follow-up at6be5514d: cleanup is complete. Owned addhost-round3 cache/build targets and VM artifact directory removed; worktree clean. The compact a3 TMPDIR was a preexisting directory, rather than a direct symlink alias; only the symlink this run added inside it was removed. Its tested107-byte socket paths and all gate outputs remain valid; preexisting contents were preserved.
How to recheck (set CARGO_TARGET_DIR to your own HOME-cache scratch first):
Independent delta verification started on review-addhost-fixes2 at target
6be5514d7f; prior review headff0285f6preserved locally as review-addhost-fixes2-prior-ff0285f6. Scope only 9cdab5c7..6be5514d, excluding dev merges, and V1–V4 acceptance. Read #169 history/report/contracts; no linked parent. No implementation fixes, push, deploy or perf VM. Functional checks local; no build-host timing claims.Starting host provisioning start-fresh on addhost-start-fresh, base
6be5514d7f(origin/addhost-fixes2), as explicitly assigned. Read #169 history (165 comments); no parent linked. Read CLAUDE/DECISIONS/ARCHITECTURE/PROCESS and origin/dev TASTE “Offer the way forward, not a chore.” Owner live fixtures will remain untouched; isolated real filesystem/shell fixtures only.Additive contract proposal: pre-enrollment check returns typed stale_previous_attempt with bounded public findings, or refuses live/foreign data. Same-socket start-fresh action is allowed only for that paused target, revalidates all ownership/liveness before mutation, and continues its existing attempt. Exact UI copy for later pass: title “This host has leftovers from an earlier setup”; body “infraternal didn't finish setting up here before. Start fresh to remove what it left behind and continue.”; primary “Start fresh”; secondary “Back”; paths only in Details. Live copy “Setup is already running on this host”; offer to wait.
Reproduced owner-hit stale socket on isolated real shell/filesystem: original rollback exits72 and leaves the journal; Rust regression stale_previous_attempt_socket_can_be_retired_without_manual_cleanup fails at rollback().is_ok() (0 pass/1 fail). Evidence evidence/addhost-start-fresh/stale-before.log and shell-before.log. No owner fixture read/modified.
Recovery reuses the inode-bound host/transaction fresh journal and existing rollback; older unbound formats are refused explicitly. Foreign base entries retain ownership receipts and remain untouched. Read-only classification is folded into the existing SSH check (no new round trip). Shared directory flock serializes check/recovery/new installer commands without an unowned lock file; actual listeners and installation executable links are inspected, never argv/environment. No new library or foundational engine.
Independent delta acceptance committed locally at
2528278d, review-addhost-fixes2. Prior V1/V2/V3/V4 test functions unchanged from 2bfa2da3/5632b02b/cabed7b6/0e9717ae; all PASS. Add-host50/0; real-shell install41/0; exact original V4 1/0; cancellation/SQLite BUSY2/0; retirement-store4/0; tracer compiler exit0. Evidence/every raw log: evidence/review-addhost-fixes2-delta/acceptance.md. No new confirmed source finding. Target fast gate continues; no implementation fix, push, deploy or perf VM.Independent functional-mode evidence committed locally at58ed72a6. Real daemon/shim refusal-sentinel harness exits0: no RSS/scheduler calls, all24 stop/restart assertions PASS, 4,710,396 exact replayed payload bytes. Evidence/review-addhost-fixes2-delta/functional-mode.md and raw log. No source change or performance claim. Full target fast gate still running; no new confirmed finding.
Completed additive action/event schema at5d52dfc3. ssh.start-fresh declared once (D16/D198/D202); stale_previous_attempt and setup_running typed states plus bounded resumable findings. Findings live on events, preserving the120-byte Failure layout and existing wire fields. Exact isolated commit gates: cargo fmt --all --check, SSH all-target strict Clippy, complete SSH suite pass (raw schema-{clippy,tests}.log under evidence/addhost-start-fresh). Rejected automatic deletion and client-selected paths; ownership remains host-local.
Resuming host provisioning start-fresh on addhost-start-fresh at
5d52dfc3d8; assigned base6be5514d7f120380bb57b20e0b92ad7329dee0c4. Read current #169 body/all171 comments and prior state; no parent linked. Schema commit retained; uncommitted ownership shell, runner/API wiring and fixtures continue from prior sign-in interruption. Owner live fixtures remain untouched. Review ownership/liveness and same-attempt recovery, per-commit gates, VM-only check measurements, final fast gate, atomic commits and branch push; no deployment.Independent review resumed on review-addhost-fixes2 at58ed72a6, target6be5514d (delta9cdab5c7..6be5514d). Read continuation commits2528278d/58ed72a6 and current169 history; no linked parent. V1–V4 acceptance and functional counter-refusal evidence already committed. Earlier fast gate interrupted before summary; restarting unchanged gate with owned HOME-cache target/compact fixture root. No source fixes, push, deploy, perf VM or shared-host performance acceptance.
Ownership review found old lock contents were not bounded before recursive rollback. Added real-filesystem regression: foreign-lock-red.log fails (observed0, expected75). Recovery now accepts only proven fresh staging artifacts (transaction/kind/lease/payload), refuses advanced or unknown old lock layouts before mutation, and leaves foreign entries untouched. Older inode-unbound receipts are refused with a clear unsupported/changed-format reason. Existing diagnostics regression also exposed obsolete “Finish its cleanup” expectations; replacing those with retry-check/live-wait classifications. No owner fixture changes or new dependencies.
Resumed exact-target fast gate encountered unchanged daemon::readiness_tests::uninstall_handoff_runs_only_after_a_successful_real_socket_write at daemon_readiness_tests.rs218: result.is_ok=true vs healthy=false (agent55 pass/1 fail/11 ignored). File has zero delta9cdab5c7..6be5514d. Exact isolated unchanged artifact now1 pass/0 fail; no118 exemption claimed. Full gate will be repeated serially with original assertions after current static build. TypeScript fast exit0, source161/0 and protocol36/0/34462 assertions. No new confirmed delta finding.
Completed ownership lease prerequisite97ae5f1c. Directory flock serializes shell mutation; bounded180s lease holds the transaction across SSH command gaps and retires only after matching PID birth + exact inherited directory descriptor. Real between-command regression, complete SSH suite and strict Clippy/fmt PASS (lease-{tests,clippy}.log). Initial unchanged process group pipe test timed out under contention; whole suite replay passes with CPUWeight10000/RUST_TEST_THREADS4, no assertion change or exemption. Next independent slice: ownership/liveness classification and explicit recovery.
Further ownership review: a foreign root file survived, but recovery still removed a stale socket before ordinary rollback refused. Strict (c) requires refusal before any removal. New failing-before evidence foreign-base-red.log proves missing socket; classification now validates the complete root inventory first and refuses unjournaled entries unchanged. All12 real filesystem/shell cases pass (shell-final.log), including listener/journal changes after observation. Native proc-link traversal replaces per-descriptor subprocesses; fixed records only, bounded process table, no argv/environment inspection. No owner fixtures changed.
MERGE (no P0/P1). Current confirmed findings P0=0,P1=0,P2=0,P3=0. V1/V2/V3/V4 all PASS unchanged; no new delta finding. Scope9cdab5c7..6be5514d only. Local review-addhost-fixes2 headd9cbd38ae68d9ce8e0cfaa6db0db440513cf38d1; evidence/verdict and full verbatim logs/hashes in evidence/review-addhost-fixes2-delta/. Continued2528278d/58ed72a6; final evidence committed atomically atd9cbd38a.
Independent final scripts/verify fast exit1 solely inherited static cap; native679 passed/0 failed/68 ignored. Exact x86 stripped static PIE/no PT_INTERP20132112B (+24096 versus earlier reviewed artifact); ARM not reached, no new ARM delta. D205 size growth reported, no cap/profile/dependency change. Separate TypeScript fast exit0, source161/0, protocol36/0/34462 assertions, compiler/build/transfer PASS. Earlier unchanged uninstall-handoff failure retained; exact isolated1/0 and full serial test row PASS, no118 exemption.
Verbatim final gate summary:
Verdict: merge. No implementation fixes, push, deploy, perf VM or issue closure. Performance not independently remeasured; shared-host gate durations are not performance acceptance. Owned build/fixture scratch is being removed after retaining evidence.
Completed recovery module987ff272. Stale leftovers → typed observation → explicit exact-observation recovery; live listener/process/lock, foreign root/lock/socket contents and older unbound journals refuse before removal. Twelve real filesystem/shell regressions pass; full SSH suite, fmt and strict SSH/fuzz/server Clippy pass (recovery-*-final.log, shell-final.log). Decoder fuzz target updated/compiled; no instrumented fuzz acceptance claimed. Initial fuzz-only feature selection exposed inherited remote-without-ssh dead-code errors; normal remote+ssh union passes without source exemptions. Original rollback is reused, with fixed numeric diagnostics retained. Next: same-attempt runner/WebSocket action and public recovery logging.
Milestone fast failure is a fixture path, not a load exemption: daemon_gone_tests.rs104 produced108-byte Unix socket with TMPDIR=.../asf (SUN_LEN requires<108). Exact milestone test binary replay passes1/0 using newly owned compact .../af (107-byte path), gone-compact-replay.log. Final gate will use af; no source/assertion/deadline change. Owner fixtures untouched. Additional preflight regressions acquisition-red.log (mismatched acquisition + valid journal) and foreign-marker-red.log (unowned private root) fail before fixes; latest shell-final.log14/0 now refuses both before removal.
UI for start fresh pushed on addhost-start-fresh-ui:
834c75f3(leftovers + running screens, double, specs, ink captures) and9305932b(toast icon). Gates: bun run check clean, bun test src 162 pass, build + transfer-budget ok, new specs pass. Wire assumed for start fresh: {"action":"start-fresh","target":,"challenge":} on the open socket; failed event carries failure.kind stale_previous_attempt|setup_running and top-level leftovers {challenge, mode, findings}.Completed preflight45c6d316: mismatched acquisition receipts and unowned private roots now refuse before removal; failing-before evidence retained,14 shell cases/fmt/strict SSH Clippy PASS. Completed same-attempt action and logging at
436dfb97; SSH103/0 (8 opt-in ignored), remote bootstrap41/0 (3 opt-in ignored), strict touched-crate Clippy/fmt PASS. Tests prove one invitation/attempt, exact owner/target/challenge admission, replay rejection, off drains to0 with intact leftovers, and public-only journal/outcome logs. Contract/copy recorded ate5b7c02f. No UI changes; later pass uses typed leftovers. Final head gate follows milestone fixture-path correction; no test exemptions.Independent start-fresh verification started on local review-start-fresh at base
e5b7c02fb7. Scope: origin/addhost-fixes2..e5b7c02f backend and origin/addhost-fixes2..f60ed12f screen only. Read #169 history; no parent linked. Copied fixture tests only, owner live files remain read-only; no push/deploy. 35-minute review time box.Independent finding SF-1 P1: start fresh follows a symlink in an ancestor of the expected root. Real-shell regression test_review_symlink_ancestor_refuses_before_removing_leftovers uses local-alias -> external-local, with real share/infraternal directories and an exact inode-bound journal. Recovery returns 0 (expected refused 71) and removes the redirected directory and journal. setup_guard checks only dirname(base); Installation::script_for checks only the final base. Smallest fix: reject symlink ancestors of the fixed root before any inspection/removal (including rootless ~/.local), and retain the regression. Owner copied old shape otherwise passes after copy-only inode translation; owner originals untouched. HOLD pending SF-1.
SF-1 P1 failing real-shell regression committed at
0ce5757e2b, local review-start-fresh. Explicit run: 0 passed/1 failed, recovery returned0 instead of refused71. fmt and diff checks pass; Python-only test change, no Rust/TypeScript implementation change. Production-build functional UI checks pass3/3 on unique4379; source/copy tests37/37 and bun run check pass. Owner old-shape copy: inspect stale + recovery0; originals unchanged. Final fast gate is running; no performance claim from build-host durations.Verification matrix committed at
cfac58daf2: SSH install48/48 (including other-target/other-batch challenge refusal, single use, same-attempt completion and three personal-file byte/inode/mode sentinels); fmt and strict SSH all-target Clippy pass. Additional real-shell safety cases7/7 pass, including root uid marker refusal and real dead/live directory-flock holders. Narrow cp-a old-shape replay succeeds with copy-only inode translation; original inode/uid/mode/size/mtime/ctime fingerprints unchanged. Functional UI3/3, capture harness2/2, eight fresh production-build desktop/phone PNGs retained. SF-1 P1 remains, HOLD; no implementation fix.Evidence committed at1b53612afd61a85a1d42df6f40bd8e2a01041d06: report/checklist, eight fresh production-build PNGs and functional logs. Milestone fast gate: fmt/VT fmt/workspace Clippy/cargo-deny pass; agent gone fixture failed SUN_LEN at daemon_gone_tests.rs104 with the owned rsf TMPDIR. Stopped only the milestone scope during static compilation to start exact committed-head fast gate with compact rf TMPDIR (one character shorter); no source/assertion/deadline change or load exemption. Final output is being retained verbatim. HOLD still follows SF-1 P1 independently of gate status.
Final review source/evidence head1b53612a is committed locally; no screen implementation remains in the worktree. SF-1 P1 failing regression0ce5757e and passing matrixcfac58da are separate atomic commits. Full exact-head fast gate is running after the short fixture-root correction; native targeted ownership/confirmation tests48/48 and all additional functional proofs are complete. Report/checklist: evidence/review-start-fresh/README.md. HOLD; owner old shape supported, originals unchanged.
Completed liveness refinement
b52d57a1: bound sockets before listen and accepted connections after listener closure are live kernel entries, not stale files. Failing-before bound-socket-red.log reproduced0 vs73; all15 real-filesystem shell cases/fmt/strict SSH Clippy now PASS. Shared action summary now matches the approved primary “Start fresh” (D16/D202); existing model/action tests pass with the approved metadata expectation. Milestone static ELF is20175632B vs16404816B cap; supplied base6be5514d measured20132112B, delta43520B. D243 is not waived; final gate will report any inherited cap failure, with complete verbatim output. No deployment or owner fixture access.Final start-fresh gate output verbatim, part 1/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Final start-fresh gate output verbatim, part 2/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Final start-fresh gate output verbatim, part 3/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Final start-fresh gate output verbatim, part 4/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Final start-fresh gate output verbatim, part 5/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Final start-fresh gate output verbatim, part 6/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Final start-fresh gate output verbatim, part 7/7. Reviewed source head1b53612afd61a85a1d42df6f40bd8e2a01041d06; final evidence-only head46421f265d6dca97b611c64df99f317228dbef0e. scripts/verify fast exit1 (static cap). Raw functional transcript durations are not accepted performance measurements.
Verdict: HOLD — SF-1 P1 (P0=0,P1=1). Start fresh follows a symlink ancestor of the fixed installation root and removes redirected leftovers; it checks only the final base and immediate parent. Failing real-shell test0ce5757e returns0, expected refused71. Smallest fix: reject symlink ancestors (including rootless ~/.local) before inspection/removal; retain the regression. No implementation fix.
Final local review-start-fresh head46421f265d6dca97b611c64df99f317228dbef0e; no push/deploy/issue closure. Passing verification matrixcfac58da; report and fresh screenshots1b53612a; final gate evidence46421f26. Source unchanged after the exact-source gate; the final commit adds transcripts only.
Owner old shape: narrow cp-a copy has33B marker/85B sibling journal/one nested stale socket. Exact copy refuses75 because its inode differs; copy-only inode translation is recognised as leftovers and recovery succeeds, foreign sibling intact. Original inode/uid/mode/size/mtime/ctime fingerprints unchanged.
Real-shell additional ownership/liveness cases7/7; existing shell14/14 before SF-1. Rust install48/48 includes wrong/random/reused/other-target/other-batch challenges, same-attempt completion with one invitation and unchanged personal-file bytes/inodes/modes. Enrollment manager/payload/readiness backend are synthetic; no new full SSH enrollment claim. UI production-build functional3/3 and capture2/2; screen offers Start fresh only for leftovers, running state offers Try again; eight desktop/phone captures retained. Browser plugin unavailable; repository Playwright, unique4379/4383. TypeScript fast exit0:161 source tests,36 protocol tests/34462 assertions, compiler/build/transfer checks pass.
Final scripts/verify fast exit1: native689 passed/0 failed/68 ignored; all rows except static pass. x86 stripped static PIE/no PT_INTERP20175632B exceeds unchanged16404816B cap; ARM and dispatcher TS not reached. No gate waiver, budget/dependency/profile change or perf-host/shared-host timing acceptance. First milestone SUN_LEN fixture failure retained; compact-root final run passes it, no118 exemption. Complete195996B transcript quoted verbatim in preceding parts, SHA256cd0096102dfacfd284f013174e705e2195f41405607ec957968177a12da8cadf; evidence/review-start-fresh/verify-fast-final.log. Verbatim summary:
Report/checklist/reproduction: evidence/review-start-fresh/README.md and validation-command.txt. Owned fixture/build scratch cleanup follows; owner live state remains untouched.
Cleanup complete: owned review-start-fresh build/fixture scratch, compact rf/rsf fixture roots and gate-only target directory removed. Local review-start-fresh is clean at46421f265d6dca97b611c64df99f317228dbef0e. HOLD remains for SF-1 P1. Verbatim gate output/summary and final counts are in the preceding finish comment/7 transcript parts and committed evidence; no owner fixture, push, deploy or issue closure.
Orchestrator: start-fresh review (review-start-fresh
0ce5757e) HOLD on SF-1 (P1): recovery followed a symlinked directory below home. Fixed on release-prod-5 at5bf3f42e: every directory from / to the setup root must be real, except the home directory or one above it (keeps /home -> /var/home hosts working). The review test passes; all 15 previous_attempt shell cases pass; extra checks: a symlinked home is allowed, a symlinked .local below home is refused (exit 71). The owner real leftovers shape passed the review on a copy. The rest of the review (ownership, running setup refused, single-use challenge, setup continues, screen) passed.Source complete at
a83c5d91; evidence/checklist checkpoint recorded on branch head. Final-source booked-VM checker:101 paired samples, median addition4.438ms, no extra SSH roundtrip; stale31 samples p50=74.745ms/p99=89.545ms; lease1908KiB×8=15264KiB,180s ceiling. Separate real NetBird SSH command-gap assertions PASS: lease survives first-command closure and exact retirement succeeds. Raw logs/input hashes committed in evidence/addhost-start-fresh. Full TypeScript161/0 + protocol36/0 (34462 assertions), compiler/build/transfer PASS. Final exact-head scripts/verify fast now runs with compact af scratch, original VM RTT gate and unchanged D243 cap; branch push follows with any cap failure explicitly recorded.Orchestrator: DEPLOYED to production at 12:08 UTC (17:38 IST): release-prod-5 @
5bf3f42e= live4d1b1950+ add-host rounds 2-3 (#169, reviewed MERGE) + start fresh (server + screen; review SF-1 fixed in5bf3f42e) + CA key memory on static builds (#154, D244). sha256 prefixes x86 2f2b4a5a / arm 7cc6e2cd match on host; healthz 200, public 200, 0 errors in journal; VmLck 1024 kB (CA key memory locked under the 8 MiB limit). Restart re-locked the CA: owner unlock needed. Owner test: add infraternal-dev (leftovers untouched) -> expect "This host has leftovers from an earlier setup" -> Start fresh.Starting D250 production backmerge on dev-backmerge-prod5, base
ee104344d0= origin/dev (D215 already integrated). Read #169 history; no linked parent. Merge production5bf3f42ethen start-fresh84be6062separately; retain both histories, D243 caps, D244 static key-memory gate. Push only task branch; no deployment. Functional checks local, timing only on booked perf VM.Production merge has 8 conflicts, now resolved retaining signed-payload and readiness/Start fresh UI paths. Real ownership shell: 15/15 PASS including SF-1. Migration 0023 has no collision with dev 0020/0021/0022. Tracer first check exposed the new STEP_ADVICE map missing dev architecture steps; integration retains both labels. D244 diagnostic must enter dev’s existing action registry: propose additive hidden __key-memory-check action and KeyMemoryCheck runtime handler, compiled only with remote, preserving its imported JSON/result and low-memlock regression. Reject restoring production’s handwritten CLI enum or bypassing registry parsing. D16/D198/D202; D243 cap values unchanged.
Integration findings: imported key-memory CLI test failed 0/1 before registry wiring, passes 1/1 after (key-memory-before.log/native-prod5.log). Automatic merge omitted #[cfg(test)] on readiness_exit_tests; restored after compiler unused-import evidence. Strict Clippy caught dispatcher >100 lines; diagnostic execution moved into its own handler; Clippy replay PASS. Added real SQLite live-production history regression for applied migration 23 above the dev 20/21 gap. Browser first run:26 pass/14 fail, including 30s whole-test limits and later ECONNREFUSED ::1:4967 against the IPv4 proxy. Replay uses a job-only config with 127.0.0.1 plus 120s whole-test envelope; all original per-assertion limits/functional assertions retained. Initial product build ENOENT came from my concurrent builds to dist; serial replay pending. No timing evidence from functional durations, no gate exemption.
Native fixture findings: short scratch symlink bp5 was rejected by strict replacement ancestry and canonical-path assertions (native-prod5-2.log:61 pass/7 fail); replacing it with a real private directory clears CLI68/0. Gone-session rendezvous is still exactly108 bytes with the retained96-bit fixture name; shortening only the owned scratch root to b5 yields107. Private-readiness regression encountered dev’s mandatory terminal.replay_complete at out_seq0 before output; adapt the production fixture to assert that fence explicitly, retaining all EOF/credit/output/exit assertions. No relay behavior, UUID isolation, timeout or #118 waiver changed. Evidence native-prod5-3.log, native-prod5-4.log.
Production merge committed/pushed as
f2a4b36d28; parentsee104344and live5bf3f42e. Eight conflicts combine dev D215/CLI/signed-payload with production readiness/Start fresh/retirement/D244/ended strip. D243 size function unchanged. Gates: strict touched Clippy/fmt PASS; CLI68/0, shell15/0, install48/0, payload2/0, corrected migration8/0, shim79/0, compile-out25/0; browser40/0; TS176/0 + protocol36/0, check/build/transfer PASS. Evidence/reviewer checklist/captures: evidence/dev-backmerge-prod5. Earlier red transcripts retained. Later start-fresh84be6062 merge is staged cleanly; verify bound-socket refusal and SF-1 together before its separate commit. Final static/full head gate still pending; no deployment or dev/main update.Later start-fresh merge committed/pushed separately as62b3950aac6bd982fef47ad1ec39abf1d211052e; parents
f2a4b36dand84be6062.b52d57a1bound-socket refusal anda83c5d91shared label retained with production SF-1. Combined ownership shell16/16, strict SSH Clippy/fmt/all SSH tests PASS (explicit existing tokio/test-util for standalone paused-clock fixture; no dependency/runtime change). Both requested upstream heads are ancestors. Final scripts/verify fast running on62b3950a with owned real compact scratch,4 test threads, unique4967, original RTT probe delegated to208/shared lock. Both static targets/key-memory will still run if D243 size fails. No dev/main update or deployment.Source-head62b3950a full fast exit1 solely at D243: x8622438672B >16404816B; ARM19871968B >13817816B. Both stripped/static/no PT_INTERP. X86 shipping key initialization=1 under8388608B memlock; ARM runtime not checked on x86. Fmt/VT fmt/CLI compile-out/Clippy/tests/cargo-deny PASS; native958/0/116ignored, no118 exemption. TS dispatcher skipped after static failure; separate TS176/0 + protocol36/0/build/transfer and browser40/0 already pass on unchanged frontend. Full output and hashes saved under evidence/dev-backmerge-prod5. Both merges pushed; no size waiver or deployment. Final evidence-only commit and required branch-head replay follow.
Evidence correction committed/pushed6b978074abcb49a28aaeb397b5a943b6d85b9421: repository ignores .log, so earlier recursive staging omitted raw logs. Explicitly tracked only this job’s21 transcripts (8795 lines), leaving ignore rules unchanged. Final source remains62b3950a; diff to6b978074 is evidence only. The prior evidence-head replay was stopped only by owned process IDs; final scripts/verify fast restarted on6b978074. Source-head full output already tracked and hashed, all non-static rows PASS; final-head output will be quoted verbatim here. No source change, size waiver, dev/main update or deployment.
Final branch-head scripts/verify fast output verbatim, part 1/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 2/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 3/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 4/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 5/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 6/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 7/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 8/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 9/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 10/10. Head
6b978074ab; exit 1 solely at unchanged D243 static sizes. Complete output 259656 bytes, SHA256 28657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a. RTT is measured on VM208/shared lock; functional transcript durations are not performance evidence.Final branch-head separate TypeScript gate, verbatim. Head6b978074; exit0. The fast dispatcher stops after D243 static failure, so this explicit ts_verify_fast replay establishes the exact-head compiler/tests/build/transfer result. Functional durations are not performance evidence.
Finished source integration and pushed only dev-backmerge-prod5 at
6b978074ab. Separate merge commits:f2a4b36d(live production5bf3f42e into devee104344), then62b3950a (later start-fresh84be6062); both upstream tips remain ancestors. Evidence commits7e01cb2f/6b978074 change no source. No dev/main update, deployment or issue closure.All code/functional gates pass: native958/0/116ignored, strict workspace Clippy/fmt/VT fmt/compile-out/cargo-deny; ownership shell16/16; browser40/40 with22 captures; final-head TypeScript176/0 + protocol36/0/34462 assertions, compiler/build/checksums/transfer. No118 exemption. Captures are optimized browser-build fixtures, not physical iPhone proof or an author visual verdict. Author checklist: evidence/dev-backmerge-prod5/reviewer-checklist.md.
Final scripts/verify fast on6b978074 exits1 solely at unchanged D243 sizes: x8622438672B >16404816B; ARM19871968B >13817816B. Both stripped/static/no PT_INTERP. Native x86 key-memory initialization=1 under8388608B memlock; ARM runtime requires native ARM and is not claimed. Release profile/features/dependencies/caps unchanged. Needed from orchestrator: a size reduction preserving D243 before a green fast-forward; this branch is not represented as green.
Exact-head RTT: pinned infraternal-perf-test,4cores, load0.23/0.17/0.37, shared lock; original3-sample300ms probe PASS (p50=308.24ms,p99=310.35ms). No build-host performance numbers accepted. Full259656-byte final fast output is quoted verbatim in the preceding10 parts, SHA25628657059fa161cedefd974de05b7814f97185e649a0d88911a574091effd181a; exact-head TS output follows it. Committed source-head full transcript and hashes plus corrected raw regressions: evidence/dev-backmerge-prod5/. Local exact-head artifacts verify-fast-final.log/ts-final.log are also retained there; no further source commit follows the final gate.
How to recheck:
Verbatim final Rust gate summary:
Owned build/fixture scratch and VM input cleanup follow; the final cleanup comment confirms the tracked worktree state.
Cleanup complete: owned dev-backmerge-prod5 build/log scratch, real bp5/b5 fixture roots and VM input directory removed. Final gate scope is inactive/dead; no shared process or owner live fixture was touched. Branch remains pushed at
6b978074ab; tracked worktree clean. Exact-head fast/TS logs retained locally in evidence/dev-backmerge-prod5 and quoted verbatim above; source-head/raw regression logs/captures are committed. D243 remains unwaived: final fast exit1 only at both static size caps; no dev/main update, deployment or issue closure.Continuation started on dev-backmerge-prod5 at
6b978074ab, merging origin/dev53d7af1ba6. Preserve production add-host/Start fresh and round-4 signed replacement, anonymous running image, SSH prompt and bootstrap helper behavior/tests. D243 caps remain unchanged. Push branch only; no deploy or dev update. Previous #169 continuation evidence inspected; original issue has no parent.One textual conflict: crates/infraternal/src/restart_signature_audit_tests.rs was added independently. Retained all four prior admission cases, migrated positive admission to round-4 signed immutable fixtures, removed obsolete UP-01/02/03 ignores because round 4 fixes them, retained the new ignored optimized measurement and this branch's INFRATERNAL_TEST_STATE_ROOT override. No test deleted. bootstrap_runtime.rs retains both exact kernel payload resolution and /proc/self/exe SSH prompt materialization; production retirement/Start fresh hooks remain. D243 size checker unchanged from
6b978074; D244 exercises both static providers even after size failure. Source diff whitespace check passes; imported upstream verbatim logs have pre-existing whitespace, deliberately preserved. Functional native/TS gates running; RTT routed unchanged to VM208 under flock.Tracer compiler, 176 source tests, production build/checksums/transfer pass. Initial protocol run:35 pass/1 fail at stream.test.ts differential 30000ms timeout (ts.log364-365); Rust builds were concurrently using the same target directory. Retain this transcript and rerun unchanged 36-test protocol suite serially after native compilation; no timeout/assertion change or #118 exemption. Ownership shell16/16 passes including SF-1 and bound-socket refusal (ownership-shell.log). Native touched-crate strict Clippy/fmt pass (native.log283); tests are compiling.
Completed functional production-build browser replay:40/40 pass on unique4977 (browser.log), retaining all add-host, Start fresh and ended-strip assertions. Captures retained under evidence/dev-backmerge-prod5/dev-continuation/captures; generated historical captures restored without changing source/index. Optimized browser build with provisioning test doubles, not physical iPhone or an author visual-quality verdict. Native CLI75/0 includes activated signature/inode/refusal regressions and same-PID/PATH recovery; SSH askpass2/0, real abrupt-stop24-cycle and vt_credit gates pass so far. RTT setup passes on locked VM208,4cores, load0.28/0.56/0.84; exact unchanged probe SHA119b87ef39ee7022c76dbc641a1428dd5eae7254265e6ddeea6697b56a4a8339. Full native/final gate pending; no size waiver.
Unchanged full TypeScript replay now passes (ts-serial.log, exit0):176 tracer source tests,36 protocol tests/34462 assertions;134 streams/11439 schedules/zero divergences; compiler/checksums/build/transfer pass. Critical gzip295019/307200B and Brotli245740/256000B. Prior30s differential timeout retained without assertion/deadline changes. Browser40/40 and ownership shell16/16 pass. Real sealed-anonymous infraternal-agent PTY oracle also passes open/input/output/control-fd isolation (anonymous-pty.log). Native server suite continues with no failures so far.
Merge committed/pushed
5a4414a8c8; parents6b978074and53d7af1b. origin/dev53d7af1bis an ancestor, so this result permits its fast-forward. Every conflict test retained; signed fixtures and isolated scratch root combined.Touched strict Clippy/fmt/native gates exit 0: reported 861 pass / 0 fail / 86 ignored across 61 rows (CLI 75, server 413, shim 80); SSH/restart/release suites pass. Ownership 16/16, browser 40/40, TS 176 + protocol 36 / 34,462 assertions pass; anonymous real PTY passes. Evidence-only commit follows; final exact-head fast/static gate then records the unchanged D243 failure if present. No dev/main update or deployment.
Evidence committed/pushed separately at
873ae89609. No source changes after merge5a4414a8. Native/TS/browser/ownership/anonymous-PTY transcripts, 26 capture artifacts, hashes and author checklist are committed under evidence/dev-backmerge-prod5/dev-continuation. bash -n passes for all continuation gate scripts. Final scripts/verify fast is running on this pushed head with a private real compact fixture root, unique 4977, four test threads and unchanged RTT probe delegated to locked VM208. D243 caps remain unchanged; both static architectures/key-memory checks follow native workspace gates.Remote dev moved again to
13f1617cduring final verification, only DECISIONS.md (+28 lines: D257-D259) and TASTE (+1 owner rule). Read those additions and merged them verbatim in separate mergef82da26629, pushed branch only. Both current dev13f1617cand original requested53d7af1bare ancestors. Source suites/caps unchanged; fmt and staged whitespace checks pass.Stopped only the owned older-head gate PIDs 785289/785260/782223/782190 and retained its interrupted transcript. Final scripts/verify fast restarted on
f82da266; no earlier interrupted run is claimed as final evidence. The source remains the tested merge5a4414a8. No dev/main update or deployment.First full gate on
f82da266: fmt, compile-out and workspace Clippy pass. cargo test stops at agent_queues initial terminal.open refusal: resource_unavailable, before saturation assertions (verify-fast-final.log204-232). The earlier complete targeted suite passed this test; no cause or #118 exemption claimed. Retain the failure and rerun the exact compiled artifact unchanged, then rerun the complete workspace gate on the reconciled final head. Static builds continue in this milestone despite the test row failure. Current dev added only D260 (87444825); reconcile after this cold-build milestone, with no source/size-cap change.Disk-full continuation resumed on dev-backmerge-prod5 at
f82da26629. git status is clean; no tracked partial files to restore; 168 GiB available. Read #169 body/comments and existing continuation evidence; no parent is linked. Prior interrupted gate retained (disk-full static compilation plus unresolved agent_queues initial-open failure); rerun unchanged test and full final head gate. Current origin/devd100cfe5adds only D260/D261 after already merged13f1617c; merge these owner decisions verbatim to keep fast-forward ancestry. Reuse one owned26GiB target, no dev/main update or deployment, push branch only; D243 caps unchanged.Completed/pushed merge5b1e8b70; parents
f82da266andd100cfe5. Current origin/dev and both requested upstream heads are ancestors. Incoming diff is only56 DECISIONS.md lines (D260/D261), retained verbatim. fmt/staged whitespace PASS. Tracked status clean; old interrupted final gate copied to verify-fast-disk-full.log. Functional browser40-case replay running on4977; unchanged queue saturation regression rebuilding disk-interrupted dependencies in the one existing target. No source or D243 cap change, deployment or dev update.Continuation verification: ownership shell16/16PASS (ownership-resume.log). VM208 reachability confirmed (perf-access-resume.log); RTT waiting on shared lock held byPID658049, no trust/credential changes. Started Rust-only milestone while exact-head scripts/verify fast waits on its unchanged VM precheck. Functional browser first failure: server-key case expects w-web-01 within unchanged5000ms poll; current window list lacks it (apps/tracer/test-results/.../error-context.md); retain run and replay unchanged after complete40-case suite. Native queue replay rebuilt CLI after disk interruption and is running. No failure exemption/source fix claimed.
Queue replay PASS: queues-resume.log reports1pass/0fail with unchanged saturation/byte-preservation/daemon-liveness assertions. Rust milestone fmt/nativefmt/compile-out/Clippy pass; CLI suite74pass/1fail at cold_exec closed-stdin helper's existing8s total deadline (remaining1.819989906s, restart_tests.rs150); retain evidence, replay exact compiled artifact unchanged under CPUWeight10000. No118 exception applies or is claimed. Cargo-deny passes; static compilation continues. Browser failures now two existing5000ms polls; complete/replay pending. No source change.
Cold-exec restart exact-artifact replay PASS1/0 (restart-resume-replay.log), unchanged source/assertions/deadlines, CPUWeight10000. Earlier CLI timeout retained; no118 exemption or cause beyond expired existing total deadline claimed. Replaced the unused waiting RTT/full-gate attempt with serial native RUST_TEST_THREADS1 in an owned CPUWeight10000 scope (final-resume-gate.sh); final exact-head transcript remains verify-fast-final.log. Only the owned waiting netbirdPID1050391 stopped. Rust milestone continues both static targets. No budget change or build-host timing evidence.
Browser resume first full run:37pass/3fail, unique4977, unchanged assertions/whole-case120s harness. Failures: server-key window poll5000ms, pasted-key poll5000ms, no-server-key missing choice field15000ms. Retained browser-resume.log and three error-context receipts under dev-continuation/resume-browser-errors; no cause/exemption claimed. Earlier identical-source continuation run40/40 remains recorded. All signed-payload, Start fresh and ended-strip cases pass in this run. Tracer full TS check/test/build/budget now running after browser server exit to avoid dist races; follow with unchanged full browser replay under CPUWeight10000. Final native fast continues clean; VM original RTT PASS at load2.01/1.53/0.86. No source/assertion/budget changes.
Completed exact-head ts_verify_fast replay on5b1e8b70, exit0 (ts-resume.log): compiler/checksums/build/budget PASS;176tracer tests/0fail;36protocol tests/34462assertions;134streams/11439schedules/zero divergences. Critical gzip295001/307200B,Brotli245729/256000B; deferred49751/51200B,43942/45056B; largest on-demand15854/20480B,14110/18432B. Functional durations are not performance evidence. Full40-case unchanged browser replay now in owned CPUWeight10000 scope on4977 after TS build completed; first37/3failure transcript retained. Final native gate still clean; static builds continue. No source/budget changes.
Final exact-head fast has completed all native rows clean: fmt/VTfmt/CLIcompile-out/workspaceClippy/test/cargo-deny PASS, no118 exemption. CLI75/0, real daemon24-stop replay, queue saturation, SSH/restart/release, add-host/retirement/storage/migration and VT corpus retained/passing. Static row now waits for the already resumed build in the same target (no duplicated target). Weighted browser replay first formerly failing server-key case now PASS unchanged; remaining cases running. TS exact-head exit0 already recorded. No source/budget changes.
Final native totals995pass/0fail/115ignored across95test-result rows, no118 exemption. X86 static build succeeds:22464880B, stripped/static,noPT_INTERP; D243FAIL >16404816B unchanged cap. Shipping __key-memory-check PASS initialization=1 under8388608B memlock; locking/dump exclusion enabled. ARM builds continue despite x86 size refusal. Weighted browser formerly failing server-key/pasted-key/no-server-key cases allPASS unchanged (cases6/9/14); full replay continues. D243 checker SHA335990c3aa0ced17260a0aedba81f060d54271ba89cadf8e69fc5b7a4dcd3104 identical to6b978074. No source/profile/dependency/cap changes or deployment.
Exact-head ts verification on
5b1e8b70c7, exit0, verbatim. Raw log evidence/dev-backmerge-prod5/dev-continuation/ts-resume.log; SHA256 95fa66724638370459c766cea7b553e153e4c5430f5c6a454eb567216e966d1b. Functional durations are not performance evidence.Exact-head browser verification on
5b1e8b70c7, exit0, verbatim. Raw log evidence/dev-backmerge-prod5/dev-continuation/browser-resume-weighted.log; SHA256 baf18313730ae3e3205b3fae9abb155a6ad5cb4175104b5340f75d2a7cb09bbf. Functional durations are not performance evidence. 40/40 unchanged assertions pass under CPUWeight10000 on unique4977; initial37/3 log retained without source/deadline changes.Dev advanced during ARM build to92e17ea7 (#196 owner close-toast hints). Need current-dev ancestry for D250; staged conflict-free merge into5b1e8b70 retains all11incoming files verbatim alongside production add-host/ended-strip. Read diff/TASTE; no hand-built UI/style/library added. Prior exact-head5b fast native995/0/115ignored and x86key-memory/size receipts preserved; old gate stopped only by owned scope PIDs, transcript verify-fast-5b1-interrupted-dev-advance.log. Rust milestone ARM continues only as a build-cache milestone; do not claim it as final-head evidence. Run hints unit tests, tracerfull gate,3new functional close-toast cases plus40existing add-host/ended-strip cases, then final fast on committed merge head. No timing browser spec on build host, no source assertion/cap relaxation, branch push only.
Reconciliation committed/pushed2f94d955 (parents5b1e8b70/92e17ea7), all11owner files retained verbatim; origin/dev92e17ea7 is ancestor. Separate evidence commit/push
efac27a5retains four hint tests, full tracer/protocol/build/budget receipts and3functional toast cases/capture; fmt/whitespace PASS. New critical gzip295237/307200B,Brotli245642/256000B; deferred49794/51200B,43966/45056B. No source change after2f94d955. Final scripts/verify fast restarted onefac27a5(owned CPUWeight10000, RUST_TEST_THREADS1, unique4977, unchanged RTT delegated toVM208/shared flock). Repeat40inherited functional browser cases on this head in a separate owned scope. Prior5b gate explicitly interrupted by dev advance; no source/test/cap waiver, dev update or deployment.Final head
efac27a5full fast is running: original VM208 RTT probe PASS (p50=321.21ms,p99=324.88ms,3samples,load0.04/0.71/1.33,4cores/shared lock); CLI75/0 and24-stop daemon replay PASS so far. Both static build caches finished in the earlier milestone; its initial CLI timeout stays recorded as a failed test row and is not final evidence. Final-head fast verifies both static artifacts independently after all workspace tests. Current final-browser inherited40-case replay starts clean;3newtoast cases already committed/passing. No source/profile/cap changes, deployment or dev update.Final branch-head scripts/verify fast output verbatim, part 1/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 2/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 3/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 4/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 5/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 6/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 7/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 8/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 9/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 10/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final branch-head scripts/verify fast output verbatim, part 11/11. Head
efac27a598; exit1 solely at unchanged D243 static sizes. Complete output 260294 bytes, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. RTT is measured on VM208/shared lock; functional durations are not performance evidence.Final exact-head tracer/hints/protocol/build/budget output verbatim. Head
efac27a598; exit0. SHA256 012d7642bd69b0efaf42fb6028985d37cd0560506048e140a148ee5d9f759e19. Functional durations are not performance evidence.Final exact-head 40-case inherited functional browser output verbatim. Head
efac27a598; exit0. SHA256 dc2b11af74a9378e2b3166d4b4983e7bab070cf8b1218cf0789a08541f72a4f4. Functional durations are not performance evidence.Integration finished on dev-backmerge-prod5, pushed head
efac27a598. Current origin/dev92e17ea7 and both requested upstreams5bf3f42e/84be6062 are ancestors; only this branch was pushed. Merge2f94d955 retains the owner’s later #196 files verbatim, with separate evidenceefac27a5. No dev/main update, deployment or issue closure.Final checks: native995pass/0fail/115ignored; workspacefmt/VTfmt/compile-out/Clippy/cargo-denyPASS; ownership shell16/16; inherited functional browser40/40; newtoast cases3/3; hints4/4; tracer176/0; protocol36/0 with34462assertions,134streams/11439schedules/zero divergences; compiler/checksums/build/transferPASS. No118 exemption or source/deadline weakening. Earlier disk-full, CLI-timeout and37/3browser transcripts retained; unchanged final replays pass.
Final scripts/verify fast exits1 solely at unwaived D243: x8622464880B >16404816B, ARM19898568B >13817816B. Both stripped/static/noPT_INTERP. X86 shipping key-memory initialization=1 under8388608B memlock, locked/dump-excluded; ARM runtime not checked on x86. Size checker SHA335990c3aa0ced17260a0aedba81f060d54271ba89cadf8e69fc5b7a4dcd3104 unchanged from6b978074. No profile/dependency/cap change. Integration ancestry is ready; the size-reduction lane/orchestrator still needs to make D243 green.
Full260294-byte final fast output quoted verbatim in preceding11parts; SHA2563b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. Exact-head TS/browser outputs are also quoted. Final VM208 RTT: unchanged3-sample probePASS, p50=321.21ms,p99=324.88ms,4cores,load0.04/0.71/1.33,shared lock. No build-host performance claims.
Author checklist rerun: all8/8 and9/9functions from both replacement-audit parents retained; kernel image/SSH prompt/readiness/Start fresh/retirement behaviors and test sets coexist; D243/D244 unchanged; incoming owner toast reuses shared toast. Evidence/capture: evidence/dev-backmerge-prod5/dev-continuation/dev-ui-advance.md and captures/close-toast/reopen-desktop.png. Optimized browser build with synthetic canvas fixtures; no author visual-quality or real-iPhone claim.
Reproduce:
efac27a5with your own HOME-cache CARGO_TARGET_DIR and short real private INFRATERNAL_TEST_STATE_ROOT/TMPDIR; use umask022.Verbatim final Rust gate summary:
Owned scratch cleanup follows; owner live fixtures were never touched.
Cleanup and handoff complete. Pushed branch dev-backmerge-prod5 remains at
efac27a598; remote dev92e17ea7and both requested upstream heads remain ancestors. Worktree clean. Final gate scopes are inactive; owned 30 GiB target, b6 fixture scratch and VM208 job inputs removed. Only fixture-owned shim PIDs authenticated through the private rendezvous were stopped. No owner live fixture or shared process was touched.Final result remains explicit: 995 native tests pass, ownership 16/16, inherited browser 40/40, new toast 3/3, hint units 4/4, tracer 176/0 and protocol 36/0; compiler/build/transfer budgets pass. Final scripts/verify fast exits 1 solely at unchanged D243 sizes: x86 22,464,880 > 16,404,816 bytes; ARM 19,898,568 > 13,817,816 bytes. X86 key-memory locking/dump exclusion passes. Complete output is quoted verbatim above, SHA256 3b3aeea4c69790d890444ecb9b66693f02b4ba954a8ebc91129196f482da5387. No size waiver, dev/main update, deployment or issue closure.
kayg referenced this issue2026-10-28 17:53:54 +00:00
kayg referenced this issue2026-10-28 17:53:57 +00:00