Phone on LTE: liveness timeouts, then resume refused 33 times a minute (prod) #244
Open
opened 2026-10-18 10:21:41 +00:00 by kayg
·
127 comments
No Branch/Tag specified
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/infraternal#244
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Production (release 3,
5b2a1b9ee), owner's iPhone PWA on LTE, 2026-10-18 ~15:44–15:50 IST. Toasts: "The connection did not respond in time." then "The server would not continue the previous connection." Server log (close reasons from #227):So a phone on LTE hit repeated liveness timeouts (D300 deadline) and then a reconnect loop the server refused 33 times a minute. Expected: a mobile link that stalls is doubted, then resumed (D300); a refused resume falls back to a fresh attach once, never loops.
Starting #244 on resume-refused-loop, base/head
5b2a1b9ee3(release-prod-live). Issue has no comments or parent link. Reading browser/server probe and resume paths; will add failing reproductions before fixes. No push or deploy, per brief. Evidence/builds use job scratch directory.Findings at base
5b2a1b9ee:Contract proposal (#244/D300): additive control Event sub_id=0, JSON {"topic":"connection.visibility","hidden":boolean}, enabled by Hello capability bit 2. Hidden pauses browser lease probes without retiring socket/writers or changing auth checks; visible restarts unknown-RTT sampling, rejecting old correlations. Foreground browser routes use D300's two-RTT allowance and first-miss doubt, second-miss retirement. Explicit resume refusal wipes the failed capability and schedules one fresh connection through existing growing backoff; a refused fresh handshake does not repeat this fallback. Generic person-facing wording explains automatic reconnection. No new library or engine.
Step 1 committed:
a839a9ac6. Explicit resume refusal now wipes the old capability before scheduling a fresh handshake through existing backoff. Repeated fresh failures retain exponential delay. Regression failed before ([false,true,true,true]) and passes after ([false,true,false,false]); 17 connection/close-wording tests pass, tracer TypeScript check passes. Refusal wording now says the previous connection could not be restored and terminals reconnect automatically. No push/deploy per brief.Visibility regression reproduces both 5s and 30s suspension: old code disconnects all 28 streams on one socket. Lifecycle implementation in progress; building native tests in the job target directory through build-slot.
Contract step 2 committed: visibility transport notice and suspension behavior in contracts/wire.md; additive Hello bit 2 / existing Event layout, no new parser frame or dependency. Commit bisection retains old behavior until both implementations land. git diff --check passed. Contract proposal recorded in previous finding before implementation.
Further reproduction: after settling at 150ms RTT, replies varying through 150/300/600/450ms (one additional RTT on every 100th reply to model ordered retransmission delay) retire the original foreground browser lease. Visibility support alone does not fix it. With the proposed D300 two-RTT allowance and first-miss doubt on both browser peers, all 47 connection tests pass, including 28-stream hidden/foreground cases and bounded dead-peer detection. This deterministic model is not packet-level loss evidence.
Raw evidence: production journal excerpt, failing refusal regression, failing hidden-page regressions.
Contract commit
08f44356bdefines the visibility notice. Focused native liveness run passed 33 tests. The first scripts/verify iterate stopped before compiler checks in the existing latency-proxy self-test (configured 300ms, reported p50=311.42ms p99=423.16ms); no passing gate is claimed. Focused native lint/tests are running through build-slot, followed by final full gate and real-browser recovery checks.Perf-VM deterministic comparison, 60,000ms modeled per case, one socket / 28 streams, 150–600ms reply delays, every 100th reply held for one additional RTT (retransmission model; NOT packet-level loss). Same harness and baseline
5b2a1b9eevs current implementation:Machine infraternal-perf-test, x86_64, Linux 6.12.111, load 0.08/0.02/0.01; /var/lock/perf-bench.lock held only during the short run. Receipt: model output, model source, failing foreground regression. Counts are not presented as real-phone measurements. Production browser/PTY check and final native checks remain pending (build-slot queue).
Lifecycle implementation committed:
56dff47e6(contract steps08f44356band2b11b5f9b). Both peers advertise visibility capability; legacy peers receive no unknown transport notice. Hidden leases park; foreground relearns RTT without old correlations; explicit dropped hidden sockets wait until visible. Foreground browser routes use D300 two-RTT estimation and recover through a first missed allowance. Authentication expiry/revocation remain on the original independent path.Per-commit checks: fmt, server all-target clippy, 34 native liveness tests, 5 transport tests, tracer TypeScript check, 49 browser connection tests passed. Native lint/tests.
At 224 modeled streams (8 sockets × 28), same perf-VM profile: foreground closes 93 -> 1 / modeled minute; 30s suspension case 67 -> 1. Every replacement recovered; all eight final routes welcomed. One combined RTT step/retransmission still exceeds the bounded allowance at this scale; it does not become a retry loop. These are deterministic browser-side model counts, not packet-level loss or physical-phone results. Scale receipt.
Next: fixed safe resume check diagnostics/tests, production-browser functional proof, final full static gate and size comparison. No push/deploy.
Diagnostics committed:
3c29e3d96. Server logs fixedconnection_resume_refusedcheck categories (connection_active, connection_owner, connection_missing, resume_expired, token_mismatch, capacity/rotation/session-authority stages), never credentials or arbitrary error text. The original admission query, writer lock and constant-time token comparison remain; the extra state query runs only after refusal.Native tests reproduce an overlapping active socket, a wrong owner, a missing row, expiry, an incorrect token, and a committed rotation whose Welcome was lost. Every refusal remains strict and is now distinguishable. fmt and all-target server clippy passed; 9 writer/authority tests passed, 1 existing release scale measurement ignored. Verbatim check output.
The historical production refusal cannot be assigned to one category from its existing log. Same process PID rules out a restart during the observed interval; no epoch or terminal-stream ID is checked in connection resume. Static baseline/head comparison is building; full final gate and production browser proof are next. No push/deploy.
Self-review found two background retry boundary cases: an online event could bypass the hidden-page park, and a retry timer armed just before hiding could still open a socket. Both reproduced as 2 sockets where 1 was required. Fixed in
85ce1c1f8: ignore online hints while hidden and cancel pending retry timers on hide, retaining the visibility listener for immediate foreground recovery.Tracer TypeScript check and all 50 browser connection tests pass. Failing regressions, passing regressions. No new dependencies, UI components, auth relaxations or deployment.
Static comparison build is progressing; baseline x86_64 release build completed and branch build is running with the same target/toolchain recipe. Next is a real TCP loss/jitter check inside private perf-VM network/mount/PID namespaces, plus the real-browser fresh attach regression and final scripts/verify fast.
Same-recipe stripped static x86_64 comparison complete: merge base
5b2a1b9ee= 21,096,144 B (exactly the shipping ceiling); current native implementation = 21,100,176 B, delta +4,032 B (+3.94 KiB). Both built with scripts/static-cargo, locked dependencies, Rust 1.99.0 and the same job target directory/compiler wrapper.D243 justification/proposal before adjusting the exact ceiling: the 4,032 B adds the negotiated bounded visibility parser/lease scheduling and fixed secret-free resume diagnostics needed for #244. Existing Event codec/serde/runtime are reused; no crate or feature additions. This is below the brief's 16 KiB threshold. ARM measurement and full final gate remain pending; no ceiling is loosened beyond measured bytes and DECISIONS.md stays unchanged.
Real TCP netem proof is running on the perf VM with 75ms ±40ms per loopback hop, normal jitter, 0.5% packet loss and seed 244. Private mount/network/PID namespaces isolate /run, /var/log and generated fixture account files. The first attempt did not create a terminal; the fixture was aligned with the existing CA-unlock setup before taking measurements. No production state changed.
Real-browser test step committed:
728284e33. Production static server + production browser + real passkey/CA + retained PTY. Hidden CDP freezes of 5s/30s keep the socket. Removing only fixture connection state, then restarting, produces exactly one explicit refusal and one fresh Hello; an executedecho mobile-resumed-$((6*7))returnsmobile-resumed-42. TCP upgrades that fail while the server restarts are counted separately from fresh attach. Strict real-spec TypeScript check and tracer check passed.Actual TCP netem comparison on infraternal-perf-test (kernel 6.12.111, quiet load 0.00/0.00/0.00 baseline, 0.11/0.04/0.01 branch): same seed 244, 75ms ±40ms normal delay per loopback hop (TLS front and backend), 0.5% packet loss, one mobile 393×852 viewer, real PTY. Same 25s visible + 5s/30s frozen observation:
5b2a1b9ee: 2 socket closes in 62,058ms = 1.9337 closes/min; 86/17,751 packets dropped;The whole-socket
closedcount is used. The observer's extraserver_closesfield is unused: its tag comparison does not match this fixture'sserver <code>format. It does not identify which peer closed first. Initial_sockets=1 on both runs confirms no setup closes entered these counts.Paired netem receipt also contains the first functional assertion attempt; that assertion incorrectly omitted the bridge's
s0reason prefix. Corrected test then passed with actual packet loss (20 dropped): functional receipt.Namespace/profile driver, paired observer source, same-recipe x86 size comparison.
Production-build screenshot for owner review (no visual-quality claim):
ARM build and final scripts/verify fast remain pending. No push/deploy.
ARM static build complete: aarch64-unknown-linux-musl is 19,328,840 B, +3,320 B over the recorded release-3 ceiling (19,325,520 B). x86_64 is 21,100,176 B, +4,032 B against a same-recipe rebuild of merge base
5b2a1b9ee(21,096,144 B). file identifies both providers as static and stripped. No dependency or feature changes.Under D243, propose exact ceilings 21,100,176 / 19,328,840 B for #244's bounded visibility handling and fixed refusal diagnostics; no spare growth allowance. Both deltas are below 16 KiB and justified by the real TCP false-close reduction 1.9337/min -> 0/min. Updating the two exact values and their process documentation, then running scripts/verify fast on the committed head.
Full gate milestone on
7c654f8ccreached PASS for fmt, VT fmt, CLI plugin compile-out, clippy, cargo-deny and both static providers (exact new sizes and key-heap checks). The Rust test row failed in an existing gate fixture: previous_attempt_shell.py unconditionally reads INFRATERNAL_TEST_RUNTIME_HELPER, but previous_attempt_shell.rs does not set it. 32 cases fail with KeyError before exercising their oracle. The helper binary already exists at target/debug/examples/recovery-helper; Rust SSH fixtures already resolve it through tests/common::runtime_helper().Fixing this wrapper to reuse that resolver and the existing shared scratch helper, instead of requiring an undocumented caller environment. No test is skipped and no shipping code/dependency changes. Final head also includes
e42294042: two test assertions now compare counts so failure reports cannot serialize fixture cookies. Synthetic marker reproduction and strict TypeScript checks passed; real-engine success predicates remain identical. Assertion-reporting proof.Perf cleanup is complete: no process held this job's executable/runner artifacts before removing only its VM folder. Cleanup receipt. Full gate will be rerun on the final committed head after the fixture correction.
Gate fixture correction committed:
997bf36868. previous_attempt_shell.rs now passes tests/common::runtime_helper() and common::scratch() to its Python oracle. Explicit caller overrides remain supported. fmt, SSH all-target clippy, helper example build and the wrapper's 34-case Python oracle pass. No shipping code or size changes.Final scripts/verify fast is running on this committed head with CARGO_INCREMENTAL=0, CARGO_BUILD_JOBS=2 and the job's single target directory. No external runtime-helper override and no skipped test. The earlier milestone failure is retained as raw evidence; its only failed Rust row was the missing fixture environment, while both exact static size/key-heap rows passed. Final full output will replace that milestone as the completion gate.
The
997bf3686gate proceeded past the corrected Python oracle, then three existing recovery_support cases failed before their checks because absolute AF_UNIX names exceeded sun_path (the newline-home case explicitly requires a short INFRATERNAL_TEST_STATE_ROOT). No production change caused this: the fixture's home/cache prefix plus .local/share/infraternal/sessions/32-hex.sock is too long.Configured /run/user/1001/rrl244 as a short, job-owned runtime alias to /home/kayg/.cache/codex-scratch/resume-refused-loop/s. All data remains in this job's scratch directory; the alias is removed at cleanup. recovery_support now passes 11 tests with its one existing Docker-only ignore. No deadline, case or expectation was relaxed. Final full gate is rerunning on unchanged head
997bf36868with this documented fixture root.Gate follow-up on
997bf36868: recovery_support had 3 existing failures because its absolute session socket name is 117 bytes before the newline variant (Linux sun_path is 108 bytes). A global short symlink override made those 11 tests pass but correctly failed 7 canonical replacement-path checks elsewhere, so that override is removed. Narrow fixture correction: use XDG_RUNTIME_DIR for these ephemeral socket fixtures when available, retain the explicit state-root override and HOME-cache fallback. Build artifacts remain in job scratch; production code and canonical-path checks stay unchanged. Running focused fmt/clippy/recovery tests before committing, then full fast again. No tests skipped.Step complete:
4a64e2cc2limits the recovery fixture correction to ephemeral socket paths. cargo fmt --all --check and all-target SSH clippy pass; recovery_support: 11 passed, 0 failed, 1 pre-existing container-only ignore. Global symlink override removed. Full scripts/verify fast is running on this head with the normal canonical state roots.Author review on
4a64e2cc2: additive Hello capability gates the <=128-byte typed visibility Event; existing frame codec and JSON parser are reused, with no dependencies/features added. Visibility only parks probe deadlines: authentication/revocation, writer ownership, attachment/memory caps and socket limits remain enforced. Correlation IDs remain monotonic and old replies cannot renew a visible lease after suspension. Foreground proven-route retirement is bounded at 500 ms–4 s; initial unknown-RTT grace remains 2 s. Browser listeners/timers retire with the connection; pending retries and online hints remain parked while hidden. Refusal diagnostics contain fixed categories only; original eligibility query, writer transaction and constant-time token check remain authoritative. The fresh-attach path wipes refused capability buffers and retains ordinary exponential retry backoff. Real PTY marker is computed by the shell, so echoed input cannot satisfy the assertion. Failure formatting uses counts instead of bridge objects that contain fixture cookies. Existing auth/socket/path guards were not weakened. Screenshot is attached for independent visual review; this is Chromium mobile emulation, not a physical iPhone claim. Historical exact resume refusal check remains unknowable from the old aggregate logs.Full fast on
4a64e2cc2reached SSH install tests, where review_inode_reuse_after_diagnostics_preserves_live_replacement could not reproduce filesystem inode reuse in 256 allocations (93 passed, 1 failed, 1 pre-existing ignore). It failed during fixture preparation, not the preservation assertion. Immediate exact-test rerun on unchanged head passed, reaching the actual safety assertion (1 passed, 94 filtered). No waiver, test skip or code change: rerunning the full gate on the same head. Prior global alias is gone; recovery_support passes with its narrow real-runtime fixture path.fast-inode-allocation.log
inode-recheck.log
recovery-runtime-root.log
Full fast on
4a64e2cc2: every Rust gate row PASS, including the unchanged inode-reuse fixture. Frontend combined suite then found 6 failures at MuxConnection construction: earlier canvas fixtures leave a partial document without addEventListener. Focused connection suites alone had not exposed that shared-run environment. Narrow correction guards visibility listener registration/removal by method availability, consistent with existing optional browser APIs; real-browser behavior is unchanged. Keeping the failing combined transcript; rerunning the exact frontend test set and check before an atomic commit and final full gate.Step complete:
526460c2eadds only availability guards for visibility listener registration/removal in non-browser consumers. bun run check and the exact combined frontend gate suite pass: 256 pass, 0 fail, 11712 assertions. Existing six failures are now green. No production browser behavior or Rust bytes changed. scripts/verify fast is running on this new committed head; all Rust rows passed on the immediate prior head.fast-partial-document.log
partial-document-fixed.log
Finished implementation on resume-refused-loop. Head:
526460c2e6. Atomic commits retained; clean worktree. No push or deployment.Findings and behavior: browser routes used a tighter one-RTT allowance and first-miss retirement, and had no hidden-page suspension notice. Negotiated bounded visibility notices now park browser probes while hidden and relearn on visibility. Proven foreground browser routes use the D300 two-RTT/jitter allowance and first-miss doubt, with retirement bounded to 500 ms–4 s; startup grace remains 2 s. Authentication/revocation and resource caps remain enforced. Hidden retries/online hints park too.
Refused resume credentials previously survived until their 120 s expiry and were retried. They are now wiped at refusal; the next handshake attaches afresh once, retaining exponential backoff. Copy explains automatic terminal reconnection. Historical logs do not identify the exact refusal check, and cannot establish a token/ownership/expiry cause. Same PID excludes a restart within the observed interval; epoch/stream ID are not connection-resume checks. New fixed-category, secret-free diagnostics distinguish the checks for future incidents. Production close counts were terminal-stream closes, not necessarily distinct socket closes.
Real TCP netem on the locked perf VM (75 ms ±40 ms per hop, normal jitter, 0.5% loss, seed 244; 5 s and 30 s renderer freezes): before 2 socket closes/62.058 s = 1.9337/min; after 0/62.087 s = 0/min. Packet drops were 86/17751 before and 41/9134 after. Separate real browser/passkey/PTY functional proof passes with actual packet loss: one refusal, one fresh Hello, computed shell marker received after recovery. These are Chromium mobile emulation results, not a physical iPhone claim. At scale, the deterministic 8-socket/224-stream model reduced jitter closes 93→1/min and suspension closes 67→1/min; it did not establish zero closes universally.
Raw TCP measurement: receipt. Real functional proof: receipt. Screenshot for independent review: resumed terminal.
Final stripped static sizes: x86_64 21,100,176 B, +4,032 B against same-recipe merge-base rebuild; ARM 19,328,840 B, +3,320 B against recorded shipping ceiling. No dependencies/features added; exact ceilings match these bytes. Author review is recorded above. Gate fixture corrections and the failed preparations remain attached alongside successful reruns; no extra test skips or baseline waivers.
Full final gate output is attached verbatim; summary below is quoted verbatim. The command used the build-slot wrapper, the job target, CARGO_INCREMENTAL=0, CARGO_BUILD_JOBS=2 and TRACER_TEST_PORT=4444; no global state-root alias.
fast.log
final-gate-summary.txt
Cleanup: private perf namespaces/processes and VM artifacts are already removed (receipt). Local baseline worktree, 21 GiB target, temporary Node toolchain, comparison binaries and runtime alias are removed; evidence retained outside checkout.
final-cleanup.txt
Break-it review starting on review-resume-refused-loop, head/base
526460c2e6; reviewed diff base5b2a1b9ee3. Read all 24 comments; no parent linked. Checking hidden authority lifetime, refusal fallback/input/replay, lifecycle ordering and D300 host deadlines independently. Worktree DECISIONS ends at D299; checking origin/dev for D300. Findings/evidence: ~/.cache/codex-scratch/review-resume-refused-loop/. No deployment; only review branch may be pushed.Independent review finding P1: automatic refused-resume fallback silently loses offline typing. Test
review: refusal fallback preserves offline typing through the fresh writer replay fence(apps/tracer/src/connection-review-refusal-input.test.ts) drives real MuxConnection + TerminalClient callbacks: 11 never-transmitted bytes queued after disconnect; exactly 1 Resume and 1 fresh AttachTerm; after replay fence, 0 Input bytes delivered (expectednever-sent\n). New fallback dispatches onWelcome(false); terminal.ts:870 calls restartCursors(), which clears inputQueue at :1169. Previously sent/unacknowledged bytes are ambiguous on a new writer; never-sent bytes must not disappear silently. Failing reproduction. Test commit follows compiler check. D300 read from origin/dev; review branch's decision snapshot ends at D299. Full fast waiting for build slot; audit continues.Review step:
7464a7797(pushed) preserves the P1 offline-input failure; compiler check passes. Additional P2 reproduced:a refused resume forgets its capability once and fresh refusals keep backing off when online hint is false(connection-restart.test.ts). Same oracle with online=true passes. With false, explicit resume refusal then fresh refusal both schedule 50 ms (zero jitter), failing expected >50; disconnect resets retryAttempt for the stale offline hint. This reset predates #244 but still defeats growing backoff on its fresh fallback path. Failing output and passing control. No claim of measured request rate. Native hidden/logout check and paired perf measurements remain in progress.Review step
59132456f(pushed): preserves the P2 stale-offline backoff failure and passing online control. Step7777a8a18(pushed): two additional queue/visibility ordering checks pass, one socket / 28 streams / modeled 30s suspension; all 37 idle tests pass, 76 assertions, TypeScript check passes. These are task-order checks, not a physical iOS claim.Independent paired host-lease measurement on locked infraternal-perf-test, kernel 6.12.111, load 0.08/0.08/0.02 -> 0.08/0.09/0.02. Compiled exact Liveness sources from
5b2a1b9eeand526460c2einto one driver; real TCP echo, 25ms probes, private network namespaces, no shared network changes. Delay/loss from tc netem. Time from last accepted reply to doubt/retire includes up to one probe interval:No doubling found in this lease path. This is not a repeat of #234's ten-minute real host-link test and does not establish the universal mobile zero-close claim. Receipt. Full fast compiling (cold job target); no gate pass claimed.
Review controls: both native tests pass with real WebSocket/SQLite: review_hidden_socket_cannot_park_logout_revocation and review_hidden_socket_cannot_park_absolute_session_expiry. Expiry fixture updates the session after HTTP upgrade renewal and before the probe lease is created; hidden notice plus correlated proof leaves the persisted expiry unchanged. No hidden-session authority bypass demonstrated. Verbatim focused summary.
The first full gate found a style lint in the new test helper (corrected), and an existing UDP receipt fixture failed once (owned socket count 0 vs 1). Exact unchanged recheck passes: receipt. No waiver claimed; final full gate will rerun that row. Static builds are progressing. Test-only review head is currently
7777a8a18; authority-test commit awaits final lint. Paired perf driver and exact source snapshots; VM cleanup. All measurement processes exited and the owned VM directory is removed.Review step complete:
3a642b42ea(pushed to review-resume-refused-loop) adds the two native hidden/logout and hidden/absolute-expiry controls; 2 passed, 0 failed. Fmt and whitespace checks pass. Earlier helper style issue is corrected; final branch-head clippy/full gate remains pending. Four atomic review commits, no shipping-code changes. Current verdict HOLD: P0=0, P1=1 (offline input loss), P2=1 (stale-offline refusal backoff), P3=0. Final gate and independent review-vs-reviewed-head static size comparison still running; no merge/deployment.Review step:
0cedabe9f(pushed) fixes only clippy::similar_names in the new authority fixture (short_session flag versus expiry value); both assertions/setups are intact. Full native test row now PASS, including the earlier UDP counter fixture and both new hidden-session controls. Static builds PASS at exact ceilings. That attempt's only failed row was the corrected naming lint; it is retained as a preliminary transcript, not final certification: fast-final.log. Corrected-head full gate queued with a lint preflight; five atomic test-only commits. HOLD counts remain P0=0/P1=1/P2=1/P3=0. No merge/deployment.Finished break-it review: HOLD. P0=0, P1=1, P2=1, P3=0.
Reviewed
5b2a1b9ee..526460c2e. Final pushed review head:0cedabe9f8cad7256938bd71619bbd35cf5e384donreview-resume-refused-loop; five test-only commits, clean worktree. No production fixes, merge or deployment.review: refusal fallback preserves offline typing through the fresh writer replay fence(connection-review-refusal-input.test.ts). Exactly one Resume and one fresh AttachTerm; 11 never-transmitted bytes retained after disconnect become zero delivered bytes after replay confirmation.connection.ts:491activates fresh recovery;terminal.ts:870callsrestartCursors(), clearing the queue at:1168. Preserve known-unsent input separately; previously sent bytes remain ambiguous on a new writer. Failing oracle.a refused resume forgets its capability once and fresh refusals keep backing off when online hint is false(connection-restart.test.ts). Explicit resume refusal then fresh refusal both schedule 50ms with zero jitter; the true-hint control grows its delay. The reset atconnection.ts:529predates the patch and remains on the new fallback path. Failure and control. These are deterministic scheduled constants, not measured request rates.Native real-WebSocket/SQLite tests
review_hidden_socket_cannot_park_logout_revocationandreview_hidden_socket_cannot_park_absolute_session_expirypass, including no renewal by hidden notice/proof. No hidden-session authorization bypass demonstrated. Hidden parking deliberately removes probe retirement; repeated half-open cleanup and physical iOS/bfcache behavior remain uncertified. Both queued-reply/visibility task orders pass at 28 streams.Perf VM, exact native lease sources over TCP, private netem namespaces, lock held only during measurement; load 0.08/0.08/0.02 to 0.08/0.09/0.02. Baseline -> patch doubt/retire from last accepted reply (includes probe phase): LAN 273/525 -> 275/524ms; 150ms 385/767 -> 380/762ms; 150ms + 1% loss 381/763 -> 380/758ms. No doubling observed; 0 healthy retirements in each 20s window, 35 packet drops in the paired loss namespace. This is a lease driver, not the full UI/host actor or a ten-minute loss study. Receipt, exact driver/source snapshots.
Independent same-recipe stripped static x86 comparison: 21,096,144 -> 21,100,176 B (+4,032 B). ARM head 19,328,840 B. Rust 1.99.0, locked dependencies, same job target/static-cargo recipe. Size receipt.
The full gate is green, while the explicit review tests remain red. Current combined frontend invocation discovers 256 tests and omits the connection suites; an exact repeat reproduces that result. Explicit connection/close-reason invocation discovers 62 tests: 60 pass / 2 fail. No gate modifications or exclusions were added. Selector root cause is not established. Combined invocation, explicit connection invocation.
Final
scripts/verify fastused build-slot, CARGO_INCREMENTAL=0, CARGO_BUILD_JOBS=2, RUST_TEST_THREADS=4 and TRACER_TEST_PORT=4497. Full output is attached verbatim; the following summary is quoted verbatim:Appended findings. Local cleanup: own baseline, target, runtime and temporary binaries removed, zero running executable references, clean worktree and HEAD equals pushed head. Perf VM cleanup: owned directory removed and measurement processes/namespaces exited. Evidence retained outside checkout. Issue left open.
Round 2 starting on resume-refused-loop-r2, base
526460c2e6. Read all 32 comments and review head0cedabe9f; #244 has no parent link. Will retain ambiguous input in memory and offer explicit one-tap resend rather than automatically type into a fresh writer. Will fix stale-offline backoff and include connection suites in the fast gate. D300 deadlines remain unchanged. Perf VM measurement authorized by this brief; no deployment.Review tests cherry-picked through
76a2b3dc9(five review commits). Before fixes the explicit focused run reproduced both review failures: 8 pass / 2 fail.79cde72ae: refusal state survives stale offline hints until successful Welcome; ordinary outage fast recovery remains. Nine restart tests pass.d0e38f25d: fresh writer retains the existing bounded input queue and requests explicit Resend typing through shared mountToast. No automatic stale typing. Updated review oracle covers both never-sent and sent/unacknowledged 11-byte input, no delivery before replay/consent, and no duplicate on a second tap. Explicit connection/close suites: 63 pass, 0 fail, 265 assertions; tracer check passes. No additional byte buffer: existing 1 MiB cap per terminal (224 terminals = 224 MiB allocated queue capacity) unchanged.Fast gate selector fix: explicit shell-expanded ./src/connection*.test.ts in a separate Bun invocation; 55 connection tests now discovered, rather than zero. Existing frontend suite still passes. D300 deadlines, native production code and dependencies unchanged. Full fast, production screenshot/PTY proof and perf-VM repeat pending.
#244 round-2 self-review (D11/D160/D243/D262/D300)
526460c2eare the independent authority tests. D300 estimation and retirement methods unchanged.review-before.log
connections.log
Additional controls committed:
e98d1a1d4exercises retry hints during refused fallback for both online states; 11 restart tests pass, 50 assertions.e71addd6akeeps the persistent saved-typing notice truthful through later disconnects (it no longer claims a current connection).a1929b04astarts the sent-input case after seven previously acknowledged bytes and verifies fresh resend rebases to offset zero; both consent tests pass, 20 assertions. Strict tracer check passes.Gate selector commit is
35546c8ea: shell-expanded filenames discover all connection suites separately from the original frontend suite. Initial full fast has passed fmt, VT fmt and CLI compile-out and is compiling clippy dependencies. Production PTY/screenshot proof and final head fast remain pending. No push/deployment yet, per round-2 requirement to push when green.Initial full fast reached clippy PASS, then stopped the workspace test row at unchanged infraternal-agent test plugin_control::tests::retained_runtime_registry_keeps_lifetime_ownership_after_listener_abort: plugin_control.rs:1040, assertion failed: probe.try_lock().is_ok(). Result 143 passed / 1 failed / 20 existing ignores in that crate. There is no diff to infraternal-agent or its production dependencies in this round. No waiver or passing gate claimed. Rechecking the exact compiled test unchanged, then final full fast will repeat the entire workspace. Initial gate continues through required static builds.
Exact unchanged compiled lifetime-lock test immediately rechecked: 1 passed, 0 failed, 163 filtered. No source change, skip or waiver; final full workspace row must still pass. Latest unit control commit
1f29ae034sends and acknowledges a seven-byte prefix through real MuxConnection/TerminalClient callbacks before retaining the 11-byte suffix; consent tests: 2 passed, 21 assertions.aaecdf2b3updates only comments explaining the fresh-writer and refusal invariants; all 57 connection tests passed before the prefix-oracle refinement. Full-head final checks and perf proof still pending.runtime-lock-recheck.log
35c058a6acompletes the refusal-state control for an initially fresh handshake too. Expanded the same oracle across resumed/fresh, both online hints and retry hints. Three cases failed before; marking every explicit pre-Welcome server refusal fixes all of them without removing ordinary no-response outage recovery. Strict tracer check and all 61 connection tests pass, 195 assertions. D300 unchanged.Initial full fast finished: fmt, VT fmt, CLI compile-out, clippy, cargo-deny and static builds PASS; test row FAIL only at the unchanged lifetime-lock control already recorded, whose exact unchanged recheck passed. Both stripped static providers match exact ceilings: x86_64 21,100,176 B; ARM 19,328,840 B. No final gate claimed yet. Same-recipe baseline comparison and production browser asset builds are running through the one build slot, then perf-VM proof and a final full head gate.
fresh-refusal-before.log
connections-final.log
Production-browser proof on the locked perf VM found an additional presentation defect before completion: saved typing survives and the one refusal/one fresh Attach succeeds, but the Resend typing button has viewport intersection ratio 0 after resizing to 1440x900. The fixed toast was mounted under the transformed canvas, which changes its containing block. Failing receipt attached. Correcting this through an additive shared toast parent-placement option, anchored to the terminal window's positioned unscrolled body; no duplicate style or global action for the wrong window. The browser assertion remains unchanged. D300 unchanged; full head gate remains pending.
functional.log
Full fast on
aa9a3be6ehit the same unchanged lifetime-lock test again: retained_runtime_registry_keeps_lifetime_ownership_after_listener_abort, plugin_control.rs:1040, probe.try_lock().is_ok(). The test waits only for the listener task completion flag, then assumes that proves the lock is already available. Exact isolated test passed unchanged; both full workspace attempts fail this immediate final check. No waiver applies and none is used. Will change only this test to wait for the actual required lock availability within its existing 5 s timeout, retaining the assertion that a retained registry keeps the lock held. Production ownership/locking, D300 deadlines and timeout bounds stay unchanged. Re-run the full touched crate and final full head gate.Perf repeat completed meanwhile: base
526460c2e, 0 closes / 62,113 ms; round 2, 0 / 62,088 ms. Both one socket, 75 ms ±40 ms normal loopback-hop delay, seed 244, 0.5% actual packet loss, 25 s foreground plus 5/30 s suspension. Packet drops 37/8,709 base and 37/8,862 round 2. Perf VM 4 vCPU, kernel 6.12.111, loads 0.01/0.01/0.00 and 0.04/0.10/0.04. Original zero-close result retained; these are emulated mobile measurements, not a physical-phone claim.Browser step complete: shared parent-placement option







79262d682, terminal-window anchor53e2a6789, real PTY regressionaa9a3be6e. Actual browser/PTy test passes with seeded packet loss: no typing executed before consent; exactly one computed marker after Resend typing; subsequent normal typing also works. Both 5/30 s suspensions retain the socket, and forced refusal produces one Resume and one fresh Attach. Resend stays in the viewport in Ink/Paper at 1440x900, 393x852, 375x667. Production screenshots attached for independent review; no author visual-quality claim.Paired perf-VM observation: author head
526460c2e0 closes / 62,113 ms, round 2 0 / 62,088 ms; one socket each. The original zero-close result is retained. Same seed 244 / 75ms±40ms normal per loopback hop / 0.5% packet loss; actual drops 37/8,709 and 37/8,862. Kernel 6.12.111, 4 vCPU; loads 0.01/0.01/0.00 -> 0.04/0.10/0.04. These are Chromium mobile emulation results. Each foreground remote command held the lock only for its short run and released it between cases.Same-recipe stripped static x86_64 baseline and round 2: 21,100,176 B each, +0 B. ARM matches 19,328,840 B. No native shipping source or dependency changes. The only outstanding gate issue is the unchanged lifetime-lock test's immediate release assumption; its bounded observable-cleanup correction and full head gate are in progress. No push or deployment yet.
functional.log
mobile-before.log
mobile-after.log
size-comparison.log
0faee4959corrects only the lifetime-lock test's final observation. Fmt and remote/mesh all-target clippy pass; full infraternal-agent crate: 144 passed, 0 failed, 20 existing diagnostic ignores. The retained-registry must-stay-held assertion and 5 s bound remain; final wait now checks actual lock acquisition as well as listener completion. Production ownership and D300 unchanged.Screenshot review found that the old desktop captures could include an overlapping neighboring Appearance view immediately after resizing a mobile canvas; viewport intersection alone did not prove click delivery. Tightened the real-browser harness to navigate normally to Console after each resize and perform a trial click before capturing. No product code changed. Its run is queued on the perf lock (owner file: retry-join-time-r2, started 2026-10-18T17:17:43+02:00). Will retake captures and run final full fast on the committed head before pushing. The completed paired close measurement remains 0 -> 0 with identical production code.
runtime-lock-crate.log
The strengthened browser check found a real actionability defect that viewport intersection missed: at 1440 px, a settings-view element intercepts Resend typing clicks even after normal Console navigation. The parent toast is centered across a wide window extending beneath the neighboring view. Correcting the reusable parent-placement option to anchor at its window's inline start; ordinary viewport toasts stay unchanged. The trial-click assertion remains and fails after 5 s rather than an unbounded action wait. Failed receipt attached. No protocol, D300 or input-queue changes. Native full iterate has passed the corrected lifetime-lock control and is progressing through server authority tests.
functional-clickability.log
64fb14056fixes parent-toast action occlusion through the reusable component's inline-start alignment.724f14de0strengthens the real browser test: normal Console navigation, bounded trial click in every viewport/theme, fixed phase markers, guaranteed server teardown. All six trial clicks now pass (including both 1440 px cases); no execution before consent and exactly one saved command after the real tap. Actual seeded loss: 24 dropped / 5,388 packets. D300 unchanged. Updated production captures below supersede the earlier desktop captures that exposed interception; snapshots are for independent visual review.Full scripts/verify iterate passes: every native row, corrected lifetime-lock control, both hidden logout/absolute-expiry controls, existing 256 frontend tests, all 61 connection tests (195 assertions) and 37 protocol tests. This is not the final gate. Full scripts/verify fast is running on clean committed head
724f14de0b(full head SHA will be in the finish comment), followed by the final-bundle close observation and both requested branch pushes. No deployment.functional-final.log
Finished #244 round 2. Head:
724f14de0b8aa3ab808d80be40266cb931cdee1f. Pushedresume-refused-loop-r2and atomically fast-forwardedresume-refused-loopto the same head. Clean worktree; no deployment.Final-head binary and bundle on the locked perf VM: 0 closes / 62,088 ms = 0/min, one socket. Author base
526460c2e: 0 / 62,113 ms. Same seed 244, 75 ms ±40 ms normal delay per loopback hop, 0.5% configured loss, 25 s foreground plus 5/30 s suspension. Namespace drops: 37/8,709 base, 37/8,981 final. Kernel 6.12.111, 4 vCPU, loads 0.01/0.01/0.00 -> 0.01/0.02/0.09. This retains the author's zero-close result in Chromium mobile emulation; it is not a physical-phone measurement. Each foreground run held the lock only while running and released it between cases.Same-recipe stripped static x86_64: base/final 21,100,176 B, delta +0 B. Final ARM 19,328,840 B, matching its exact ceiling (D243). The two earlier failed full attempts and the passing iterate run are retained below. The existing lifetime-lock test now observes actual lock release within its original 5 s bound while preserving its retained-owner assertion; no production locking or deadline changed. Final full gate passes without a waiver or new skip.
Final gate summary quoted verbatim; full stdout/stderr is attached verbatim:
fast.log
final-gate-summary.txt
final-sizes.txt
fast-initial.log
fast-second.log
iterate.log
Final measurement, reproducible driver source, push and cleanup receipts:
mobile-after-final.log
perf-cleanup-final.log
local-cleanup.txt
pushed-refs.txt
measurement-driver-source.txt
Production browser proof and selected final captures; all six current captures are in the preceding step comment, for independent visual review:
functional-final.log



How to test:
apps/tracer, runbun test ./src/connection*.test.ts(61 cases including both review regressions).mobile-lease.spec.ts, production server/bundle and disposable state.Cleanup: zero owned executable/cwd references on the VM, owned VM folder removed; zero local owned executable references, baseline worktree, job target, comparison binaries and temporary toolchain artifacts removed. Evidence stays outside the checkout and is attached above. Issue left open.
Delta review round 2 starting: branch review-resume-refused-loop-r2, base/head
724f14de0b. Read all 46 issue comments; #244 has no parent link. Reviewing delta 526460c2e..724f14de0, preserving round-1 controls. Focus: retained typing isolation and lifetime, explicit consent/decline, races, keyboard/live announcements, refusal backoff, unchanged D300. No deployment; push only review-resume-refused-loop-r2. Findings and raw receipts will live in job scratch and be attached here.Review step
d75b816capushed on review-resume-refused-loop-r2. P2: saved typing has no decline/discard action. terminal.ts:1266-1278 produces a persistent Resend-only toast (dismissible/onClose both undefined); :1236 also withholds subsequent typing. A person unwilling to resend uncertain typing cannot keep using the same shell without closing/replacing it. Synthetic 14-byte failing test:review: saved typing can be declined without closing the live shell. Three controls pass: window-local consent, clearing on session selection including stale action, and another disconnect before consent. TypeScript check passes. 3 pass / 1 fail / 17 assertions. Full fast is running through build-slot; expected review failure will be reported, never waived.retained-lifetime.log
Review steps pushed:
35e56d217adds production disposal/stale-consent control;30830d361tightens the Welcome reset oracle. Strict TypeScript passes. Explicit connection run: 65 pass / 1 fail / 217 assertions; all 61 author tests still pass, four additional lifetime controls pass, only decline is red.Defensive audit: forceDrop after Welcome reset the counter itself, concealing a missing Welcome reset. Replacing it with an ordinary close passes all 15 restart tests. In-memory mutation removing Welcome's reset: previous oracle 15 pass / 0 fail, strengthened oracle 7 pass / 8 fail. Production source remains untouched. No additional shipping defect counted. D300 production deadline paths have no delta changes. Full fast compiling native checks; browser functional controls follow through the same single build slot.
backoff-audit.log
previous oracle under mutation
strengthened oracle under mutation
mutation source
connections.log
91992ec85pushed: production-browser checks reproduce both P2 findings. Pending typing has no decline control; Tab and Shift+Tab from the terminal remain on its textarea and add four bytes (tab + reverse-tab) to the held 17-byte marker. Test-only focus on Resend followed by Enter delivers exactly those 21 bytes, proving navigation attempts became input. Positive native-button control passes with one marker sent. Browser result: 1 pass / 2 fail; strict TypeScript passes. role=status and aria-live=polite are present; actual screen-reader speech is not certified. No window/account crossover, automatic pre-consent send, or duplicate saved marker demonstrated. Verdict currently HOLD: P0=0/P1=0/P2=2/P3=0.Tests:
review: saved typing offers a non-resend exit in the live terminal window;review: keyboard can reach Resend typing from the terminal without sending more typing; passingreview: a focused Resend typing button accepts Enter.Stopped the preliminary cold native gate before completion to finish these browser controls; it is not a gate result. Final scripts/verify fast is now running on
91992ec85with build-slot, one reused job target, two build jobs and TRACER_TEST_PORT=4498. Browser run used unique port 4499, zero artificial delay, production bundle plus fixture transport; no timing claim or visual-quality judgment.browser-final.log
appended findings
Final-head fast gate on
91992ec857: fmt, VT fmt, CLI compile-out, clippy, full native test row and cargo-deny pass. Both hidden-session revocation/absolute-expiry controls pass. x86_64 static provider passes linkage, trust, key-memory and exact-size checks: 21,100,176 B, delta +0 B against ceiling. ARM static build is running; frontend and merge-base size comparison follow. No final gate claimed yet; expected new decline regression remains red. Review counts unchanged: P0=0/P1=0/P2=2/P3=0, HOLD.Finished delta break-it review: HOLD. P0=0, P1=0, P2=2, P3=0.
Reviewed
526460c2e..724f14de0. Final review head91992ec857, four atomic test-only commits pushed only toreview-resume-refused-loop-r2. Clean worktree; no production fix, merge or deployment. Issue remains open.terminal.ts:1265-1273creates a persistent Resend-only notice, with neither dismissible control nor discard handler.:1236also holds subsequent typing behind that consent. Someone unwilling to resend uncertain typing cannot continue in the same live shell without closing/replacing it. Unit testreview: saved typing can be declined without closing the live shellretains 14 synthetic bytes and fails on missing dismissal; browser testreview: saved typing offers a non-resend exit in the live terminal windowindependently fails because no such button exists. Unit reproduction, browser receipt.review: keyboard can reach Resend typing from the terminal without sending more typingproves Tab and Shift+Tab both leave focus on the textarea, adding four navigation bytes to the held 17-byte marker. After test-only programmatic button focus, Enter delivers those exact 21 bytes. The notice supplies no app focus shortcut; the terminal captures the normal Tab path. This is specifically navigation from terminal focus, not a claim that native button Enter is broken or that every browser-chrome workaround is impossible. Passing control:review: a focused Resend typing button accepts Enter. Browser receipt.Controls and defensive audit: all 61 tests on the reviewed head still pass; four additional lifetime controls pass (window-local consent, session replacement/stale action, disconnect before consent, production dispose/stale action). No automatic pre-consent Input, account/window crossover, persistent-storage disclosure or duplicate saved marker was demonstrated. The new path retains the existing per-instance ByteQueue, with no new storage/logging path or second byte buffer; its cap remains 1 MiB per terminal (224 MiB capacity at 224 terminals). Sign-out/window closure reach dispose and clear the queue. This is source-path and state-control evidence, not a full multi-account browser certification. The polite status live-region markup passes; actual screen-reader speech was not tested.
Strengthened the backoff success oracle: forceDrop after Welcome reset the counter itself and hid a missing Welcome reset. An ordinary close now observes the reset. In-memory mutation removing that production reset: previous oracle 15 pass / 0 fail; strengthened oracle 7 pass / 8 fail; unchanged production 15 pass / 0 fail. No shipping defect added to the counts. Audit, old oracle, new oracle, mutation source.
D300 unchanged: delta native production sources and deadline/retirement estimators do not change. Both real-WebSocket/SQLite hidden logout and absolute-expiry controls pass in the full native row. No fresh latency/RSS or physical-phone measurement is claimed.
Independent same-recipe stripped/static x86 comparison: merge base
5b2a1b9ee21,096,144 B -> review head 21,100,176 B, +4,032 B, matching the already-reviewed original #244 increase. Round-2 and review changes add no native production source/dependency. ARM 19,328,840 B. Rust 1.99.0, locked dependencies, scripts/static-cargo and one reused job target. Size receipt.Production-mode bundle/build budget passes independently: critical gzip 300,989/307,200 B, deferred 49,066/51,200 B, largest on-demand chunk 15,111/20,480 B. Browser checks use the normal minified build with the repository preview flag and disposable fixture transport; no timing or visual-quality claim. Production budget.
Final scripts/verify fast exits 1 solely at the new decline regression. Every Rust/static row passes; existing frontend 256 pass; connection suites 65 pass / 1 fail. No exclusions, skips or baseline waiver added. Protocol tests after the failing connection invocation are not reached; the protocol compiler passes and protocol source is unchanged. Full stdout/stderr, verbatim, verbatim summary:
Appended findings.md. Cleanup: own baseline, target and comparison binary removed; zero executable/cwd references before deletion; browser fixture ports 4499/4500 closed; clean worktree and HEAD equals pushed head. Cleanup proof.
Round 3 starting on
resume-refused-loop-r3, base724f14de0b8aa3ab808d80be40266cb931cdee1f(origin/resume-refused-loop). Read all 52 comments on #244; the issue has no parent link. Round-2 review head is91992ec8571a5dcd3a90e178a321e2fe16b5b78f; I will bring over its tests, fix discard and keyboard reachability, and leaveapps/tracer/src/connection.tsuntouched. No deployment.Shared toast step committed as
438b74b6c. It adds a second native action, accessible action-group metadata andfocusAction()to the existingpackages/uitoast.bun run checkexits 0;git diff --checkpasses.The pre-fix retained-typing regression reproduces the P2s and the new fresh-shell rule:
connection-review-retained-lifetime.test.tsis 2 pass / 4 fail / 20 assertions. Fresh input is still withheld while the saved buffer remains mixed into the live queue. Raw outputRound-3 regression tests committed as
a4e0c43b1. The unit oracle now requires a named Discard action, retains saved bytes separately, delivers new typing to the fresh shell, and covers another refused writer. The browser oracle keeps Tab and Shift+Tab in the shell, then uses the documented focus handoff to reach both buttons.bun run checkexits 0;git diff --checkpasses. Pre-fix unit result remains 2 pass / 4 fail / 20 assertions, attached above.Production browser run: 3/4 review checks passed. The keyboard check failed because its expectation included the old saved bytes on the fresh writer; observed fresh-shell delivery was only
\t, confirming those bytes stayed isolated. Updating the assertion to check focus handoff and fresh-shell keys only.Second production browser run: 3/4 review checks passed. The terminal kept focus after both Tab and Shift+Tab; each produced a fresh-shell tab byte. The assertion counted only the first key; correcting it to expect both delivered bytes.
Correction to my previous browser note: the latest production run showed one tab byte after Tab and Shift+Tab, not two. The textarea remained focused; Shift+Tab did not add another frame. I am making the browser assertion verify the fresh-shell Tab and that Shift+Tab leaves focus with the terminal, while the saved bytes remain absent.
Production browser evidence: after a 100 ms settle, Tab and Shift+Tab delivered fresh-writer byte values
9, 27, 91, 90(Tab then ESC [ Z); the terminal textarea kept focus. Earlier immediate samples raced the Shift+Tab frame. Updating the test to await the exact sequence and assert the old saved marker is absent.Production browser review checks: 4/4 passed in 56.3 s on working tree based at
a4e0c43b1. Verified both actions are announced, Ctrl+Shift+Space focuses Resend, Tab and Shift+Tab remain terminal input, fresh typing flows while the offer stays open, and Discard does not resend saved bytes. Screenshot:scripts/verify iteratereached the TypeScript connection suite: 65 passed, 2 failed, 229 assertions. Bothconnection-review-refusal-input.test.tssent-before-drop variants expected the retained 11 bytes in the live queue; the implementation now stores them inretainedInputuntil consent. Updating the fixture assertions to match the separate buffer and checking queue state after resend.Implementation committed as
460064f26.bun run checkpassed; all connection tests passed: 67/67 across 9 files, 243 assertions. This includes both refused-resume send-state variants and the retained-input/discard cases.First
scripts/verify fastreached static builds successfully, but the Rust test row failed only in unrelatedcrates/plugins/ssh/tests/install.rs:3483:review_inode_reuse_after_diagnostics_preserves_live_replacementcould not reproduce inode reuse in 256 allocations (93 passed, 1 failed, 1 ignored in that binary). Host load average was 3.75 / 9.88 / 11.98. This is outside the gate's named host-load exceptions; I will rerun the isolated test and the full gate, with no source change unless evidence identifies a defect.Reran the exact failing test on unchanged HEAD
460064f26:review_inode_reuse_after_diagnostics_preserves_live_replacementpassed (1 passed, 94 filtered, 0.34 s). No source change. Rerunning the full fast gate to confirm the complete suite.D243 static size comparison, same locked release package and target: merge base
724f14deproduced 21,100,176 bytes; branch460064f2produced 21,100,176 bytes. Delta: 0 bytes. The branch remains exactly at the x86_64 size cap. Build used the existing per-job target under/home/kayg/.cache/codex-scratch/resume-refused-loop-r3/target.Finished #244 at
460064f2616b87575429d12187414b692281720c. Publishedresume-refused-loop-r3and fast-forwardedorigin/resume-refused-loopto the same head. Worktree is clean.Saved uncertain typing now stays separate from fresh shell input. The retained typing notice offers Resend typing and Discard; both choices resume normal typing. Ctrl+Shift+Space / ⌘⇧Space focuses the actions only while the choice is visible; Tab and Shift+Tab remain shell input. Connection tests: 67 passed, 0 failed. Production browser review: 4 passed, including the keyboard, screen reader announcement, resend and discard paths.
D243 size comparison: stripped static x86_64 branch binary 21,100,176 bytes; merge base
724f14de21,100,176 bytes; delta 0 bytes. Static gate also reports the aarch64 artifact at 19,328,840 bytes, at its cap. Fullscripts/verify fasttranscript: verify-fast-rerun.log.Successful
scripts/verify fastgate output:Round-3 delta review starting: branch review-resume-refused-loop-r3, base/head
f57cdd479. Read all 66 comments on #244; no parent link. Reviewing 724f14de0..f57cdd479 including prior review controls. Checking consent/lifetime, keyboard registry and terminal coexistence, every toast-related browser spec, 67 connection tests and final fast gate. No deployment; push only review-resume-refused-loop-r3. Evidence and findings remain in job scratch.Round-3 finding, provisional P2: keyboard focus handoff is one-way. Ctrl+Shift+Space -> Tab -> Enter on Discard (and Enter on Send again) hides the toast but leaves the terminal textarea inactive. Both new production-browser tests
review: keyboard Discard returns typing to the live shell/review: keyboard Send again returns typing to the live shellfail at terminal focus after the action. Original 4 browser controls and repeated native activation pass (5 pass / 2 fail). Strengthening the reproductions to observe post-choice delivery; all 22 toast-related spec files now running on unique port 4519, omitting only the canvas frame-time budget case because this brief forbids unlisted hosts and timing evidence must come from the perf VM. No new timing claim. Unit audit now checks zeroing of held references on Discard/session replacement/dispose: 67 pass, 247 assertions; TypeScript passes. Full gate stopped during the initial compile to finish review tests first; final fast follows committed tests.Review audit step pushed:
bf56ae517(retained-byte zeroing controls only). TypeScript check passes; connection suite 67 pass / 0 fail / 247 assertions. Mutation audit temporarily omitted the production retainedInput.fill(0) calls and restored the exact original source afterwards: original lifetime oracle 6 pass / 0 fail; strengthened oracle 3 pass / 3 fail. No shipping defect added to counts. Initial browser reproduction confirms both keyboard choices lose terminal focus; delivery-focused regressions plus storage/console/sign-out control are included in the full toast-related run.Review step
187e2e727: strengthen two-window consent with distinct markers and exact frame contents (the old identical markers could conceal swapped ownership). TypeScript check passes; 67 connection tests pass / 250 assertions. No production change. Full toast run has confirmed an existing test selector failure in add-host-prototype ten-hosts: .ui-toast-action now resolves to Review plus the hidden secondary button. This is recorded as test maintenance, not evidence of a broken visible Review button; sessions-canvas has the same selector and remains in the run.Evidence for the provisional P2 and defensive audit: initial production-browser result 5 pass / 2 fail (both keyboard choices leave Terminal input inactive); original clearing oracle under omission 6 pass / 0 fail vs strengthened oracle 3 pass / 3 fail; restored production source 67 connection tests pass / 250 assertions. Review commits
bf56ae517and187e2e727are pushed. Delivery-focused regressions and all six viewport/theme captures are in the remaining browser batch.retained-browser.log
clearing-mutation.log
clearing-mutation-old-oracle.log
connections-isolation.log
Audit correction: the two focus-only browser regressions pass their new physical-key delivery assertion, then fail only on textarea focus. Physical ASCII typing still reaches the shell. Focus state alone is insufficient evidence of input loss, so the provisional P2 is not yet confirmed as a user-visible defect. The next controls exercise committed text without keydown events and a focused-terminal positive control. All six offer geometry/click captures pass; repeated activation, storage/console/sign-out, close and Ctrl+Space controls pass. No production change.
Confirmed remaining P2 in the keyboard choice workflow: physical ASCII input is delivered after Discard/Send again, but committed text sent through the browser text-input path is dropped. Focused-textarea positive control delivers the same synthetic 漢字 string. This is not a new polish regression: the pre-delta Resend button at
724f14de0reproduces the same failure. Public renderer focus targets its editable container; text-only beforeinput handling lives on its textarea. Replaced the invalid internal-textarea focus oracle with observable delivery plus a public terminal-focus check. Final regressions will preserve this narrower, demonstrated failure.Baseline browser comparison: all four DPR-3 short-toast ratios match exactly to four decimals (head/base: 0.2520, 0.4255, 0.2101, 0.3712 versus ceiling 0.1), using the identical repaired Reopen fixture with unchanged thresholds. Canvas resize/footer/shortcut, settings, ended-strip, four sign-in snapshots, and two product/sign-out checks also fail on
724f14de0. The palette sign-out check passed on baseline and is being rechecked on head. A separate sidebar setup race captured an initial count then waited for that obsolete count; the existing populated-tree assertion now performs the wait without that tautology. No production fixes or threshold relaxation.Review steps published on review-resume-refused-loop-r3:
ae8070c00repairs the unreachable glass-toast fixture;a9c1cd438preserves the two committed-text failures and 15 passing retained-input controls;a8c02e84cuses named Review/Reopen buttons;e71893184removes a transient-count sidebar wait. TypeScript passes. Review/Reopen cases and all four sidebar cases pass. Final focused browser run: 21 pass / 5 fail (two committed-text findings, three older sign-out checks); two of the sign-out checks also fail at724f14de0, while the palette sign-out check passed once there and still fails on head, so it remains unattributed rather than counted as a new defect. No production source changed. Recovery preview is running; final scripts/verify fast and same-recipe static size comparison follow on heade718931840.text-controls.log
baseline-browser.log
baseline-ghost-fixture.patch
Production-build saved-typing captures at 1440/393/375 px, ink/paper. All six assert message-above-actions, both buttons within the window/viewport, successful trial clicks on both, actual Send again/Discard selection, and phone targets >=44 px. No visual-quality grade or physical-phone claim. Final retained-typing controls are 15 pass / 2 fail for committed text; the focused-textarea positive control passes. Ordinary physical keys still arrive after both actions.
final-browser.log
Final review head
bbc7e418f8is pushed and clean (seven test-only commits). The active recovery-card fixture repair isbbc7e418f; TypeScript and all 15 preview cases pass, including six additional audit/reviewer security controls. Baseline repeat also reproduces both recovery-selector failures and the palette sign-out failure (2/2), resolving the earlier attribution uncertainty.Coverage audit against Playwright discovery: 308 requested tests in 22 toast-related files; 302 functional cases executed with 284 passes and 18 failures. Sixteen failing cases reproduce before the delta; two preserve the remaining committed-text P2 after keyboard choices. Six runtime-budget cases remain unverified because this brief forbids the required perf venue; incidental local timing output is not evidence. The six additional recovery security cases pass. No thresholds were loosened and no production source changed. Final scripts/verify fast is queued through build-slot on this head.
browser-matrix.txt
browser-coverage.txt
recovery-final.log
baseline-preview.log
baseline-retained-input-source.txt
Additional shortcut audit: the maintained upstream input-method keymap assigns Ctrl+Shift+Space to InsertFullSpace in Composition, Conversion and Precomposition states. Primary keymap. The app registry has no duplicate and plain Ctrl+Space is preserved, but this documented text-input use prevents a universal no-collision claim. A native-browser composition reproduction is drafted outside the checkout; it will verify the isComposing precondition before treating any focus change as a defect. The in-flight native gate remains on clean
bbc7e418f; no new finding count until the runtime check.Full scripts/verify fast completed with exit 0 on
bbc7e418f8. All Rust rows pass; tracer units 256/256, connection 67/67 (250 assertions), protocol 37/37. Complete transcript retained for final attachment. Stripped static artifacts: x86_64 21,100,176 bytes; ARM64 19,328,840 bytes. Composition-control browser check is queued through build-slot; same-recipe merge-base size comparison follows. No production code changes.Second P2 confirmed and preserved in pushed test-only commit
aa416b20f0.review: active composition survives the focus chord with a resolved typing choicepasses; the pending-choice counterpart fails because compositionend occurs. Both establish native CDP composition and verify a trusted Space keydown with isComposing=true before testing lifetime. core-screen.ts:770 consumes the chord whenever hasPendingInputResend is true, without a composition guard. TypeScript passes. This is distinct from the post-choice committed-text loss: the new shortcut interrupts composition before any choice is made.The primary input-method keymap reserves Ctrl Shift Space for InsertFullSpace during Composition/Conversion/Precomposition: https://github.com/google/mozc/blob/master/src/data/keymap/ms-ime.tsv . This is a documented conflict plus a native-browser composition reproduction, not a tested OS input-method installation.
Current HOLD: P0=0/P1=0/P2=2/P3=0. Full fast passed on bbc7; the final-head repeat and full 19-case retained-input suite are scheduled sequentially after the merge-base static build. Updated aggregate functional coverage: 304 cases, 285 pass/19 fail (16 older failures plus three regression cases for the two P2s); six runtime-budget cases still unverified.
composition.log
shortcut-reservations.txt
input-method-keymap.md
Final focused browser run on
aa416b20f0: 16 pass / 3 fail, exactly the two committed-text cases and pending-composition case already reported. Both positive controls pass; six geometry/click captures pass again. All 13 recorded browser/server process IDs are gone. Fresh discovery confirms 310 tests in 22 toast-related files; aggregate functional coverage is 304 run, 285 pass/19 fail, with 16 failures reproduced before the delta. Six runtime-budget cases remain unverified; six additional recovery audit/reviewer cases pass.Same-recipe merge-base x86_64 static build completed: 21,096,144 bytes at
5b2a1b9eevs 21,100,176 for the reviewed native code (+4,032 bytes, the existing #244 cap). Baseline is stripped static-PIE. Final-head scripts/verify fast is now running; final artifact size/hash will be refreshed after it completes.final-retained.log
browser-matrix.txt
browser-coverage.txt
toast-test-list-final.txt
final-retained-process-cleanup.txt
Final-head scripts/verify fast completed with exit 0 on
aa416b20f0. All Rust rows pass; tracer 256/256, connection 67/67 (250 assertions), protocol 37/37. Final static x86_64 is 21,100,176 bytes against measured merge base 21,096,144 (+4,032); ARM64 is 19,328,840. Native hidden logout and absolute-expiry controls pass. Verbatim gate summary and complete stdout/stderr are attached below.A cross-configuration scan added real/host-facts.spec.ts and the changed real/mobile-lease.spec.ts to the coverage audit. Initial real-server run: OS/kernel case passes; phone refusal case times out; mobile sees a refusal but no fresh Hello. They are not attributed to the delta yet. Pre-delta and head controls are running with the same native binary (no native delta 724f14de0..f57cdd479). Fixed stage labels locate the old phone selector wait; source audit found the real bridge discards delayed Close frames and drops the upstream close code/reason. Zero-delay controls will separate this fixture issue from product behavior and test the stale old-copy assertion. All real fixture state is private scratch; owner fixtures are untouched. Current confirmed verdict remains HOLD, P0=0/P1=0/P2=2/P3=0 pending these controls.
fast.log
fast-summary.txt
static-size-comparison.txt
hidden-authority-controls.txt
review-source-audit.txt
Finish: HOLD — P0=0 / P1=0 / P2=2 / P3=1. Reviewed 724f14de0..f57cdd479. Eight test-only commits pushed to review-resume-refused-loop-r3; clean head
aa416b20f0. No production change, merge, deployment or issue closure.review: active composition survives the focus chord with a pending typing choicefails on compositionend; the resolved-choice control passes. Both establish CDP composition and verify trusted Space with isComposing=true. core-screen.ts:770 checks only pending typing. The upstream input-method map also reserves this chord: https://github.com/google/mozc/blob/master/src/data/keymap/ms-ime.tsv . No installed OS input-method runtime was tested.review: keyboard Discard preserves committed text input in the live shelland the Send again counterpart fail on delivered committed text; focused-fresh-terminal control passes. Physical keys work. This remaining defect also reproduces through pre-delta Resend. Canvas window clicks route through frame.ts:103 / TerminalClient.focus at terminal.ts:779 to the renderer container; text-only beforeinput forwarding belongs to its textarea. The earlier focus-only claim was corrected.Main retained-input suite: 16 pass / 3 fail. Expanded inventory: 313 original cases in 24 files, across both browser configs; 307 executed, 286 pass / 21 fail, six runtime-budget cases UNVERIFIED (brief forbids the required venue; local timings are not evidence). Eighteen failing cases also fail before the delta; the other three preserve the P2s. The mobile case additionally has the stale-copy oracle once its older bridge problem is controlled. Six extra recovery security cases and the supplemental real-phone/current-copy controls pass.
Defensive audit: held-reference checks reject removing the wipe (old oracle 6 pass/0 fail; strengthened 3 pass/3 fail). Distinct window samples, repeat activation, storage/console/sign-out/close, fresh typing, Tab/Shift+Tab and plain Ctrl+Space controls pass. Fixed stale selectors/setup oracles without weakening assertions or pixel ceilings. Original real fixtures also have an absent phone split selector and a delayed bridge that drops Close frames/code/reason; both reproduce before this delta. Registered-key phone control passes real refusal/Sessions taps. Zero injected delay preserves Close delivery and the corrected-copy real recovery flow passes. No physical-phone/WebKit or complete real multi-account certification is claimed. Screenshots are for human judgment.
D300/native liveness and connection.ts are unchanged by the reviewed delta; native hidden logout and absolute-expiry controls pass. Final stripped static x86_64 21,100,176 bytes vs measured merge base
5b2a1b9ee21,096,144: +4,032 bytes, within the existing #244 cap. ARM64 19,328,840 bytes. Same pinned compiler/recipe on both comparison builds.All recorded browser/server PIDs are gone. Owned baseline, build targets, fixture state, copied binaries and temporary harnesses removed; evidence retained outside the checkout. Reviewer checklist and chronological findings attached.
scripts/verify fast: exit 0 on the exact head above. Verbatim summary below; complete stdout/stderr is linked.Gate, size and source receipts:
fast.log
fast-summary.txt
static-size-comparison.txt
hidden-authority-controls.txt
review-source-audit.txt
Findings, audit and cleanup:
findings.md
cleanup.txt
cleanup-before-removal.txt
real-run-summary.txt
real-baseline-host-facts.spec.ts.patch
real-head-diagnostic-host-facts.spec.ts.patch
real-head-diagnostic-mobile-lease.spec.ts.patch
cleanup-final.txt
real-original-copy.log
browser-matrix.txt
browser-coverage.txt
real-test-list.txt
reviewer-checklist.txt
Real-server controls and production captures (synthetic fixture typing only):








real-host-facts.log
real-baseline.log
real-head-diagnostic.log
real-zero.log
real-phone-trigger-control.patch
real-current-copy-control.patch
Finish: branch
resend-offer-p2, head27f16f799(basef57cdd479, review tests cherry-picked from review-resume-refused-loop-r3 unchanged).Changes:
5774b258bcore-screen: the focus chord (Ctrl+Shift+Space) declines while the key event is part of an IME composition (isComposing / keyCode 229).6da182614terminal: TerminalClient.focus (the shared helper) focuses the renderer's textarea, so committed text works after Send again / Discard.27f16f799real/mobile-lease.spec.ts expects the current copy.Tests: apps/tracer/test/retained-typing-review.spec.ts 18 pass first run; the one failure (393px paper geometry) passed alone, so it was load flake. tsc clean.
scripts/verify fast: Rust gate table
fmt PASS, VT native fmt PASS, CLI plugin compile-out PASS, clippy PASS, test FAIL (101), cargo-deny PASS, static builds PASS.
The test failure is environmental, not from this change: crates/plugins/mesh node_state tests (12) panic "test directory: NotFound" because they create their dir under cwd/target (crates/plugins/mesh/target) when CARGO_TARGET_DIR is unset, and that dir does not exist in a fresh worktree. With that directory created,
cargo test --workspace --lockedpasses. Fix belongs in the mesh test helper (separate change).Starting Release 6 on
release-6, base/headc00c6364e695c6676edb5d0dcc82a6388e5a1e59(origin/release-prod-live, release 5). Read all 83 comments on #244; no parent issue is linked. Continuing from reviewedresend-offer-p2head27f16f799. No push or deployment yet.Merge audit: target
c00c6364e(release 5), source27f16f799(resend-offer-p2), common base5b2a1b9ee(release 3). The source range contains no merge commits; it carries the reviewed #244 work only.git merge-treeidentifies conflicts indocs/PROCESS.mdandscripts/verify.d/rust.sh. The source has release-3 size caps and an older CLI gate; release 5 records 21,225,872 B x86_64 / 19,427,688 B aarch64 in its manifest and checks the newer CLI group output. I will retain release-5 gate behavior and documentation, then set exact size caps from the release-6 artifacts.Baseline check:
cargo test --locked --package infraternal-plugin-mesh --libon release 5 passed 24/24 tests withCARGO_TARGET_DIRoutside the checkout. This does not reproduce the priorDirectory::newissue, so I am not cherry-picking6f7cde60a.Starting Release 6 on
release-6, base/head857823ce433bc3f729c04571e32813073b1cc37e(origin/release-prod-live). Read all 86 comments on #244; the issue has no parent link. The prior release-6 comment used the olderc00c6364ebase; this worktree is at the requested production base. Verifyingorigin/resend-offer-p2and the release-5 staging recipes before the merge. No push or deployment yet.Merge audit: target
857823ce4(release 5 + #245), source27f16f799(origin/resend-offer-p2), common base5b2a1b9ee. The source range contains 33 files and only #244 commits.git merge-treereports six both-changed files:apps/tracer/src/connection-idle.test.ts,connection-liveness.test.ts,connection.ts,terminal.ts,docs/PROCESS.md, andscripts/verify.d/rust.sh. I will preserve the #245 route-liveness changes and release-5 CLI gate behavior, then measure and set exact static caps. The release-5 artifact files report 21,225,872 B x86_64 and 19,427,688 B aarch64. A baseline mesh test with the job target outside the checkout passed 24/24, so I am not bringing in6f7cde60aunless the release gate needs it. No #230, #231 or #243 changes are in the source range.Merge test finding: tracer TypeScript check passes, but the combined connection suite is 84/93 with 9 failures in
apps/tracer/src/connection-idle.test.ts. The release-5 test cases assert retirement after 276 ms or within 725 ms using the previousmax(250, srtt + 4*rttvar + 50)allowance. #244 changes that path to two D300 allowances (2 * clamp(2*srtt + 4*rttvar + 75, 250, 2000)); the failures are the now-stale deterministic ceilings after a resumed browser stall. Visibility, refusal-backoff and retained-input tests pass. I will update only those browser test bounds to D300's measured contract shape and rerun the complete connection suite; no host timing claim is inferred from this model.Merged connection coverage now passes: 93/93 tests across 9 files (339 expectations). The tracer TypeScript check passes. Updated the deterministic idle test bounds for D300's two-allowance window; no runtime timing claim is taken from this host. Proceeding to iterate gate and static size measurement.
Starting Release 6 on
release-6, base857823ce433bc3f729c04571e32813073b1cc37e(origin/release-prod-live). Read all 90 comments on #244; no parent issue is linked. Continuing from reviewed #244 head27f16f799; #243 shared-layout merge follows it. No merge or deployment yet.Finding: the merge-tree audit from release-6 base
857823ce4to #244 head27f16f799reports content conflicts inapps/tracer/src/connection-idle.test.ts,apps/tracer/src/connection.ts,docs/PROCESS.md, andscripts/verify.d/rust.sh;connection-liveness.test.tsandterminal.tsauto-merge. I will retain #245 route-bound liveness behavior and the reviewed #244 refusal fallback, focus handling, retained typing, and current gate behavior. Release-5 manifest sizes are 21,225,872 B x86_64 and 19,427,688 B aarch64. No merge result has been committed.#244 finding: the merged connection-idle unit file currently reports 54 pass / 9 fail. Six stalled-reply cases and two late-timer cases expect route retirement at 276 ms; the D300 two-allowance estimator keeps the route through that window. The continuous-typing case retires at 750 ms against its inherited 725 ms bound. Full output: connection-idle-merge-regression.log. I’m reconciling those bounds with D300 while retaining #245’s correlated-reply, stall-grace, and route-bound confirmation behavior before committing the merge.
#244 finding: the first
scripts/verify iteraterun reached the Rust test row, then failed only the real-binary daemon oracle because my configured test-state path produced a 112-byte AF_UNIX path (scripts/agent_fixture.py:55). This is a test-root setup error, not a code or permitted host-load failure. Full output: merge-244-iterate.log. I’m rerunning with the same scratch state under the shorterrelease-6/tpath.Resuming
release-6at production base857823ce433bc3f729c04571e32813073b1cc37e. The #244 merge is active with source27f16f7994b77be26ebd9fcd6cd4bbaec22d0e89; its reviewed changes are staged, with two unstaged liveness reconciliations preserving #245 behavior. I’m finishing that merge, then integrating #2433603f0b86and the required mesh test fix6f7cde60a. No push or deployment yet.Finding: the corrected
scripts/verify iteraterun still exited 1 at the workspace test row. Incrates/plugins/ssh/tests/recovery_support.rs, 8 passed / 3 failed / 1 ignored: the cached-helper case panicked withpath must be shorter than SUN_LEN; two live-session socket cases sawUnexpectedEofafter childAF_UNIX path too longerrors. The isolatedprevious_attempt_shelloracle passes 1/1 withINFRATERNAL_TEST_RUNTIME_HELPER=/home/kayg/.cache/codex-scratch/release-6/target/debug/examples/recovery-helper, so the release-brief helper condition is not the failing row. I’m correcting the SSH fixture root configuration; no product fix is inferred.Follow-up finding: a short
/tmp/r6-244...root fixed the AF_UNIX length but made sixinfraternalrestart tests fail withEXDEV/PermissionDeniedbecause their replacement files must stay on the cargo target filesystem. I’m keeping the canonical cache root for the workspace and adding a test-only short-root override forrecovery_support, whose fixtures alone need the shorter socket path.Finding: the split-root iterate run passed the earlier fixture-path cases, then failed one SSH install fixture:
review_inode_reuse_after_diagnostics_preserves_live_replacementcould not reproduce inode reuse within 256 allocations (crates/plugins/ssh/tests/install.rs; 93 passed / 1 failed / 1 ignored in that target). I’m isolating the fixture and checking its filesystem assumptions before classification; no product fix is inferred.Finding: the workspace iterate run reported crates/plugins/ssh/tests/install.rs::review_inode_reuse_after_diagnostics_preserves_live_replacement failing to reproduce inode reuse in 256 allocations. The isolated test passes 1/1 on the same ext4 filesystem (0.73 s), so this is not yet classified as a branch regression. Full iterate is queued for a rerun; evidence: verify-iterate-split-root.log and install-inode-isolated.log under the release-6 scratch evidence directory.
Finding: scripts/verify iterate reported
runtime_lock::tests::session_inspection_finds_bound_endpoints_by_identityfailing at crates/plugins/remote/src/runtime_lock_tests.rs:266:left: Err(73),right: Ok(()). The previous workspace run failed a different inode-reuse fixture; both tests pass or fail depending on the run, so I am isolating this case before another full iterate. Full log: verify-iterate-rerun.log in release-6 scratch evidence.Update: the failing runtime_lock test passes alone with the workspace feature set (
agent,server,test-support,tokio/rt-multi-thread): 1 passed, 112 filtered out. The full iterate failure is not reproduced in isolation; I am running another full iterate. Exact isolated command/output is insession-inspection-isolated.log; full failure is inverify-iterate-rerun.log.Completed #244 merge on
release-6:fc31822ccfecb78426444e31dc7b128cca94a398(parents857823ce4,27f16f799). The merge keeps #245’s one-shot route-stall grace and reconciles it with the refused-resume backoff and two-miss deadline.bun run checkpassed; the targeted connection suite passed 93/93; the isolated runtime-lock test passed 1/1.The full iterate run passed. Earlier full runs exposed two intermittent fixtures (inode reuse and runtime-lock inspection); both passed when isolated, and the third full run passed without exceptions.
Gate output:
Evidence: verify-iterate-third.log, session-inspection-isolated.log. Static sizes for this merge are the next step.
Static size measurement after #244, built with the pinned release toolchain and Zig under the release-6 target directory. Both binaries are stripped and static; x86_64 is static-pie.
Both deltas are below D243’s 16 KB justification threshold. The final caps will be set to the measured release-6 binaries after #243.
Receipt: static-sizes-after-244.log.
Resuming Release 6 work on
release-6atbbefb76e8b4f86554b689e690aa84b206f95ef7b, based on production857823ce433bc3f729c04571e32813073b1cc37e. #244 and #243 are merged, the size ceilings are measured, and the required mesh fixture fix is committed. No push or deployment yet. Continuing with release artifacts, perf-VM database/browser staging, final fast gate, and issue receipts.Artifact packaging finding: the static build succeeded and produced x86_64 21,344,304 B and aarch64 19,535,600 B binaries, but the first manifest pass exited 1 because
sha256sum *encountered theweb/fonts/directory. The partial packaging log is attached: artifact-build.log. I’m switching the scratch manifest step to hash every regular file recursively and rerunning packaging/checksum verification; no repository files changed.Release artifact step complete at current head
bbefb76e8b4f86554b689e690aa84b206f95ef7b.scripts/package-releaseproduced the server bundle and both x86_64/aarch64 host and session payloads; the web distribution has 223 files. Bundle and recursive web SHA256 checks passed locally and on the perf VM. Static sizes: x86_64 21,344,304 B (ccc0cb3a…e015); aarch64 19,535,600 B (3e460eb1…8d7c). Web tar SHA256cb85b883…fea86. Evidence: manifest, bundle checks, web checks, perf-VM hashes. Next: v27 database staging and real-browser specs.Database staging finding: Release 5 created schema v27 and passed the initial health check, then the synthetic mesh seed hit the schema's unique
mesh_nodes.dataplane_hostconstraint on row two; SQLite rolled the transaction back. Receipt: perf-db-stage.log. I’m assigning distinct dataplane host values to all four fixture rows and rerunning the v27-copy-to-v28 startup proof.Database staging follow-up: the host-name uniqueness fix allowed all four fixture inserts, but the proof incorrectly expected four total mesh rows. The release-5 server already creates one
servernode; inspection shows integrityok, 5 total rows, and exactly 4release6-mesh-*fixture rows. Receipts: failed proof, read-only schema inspection. I’m making the prep script recreate only its own staging databases before seeding and assert the four named fixture rows while preserving the server-created node.Production-shaped database staging passed on the perf VM. Release 5 started from an isolated directory and created schema v27; four uniquely named mesh rows were added beside its existing server row. On a copy, the Release 6 server applied migrations 28/28,
integrity_check=ok, retained all four fixture rows andcanvas_heads, and served the production shell with HTTP 200. Both server PIDs are confirmed gone. Receipts: migration/startup result, process cleanup.Signed-staging safety finding: preflight review caught an outer EXIT trap that could erase the next job’s
/var/lock/perf-bench.ownermarker after our lock was released. I interrupted the local staging build before any NetBird connection or perf-VM lock acquisition, removed that outer cleanup, and regenerated the wrapper so only the measurement-shell trap clears its marker while holding the lock. No remote staging transcript was created. Review receipt: signed-staging-owner-cleanup-review.txt.Perf-VM staging finding: the signed deployment harness acquired the lock and ran on
infraternal-perf-test(kernel 6.12.111+deb13-amd64), then stopped atdeploy_staging_tests.rs:97because its fixture guard only accepts/root/infraternal-bench/deploy/...; this task's isolated root is/root/infraternal-bench/release-6/signed/.... The remote run directory was removed (STAGING teardown_directory_absent=true). Receipt: signed-staging-perf.log. I’ll generalize the test and Python helper guards to accept a normalized per-jobsigned/root while preserving containment and traversal checks.Browser-harness finding: the first perf-VM suite stopped before Chromium launch because
playwright.real.config.tscould not resolvetest/evidence-path. The scratch uploader pre-createdapps/tracer/testand copied the source directory into it, nesting the suite undertest/test. Receipt: perf-browser-suites.log. I’m changing the scratch copy to place the source directory contents directly atapps/tracer/test, then will rerun all required specs; no timing result is claimed from this failed setup.Perf browser harness finding: Playwright discovered all 11
layout-reviewcases, but each failed before server startup because the runner setINFRATERNA_BINwhileapps/tracer/real/server.ts:114readsINFRATERNAL_BIN; it therefore tried the absent repository target path. Receipt: perf-browser-suites-final.log. I’m correcting the harness variable and will rerun the suites; no product test result or timing claim is taken from this setup run.Perf browser harness follow-up: the corrected binary path reached the browser evidence guard, which rejected
$job/evidencebecause the copied test tree treats/root/infraternal-bench/release-6as its checkout. This is an evidence-path setup failure before the spec starts. Receipt: perf-browser-suites-final2.log. I’m moving perf-VM browser receipts to/root/.cache/codex-scratch/release-6/evidence, outside the copied checkout, and will rerun.Real-server mobile resume spec failed on perf VM: layout suites passed; the #244 spec recorded 1 refused resume Close, then 0 fresh Hello frames with resume disabled within 30 seconds at apps/tracer/real/mobile-lease.spec.ts:73. This leaves the Send again/Discard checks unrun. Machine: infraternal-perf-test, 4 vCPU, kernel 6.12.111+deb13-amd64; suite load before 4.17 1.91 0.74, after 1.45 1.55 0.70. Evidence: perf browser suite log.
Second perf-VM run confirms the #244 failure. It recorded the refusal on bridge 3 with
helloResume=true, then no freshhelloResume=falsebridge; at failure the page was visible and the terminal status wasconnecting. The 30-second fresh-Hello poll failed again. Machine: infraternal-perf-test, 4 vCPU, kernel 6.12.111+deb13-amd64; load before 0.00 0.20 0.35, after 0.38 0.29 0.37. Evidence: mobile resume diagnostic log.Real-server layout gate complete on infraternal-perf-test (4 vCPU, kernel 6.12.111+deb13-amd64): layout-review 11/11, layout-sync 2/2, layout-phone 7/7. The ten-window sync case used a 300 ms RTT profile and measured 11.7 ms work, 17.7 ms edit-to-paint, and 329.5 ms remote paint. Loads before/after: review 0.09 0.07 0.02 / 3.09 0.99 0.35; sync 3.09 0.99 0.35 / 3.16 1.19 0.43; phone 3.16 1.19 0.43 / 4.17 1.91 0.74. Suite logs and desktop/phone ten-window screenshots are attached: review, sync, phone, desktop screenshot, phone screenshot.
Perf-VM frame diagnostics identify the cause in
apps/tracer/real/server.ts: the bridge decoded the server's stream-zero refused-resume Close, then Chromium observed WebSocket close code 1000 with no reason. Subsequent bridges b4–b10 sent Hello+Resume and received another Close; none sent a fresh Hello. The relay drops the already received, delayed protocol frame afterbridge.closedbecomes true and closes the browser route normally. This is a harness ordering bug; the client fallback remains unverified until the relay forwards in-flight frames before closing the browser socket. Evidence: WebSocket diagnostic log.The first relay adjustment was insufficient: the post-change
mobile-leasereal-server run still produced 0 fresh non-resume Hellos and stopped at the same assertion. Inspection shows the upstream close handler closes the browser socket immediately, before the configured 75 ms downstream frame delay expires. The relay must drain already received server frames before closing that route. Evidence: post-change mobile lease log.Release 6 browser run on infraternal-perf-test (4 vCPU, kernel 6.12.111+deb13-amd64): layout-review passed 11/11. Layout-sync functional assertions passed, but its timing budget failed once: work 18.8 ms > 16 ms; edit-to-paint 24.7 ms; remote paint 348.3 ms vs receipt 333.9 ms (the receipt-based bound passed). Load before/after was 4.56 2.65 1.44 / 4.94 2.95 1.59. I will retry this timing case after target load settles. Evidence: layout-sync high-load run.
Finding: the retained-typing retry did not reach resume behavior. The fixture navigated to
/?theme=...without the product-canvas test route; the default server-held canvas had no saved window for this fresh test account. Captured Playwright context shows no.cl[data-view=console] .ui-windowand a timeout at the initial live-window assertion. I’m updating the fixture URL and rerunning with the preview build. Log: retained-typing-retry.log.Finding: after adding the product-canvas test route, the full retained-typing suite reached all 19 cases: 18 passed and the sign-out/storage case failed only after reload. The fixture URL included
fresh=1, which tells the test double to restore preview authentication on reload; the observed page remained signed in. I’m removingfresh=1(each test already resets the test double before opening the page) and rerunning the full suite. Evidence: full run, failure context.Retained-typing browser gate: 19/19 passed in 52.1s, one worker, port 4360. Perf VM: 4 vCPU; load average before 0.83 0.94 1.08, after 2.96 1.48 1.25. The fixture uses the product-canvas test route without
.
fresh=1, preserving sign-out across reload. Run log. Offer screenshots:Committed retained-typing fixture routing as
b03803f56. D308 server-held layout made the previous bare-route setup windowless; the preview route now supplies a test layout and preserves sign-out on reload. Gates: retained browser suite 19/19 in 52.1s on port 4360;bun run checkpassed with no diagnostics.Committed the delayed-frame drain fix as
fc4b466ca. When the real-server bridge delays already-received frames, it now sends them before closing the browser side; this preserves the refused-resume Close ordering and allows fresh attach. With the fix present: mobile-lease 1/1 passed; layout-review 11/11, layout-sync 2/2 and layout-phone 7/7 passed; retained-typing 19/19 passed;bun run checkpassed. Mobile lease log · resumed state.Release 6 complete. Head
8f8c84e37b40cb6fb3159815b502debeb837694dis pushed torelease-6; no deployment was made.Integration and conflict resolutions
27f16f799) asfc31822cconto production base857823ce4. Inconnection.ts, retained the base #245 one-shot local-stall grace and route confirmation while adding the refused-resume fallback. The timer and socket-reply paths use the same two-miss deadline; subscriptions reconnect. Retained pending typing with explicit Send again/Discard and foreground recovery after background suspension.3603f0b86) as5d110f803. Preserved #245 liveness handling, #244 resume recovery, and terminal focus/fit lifetimes across the layout integration. Kept the mobile-visibility and canvas socket test modules in the verifier. The layout branch's migration0026_canvas_layout.sqlwas renumbered to0028_canvas_layout.sql; shipped production migrations 0026 and 0027 were left unchanged. The production-shaped 0027→0028 and fresh ordered 0001→0028 migration test passed (1/1).bbefb76e8(patch-identical to dev commit6f7cde60a). Later test-only fixes areb03803f56,fc4b466ca,b9ca6ced0,f43d6caad, and8f8c84e37.Size against release 5 (stripped static binaries; bytes)
D308's x86_64 stop line was 21,369,232 bytes (+140 KiB); final is 24,928 bytes below it. The final fast gate confirms both exact size caps.
Artifact SHA-256 (all packaged checksums verified)
ccc0cb3a0f9aba1c8b901eb2f150c383a09b18f101ea90e5f89bc53dee99e0153e460eb1682bdf8eb28fe60a592dd55f3886d8f80e80f14802eac636908d8d7ccb85b883a72fc9b8cb033ee2717eca07ff32276f9b59185d0a999853a5bfea86ccc0cb3a0f9aba1c8b901eb2f150c383a09b18f101ea90e5f89bc53dee99e01524f652c770683fe25f80de12cae34ca3dde6965df6e00cb6f14a3bde1b829cfb887475d130a9a3f088d4502ea8120ad085ac3b9815203aa1fc55a8f3cd6a26180f357e9f573aaff647d7521eb22483f6b6a807f0967c0fb58656175a61d171e95489f111cc722950875a92784f8ac6ca0b97d5c241d1cd75e4b9054c41a8220979b877570df6dac5d4bb275a9c9f518f2d00f4b4192ad76379011651b416e51abe7a3450c55e49c0ba39fd8eb04cc15153debe6ae8d3bccba1d2c14dedd265b9Browser and staging results
infraternal.canvas.<account>localStorage layout migrated to the server and appeared in a second phone context; restart retained it.Final
scripts/verify fastoutputReceipts and screenshots:
Deployed to production as release 6 at
8f8c84e37(orchestrator, 2026-10-19 ~19:47 IST). Contents: #244 phone resume / refused-session recovery with Send again / Discard, and #243 server-held canvas layout shared across devices (D308 size accepted). healthz ok, new MainPID 124534 (server restarted, so the CA must be unlocked). Backup /var/backups/infraternal/20261019-161717-8f8c84e37…. release-prod-live →8f8c84e37, merged into dev (8a6949a07).Owner test: unlock the CA; open the canvas on desktop and phone: both show the same windows; move/split/close on one, it follows on the other; reload keeps it. Phone: background the app for a minute, come back: no resume loop; if the old shell is gone, the offer reads "New shell. Offline typing may not have arrived." with Send again / Discard.
kayg referenced this issue2026-10-28 17:53:54 +00:00
kayg referenced this issue2026-10-28 17:53:57 +00:00