Second device stalls and terminal streams close with an invalid protocol frame at 60 windows #250
Open
opened 2026-10-20 15:01:36 +00:00 by kayg
·
285 comments
No Branch/Tag specified
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/infraternal#250
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found while qualifying #243 (layout recovery). Inherited from production code; not caused by #243.
Symptom. The real-server browser test with 60 window memberships across two devices (desktop + phone) fails intermittently, about 1 in 2 isolated runs, on both the exact production-equivalent build (
bd5ee1dd7server + production UI) and the #243 fix head90c7d4c2b:Why it matters. A second device can silently stop receiving layout changes, and terminal streams close with a protocol error under many open windows. The owner's production account has 44+ windows.
Evidence (from #243):
Next: root cause (which frame is invalid and who sends it; why the layout stream stalls instead of resyncing), a deterministic reproduction, a fix, and the test made reliable.
Starting #250 on branch
sixty-window-stall, base85b0285b733bac4c94d7540c58d4521d1a3410c4(production line). Read #250 (no prior comments) and retrieving linked #243 history/evidence. Will reproduce the terminal frame failure and silent layout stall separately, add regression coverage, and qualify shipping static builds with ten consecutive before/after scale runs. No deployment.Three deterministic production-code failures reproduced before product edits:
connection-unattached-close.test.ts: open a local stream on an already-welcomed socket, dispose it before its welcome microtask; browser emits Close(code1000) without any preceding AttachTerm.websocket.rsrequires an attached stream for Close. Failing output.terminal-fit-lifecycle.test.ts: 10000×10000 CSS box / 8×16 cells fits1250×625; server allows only500×300. Failing output. Actual scale-frame attribution remains pending.server-store.test.ts:100 noncontiguous revision events followed by a worker-withdrawal event schedule no recovery at all.ServerCanvasStore.#eventignores anything rejected byLayoutJournal.accept, including error events and revision gaps. Failing output. Linked production diagnostic has phone at57 with no received layout event above57, whereas desktop receives snapshot62; this is absent delivery, not merely a rejected reducer operation.Baseline static build is running after staging the pinned compiler; no production source change yet, no benchmark claims.
Baseline shipping static x86_64 at85b0285b7:21,387,184 B, SHA256bc3e7b2fa2bde97d06b548981954a96d68f4a8524fed2b41a7c80d99348de58b. Original browser sources bundled separately before repair. On locked perf VM:ten consecutive unchanged sixty-membership cases passed10/10, but emitted171 invalid-protocol control Close frames and7073 transient shim-disconnected stream Close frames. Thus a passing outer assertion does not prove healthy terminal protocol. Full frame metadata + output. Initial harness attempts failed before reaching product code (CLI version discovery and runtime-path length); excluded from count. Corrected to pinned CLI and a private /run mount for root fixture isolation.
Actual trace attribution:browser sends ordinary Credit/Resize (80×24) already queued behind AttachTerm; host sends transient Close(1,terminal shim disconnected); websocket.rs1351–1374 removes the attachment but only inserts an ended-stream record for4404. The next queued Credit/Resize hits missing attachment and is blamed as an invalid frame, closing every stream and layout on that socket. This is separate from the newly found unbound browser Close and oversized-fit bugs. Native regression for the transient-close ordering is compiling before any native production edit. The scale spec refers all60 memberships to one PTY; shim session.rs39,575–577 has32 peer slots, while each attachment opens data and ownership peers (daemon.rs1169–1177), explaining the upstream pressure. No limit or engine change proposed.
Silent layout recovery reproduced deterministically against the same shipping server/browser:withdraw canvas plugin, restore it, commit view.update. Phone remains revision3 while server advances4; healthy socket never reconnects. Failing real-server test. Worker withdrawal uses Event type:error, ignored by the browser store; no future snapshot is requested. Same-ID resubscription with one250ms→5s timer will restore it without consuming additional server subscription slots.
Completed/pushed e32d8284e:do not send Close for a local browser stream that never sent AttachTerm. Reuses existing writer bookkeeping and retains strict server validation. Exact isolated commit gate:tracer check +101 connection tests pass (389 assertions); original regression emitted one unbound Close, repaired code emits none. Gate. Native transient-close regression, same-ID layout recovery and shipping qualification remain in progress.
Completed/pushed789dbede2:reusable disposable subscription handle now refreshes the same ID. Exact isolated gate:tracer check +102 connection tests pass;100 replacements keep one ID and one socket, disposal blocks further refreshes. Gate.
Native transient-close ordering reproduces before the production edit (failure); remembering every host-closed stream makes all3 native session-close tests pass (after). Native clippy/commit still pending.
Completed/pushed38e4317b2:native host closes now enter the existing bounded ended-stream set for transient and final closes. Unknown-stream validation and32-entry bound retained. Fmt, workspace all-target clippy and3 native authenticated socket controls pass.
Completed/pusheddaad6268e:layout errors/gaps/receipt high-water marks recover through same-ID resubscription, one coalesced250ms→5s timer, cleared by caught-up operations or a valid snapshot. Pending operations remain immutable across resubscription. Tracer check and20 focused store/sync tests pass (79 assertions). Tests.
Baseline performance on locked perf VM:full wire send-to-both-subscribers event p50/p99 at1,10,60 memberships=2.275/10.368,1.904/6.040,3.498/41.002 ms (200 samples/row,2 concurrent writers). Raw output. Credited50MiB LF output,3 runs=9.959,10.186,10.492 MiB/s; median10.186. Raw output. Earlier gesture benchmark produced no usable timing (remote move absent); native throughput first needed a private passwd/run namespace so its expected zsh fixture prompt was present. No shared system files changed.
Scope attribution:all8573 Resize frames in the first ten baseline runs had at most80 columns and42 rows. Oversized-fit mismatch is an adjacent finding, not #250's observed trigger; the exploratory size-clamp change/test has been removed from this branch. Root fixes preserve current rendering/geometry behavior.
Strengthened60-membership gate on the same shipping baseline:0/10 pass when invalid-protocol socket closes are rejected. The original convergence-only gate was10/10 while emitting171 such closes; the stronger assertion is needed to expose the protocol failure reliably. No retry/timeout weakening. Ten failures.
Iteration gate passed fmt, VT fmt, CLI compile-out, clippy, runtime helper, inode fixture and cargo-deny, but the unchanged daemon_gone_tests fixture failed SUN_LEN:TMPDIR38 B + fixture/socket73 B=111 B. A private physical34-byte TMPDIR makes the exact socket107 B and the unchanged test passes. No source change/waiver; final fast will use the corrected root. Original gate, exact passing control.
Both-architecture same-recipe comparison is running:baseline aarch64 first, then repaired x86_64/aarch64 and production browser bundle. No deployment.
Contract documentation clarification proposed under the authorized recovery brief:after worker withdrawal/error, a skipped revision, or a receipt beyond the local canonical head, replace the same subscription ID to obtain a canonical snapshot. Coalesce retries with positive capped backoff, stop them when caught up, preserve immutable pending intents. Existing Subscribe/Event/Result shapes and authorization remain unchanged.
Also clarifying the existing nonzero Close rule in contracts/wire.md:after any host/server stream Close, discard already-in-flight terminal input/credit/resize until browser Close or reattach; keep the existing bounded retirement set and strict unknown-stream admission. Local browser stream registration alone does not open a wire stream. No frame encoding, field, limit or authorization changes. This documents the #250 race fix rather than changing a foundational choice.
Continuation start on
sixty-window-stall-cont, base/head32a98b1ebb84a575d633987b383c8501e3e1d266(continuingsixty-window-stall). Read all 8 #250 comments and retrieved all 233 parent #243 comments. Retaining existing fixes and baseline receipts. Will replace the WIP test commit, complete the subscription-contract clarification, repeat strict shipping-static baseline/head qualification, measure after-results on the named locked perf VM, compare both static architectures with85b0285b7, and run final fast with a short private TMPDIR. Push each atomic commit; no deploy.Completed/pushed
b20db48bc: clarified same-ID subscription generation replacement and healthy-socket withdrawal recovery in contracts/plugin.md §5. contracts/wire.md already contains both proposed #250 clarifications from78e39de06(§4 and Account canvas layout); retained them. No frame, field, limit or authorization change. Documentation gate: cargo fmt --all -- --check and git diff --check exit 0. Strict tracer/spec type checks and 86 focused connection/layout tests also pass; shipping comparison and strict ten-run baseline are active.Fresh strict shipping-baseline qualification (
85b0285b7binary + original production UI): 0/10 pass, retries=0, on locked infraternal-perf-test (4 CPUs), load 0.07/0.20/0.53 initially. All ten reach the protocol-close assertion. Per-run invalid-protocol control Close / all invalid-protocol Close / transient shim-disconnect Close counts:Totals: 155 control protocol closes, 7050 transient closes. The fixture now retains every Close, independently of timing traces, so an earlier close cannot be hidden by a later one. Head comparison pending; no retry or timeout weakening.
baseline-counts.json
baseline-ten.log
Both-architecture shipping comparison complete, same scripts/static-cargo --locked --release recipe against
85b0285b7: x86_64 baseline/head 21,387,184 / 21,387,184 B (+0); aarch64 19,559,112 / 19,559,112 B (+0). Rebuilt x86 baseline is byte-identical to the original receipt (bc3e7b2f…). Hashes for all four files are attached. Exact D243 caps remain unchanged. Production tracer build + transfer-budget check exit 0. No dependency, feature, profile or migration change. Head real-server controls and final fast are active.static-sizes.json
build-compare.log
Completed/pushed
8f2930350: replaces the saved WIP with the atomic real-server test commit. The shared harness retains every Close when requested, the scale gate explicitly enables decoding and rejects all protocol closes, and unique per-run receipts retain counts. Includes the same-socket subscription withdrawal control and event benchmark used by the baseline. Shipping static head + production UI: 2/2 controls pass (strict sixty memberships; withdrawal/catch-up without another socket). Strict tracer/spec types, 86 focused tests, fmt and diff checks pass.Ten consecutive strict head runs and final fast are active. The final head tree matches the source tree captured at gate start exactly; no source edits since qualification started.
sanity.log
ts-check.log
focused-tests.log
Head
8f2930350shipping static + production tracer: strict sixty-membership gate 10/10 pass in a row, retries=0. Baseline85b0285b7was 0/10. Per-run invalid-protocol control Close / all invalid-protocol Close / transient shim-disconnect Close counts:Totals: 0 protocol closes, 7028 transient closes; 0 page errors. Baseline totals: 155 control protocol closes, 7,050 transient closes. Transient pressure remains expected in this deliberate 60-memberships-to-one-PTY fixture (the shim has 32 peer slots, as recorded above); it no longer ends sibling streams or strands layout. Both devices finish canonical revision 196 on every head run. Screenshots are attached for owner review; no visual-quality rating.
head-counts.json


head-ten.log
Perf VM after-measurements complete on infraternal-perf-test, x86_64, 4 CPUs; every run held the shared lock, with no detached measurement process. Same 200-event/two-writer benchmark at 1/10/60 memberships; p50/p99 ms:
Recorded load averages: original event baseline 0.26/0.67/0.52→0.37/0.68/0.52; first head 3.01/2.13/1.07→3.26/2.21/1.11. Paired control started baseline at1.04/1.76/1.03 and head at1.33/1.80/1.05. Retain all samples; these do not establish a latency improvement.
Credited 50 MiB LF output, credit batch65536, 3 runs (same original recipe): paired baseline8.353,8.708,8.161 MiB/s (median8.353); paired head8.300,8.128,7.842 (median8.128). Original saved baseline was9.959,10.186,10.492 (median10.186). Paired output load baseline0.68/1.57/1.01; head1.07/1.60/1.03. Both current binaries are below D41's10 MB/s (9.537 MiB/s) budget; performance is not qualified by this window. The first head attempt used an additional budget assertion and stopped after8.733 MiB/s; its failure is attached, then the exact original three-run recipe was used for both binaries. No benchmark threshold or production source was changed. The common baseline slowdown is recorded, not waived or attributed to this fix.
Final fast is now in workspace tests; all earlier rows have passed so far.
performance-summary.json
events-after.log
events-paired.log
throughput-after.log
throughput-paired.log
First final fast on
8f2930350exited1: all9 Rust gate rows PASS, including workspace tests and both static targets. Two TypeScript property tests could not start their real Rust layout oracle: the shared sccache daemon tried to create compiler temporary directories under removed /home/kayg/.cache/codex-scratch/q9 (ENOENT). No reducer comparison ran, and no source change is made.Using a job-specific SCCACHE_SERVER_UDS socket with this job's physical TMPDIR instead; caching remains enabled. The shared daemon and its former directory are untouched. Rechecking the two exact oracle tests through the build slot, then rerunning the complete fast gate.
fast-attempt-1.log
ts-gate-failures.txt
Compiler-cache isolation control completed on unchanged
8f2930350: both exact real Rust reducer oracle property tests pass, 8,192 generated operations, zero divergences. A job-specific Unix socket selects a separate sccache daemon with this job's stable physical TMPDIR; the shared cache store remains enabled and the shared daemon was not stopped or changed. No repository source change. Full scripts/verify fast rerun is active on the same pushed head.oracle-recheck.log
Finished continuation on
sixty-window-stall-cont, pushed HEAD8f29303503fbf2fb67e6ec4454aefb8397d61caf(fix base85b0285b733bac4c94d7540c58d4521d1a3410c4). Worktree clean; no deploy; issue stays open.Contract documentation is complete: existing wire §4 / Account canvas layout rules retained, plugin subscription replacement clarified in
b20db48bc. WIP replaced by atomic test commit8f2930350. No frame, field, limit, authorization, dependency or migration change.Shipping static + production UI strict scale qualification: baseline0/10, head10/10 consecutively, retries0. Invalid-protocol control closes155→0; all head protocol closes0. Transient shim-disconnect closes7050→7028 remain expected for60 memberships referring to one PTY; both devices end revision196 in every head run, page errors0. Same-socket worker-withdrawal recovery passes. Raw per-run receipts:
baseline-receipts.tar.gz
head-receipts.tar.gz
D243 same-recipe baseline/head sizes: x86_64 21,387,184/21,387,184 B; aarch64 19,559,112/19,559,112 B. Delta +0/+0. Final fast shipping files match the benchmarked hashes exactly.
Performance qualification remains incomplete: paired credited50 MiB LF throughput baseline8.353/8.708/8.161 MiB/s, head8.300/8.128/7.842; medians8.353/8.128. Both below D41's10 MB/s budget in this VM window. All original/after/paired event p50/p99 samples, load averages and the initial budget-check failure are preserved in the performance comment above; no performance pass or latency improvement is claimed.
Final
scripts/verify fastEXIT0 on this exact head. Successful environment used a private physical34-byte TMPDIR/state root, the built recovery-helper, RUST_TEST_THREADS=1, TRACER_TEST_PORT=4798 and a job-specific SCCACHE_SERVER_UDS; caching stayed enabled. The earlier shared-daemon ENOENT is resolved without touching that daemon. Verbatim gate output:fast.log
final-gate-summary.txt
final-state.json
Self-review completed. No remaining job binary processes; private compiler cache stopped; local build target, shipping copies and runtime roots plus perf-VM temporary files removed. Evidence retained outside the checkout and attached:
final-self-review.md
local-cleanup.json
vm-cleanup.json
Verification review start: branch
review-sixty-window-stall, requested base/head8f29303503fbf2fb67e6ec4454aefb8397d61caf, production comparison85b0285b733bac4c94d7540c58d4521d1a3410c4. Worktree initially held superseded WIP32a98b1eb; aligned clean branch to requested head. Reading #250 and parent #243 history; preserving prior author receipts. Test-only review; no product edits or deploy. Will verify bounded retirement, unknown/reused IDs, layout retry/pending ordering, strict scale evidence, static size and interleaved six-pair credited output on named perf VM.Review step pushed:
132e66d05adds test-only coverage for ordered pending inputs/stale receipts across three replacements and for sixty persistently withdrawn subscriptions. Focused browser tests:85 pass,0 fail,253 assertions; tracer type check exit0. Virtual scheduling produces15 refreshes/subscriber in the first minute (900 requests total), then12 requests/s across60 independent subscribers. Snapshots stop every recovery timer. This is a scheduling/request-count result, not host performance timing; real UI stores are per account/device rather than per window.Bound inspection:the ended-stream set refuses its33rd distinct entry; it does not evict. Unknown IDs are not absorbed; reattach/Close remove the entry before normal validation. Existing native32-writer cap and browser28-writer rotation bound valid unique stream IDs per socket. Authenticated socket/overflow controls and final gate are waiting for a host build slot; no product edits or deploy.
P1 review finding:
closeStreamstill emits an unbound Close during reconnect.connection.ts:245checks retainedwritersUsed; that set survives a resumed connection, while server attachments are per socket (websocket.rs:631,2126). Reproduction: attach window on socket A, retain sibling, reconnect/send Hello on socket B, dispose first window before its replacement AttachTerm. B emits1 Close and0 AttachTerm for the removed window. Strict server handling returns an error for the missing attachment and ends the shared socket.Test-only regression is committed/pushed on
review-sixty-window-stall; original never-attached control passes, reconnect case fails (1 pass,1 fail,6 assertions). This is a remaining #250 ordering case; production also sent Close here, so no claim that the branch newly introduced it. Repair should distinguish current-socket attachment from retained writer identity and preserve strict unknown-stream validation. No product edits.reconnect-close.log
Review status:confirmed P1 reconnect Close regression committed/pushed as
776d4996a; tracer types pass. Production control confirms it is inherited (production0/2 controls vs head1/2). Native controls are written/formatted but remain uncommitted until their checks run. The host build-slot wrapper has not acquired a slot since18:51 CEST; at19:08 its three configured slots are held by release-9, app-windows-r6 and readd-minimal-r3. Continuing to wait; no slots, jobs or host settings changed. Static artifacts, interleaved perf-VM measurements and final fast are pending this shared build capacity.Independent strict production scale control:0/10 passes, retries0, on locked infraternal-perf-test (4 cores). All10 receipts reach canonical revision196 on both devices and contain protocol closes; total145 protocol closes,0 page errors. This verifies the strengthened assertion rather than a fixture-startup failure. Reused saved production binary SHA256bc3e7b2f… (21,387,184 B) and its saved production bundle, copied into this review’s private VM directory; original artifacts unchanged. Fresh same-recipe build comparison is still running on the host. Review head remains776d4996a; no product edits.
baseline-counts.json
baseline-ten.log
baseline-receipts.tar.gz
Native review controls committed/pushed separately:
ea6f370129f9151f93b932299a226f388af27e05(retirement scope/reuse/overflow unit controls),6c8af9e9c1b7ce83eee0eb9b3a48abbfa6c624b1(authenticated socket unknown-stream/overflow controls). Corrected focused run:4 pass,0 fail; touched-crate all-target clippy and fmt pass. The socket controls confirm that unrelated Credit/Close end the socket, and the33rd unacknowledged ended ID refuses immediately without eviction. Both use logical Close1011 with the protocol-error reason; transport receive failures use1002.Initial iterate passed every other Rust gate row and every other native test, but failed those two new socket controls because I had asserted1002. Their expectations were corrected, then all4 focused controls passed; that first iterate is not a green gate. The first native commit body’s broad wording about workspace tests refers to those other passing native controls, not an overall green iterate. Final full fast on the new branch head is still pending. No product edits. Same-recipe production static build is active, followed by full head fast.
corrected-native-controls.log
corrected-clippy.log
Fresh production build completed with the same scripts/static-cargo --locked --release x86_64-musl recipe:21,387,184 B, SHA256bc3e7b2fa2bde97d06b548981954a96d68f4a8524fed2b41a7c80d99348de58b. Byte-identical to the saved production artifact used by the independent0/10 strict baseline run. Production bundle rebuilt separately as well.
Final scripts/verify fast is active on clean/pushed review head6c8af9e9c. Every native test passed, including all4 added controls; fmt, compile-out, clippy, runtime helper, inode fixture and cargo-deny passed. Static builds are still running. The reconnect regression remains committed/red; final gate output, head scale reruns and six interleaved throughput pairs remain pending. No product edits or deploy.
build-baseline.log
Independent strict head scale qualification:10/10 passes, retries0, on the locked4-core perf VM. Every receipt has desktop/phone revision196/196,0 protocol closes and0 page errors. Independent production baseline was0/10 with145 protocol closes. Screenshots are attached for owner review; no visual-quality judgment.
Fresh same-recipe x86_64 binaries are exactly21,387,184 B each, delta0 B. Baseline SHA256bc3e7b2f… and headed756796… match the author’s recorded images exactly. Tests run on review head6c8af9e9c (product tree unchanged from8f2930350). The inherited reconnect-disposal P1 remains separately reproduced by776d4996a; normal scale success does not cover that lifecycle path.
Six interleaved credited50 MiB LF baseline/head pairs are now running. Final fast’s native rows pass; remaining static/frontend rows are active.
head-counts.json


head-ten.log
head-receipts.tar.gz
Perf review complete on locked infraternal-perf-test, x86_64,4 CPUs, no detached measurement processes. Credited50 MiB LF, credit batch65536, RTT0, six fresh-process pairs in exact ABAB order (baseline then head for each pair). MiB/s:
Head median+2.17%; ranges overlap widely, with no throughput regression observed in this window. All6 head samples exceed D41’s10 MB/s (9.537 MiB/s);1/6 baseline samples is below it. Recorded1/5/15-min load ranges:1.81–2.17 /1.32–1.45 /0.79–0.86. Raw run order and every sample/load are attached; no samples discarded.
The real-server withdrawn-subscription recovery control also passes1/1, retaining the healthy socket. Exact x86 baseline/head bytes21,387,184/21,387,184, delta0; hashes match the author’s images. Review head6c8af9e9c is pushed and contains tests only. The inherited reconnect-disposal P1 remains; final scripts/verify fast is still in remaining static checks. VM process check found0 job binary/browser processes after measurements.
summary.json
throughput.log
recovery.log
Verification complete. Verdict:P1 ×1 on requested delta85b0285b7..8f2930350. Test-only review head
6c8af9e9c1b7ce83eee0eb9b3a48abbfa6c624b1is pushed toreview-sixty-window-stall; worktree clean. No product edits or deployment. Issue remains open.P1 — unbound Close during reconnect (
connection.ts:245, failing test776d4996a). A window attached on socketA remains in writersUsed. After replacement socketB opens/sends Hello, closing that window before its replacement AttachTerm emits1 unbound Close. Strict server handling ends B and its sibling streams with logical Close1011/protocol-error reason. Original never-attached disposal control passes; reconnect control fails. Production control confirms this case is inherited. Distinguish current-socket introduction from retained writer identity while preserving strict unknown-stream handling.Other claims verified:
Final
scripts/verify fastexit1 solely for the committed reconnect regression. All9 Rust rows pass. The deliberate failure stops frontend dispatch; remaining terminal17/0,protocol39/0 and production bundle/transfer checks were run separately and pass. Initial iterate’s two incorrect1002 expectations were corrected to logical1011 before native commits; that initial run is not green.Verbatim final gate summary; full transcript attached:
Self-review complete. Private compiler cache stopped; temporary checkout,build targets,shipping copies,bundles and socket root removed. VM artifacts removed;owned binary/browser process checks empty. Evidence remains outside the checkout and on this issue. Recipes are attached as text because the helper refused shell-script files with422.
fast.log
final-gate-summary.txt
final-state.json
self-review.md
review-findings.md
protocol-units.log
terminal-units.log
transfer-budget.log
vm-cleanup-check.json
vm-cleanup.log
local-cleanup.json
run-vm.txt
run-env.txt
reconnect-baseline.log
iterate.log
Starting #250 on branch
sixty-window-reconnect-close, requested base8f29303503fbf2fb67e6ec4454aefb8397d61caf(sixty-window-stall). The worktree was at stale WIP32a98b1; its three-file difference from the requested head is limited to the WIP layout gate, removal of the test-server close capture, and removal of the plugin subscription clarification. I reviewed all 28 #250 comments and the linked #243 history. The open review finding is776d4996a: a prior-socket writer record can cause an unbound Close after reconnect. The review also measured 900 layout refreshes in the first minute across 60 withdrawn subscribers, then 60 every five seconds. I will align this branch to the requested head, reproduce the failing regression, fix the socket-local attachment check while retaining strict server admission, then add jitter if it is size-neutral. No push or deployment.Finding reproduced on the requested
8f2930350base after cherry-picking776d4996aas3a4878f88:bun test src/connection-unattached-close.test.tsreports 1 pass / 1 fail (6 assertions). The reconnect regression fails because the replacement socket records Close for the retired window before any AttachTerm on that socket; the never-attached control passes. Full test output is outside the checkout at$HOME/.cache/codex-scratch/sixty-window-reconnect-close/evidence/failing-close-test.log. Server admission remains unchanged and strict.Completed reconnect fix
0fe236563: Close is sent only after this WebSocket successfully sent AttachTerm for the stream; reconnect clears the socket-local set while retaining the writer set used for rotation. Strict server unknown-stream handling is unchanged. The cherry-picked regression failed before the fix (1 pass / 1 fail); afterward the connection/layout suite passed 195/195 tests across 28 files, andbun run checkexited 0. Tracer dependencies were installed from the frozen lockfile with--no-save; no lockfile change. The exact logs are outside the checkout under$HOME/.cache/codex-scratch/sixty-window-reconnect-close/evidence/. Layout retry jitter remains.Finding reproduced with a virtual clock across 60 independent layout stores: 100 withdrawal events coalesce to one timer per store, but all first delays land in one 250 ms slot. The unchanged schedule produces 900 refreshes in the first 60 seconds and 60 more in the following five seconds. The new distribution regression currently reports 17 pass / 1 fail; tracer type checking passes. This is request scheduling, not a host timing measurement. Raw test output:
$HOME/.cache/codex-scratch/sixty-window-reconnect-close/evidence/failing-layout-jitter-test.log.Completed retry jitter in
4c8cd1bb3. For 60 virtual-clock subscribers, the first-minute retry count fell from 900 to 753, and the following five-second window fell from 60 to 48; first-delay slots are 60 instead of 1. The focused layout test passes 18/18 andbun run checkpasses. Baseline and after logs: baseline, after.Static size step complete on
4c8cd1bb3. The shipping stripped binaries match the8f2930350baseline caps: x86_64 21,387,184 bytes (delta 0); aarch64 19,559,112 bytes (delta 0). Both are statically linked. Build receipts: x86_64, aarch64.Focused tracer checks completed on
4c8cd1bb3:bun run checkpassed; connection/layout tests passed with 196 pass, 0 fail across 28 files. Log: focused-connection-layout-final.log.Strict sixty-membership real-server gate passed on
4c8cd1bb3with the production tracer bundle and shipping static x86_64 binary: 10/10 repeats; all 10 per-run receipts report 0 invalid-protocol closes. Evidence: gate log, close summary, desktop screenshot, phone screenshot.verify iteratereached Rust tests; 98 passed and one failed:restart::tests::reload_reopens_install_path_when_the_running_inode_was_replacedreturnedEXDEVwhile hard-linking the Cargo test executable. The state root was/run/user/1001/swc(tmpfs), while the test executable is on/home; this is a test setup cross-device link, not a host-load timeout. I will rerun with a short state-root alias backed by this job’s same-filesystem scratch. Receipt: verify-iterate.log.The first 10× harness invocation did not reach the server: all 10 repeats reported
ENOENTbecause the environment override was misspelled, so Playwright fell back to the absent worktreetargetbinary. I corrected the variable names to the source-definedINFRATERNAL_*names; the subsequent 10/10 run against the static binary passed with 0 invalid-protocol closes. Failed invocation receipt: real-sixty-10x.log.The same-filesystem rerun exposed a second setup issue: the short
sw250alias was a symlink, and restart-path tests intentionally reject symlinked ancestors or compare the literal install path. Eight restart tests failed; this is not a product regression. I will rerun with the physical job scratch root, which is 60 bytes and keeps the longest restart socket path at 105 bytes. Separately,cargo-denyfound RUSTSEC-2026-0382 in the unchanged lockfile:rustls-webpki 0.103.15; RustSec says0.103.16is patched (issued 2026-10-20). I’ll update only that existing package’s lock entry in its own commit. Evidence: iterate log, RustSec advisory, upstream advisory.The physical job root removed symlink/cross-device effects, but two tests still failed because the root’s 60-byte path made
setup-tokenandremote_serversocket names exceed Linuxsun_path; the Rust suite reported 97 pass, 2 fail. I’ll rerun with a mode-700 physical runtime directory at/home/kayg/.cache/codex-scratch/sw250(same filesystem as the target, 37-byte path), and remove it after verification. Receipt: verify-iterate-physical.log.The 37-byte state root fixed the restart and token sockets; the remaining failing test was
infraternal-agent::daemon::gone_tests::a_vanished_session_gets_a_final_close_and_the_channel_stays_up. It builds its socket underTMPDIR; my previousTMPDIRwas the 64-byte job-localtmppath, exceedingsun_path. I’ll rerun with bothTMPDIRandINFRATERNAL_TEST_STATE_ROOTset to the 37-byte private physical path. Rust tests were 147 pass, 1 fail, 20 ignored before this correction; cargo-deny passed. Receipt: verify-iterate-short-root.log.Committed the RUSTSEC fix as
9cdc12901, changing onlyCargo.locktorustls-webpki 0.103.16.cargo check --locked -p infraternalandcargo deny check advisoriespass; the feature tree confirms the existingrustlsdependency, with no added crate or feature. Receipts: check, advisory gate, dependency tree.The short physical root passed the restart and token socket tests. One agent session fixture still failed because its path is
TMPDIR/g<24 hex>/sessions/<32 hex>.sock: with the 37-byte root that totals exactly 108 bytes, while Linux requires less than 108. RustSec and other gate rows passed; Rust test result was 147 passed, 1 failed, 20 ignored. I’ll shorten the private physical root to 36 bytes and rerun. Receipt: verify-iterate-final.log.Correction to the previous path count:
daemon_gone_testsappends/g<24 hex>/sessions/<32 hex>.sock; the 37-byte root made a 110-byte path (not 108). The new physical root/home/kayg/.cache/codex-scratch/ais 33 bytes, yielding 106 bytes, mode 700, on the same filesystem as the worktree and Cargo target. The prior gate receipt remains here.verify iteratepassed at9cdc12901. Rust tests passed with the 33-byte physical state root, cargo-deny passed forrustls-webpki 0.103.16, and the tracer compiler/precompression/bundle transfer checks passed. Gate output:Receipt: verify-iterate-pass.log.
The first final fast attempt passed Rust tests, cargo-deny, SSH inode reuse and the non-static rows, then failed x86_64 static compilation because the shared
sccacheserver tried to create/home/kayg/.cache/mr9/sccache…, a temp root from another job that no longer exists. This is build-environment state; no source or size result was produced. I’ll measure both static architectures with the samescripts/static-cargorecipe and the wrapper unset using this job’s TMPDIR, then rerun the final fast gate. Receipt: verify-fast.log.Static size after the rustls-webpki advisory patch: x86_64 21,388,208 bytes (+1,024 vs
8f2930350); aarch64 19,560,520 bytes (+1,408). Both deltas are under the 16 KiB review threshold. The exact D243 caps must be ratcheted to these measured sizes for the final gate. Build log: static-post-patch-no-sccache.log. I’ll update the caps and size history, then run the final fast gate.Committed
f1738e40f: updated D243 ceilings and PROCESS.md history to the measured static outputs. The focused static binary checks passed:PT_INTERP: absent; size: 21388208 bytes; cap: 21388208 bytes; delta: +0 bytes
PT_INTERP: absent; size: 19560520 bytes; cap: 19560520 bytes; delta: +0 bytes
git diff --checkandbash -n scripts/verify.d/rust.shpassed. Static build receipt: static-post-patch-no-sccache.log. The finalscripts/verify fastremains to run.Self-review found the layout timer tests restore
globalThis.randominstead ofMath.random:server-store.test.tscapturesrandom: Math.random, replacesMath.random, then callsObject.assign(globalThis, original)in both retry tests. This leaks a deterministic RNG into later tests. I’m adding a regression assertion and will restore the original function directly.The new regression assertions fail before the fix:
server-store.test.tsreports 18 pass, 2 fail; both failures showMath.randomdiffers from the module's original function after the virtual retry tests. Receipt: random-cleanup-repro.log.Committed
4724c8baa: both virtual timer tests now restoreMath.randomdirectly and assert cleanup. The focused server-store suite passes: 20 pass, 0 fail, 138 expect() calls.bun run checkcompleted without diagnostics. Receipts: random-cleanup-fixed.log, bun-check-after-cleanup.log. I’ll rerunscripts/verify faston this head.The final fast run at
4724c8baafailed the Rusttestrow: 609 passed, 1 failed, 52 ignored.bootstrap_runtime::remote_backend::tests::breakit2_live::live_fixture_keeps_authority_after_connection_retirementpanicked atcrates/infraternal-server/src/bootstrap_breakit2_tests.rs:29withStorage(PoolTimedOut)while opening its private SQLite fixture. This is outside the documented #118 timeout exceptions; all other Rust rows, static size checks, and the tracer checks completed. Full receipt: verify-fast-final-4724c8baa.log. I’m checking the fixture pool setup and will rerun verification after resolving the timeout.The failing workspace test passes when rerun alone with the same workspace feature context:
live_fixture_keeps_authority_after_connection_retirement1 passed, 0 failed. The earlier PoolTimedOut was transient under concurrent host load; this is outside the documented #118 exception list, so I will rerun the full fast gate rather than waive the row. Receipt: breakit2-timeout-isolated.log.Finished at head
4724c8baa(no push or deploy).The reconnect fix sends Close only when that stream sent AttachTerm on the current WebSocket; the server remains strict. The real-server 60-membership gate passed 10/10 with 0 protocol closes; screenshots and per-run receipts are attached in earlier comments. Layout retry schedule changed from 900 to 753 requests in the first virtual minute and from 60 to 48 in the following virtual five seconds; all 60 subscribers received distinct initial delay slots. This is virtual scheduling evidence, not a host timing measurement.
Stripped static sizes versus
8f2930350: x86_64 21,388,208 bytes (+1,024), aarch64 19,560,520 bytes (+1,408). The exact D243 caps match. Self-review also caught and fixed RNG cleanup in the timer tests; the new assertions reproduced the leak before the fix and now pass.One full run had a single
PoolTimedOutinlive_fixture_keeps_authority_after_connection_retirement; the exact workspace test passed alone, and the subsequent full run passed without waiving the row.Final
scripts/verify fastoutput:Exit status: 0. Full receipt: verify-fast-final-4724c8baa-retry.log.
Starting independent break-it review on
review-sixty-window-reconnect, target head4724c8baa02e71d5e0b00f96dff83c3421861f2f(review delta from8f2930350through4724c8baa). Worktree is clean. I’m checking reconnect close timing, stream retirement/reattach, strict server admission, layout retry bounds and fairness, the real-server 60-membership gate, andscripts/verify fast. No source changes or deployment.Focused review step at
4724c8baa:bun test src/connection-unattached-close.test.ts src/canvas/model/server-store.test.tspassed 22 tests / 144 assertions. The virtual 60-store retry schedule reported first-minute=753, following_5s=48, first_delay_slots=60; the implementation bounds each first delay to [250, 375) ms. Server inspection confirms Close still requires an attached odd stream (websocket.rsremoves fromattachmentswithok_or(())?), and the socket-local set is cleared on both replacement and disconnect. Receipt: focused-tracer-tests.log. No finding from this step.Full
scripts/verify fastcompleted at4724c8baa02e71d5e0b00f96dff83c3421861f2f, exit status 0. Verbatim summary:The static-build row took 1474 s. Full transcript: verify-fast.log.
Real-server 60-membership gate: 10/10 passed (2.6m). Every run had 0 page errors, 0 invalid-protocol socket closes, and desktop/phone at canonical revision 196 with 2 views. The focused virtual retry schedule measured 60 distinct first-retry slots, 753 refreshes in the first minute, and 48 in the next five seconds; delay range was 250–<375 ms. Post-reconnect Close-to-server probe is queued separately.
Artifacts: 10x run log, desktop screenshot, phone screenshot; per-run receipts: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.
Reconnect close probes passed. With the production server, both terminal streams reattached after a server restart; closing one delivered its stream ID in the replacement socket's client Close frames, left the socket open, and retained the sibling attachment. A synthetic ordering probe closed a retained stream before replacement Welcome, then delivered Welcome: the retired stream emitted 0 frames and the survivor reattached exactly once. The strict server Close branch remains unchanged (
crates/infraternal-server/src/websocket.rs:2126-2127). Production-server probe; pre-Welcome ordering probe. The two-restart variant is still queued.Two-replacement synthetic probe passed: both streams attached on each replacement socket; on the third socket, closing one emitted AttachTerm then Close while its sibling reattached once. Probe receipt. The production-server probe after one restart already passed; the production-server double-restart variant remains queued behind the shared build slots.
Finished review at HEAD
4724c8baa02e71d5e0b00f96dff83c3421861f2f. Findings: none (P0–P3). Verdict: MERGE. No code under review was modified; worktree is clean.Checks: focused tracer tests passed (22 tests, 144 expectations). The 60-membership real-server test passed 10/10; every run had 0 page errors, 0 invalid-protocol closes, and both clients at canonical revision 196. The virtual retry schedule spread all 60 first retries across distinct slots (250–<375 ms), with 753 refreshes in the first minute and 48 in the following five seconds. A production-server reconnect probe received Close for one reattached stream while keeping its sibling and socket alive. A pre-Welcome probe confirmed a retired stream is not reattached; a synthetic two-replacement probe emitted AttachTerm then Close on the latest socket. The server’s strict Close branch remains unchanged (
websocket.rs:2126-2127). Evidence is attached in the preceding comments. The extra production-server double-restart run remained queued for 8m39 without acquiring a build slot and was stopped after the two-replacement probe passed.scripts/verify fastexited 0. Verbatim gate summary:Release 9.2 integration is running on release-9-2 from
411485164; progress, interplay findings and final artifact/gate receipts are tracked on #24. No deploy. #240 inclusion remains conditional on its completed finish record after #250/#243 are integrated and gated.Integrated into release-9-2 at
c996c8d2bd; final fast, layout/review and real-server checks PASS; 60-membership gate 10/10, protocol closes 0. Artifact hashes and complete final receipts on #24. No deploy.Orchestrator: release 9.2 deployed at
c996c8d2b(release 9.1 + #250 + #243 p2/p3 + unified layout snapshot recovery). #240 was not ready and goes into the next release./proc/PID/exesha256 edb2b0c0…7f0 matches the release artifact;/healthzok; no errors in the journal.release-prod-live→c996c8d2b.EndedStreams(6dd98ce4a) stays bounded. A transient close reattaches on the same stream id (AttachTerm clears the entry), and a window close sends Close becauseattachedOnSocketstill holds the id. The layout#resyncingflag cannot stick: errors and gaps keep a single jittered timer (250 ms → 5 s) until a snapshot arrives.Hunt 9.3: browser layout sync (read-only, prod
abedd62b0)Scope:
apps/tracer/src/canvas/model/{server-store,journal,reconcile,sync}.ts,apps/tracer/src/connection.ts(subscribe/resync, call rejection), servercrates/infraternal-server/src/canvas_store.rs(events/receipts) and the subscription generation stamp ininfraternal-plugin-api.Branch
hunt-sync-93(baseabedd62b0, head1b66e86a3), tests only, no fixes. Probe file:apps/tracer/src/canvas/model/hunt-sync-93.test.ts.1. P1: two tabs overwrite each other's durable journal, so offline edits are lost (failing test)
server-store.ts:29-33(both tabs usecanvasKey(account)),account-storage.ts:65-68(flushwrites the whole journal: canonical, pending, unresolved). Nothing listens forstorageevents or merges.isWelcomed === false) and the person edits in A, which writes its pending op to the key. B writes its cache later (any edit, its 300 ms coalesced write, orflushon hide). iOS discards background tab A. A reloads and restores B's journal.pendingandunresolvedsets. Resending B's ops is harmless because the server dedups byop_idreceipt (canvas_store.rs:402). But A now holds B's placeholder IDs as unresolved, andjournal.ts:45holds A's queue behind an unresolved open. I have not checked whether that queue drains.an offline edit in one tab survives another tab writing the shared journal cachegets "Console" where it expects "Offline edit".2. P2: after a reconnect, an already-applied edit is painted over newer shared state (failing test)
journal.ts:25-28. A snapshot replaces canonical but carries noop_ids, so it cannot prune pending entries it already contains.journal.ts:37(restart) andserver-store.ts:107(the epoch guard) throw away the late receipt.connection.ts:570rejects the call, andfail()runs). On reconnect, the snapshot at r3 is accepted, but the pending rename is still unacknowledged, socurrent()replays it on top of r3. The journal resends it, and the server returns the deduplicated receipt r2, which only then prunes it.#resyncing, by an unresolved open at the head of the queue, or by 4 entries already in flight. The test suite exercises open/close andreducefor an already-present window, and those paths are idempotent. A move, resize or weight change is not, so for those the window visibly jumps back.an edit the reconnect snapshot already contains is not painted over newer shared stategets "Mine" where it expects "Theirs".3. P3: an error toast on every brief drop while an edit is in flight
connection.ts:570rejects all calls with "connection lost".server-store.ts:120-123then runsfail()and callsrefused()once per in-flight call (up to 4), andproduct.ts:128shows the error toast "Layout changes are waiting to sync. Reconnect to try again." on each call.4. P3: on a reconnect over a slow link, the client keeps subscribing again
server-store.ts:34-42together with:85-96.onOpenarms the resync timer withtarget = Infinityand a 500-750 ms delay. If the snapshot has not arrived by then, the timer runs#off.resync(), which sends another Subscribe and callsjournal.restart()(epoch++).onOpenre-arms at about 1-1.5 s, then 2-3 s, and so on.canvas_store.rs:91-96), takes longer than about 600 ms.5. P3: a revision gap freezes shared layout for at least 250-375 ms plus a round trip
server-store.ts:61-64. After a gap, every operation is dropped while#resyncingis set, and the timer must fire before the Subscribe goes out. The first gap could subscribe again at once and back off only after that. Low impact, because gaps are rare: the server serves from a cursor in order (canvas_store.rs:61-111), so a gap needs a dropped or malformed event.Checked, no defect found
canvas_store.rs:66) on one FIFO socket. A repeated Subscribe replaces the old one by generation (plugin-api lib.rs:126-137). An op at or below canonical is ignored (journal.ts:24,30).connection.ts:441,570),restart()voids late receipts, and resends are safe because of theop_idreceipts.disposeduring a pending resync clears the timer and voids the in-flight promises through#disposed.setTimeoutcalls, and liveness usesperformance.now(). No wall-clock comparison on this path.onOpenpath covers this. Minor waste: while offline after a failed call, the resync timer keeps waking every 5 s or less, andresync()does nothing when not welcomed (connection.ts:296).Gate: not run. This branch adds failing tests on purpose, so
scripts/verify fastwould be red by design. No fix and no deploy.fix-sync-journal: both hunt-9.3 sync defects fixed (P1 + P2)
Branch
fix-sync-journal(base1b66e86a3= hunt-sync-93 on prodabedd62b0), headc64797a0b. Pushed this branch only. Nothing went to dev and nothing was deployed.P1, tabs overwrote the shared journal (
08e814f93)infraternal.canvas.<account>stays. It is only the offline drawing cache, where the last writer winning does no harm.infraternal.canvas-journal.<account>.<page>(the page id is random for each store). Only that page writes it.op_idover the newest canonical revision (mergeLayoutCachesin cache.ts).op_idat most once (canvas_store.rs receipts).clearAccountStorage) now also removes the per-page keys.P2, reconnect snapshot repainted an applied edit (
c64797a0b, client only, no wire change)held.op_id, but it is not painted until its receipt comes back:subscribesends the client's pendingop_ids.canvas_receiptsalready holds.contracts/wire.md(canvas.layout.snapshot), the subscription filter andcanvas_store.rs::events.Tests (apps/tracer)
scripts/verify fastis running now. Its output follows in a separate comment.What is left
fix-sync-journal: handover at the deadline (13:54 CEST / 17:24 IST)
Head is now
5aa52818d. It addstest(tracer): a held layout edit resolves by its receipt (#250 P2), a direct test of the receipt path. A held edit whose receipt lands at or below the snapshot revision is pruned. One whose receipt is newer is painted again. Both cases still resend the sameop_id.scripts/verify fastis not finished. It started at 13:36 CEST in.worktrees/fix-sync-journalfrom a cold Rust build and is still in the Rust test stage. Everytest result:line so far readsok. N passed; 0 failed. No row table has been printed yet. The full output is in the orchestrator scratchpad atverify.txt.Next steps:
verify.txtand post the table here verbatim, or re-runscripts/verify fastin the worktree.08e814f93(per-page journals) andc64797a0b(held entries).op_ids, snapshot answers the ones already applied) only if the one-round-trip gap after a reconnect matters in practice.Only
fix-sync-journalwas pushed. Nothing went to dev and nothing was deployed.scripts/verify faston5aa52818d: cut off by my time limit, no resultMy 20-minute
timeout 1200stoppedscripts/verify fast(exit=124) duringtests/vt_credit.rs, before it printed a table. Every Rusttest result:line up to that point readok. N passed; 0 failed. The last ones:The gate has not passed yet. Next step: re-run
scripts/verify fastin.worktrees/fix-sync-journalwith no timeout. The build cache is warm now.Break-it review started: branch review-sync-journal, base/review head
5aa52818d7; reviewing only 1b66e86a3..5aa52818d. Independent view formed from diff before reading author comments. Investigating read/delete concurrency, failed-write adoption, dropped-anchor dependencies and held-edit ordering. Frozen tracer install passed. Full scripts/verify fast running on the unmodified review head. No fixes or deploy.Independent review: HOLD. Five deterministic Bun regressions now fail on unchanged
5aa52818d(0 pass / 5 fail, 10 assertions); tests are staged outside the checkout until the exact-head fast run finishes. No author comments were read before the independent hypotheses were recorded.P1 — concurrent durable edit deleted: server-store.ts:38–45. B reads A’s key, then A saves+flushes a rename before B writes its union. B deletes A’s newer key and flushes its older drawing; A is discarded. Reload returns Console instead of Edit flushed by live page. Storage-call interleaving models independent pages; no operation is interrupted.
P1 — failed replacement still deletes source: server-store.ts:45. A’s per-page key holds the only durable edit while another page has overwritten the shared drawing. setItem throws QuotaExceededError; writeStoredJson swallows it, then removeStored deletes A’s key. Expected retained source; actual null.
P1 — dropped placeholder loses resolved child: cache.ts:62–67. Offline parent birth is unresolved; child split already references a real session and waits behind it. Adoption drops parent but retains split(anchor_id=parent). Reduction silently omits child; validation accepts empty layout. Expected resolved-child; actual [].
P2 — held receipt changes projection without notifying: journal.ts:75–79 / server-store.ts:122. Send rename, reconnect+snapshot hides it, resolve current-epoch retry at revision2 over canonical1. current() becomes Mine but subscriber’s last drawing stays Console; acknowledge returns false because nothing pruned. Repair depends on later event/snapshot.
P2 — live-tab write recreates signed-out journal: server-store.ts:110,139–141 / account-storage.ts:65. A disposes and clears account storage; B remains live and flushes, recreating one per-page key (expected0, actual1). Cross-tab invalidation is absent. The shared-cache form is inherited; this delta extends it to the new durable journal. D245 remains unmet across tabs.
review-probe.log
Full fast still compiling native code; no fixes/deploy. Atomic failing-test commits will follow.
Additional P1: cache.ts:60–62 marks an op_id seen before dropping an unresolved copy, suppressing a later completed copy of the same birth. Sequence: offline unresolved birth is persisted to both caches; target resolves to real-session; shared drawing flush succeeds while the per-page write fails (or page stops between flushes). On reload the stale per-page entry is skipped but reserves its op_id; the complete shared entry is then ignored. Expected real-session membership; actual no windows. Separate regression fails.
reference-copy.log
Controls: 4 pass / 0 fail / 311 assertions. Interruption after successful union write leaves two safe copies; 100 subsequent reloads retain exactly 1 journal key and 1 pending op. Fully blocked localStorage (SecurityError getter) preserves live in-memory edits. Two reconnects reject both older-epoch receipts; current retry prunes by its actual revision. A held edit with no Result resolves normally via its ordered event. No claim of a browser private-mode run or performance measurement.
controls.log
all-review-probes.log
Strict probe types pass. Existing store harness plus first five regressions:23 pass /5 fail. Keeping requested exact-head gate unmodified until completion, then committing each failing case separately.
Exact-head fast progress: server tests621 passed /1 failed /52 ignored. The only failure is auth::tests::socket_tests::review_authority_cap::every_socket_in_a_full_place_storm_fails_closed_without_a_reply, expecting16 closes but observing9 (review_authority_cap.rs:162). Gate continues to static checks.
Detached origin/dev control
ee3dab0185: the same exact socket test passes1/1 in isolation. Therefore the brief’s same-failure baseline exception is NOT satisfied; no waiver claimed. Rechecking the exact review-head executable in isolation and final head gate will use RUST_TEST_THREADS=1. This native source is unchanged in 1b66e86a3..5aa52818d. No production edits.baseline-auth-cap.log
baseline-context.txt
Requested exact-head scripts/verify fast completed at unchanged5aa52818d7864ec00c5616efec9f8ece25992e99, exit1. Eight Rust rows PASS; test FAIL(101) solely for the socket-cap case already recorded (621/1/52 in the server batch). Both detached dev and exact-head isolated controls pass, so no baseline waiver is claimed. Frontend dispatch is skipped when the Rust function returns failure. No checkout changes occurred during this run. Verbatim summary:
verify-fast-5aa52818d.log
verify-fast-5aa52818d-summary.txt
head-auth-cap-isolated.log
Review regression committed/pushed:
1a96522652on review-sync-journal (adoption-race). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.Review regression committed/pushed:
4d88a9d94bon review-sync-journal (adoption-quota). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.Review regression committed/pushed:
fbaf4a6b6fon review-sync-journal (adoption-child). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.Review regression committed/pushed:
83e97f9eb4on review-sync-journal (adoption-reference). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.Review regression committed/pushed:
78f8db3ef8on review-sync-journal (held-paint). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.Review regression committed/pushed:
4c9320601fon review-sync-journal (signout-write). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.All six independent failing-test commits are pushed to review-sync-journal, head4c9320601f187c8847199bfdeae695a0f2d1b818; worktree clean. Only server-store.test.ts changed (+108/-1), reusing the existing fixture. Combined store suite:23 pass /6 intentional fail /177 assertions; strict tracer bun run check and git diff --check pass. Each commit separately passed fmt/types/diff checks and its one isolated expected failure.
P1 tests:1a9652265 concurrent adoption deletion;4d88a9d94 failed replacement deletion;fbaf4a6b6 missing-anchor child loss;83e97f9eb stale unresolved duplicate suppresses completed birth. P2 tests:78f8db3ef held receipt notification;4c9320601 live-tab write after sign-out (shared-cache form inherited). Test lines417,439,456,472,494,510 respectively. Product finding locations remain on reviewed head:server-store.ts:38/45,cache.ts:60/62,journal.ts:75/server-store.ts:122,server-store.ts:140–142/account-storage.ts:64.
Required exact5aa52818d fast has completed and its failed native row plus verbatim table are attached above. Merge-base static comparison is queued behind the host build slot; final review-head fast will follow with RUST_TEST_THREADS=1. No fixes/deploy.
committed-store-tests.log
final-check.log
commit-map.json
Additional P2 confirmed while validation waits for build capacity: cache.ts:57–67 / journal.ts:23,115–119. The merged startup state never prunes a pending entry whose known receipt revision is at or below the newest canonical revision. Sequence:A saves Mine at canonical1 and receives receipt2 before its ordered event;B persists a snapshot3 named Theirs;A flushes its older own key again;C reloads both journals offline. C correctly selects canonical3 but draws Mine instead of Theirs by replaying the already-acknowledged entry. A subsequent valid snapshot prunes it, so this is incorrect cached drawing rather than a new server overwrite. Independent regression fails0/1,2 assertions; seventh atomic test commit is being checked. Verdict remains HOLD, now4 P1 +3 P2.
adoption-prune-probe.log
Review regression committed/pushed:
8bc0cfc6f8on review-sync-journal (adoption-prune). Test-only change; reused existing store fixture. cargo fmt --all --check, tracer bun run check and git diff --check pass. Focused requested regression:0 pass /1 fail (intentional on reviewed implementation). Exact failure receipt will be attached in final review. No fixes/deploy.D243 same-recipe x86_64 comparison completed: base1b66e86a3 and reviewed head5aa52818d each21,389,296 B, delta0 B. Both SHA256 e98f7dfafe827f89fc94d442235f886000a253a0cfbd1213983b864db46b7bf0: byte-identical. Baseline file is stripped static PIE with no PT_INTERP. Recipe:scripts/static-cargo build --locked --release --package infraternal --target x86_64-unknown-linux-musl, one shared job target. No native/build-input change in the review commits.
Final review head8bc0cfc6f87c57ee8305627167a439c8ea8c4dd2 is pushed/clean (seven independent failing-test commits). Updated store result23 pass /7 intentional fail /179 assertions; unchanged cache/journal/hunt tests7/7 pass. Final full fast is queued with RUST_TEST_THREADS=1 and TRACER_TEST_PORT=4287, followed by independent production bundle/transfer checks because intentional failures stop the frontend chain. No fixes/deploy.
static-size-comparison.json
baseline-static-size.log
committed-store-tests.log
unchanged-focused-tests.log
Finished review: HOLD — 4 P1, 3 P2. Pushed
review-sync-journalat8bc0cfc6f87c57ee8305627167a439c8ea8c4dd2; remote verified, worktree clean. Seven atomic commits change onlyserver-store.test.ts(+125/-1). No fixes or deployment. Full sequences, test lines and commit mapping are attached infinal-findings.txt.server-store.ts:38–45: B reads A, then deletes A's newer durable write; reload loses the edit.server-store.ts:45: replacement write throws quota error, but the only durable source is deleted.cache.ts:62–67: dropped unresolved parent leaves a resolved child split with no anchor; child disappears.cache.ts:60–62: stale unresolved duplicate reserves its ID, suppressing the later completed birth.journal.ts:75–79/server-store.ts:122: newer held receipt changes current projection without notifying drawing subscribers.server-store.ts:140–142/account-storage.ts:64: another live page recreates data after sign-out; shared-cache form inherited.cache.ts:57–67: known receipt 2 replays over cached canonical 3 until a snapshot prunes it.Focused store tests: 23 existing pass, 7 deliberate fail,179 assertions. Unchanged cache/journal/hunt tests 7/7 pass. Four positive control tests pass (311 assertions): successful-write interruption plus a 100-reload key/ID bound; blocked storage; second-reconnect receipts; event recovery without a Result. Tests model storage/transport interleavings in Bun; no real private-mode browser or performance qualification is claimed.
D243 fresh same-recipe x86_64 comparison: baseline
1b66e86a3and reviewed head5aa52818dboth 21,389,296 B; delta 0 B, byte-identical. Final review-head artifact also matches. Temporary baseline, build target and probe scripts removed; 0 job-owned native executables remain. Evidence retained outside the checkout and attached.Requested
scripts/verify faston unchanged5aa52818dcompleted with exit 1. Its sole native failure observed 9/16 socket closes; detached dev and exact-head isolated controls pass, so no matching-failure waiver was claimed. Verbatim summary:Final full
scripts/verify faston pushed8bc0cfc6f, withRUST_TEST_THREADS=1,TRACER_TEST_PORT=4287: all 9 Rust rows PASS, including the earlier socket case; frontend 348 pass / 7 intentional fail (only the committed regressions). Production build and transfer checks run separately and both exit 0. Critical transfer: gzip 306880/307200 B; Brotli 255714/256000 B. Verbatim final gate summary:The issue stays open.
final-findings.txt
verify-fast-review-head.log
verify-fast-review-head-summary.txt
final-gate-status.json
final-bundle.log
static-size-comparison.json
commit-map.json
self-review.txt
cleanup.json
controls.log
Starting round 2 on fix-sync-journal-r2, base
8bc0cfc6f8(origin/review-sync-journal). Read #250 hunt, round-1 and seven-regression review, plus parent #243 history. Reproducing the seven failures before edits. Design direction: immutable journal generations, publish replacement before deleting only immutable source generations; failed writes retain sources. This removes live-owner deletion races without locks or a new dependency. Merge must select completed duplicates, preserve resolved children, and prune covered receipts. No wire change or deployment.Pushed atomic fixes:
260bded78journal receipt restoration/notifications (11 focused tests, 68 assertions);49edfa63fcompleted-birth deduplication and resolved-child recovery (4 review/cancellation tests, 8 assertions; 7 cache/journal/hunt controls, 22 assertions);503b05301immutable storage generations (6 adoption/replacement controls, 11 assertions). Baseline reproduced 25 pass / 7 fail. Failed replacement writes now retire nothing; live writes use fresh keys so adoption cannot remove their newer generation. Legacy mutable keys are retained until sign-out. Finishing cross-page sign-out revocation and bounded-generation controls before full gate. No native/dependency/wire changes.Pushed sign-out lifetime/writer fixes
e9f4e3eba,5dfb51503, delayed-event fix1d17c48ae, startup-publication fixafdbe8be3, and bounded-generation controls3c08a9912. The requested model/account-storage/offline-drawing suite passed 76 tests / 8838 assertions (real Rust reducer comparisons included); storage controls subsequently pass 39 tests / 506 assertions, plus a 6-test account lifetime run. All seven review regressions and both hunt tests pass. Additional self-review startup-signout regression reproduced before fix: disposal before subscription setup interrupted the sweep; optional subscription teardown and startup cancellation repair it. 100 offline reloads retain exactly one journal key and the original operation ID. Final scripts/verify fast is queued through the build slot at HEADafdbe8be3, RUST_TEST_THREADS=1, TRACER_TEST_PORT=4292. No timing claims, dependency changes, wire changes or deployment.Functional receipts (build-host timings are not performance evidence):
Before: seven reproduced failures
Requested model/account/offline suite: 76 pass, 8838 assertions
Storage/startup controls: 39 pass, 506 assertions
Self-review: checked D180/D183 receipt identity, ordered event/snapshot recovery, failed storage writes, immutable generation ownership, bounded reloads, placeholder dependencies, complete account sweep, late receipts/disposal, module role/invariant comments and unchanged wire/native inputs. Final gate and same-recipe native size comparison remain pending.
Self-review found and reproduced a recovery identity conflict: a live owner could later send its original split under the ID reused by an adopter for a recovered open (different immutable inputs). Fixed in the latest pushed child-identity commit: recovery uses a deterministic disjoint version-0 spelling of the original version-4 UUID; existing recovered entries, including receipts/sent inputs, win. No wire format change (op_id remains a string). Repeated restore is bounded. Focused storage/cache/hunt run: 46 pass / 0 fail / 523 assertions; tracer check, fmt and diff checks pass. Full fast remains in native compile-out checks; all TypeScript changes will be included when its frontend phase runs.
Final fast has reached the known native socket-cap flake: auth::tests::socket_tests::review_authority_cap::every_socket_in_a_full_place_storm_fails_closed_without_a_reply is FAILED. Gate remains running, exact assertion is pending the server batch footer. The brief explicitly says to report this named flake rather than chase it here; no native source has changed. Clippy/compile-out/runtime-helper rows passed. Will retain full gate output and run the unchanged frontend gate separately if native failure skips that phase.
Production bundle finding fixed in the latest pushed module-extraction commit: account lifetime implementation had entered critical JS, measuring gzip 307120/307200 B and Brotli 256086/256000 B (86 B over). Moving implementation to canvas/model/storage-lifetime.ts keeps key creation and the sign-out registry in account-storage.ts. New production measurement: critical gzip 306955/307200 B, Brotli 255741/256000 B; deferred gzip 58474/61440 B, Brotli 51800/55296 B. All caps unchanged. Tracer check + 46 focused tests / 523 assertions pass. No native inputs changed; final fast remains in the server test batch with only the previously named socket-cap failure so far. Frontend gate will run separately after the full native gate finishes.
Exact known socket-cap flake confirmed in final fast: auth::tests::socket_tests::review_authority_cap::every_socket_in_a_full_place_storm_fails_closed_without_a_reply, review_authority_cap.rs:162; left 9, right 16. Server batch: 621 passed / 1 failed / 52 ignored. No native source or test was changed, and no baseline waiver is claimed; the brief expressly says to report this case without chasing it. Inode fixture and cargo-deny pass. Static builds are active. Removed this job's now-unused debug/incremental cache after the native test phase to recover disk space. Latest pushed head is
95675d1cc; frontend lifetime extraction has already passed production transfer budgets and 46 focused tests. Full frontend gate remains scheduled after native fast completes.Completed/pushed
664fe59b1: canvas/model/cache.ts gives recovered child opens a stable disjoint retry ID; server-store.test.ts covers a live owner later sending its original split. Completed/pushed95675d1cc: extracts storage-lifetime.ts so lifetime checks load with the canvas, while account-storage.ts remains the single owner of key construction and sign-out registration. Tracer check, fmt/diff checks and 46 storage/cache/hunt tests pass (523 assertions). Production critical JS is 306955/307200 B gzip and 255741/256000 B Brotli; deferred is 58474/61440 and 51800/55296 B. The initial Brotli overflow was 86 B; no cap changed. Final fast is building static targets; only the named native 9/16 socket-close flake has failed. Full frontend verification and fresh merge-base static size comparison follow this run.Additional recovery dependency finding reproduced/fixed in the latest pushed commit: cache.ts reused an existing recovered open at its foreign source position, behind a target intent that originally followed the split. That sent the target before its birth. cache.test.ts now checks recovered-open-before-target order and the resulting new target; before fix 0 pass / 1 fail, after fix both assertions pass. Existing recovery input/receipt metadata stays immutable and the recovered ID is emitted once. Full focused run: 47 pass / 0 fail / 525 assertions; tracer check + fmt/diff checks pass. Current critical transfer: gzip 306955/307200 B, Brotli 255864/256000 B. Native source remains unchanged; aarch64 static build is still active.
Full scripts/verify fast completed with exit 1. Exactly one known native socket-cap assertion failed (9 closes versus 16); every other Rust row passed, including stripped static x86_64 21389296 B and aarch64 19561656 B. No cap changed; no native source changed. Full frontend function is now running separately on pushed HEAD
45e8bd4ea, because Rust failure skips frontend dispatch.Verbatim final fast output:
verify-fast.log
socket-flake.txt
recovery-order-before.log
storage-controls.log
bundle-before-split.log
Full frontend gate completed on pushed HEAD
45e8bd4eac: 526 tests pass across the five stages, zero failures. Includes all seven review regressions, both hunt tests, real Rust reducer comparison, connection tests, production build and unchanged transfer/font caps. The separate run is required because the known native 9/16 socket assertion makes fast skip frontend dispatch. Fresh merge-base static comparison is queued through build-slot.Verbatim stage summaries:
frontend-gate.log
Finished round 2 on
fix-sync-journal-r2, pushed HEAD45e8bd4eac32a5dd1b209cdf7d2a1d67732a7e90, base8bc0cfc6f87c57ee8305627167a439c8ea8c4dd2. Remote verified; checkout clean. Eleven atomic commits; no dev/main push or deployment. Issue stays open.All seven review regressions and both hunt tests pass. Immutable generations retire sources only after a successful replacement; legacy mutable keys remain until sign-out. Completed births, resolved children, recovery IDs/dependency order, covered receipts, held repaint and account lifetime revocation are covered. Additional controls: 100 offline reloads retain one journal key and the same operation ID; interrupted cleanup and blocked storage preserve the intended boundaries. Focused run: 47 pass / 0 fail / 525 assertions.
Full frontend gate on final HEAD: 526 pass / 0 fail (2 metadata, 363 tracer/canvas, 105 connection, 17 terminal, 39 protocol). Frozen install, TypeScript checks, real Rust reducer comparison, production build and every transfer/font cap pass. Critical gzip 306948/307200 B, Brotli 255886/256000 B; deferred 58498/61440 and 51872/55296 B. No caps changed.
Fresh D243 comparison with the same pinned static recipe: merge base/head x86_64 each 21389296 B, delta 0 B, byte-identical SHA-256
e98f7dfafe827f89fc94d442235f886000a253a0cfbd1213983b864db46b7bf0. Head aarch64 is 19561656 B and passes its exact cap. No native/build input, dependency, profile, migration or wire change.Full
scripts/verify fastcompleted with exit 1 solely for the brief's known native flake:auth::tests::socket_tests::review_authority_cap::every_socket_in_a_full_place_storm_fails_closed_without_a_reply, observed 9 closes / expected 16. No native edits or baseline waiver; not chased per brief. Every other Rust row passed. Its failure skips frontend dispatch, so the entire frontend function was run separately and passed as above. Verbatim final fast output:Author checklist completed. Tests model storage/transport interleavings in Bun; no real private-mode browser or runtime performance qualification is claimed. Own cache PID 200160 is gone; no job-scratch executable processes remain. Temporary baseline, target, bundle, saved binaries and compiler scratch removed. Evidence remains outside the checkout and is attached below.
How to test on a build of this branch:
Seven failures before fixes
Focused controls
Full fast transcript
Exact socket assertion
frontend-gate.log
static-size-comparison.json
baseline-static.log
author-checklist.txt
cleanup.json
Round-2 verification start: branch review-sync-journal-r2, base/head
45e8bd4eac; reviewing 8bc0cfc6f..45e8bd4ea. Read contract and diff before author comments. Retrieved #250 and parent #243 history. Independent concerns to exercise: concurrent lifetime initialization, check/write sign-out interleavings, cleanup interruption, lost receipts with later snapshots, and offline journal growth/write amplification. Tests only; no fixes or deployment.Round-2 findings, tests only (review-sync-journal-r2):
4812ebc6bexpects B's edit and receives Console.35094f0d3observes 1 journal, expected 0.5b69807aereloads Console instead of Only durable copy.a97648a22observes one retained hidden edit. The current snapshot protocol has no operation IDs, so this contract needs explicit resolution semantics; no wire change made in review.New three-tab control passes when edits affect distinct fields. Concurrent changes to x/y are one cell intent and legitimately conflict, so that exploratory expectation was discarded. QuotaExceededError/SecurityError at each of three adoption writes pass 6/6. A separate removal-recovery test observes 21 generations after 20 failed removals plus storage recovery; retirement forgets failed keys. Commit/check in progress. Initial type check lacked installed node types; frozen install followed by tracer type checks passed. No production changes/deploy.
lifetime-race.log
signout-boundary.log
unreadable-source.log
lost-receipts.log
Completed/pushed test steps:
a8adafd2aadds three-tab publication and six refusal controls plus the failing cleanup-retry regression;b355d4b7fadds eviction-after-every-mutation and 1024-edit offline replay controls. Both tracer type checks pass. Focused final suite: 56 pass, 5 deliberate fail, 562 assertions. The 1024-edit case retains 1024 pending entries, one generation, 287408 UTF-16 journal bytes.P2 bounded-storage finding: server-store.ts:64–66 / account-storage.ts:39 silently forget failed removals. After twenty failed retirements, a recovered successful flush leaves 21 generations, expected one. A reload can compact them, but the live writer never retries the orphan keys.
P2 inherited latency finding: server-store.ts:130,159–161 / account-storage.ts:98 rewrites the complete pending journal and drawing on the UI thread. On locked infraternal-perf-test, 4 CPUs, Chromium 149.0.7827.55, load 0.04/0.01/0.00 to 0.12/0.03/0.01, 60 windows: pending 0/128/1024/8192/32768 gives save p99 2.2/1.8/2.9/3.7/14.1 ms and flush median 0.8/1.6/5.1/39.1/119.4 ms (7 flush samples each). At 32768, no durable generation fits beside the shared cache. Round-1 baseline also exceeds 16 ms: 8192/32768 flush medians 35.5/104.4 ms. This is inherited, not established as a new regression. Perf-only committed test
800cd898buses eight actual new edits at 8192 pending, observes seven measured flushes 23.4–38.6 ms and fails <=16 ms. It skips shared-host timing runs. Perf test source was transpiled with external imports for the VM runner; initial untranspiled harness attempts failed before running tests and are excluded.Real Chromium browsing contexts independently confirm both P1 races using debugger pauses between actual storage calls: concurrent first tabs return Console instead of B edit; sign-out between valid() and setItem leaves one journal. These are not only same-context mock interleavings. WebKit is absent on the perf VM; no Safari private-mode qualification is claimed. All-storage-blocked behavior is covered by SecurityError controls.
Native UUID validation test and final fast are queued behind the host build slots. No production changes or deployment.
focused-final.log
eviction-offline.log
perf-regression-js.log
per-edit-perf-final.log
per-edit-perf-baseline.log
real-browser-races.log
Additional deterministic sequence for the existing P1 unsafe-retirement finding: two valid immutable sources each contain33 columns/unsent opens; their66-column union exceeds the decoder's64-column limit. cache.ts:94–95 falls back to the first source, but server-store.ts:55,63–64 still retires both sources. A fresh durable generation contains only33 of66 unsent IDs. This requires no storage failure. Test server-store.test.ts:793–813 reproduces33 missing IDs; tracer type check/atomic commit in progress. No demand to draw an oversized document: the assertion only requires omitted unsent entries to remain durable until explicit server resolution.
Real three-concurrent-adopter browser control passed: A last, x7, third view present, one compacted generation. Native gate still queued; no fixes/deploy.
merge-fallback.log
real-three-tabs.log
review-checklist.txt
Verdict: HOLD — 3 P1, 3 P2. No fixes or deployment. Reviewed production delta8bc0cfc6f..45e8bd4ea; test branch review-sync-journal-r2 currently pushed at
a2bab6216d. Final native gate/size receipts are still pending host build capacity; this is the finding record, not the finish notice.apps/tracer/src/canvas/model/storage-lifetime.ts:15–18,24,server-store.ts:87: A and B both read a missing marker; B publishes its nonce, then A publishes another. B becomes invalid without sign-out and its save returns before capture. Test4812ebc6b observes Console instead of B's edit. Independent real Chromium pages reproduce the same sequence with a debugger pause between storage calls.server-store.ts:58–61: B reads the valid lifetime; A removes the lifetime and finishes the complete sign-out sweep; B publishes its generation. One private journal survives. Test35094f0d3 and real independent Chromium pages both observe1 key, expected0. Storage events arrive too late to make the check/write pair atomic.server-store.ts:44–55,61–64,cache.ts:94–95. Two reproduced sequences: (a) reading the sole durable source throws SecurityError; the shared cache is older; successful replacement omits and deletes the source (5b69807ae). (b) two valid sources each contain33 columns; their66-column union exceeds the64-column projection cap, so merge falls back to the first source but adoption deletes both (a2bab6216).33 unsent open identities disappear before any server resolution. This test requires preserving the retry entries, not drawing an oversized layout;31 of those33 could still fit before server refusal of the final2.journal.ts:35–36,83–86: send an edit, lose its receipt, reconnect, then accept100 newer snapshots (including its applied state and a newer remote edit). The same entry remains held forever. Testa97648a22. The existing wire snapshot lacks operation identities; no resolution semantics or wire fix is chosen in review.server-store.ts:62–64,account-storage.ts:39:20 retirements fail, storage recovers, then a successful flush leaves21 generations. The live page does not retry orphan removal. Testa8adafd2a.server-store.ts:133,163–164,account-storage.ts:98: on locked4-vCPU perf VM, real Chromium149,60 windows and8192 pending edits, seven new-edit flushes take23.4–38.6ms; perf-only test800cd898b fails <=16ms. Separate31-save/7-flush scale series at8192/32768 entries gives save p99 3.7/14.1ms and flush median39.1/119.4ms. Round-1 baseline flush medians35.5/104.4ms also fail; this is inherited, not established as a new regression. Test runs only with INFRATERNAL_PERF_VM=1; shared-host timings are not evidence.Positive controls: all7 round-1 regressions pass; three truly concurrent adopters recover A's later name, B's cell and C's added view, then compact to1 generation;6/6 quota/security failures at adoption writes retain the durable source; every adoption mutation interruption preserves the edit;1024 offline edits replay after eviction (1024 pending entries,287408 UTF-16 journal bytes). Focused suite before the final fallback case:56 pass/5 deliberate fail; final expected intentional count is6 Bun failures plus1 perf-browser failure. Real SQLite version-0/version-4 receipt validation test is added and awaiting its gate before commit. No actual Safari private-mode qualification: WebKit is absent on the named perf VM; blocked-storage semantics are modeled with SecurityError.
Final fast output and fresh stripped-static size comparison will follow after the build-slot queue. Named socket-close flake will be recorded without investigation. Issue remains open.
lifetime-race.log
signout-boundary.log
unreadable-source.log
lost-receipts.log
focused-final.log
eviction-offline.log
perf-regression-js.log
per-edit-perf-final.log
per-edit-perf-baseline.log
merge-fallback.log
real-three-tabs.log
review-checklist.txt
browser-harnesses.json
Additional P1: startup storage failure permanently bypasses peer sign-out.
storage-lifetime.ts:20,24,26–27: initial lifetime setItem throws, setting nonce undefined; storage later recovers and the page writes journals successfully. Another context signs out and removes all existing account keys, but there is no lifetime marker to remove, so no matching storage event can ever arrive. Even after every generated storage event is delivered, the old tab's valid() stays true and a subsequent save recreates one journal plus the drawing. This is independent of the check/write race: there is no concurrent write or delayed event in this reproduction. The new server-store.test.ts case fails expected0/actual1 journal; type check and atomic commit are in progress. Current finding count becomes4 P1 +3 P2 (8 intentional regressions, including both source-omission sequences and the perf-only budget case). Native validation is compiling; no fixes/deploy.missing-marker-signout.log
Completed/pushed native recovered-ID validation test at
b2ca1df465. Real SQLite store accepts version-4 and version-0 IDs with independent permanent receipts; identical retries retain revisions and changed inputs are refused. Fmt, touched-server all-target clippy, and the single native test pass. Branch review-sync-journal-r2; final full fast starts now. No production changes or deployment.Additional inherited P1 contract gap: a completed offline save is lost on an abrupt stop before the coalesced writes.
server-store.ts:87–89,133,account-storage.ts:98–99: capture changes memory and save resolves, but both persistent copies still wait for the300ms callbacks. Cancel the page's scheduled work without invoking disposal, then construct a fresh page solely from stored state: expected Captured before stop, restored Console. New server-store.test.ts:847–862 reproduces this with virtual callbacks, no wall-clock assertion. Normal visibility/pagehide transitions ARE explicitly flushed (canvas/create.ts:187–189); this finding is specifically the brief's abrupt-stop boundary, not graceful tab backgrounding. The current unqualified no-edit-loss contract does not carve out completed saves in that window. No production fix chosen.Current verdict: HOLD —5 P1,3 P2;9 intentional regressions (8 unit,1 perf-only). Type check/atomic push of the latest test are in progress. Full fast began on
b2ca1df46; its checked Rust inputs have not changed, and this TypeScript-only test was added before the frontend phase. All Rust cases reported so far pass, including the named socket-storm flake and vt_credit. Raw final gate and static comparison remain pending. No deployment.stop-before-flush.log
Native milestone on review-sync-journal-r2, pushed HEAD
ab50302bb9: all Rust test batches passed; vt_credit and the named full-place socket-storm case both passed. Fmt, VT fmt, CLI compile-out, workspace clippy, SSH helper, inode fixture and cargo-deny passed. Static builds are now running, then frontend. Final TS-only abrupt-stop regression was added before frontend began; no previously checked Rust input changed. No fixes or deployment.Completed test-batch footers, verbatim:
Completed/pushed
e157f9b01on review-sync-journal-r2: the abrupt-stop durability regression now observes stored state after save resolves, without replacing persistence timers. It still fails (expected Captured before stop, restored Console;2 assertions), while allowing a future save implementation to await asynchronous local persistence. Tracer type check and diff check pass. No production change. Full fast has passed every Rust test and is building both static targets; frontend has not begun and will include this final fixture. Current head e157f9b01,12 atomic review commits, no fixes/deploy.Full scripts/verify fast completed with exit1 on the final review tree (HEAD
e157f9b01). Every Rust row PASS; named socket-storm case and vt_credit both PASS, no waiver. Frontend has372 pass/8 deliberate review failures, no other failures. Native inputs stayed unchanged while final TypeScript regressions were added before frontend began. Both exact static caps pass:21389296 B x86_64 and19561656 B aarch64. The intentional failures stop frontend dispatch before connection/terminal/build/transfer/protocol stages; those stages are being verified separately, followed by fresh same-recipe baseline size comparison. No fixes/deploy; not the finish notice yet.Verbatim full fast summary:
verify-fast.log
verify-fast-summary.txt
Starting #250 P3 sync follow-up on branch
sync-p3s, baseabedd62b068cbd0af0c51488abb0a6e0ddf4ce4e(origin/release-prod-live). Read the #250 sync hunt findings 3–5 and prior issue state. I’ll add failing Bun regressions first with simulated 200 ms RTT, then address drop recovery messaging, single-snapshot resubscription, and immediate gap recovery. Journal persistence edits remain onfix-sync-journal-r2and are out of scope. No deploy.P3 sync regressions reproduced before implementation on
abedd62b0.INFRATERAL_TEST_RTT=200 bun test src/canvas/model/server-store.test.tsreports 23 pass / 3 fail / 168 assertions: reconnect resubscribes at 500 ms before the modeled 600 ms snapshot; a revision gap sends no immediate resubscribe; and a dropped call invokes the notice callback immediately. Tests are inapps/tracer/src/canvas/model/server-store.test.ts:420–507. Failing outputFinding: the #250 task specifies a fresh-link notice budget of three measured RTTs + 50 ms, while DECISIONS.md §267 (line 2654) amends D265 to four RTTs + 50 ms. I am following the explicit #250 task budget and have not changed decision text. Recording this discrepancy for review.
Step complete:
902957058adds retained measured RTT and the current resume/fresh recovery class to MuxConnection. Focused connection tests: 16 pass, 0 fail, 68 assertions.bun run check: exit 0.Step complete:
e99bc6c94updates the server canvas recovery state machine. The 200 ms RTT focused suite reports 26 pass, 0 fail, 176 assertions. It covers a 600 ms outstanding snapshot with no duplicate Subscribe, immediate gap resync, and quiet resumed recovery through 250 ms versus notice at 650 ms for a fresh link.bun run check: exit 0.Step complete:
3b94b1f28maps delayed layout recovery to the shared neutral clock toast and preserves error tone for definitive refusals. The tracer TypeScript check passed before commit. Production browser screenshots are queued in build slot 4 at port 4253.Correction to step 3 note: the wrapper uses BUILD_SLOTS and defaults to three slots; my wording “build slot 4” was inaccurate. The screenshot job remains queued; the local health check on 4253 returned 000, and no preview server has started yet.
Progress: the production screenshot run remains queued behind the four active build slots. The local health endpoint on port 4253 is not listening yet, so no browser evidence was produced. The queued job will continue when a slot is released; no owner action is needed.
Finding: the first production screenshot run hit Playwright's default 30 s test timeout while starting the fifth viewport context; the error context is in
evidence/playwright/p3s-toast-visual-capture-t-1e5d7-e-shared-production-surface/error-context.md. Four screenshots were captured before the timeout. No UI assertion failed. I raised the temporary capture test timeout to 120 s and am rerunning it.Production UI review evidence: the screenshot-only Playwright capture passed (1 test, 0 failures, six viewport/theme states; 39.9 s) against the production tracer bundle at TRACER_TEST_PORT=4253. Attached states:
Gate status:
scripts/verify iterateis queued through build-slot at pushed HEAD3b94b1f28. Its log is still 0 bytes; the gate has not started and has no failure output. Waiting for a shared slot to be released.Finding from
scripts/verify iterate: Rust gate test failed atcrates/infraternal-agent/src/daemon_gone_tests.rs:119withInvalidInput: path must be shorter than SUN_LEN. The run usedTMPDIR=/home/kayg/.cache/codex-scratch/sync-p3s/tmp; no Rust files are changed. I will rerun the focused test with a shorter scratch alias, then rerun iterate.The vanished-session socket test passed with the short scratch TMPDIR: 1 passed, 0 failed, 75 filtered. The earlier SUN_LEN failure was caused by the longer job TMPDIR; iterate is rerunning with the short alias.
Starting sync-journal-r3 at
e157f9b01a(origin/review-sync-journal-r2). Retrieved #250 and parent #243 history, read round-2 verdict and hunt. Baseline focused tests: 52 pass / 8 fail / 547 assertions. Includes the two added P1 cases (failed initial marker and completed save before flush). Implementing the approved per-account Web Lock, conservative shared-key fallback, entry-level durable writes, and bounded receiptless recovery. No deploy or wire/dependency change planned. D106 sets iOS/Safari 17. D259 recovery allowance interpreted as 50 ms after the reconnect snapshot returns (the round-trip leg has completed).Pushed
9fa090cdfbounded held-edit recovery,295a6beadaccount lock/sign-out boundary,0db0decaffixture correction. Recovery slice: 8 pass / 28 assertions. Account/lock slice after correction: 7 pass / 26 assertions; preceding commit mistakenly described the 6-pass/1-fail fixture run as passing (raw log retained); corrected immediately without rewriting history. Tracer check and fmt pass. WebKit release notes confirm Web Locks shipped in Safari 15.4: https://webkit.org/blog/12445/new-webkit-features-in-safari-15-4/ ; D106 floor is Safari 17. Current uncommitted serialized store passes all 61 focused journal/cache/store/hunt tests (556 assertions), including all eight intentional baseline failures. Full-state per-save writes are deliberately the next optimization step; no latency claim yet. Source retirement now requires replacement coverage, including explicit settled operation identities, and failures remain retryable.Pushed
fd84440c0locked store;54cfefe01journal change tracking/projection;a9bb38e0cimmutable append storage;1a83e9e46production integration and durable sent markers. Current focused controls: 65 pass / 612 assertions (additional D259 deterministic expiry test included, not yet committed). Perf VM qualification: 4 vCPU, Chromium, 60 windows, 8192 pending entries, 7 measured new edits; completed saves 2.5–3.3 ms, awaited flushes 0.6–1.1 ms, 2,362,938 stored UTF-16 bytes. Load 0.04/0.01/0.00 before and after. Both save and flush <=16 ms. Native Web Locks enabled. Baseline review measured 23.4–38.6 ms flushes. perf-journal.logProduction build is waiting for a host build slot. Independent-tab functional tests and final fast/static comparison follow. No deploy.
The short-TMPDIR iterate rerun reached the workspace suite: 620 passed, 2 failed, 52 ignored. Failures were
auth::tests::socket_tests::review_authority_cap::every_socket_in_a_full_place_storm_fails_closed_without_a_reply(15/16 closed) andbootstrap_runtime::remote_backend::tests::retirement_contention::cancelled_invitation_retires_after_sqlite_busy_timeout. The socket-cap test passed alone (1 passed in 2.20s). The first retirement rerun selected zero tests because theremotefeature was missing; a matching-feature rerun is queued. Evidence: iterate log, socket-cap focused log.Both workspace failures passed when run alone: the socket-cap test 1/1 in 2.20s, and the remote-enabled SQLite retirement test 1/1 in 6.66s. The latter requires
--features remote; rerun evidence: retirement test. I’m repeating the full iterate gate to confirm the workspace result.Pushed through
eda6510f1on sync-journal-r3; focused head controls: 83 pass / 690 assertions. All 8 intentional baseline failures now pass, plus earlier review/hunt tests. Additional source checks cover an unexpected multi-entry operation key and the fallback metadata key; both preserve uncovered entries. Legacy shared-cache checkpoint quota regression reproduced before repair (expected 2 retry entries, only 1 restored);ae9fb8eb0retains the sole legacy source and writes metadata separately.a6bec85b9applies coverage to every replacement;eda6510f1rebases the cached projection over unrelated remote fields after its failing local-rename/remote-cell regression.92e506c30preserves fresh lifetimes through delayed origin-wide clear events.Real VM qualification (4 vCPU, Chromium 149.0.7827.55): 60 windows, 8192 pending, 7 measured new edits per case. Checkpoint seed: save max 3.7 ms / awaited flush max 1.0 ms. Separate operation keys under actual browser quota pressure: checkpoint count 0, entry count 8192; save max 4.3 ms / flush max 4.5 ms. Origin stores 9399484 UTF-16 bytes including quota controls. Native Web Locks enabled; three independent-tab browser controls pass. Load 0.08/0.04/0.01 to 0.15/0.05/0.01. Final fixture now additionally checks restoration of the last completed save.
Initial production transfer: gzip 307195/307200 B, Brotli 256034/256000 B (34 B over);
871ca3fc1removes nested promise paths instead of changing caps. Deferred was 60171/61440 gzip and 53337/55296 Brotli. Iteration gate is in native tests after fmt/compile-out/clippy; first 99-test native batch passes. Final fast, fresh static comparison and final transfer qualification still pending. No native source, dependency, wire or deployment change.perf-browser-capacity-final.log
projection-rebase-before.log
multi-entry-before.log
shared-quota-before.log
The second iterate run completed with 621 passed, 1 failed, 52 ignored. The SQLite retirement case passed in the full run; the only failure was
auth::tests::socket_tests::review_authority_cap::every_socket_in_a_full_place_storm_fails_closed_without_a_reply(10/16 closed). It passes alone in 2.20s. I’m checking the same test on detachedorigin/devunder the current host load. Evidence: second iterate log.The exact socket-cap test passed on detached
origin/devunder the same host window (1 passed in 2.02s); it also passed alone on this branch (1 passed in 2.20s). Only the full parallel suite times out (10/16 closed at the 15s deadline). Evidence: baseline test. I’m rerunning iterate withRUST_TEST_THREADS=2to reduce in-process contention.Round-3 progress: pushed through
9a4f5bd6e. A strengthened real-browser cold restore exposed localStorage enumeration ordering (latest name became an older pending name).9aa502875reserves durable sequence numbers and seeds replay from the newest complete checkpoint.aed2eeb94isolates locked metadata from the legacy key, so a fallback writer cannot have a newer source overwritten.7638e3a4apreserves used recovery attempts/held status when old copies reappear. No wire/dependency changes.Focused tests: 90 pass, 0 fail, 714 assertions; tracer TypeScript check passes. The new ordering and held-copy regressions failed before their fixes.
Locked perf VM, 4 vCPU, 60 windows × 8192 pending: 5/5 browser cases pass. Seven measured new edits per storage mode, maximum save 4.4ms, maximum awaited flush 2.7ms, both below 16ms. Fresh stores restore “Measured 7” in checkpoint and quota-constrained 8192-entry modes. Stored UTF-16 bytes: 4,693,784 and 9,111,368 (latter includes quota controls); journal bytes: 2,360,664 and 4,891,588. Native cross-tab/sign-out/fallback cases pass. This receipt supersedes earlier browser results that did not assert post-write cold replay.
perf-browser-ordered-final.log
ordering-browser-before.log
metadata-isolation.log
held-merge-before.log
Final frontend gate is waiting for a build slot. The initial native iterate failure was a Unix-socket path length caused by this job’s TMPDIR, corrected using a short alias to the same scratch directory; that native test passes with the corrected environment. Full fast and static-size comparison remain pending.
Pushed
95b448b74: trimmed the account-storage critical path without raising transfer limits. Previous frontend run: all 524 tracer tests pass (402 + 105 + 17), but critical transfer was 307209/307200 gzip and 256086/256000 Brotli. Shared prefix discovery, unused cleanup arguments and redundant filters removed; 67 affected tests (580 assertions) and tracer typecheck pass. Frontend/full fast rerun queued for a host slot.Final current-code perf-VM browser run: 5/5 pass, 60 windows × 8192 pending, maximum save 4.2ms, awaited flush 1.6ms; cold restore and native cross-tab/sign-out remain green. No build-host performance claim.
perf-browser-head-final.log
frontend-size-before-trim.log
Pushed
425bbec55: simplified the unavailable-lock path and immediate lifetime revocation without a lazy import (offline sign-out must not require fetching another chunk). 69 affected tests pass, 588 assertions; tracer TypeScript check passes. Source ordering/coverage and retry semantics remain unchanged.Refreshed native-browser proof for this exact head: 5/5 pass. At 60 windows × 8192 pending, seven measured inputs in each storage mode; latest maxima are save 5.1ms and awaited flush 1.8ms, both below 16ms. Both cold restores recover the latest name. This supersedes the prior head’s performance values.
perf-browser-head-final.log
compact-lock-tests.log
Final frontend/full fast is still queued behind host build slots. No new failure is being waived; transfer budget, final static gate and fresh base comparison remain required before finish.
Capacity update: final frontend/full fast for pushed head
425bbec55has been queued in the host build-slot wrapper since 23:15 CEST (23:38 now). All four slots remain occupied. Continuing to wait; no gate exception or additional slot is being used. Native-browser/performance and affected TypeScript/model checks are already recorded above. Final transfer check, full fast, fresh stripped base comparison and production screenshots still remain.Pushed
1970667b3and83e876dde: grouped acknowledgement updates and removed the redundant post-settlement revision predicate. The two-pass settlement/filter invariant remains (later duplicate copies can settle earlier ones). 90 focused tests pass, 714 assertions; tracer TypeScript check passes.The production build on
425bbec55passed all 524 tracer tests and gzip transfer (307167/307200), but Brotli was 256004/256000: 4 bytes over. Limits stay unchanged. The new head is queued for a frontend rebuild, full fast and static comparison. No native source/dependency/wire change.frontend-425bbec55.log
prune-trim-tests.log
Branch-head
scripts/verify fastpassed (exit 0) at3b94b1f28de4b7e5bf0d0241bee38acbed5646ad; run usedRUST_TEST_THREADS=2,CARGO_BUILD_JOBS=2. Both previously reported server tests passed. Static sizes: x86_64 21,389,296 bytes; aarch64 19,561,656 bytes. TypeScript protocol suite: 39 pass, 0 fail, 36,135 expectations. Full transcript: verify-fast-final.log.Verbatim Rust gate summary:
Head
83e876ddepreflight production build passes unchanged limits: critical gzip 307173/307200 bytes, Brotli 255890/256000; deferred gzip 61092/61440, Brotli 54173/55296. Full scripts/verify fast is running: fmt, VT fmt, plugin compile-out and clippy passed; native tests continue without a recorded failure so far.Final exact-head browser run: 5/5 pass on the locked 4-vCPU perf VM, Chrome 149.0.7827.55, load 0.00/0.00/0.00. At 60 windows × 8192 pending, seven measured new edits per mode: maximum save 4.6ms, maximum awaited flush 3.9ms. Both modes cold-restore “Measured 7”. Checkpoint/entry journal UTF-16 byte counts 2,360,664/4,891,588; origin totals 4,693,784/9,111,368 (second includes quota controls). No build-host timing is performance evidence.
Four production-build screenshots are attached for independent visual review: ink/paper, desktop 1440×900 and touch-phone emulation 393×852, DPR 1/2. Driver exposes only the existing shared toast on the real production core; recovery behavior is verified separately by model and native-browser tests. Text and viewport bounds pass; phone pointer is coarse. Author makes no visual-quality judgment. Owned mock-server PID 2289342 is confirmed gone.
perf-browser-final-head.log




frontend-final.log
notice-capture-final.log
ui-production-build.txt
Final full-gate transcript, fresh stripped-static base comparison and cleanup remain before finish.
The first merge-base size build stopped before Rust compilation: its separate target lacked
auth-tools/zig/zig(sccache: Compiler not supportedfrom.cargo/musl-cc.sh). I’m initializing the baseline target withscripts/auth-toolchainand retrying. Evidence: build log.Full fast milestone on
83e876dde: fmt, VT fmt, CLI plugin compile-out, clippy, SSH runtime helper, workspace tests, SSH inode fixture and cargo-deny have completed without a failure waiver. The real backend recovered-version-zero receipt regression passes. Static release builds/linkage/key-memory/size checks are running now; final TypeScript/protocol phase and fresh baseline comparison follow.Live transcript: /home/kayg/.cache/codex-scratch/sync-journal-r3/evidence/verify-fast-final.log (full verbatim attachment follows when complete). Perf-VM job directory is removed; its shared dependency target was retained. No deployment or protected-branch push.
Size comparison complete at HEAD
3b94b1f28de4b7e5bf0d0241bee38acbed5646ad. The identicalscripts/static-cargo build --locked --release --package infraternal --target x86_64-unknown-linux-muslrecipe produced stripped static binaries of 21,389,296 bytes at merge baseabedd62b068cbd0af0c51488abb0a6e0ddf4ce4eand branch head, delta 0 bytes. Baseline build: 27m 06s. Receipt: size-baseline-x86.log.83e876ddex86_64 static milestone: production trust PASS; ELF is static-pie linked and stripped; PT_INTERP absent. Size 21,389,296 bytes, exactly the unchanged 21,389,296-byte cap. Key-memory initialization=1, locked pages and dump exclusion enabled. Aarch64 static build is running. Freshe157f9b01base comparison remains after the final gate (the zero cap delta is not that comparison).Verbatim check lines from /home/kayg/.cache/codex-scratch/sync-journal-r3/evidence/verify-fast-final.log:
The real-engine check is waiting for a host build slot. The queued production tracer build has not started; all four regular slots remain occupied by active runs. No failure occurred. Need one slot to be released; the build-slot wrapper is still waiting.
Finished at branch head
3b94b1f28de4b7e5bf0d0241bee38acbed5646ad(sync-p3s, pushed toorigin/sync-p3s; worktree clean).The simulated 200 ms RTT Bun suites passed: 42 tests, 244 assertions. The production real-server layout sync case passed 1/1 with
REAL_ONE_WAY_MS=100(100 ms each way, 200 ms RTT), in 31.0 s. Ten windows restored on a fresh phone with matching order; concurrent edits converged, and offline reconnect reconciled to seven windows. Receipts: real-layout-sync.log, functional.json, desktop screenshot, phone screenshot.Stripped static x86_64 size: merge base
abedd62b068cbd0af0c51488abb0a6e0ddf4ce4e21,389,296 bytes; branch 21,389,296 bytes; delta 0 bytes. Baseline build receipt.Final
scripts/verify fastexited 0. Gate output:Full final fast transcript.
Current pushed head
1ee6a8769(after503b4d8d3and293acab5e). All 103 focused journal/account/inventory tests pass, 764 assertions; tracer TypeScript check passes.New reproduced ordering finding: a transient failure to enumerate source keys was treated as an empty complete source set. A later saved name restored as Earlier instead of Latest, despite both records remaining.
293acab5epropagates enumeration failure and marks the read incomplete; new edits append without publishing a complete checkpoint. Regression fails before and passes after.503b4d8d3moves journal-only enumeration out of the critical module and replaces scalar-prefix scans with direct key removals. Measured fixture work: 122 key reads for 38 keys before; 37 after, with peer data intact.1ee6a8769moves the unchanged JSON cache controller alongside its only production consumers (deferred canvas/inventory), leaving sign-out dependencies and lock transactions intact.Full fast on
83e876ddepassed every Rust row, both exact static caps (21,389,296 / 19,561,656 bytes), typecheck and all 524 tracer tests, but its final critical Brotli was 256017/256000. An earlier build of that same head was 255890. The293acab5epreflight passes the actual limits (gzip 307123/307200; Brotli 255944/256000), but lacks enough room for the observed metadata variation. Repeated preflights with extra room run before the next full fast. Limits are unchanged. No native or wire changes.verify-fast-83e876dde-before-trim.log
cleanup-scan-before.log
cleanup-scan-after.log
listing-recovery-before.log
listing-recovery-after.log
New-head browser refresh, final full fast, fresh static base comparison and local cleanup remain. Earlier browser/UI receipts are labeled with their actual head
83e876dde.Completed native-browser regression step; pushed
cf98d289ce. The production store now has real-page closure coverage for source-key enumeration failure (the low-level before/fix receipt is in the preceding comment). Type checking and all six Chromium cases pass.Exact-head measurement: perf VM infraternal-perf-test, 4 vCPU, Chrome 149.0.7827.55, exclusive measurement lock; load 0.13/0.03/0.01 before and 0.20/0.05/0.02 after. At 60 windows and 8192 pending edits, seven measured new edits in each storage mode: checkpoint max completed save 4.6ms / awaited flush 2.1ms; quota-constrained 8192-entry mode max save 4.2ms / flush 3.0ms (16ms ceiling). Both cold restores recover Measured 7 after eight real completed saves. Native-lock concurrent first tabs, sign-out vs queued writer, rejected-lock conservative fallback, and unavailable-source-listing restore all pass.
perf-final.log
Two production transfer preflights and full scripts/verify fast are running on this head; the same-recipe stripped static baseline comparison follows. No deploy.
Production screenshot refresh for
cf98d289ce. The capture driver mounts the existing packages/ui toast on the real production core, in touch phone 393x852 DPR2 and desktop 1440x900 DPR1, ink and paper. It is a notice/geometry capture, not the end-to-end recovery flow; six native-browser state/performance cases cover journal behavior separately. Notice text is present once, within the visual viewport in all four captures. No author visual-quality verdict. Owned mock-server PID 2291357 is confirmed gone; the job directory on the perf VM is removed without touching the shared browser installation.notice-capture-final.log




ui-production-build.txt
Full gate is now running after two passing repository transfer preflights (critical Brotli 255955 and 255815 / 256000 bytes).
Milestone: scripts/verify fast PASSED on pushed head
cf98d289ce. All native rows pass; tracer 404 + 105 + 17 = 526 tests and protocol 39 tests pass. No baseline exception was needed. Full raw output is attached verbatim; the following blocks are exact excerpts (build-host check durations are not performance evidence).verify-fast-final.log
gate-final-summary.txt
The wrapper is now building the
e157f9b01merge-base stripped static binary with the same scripts/static-cargo recipe and one target directory. Size delta and local cleanup are pending; no deploy.Additional concrete recovery ordering finding, fixed and pushed at
573dc412e3. After D259 expiry but before its coalesced settlement flush, the already-issued retry receipt replaced changes[op_id]=settled with a pending receipt entry. Cold restoration revived one expired retry (expected 0). New production-store regression failed before the fix. Acknowledgement now ignores settled identities; ordered server events remain authoritative. Forcing an immediate flush was rejected because it narrows rather than removes the interleaving.102 focused tests, 766 assertions and TypeScript checking pass.
late-receipt-store-before.log
late-receipt-after.log
The full green gate previously recorded is for
cf98d289c. A fresh final-head gate and perf receipt will follow for this correction; the unchanged native merge-base size build is finishing first. No deploy.Exact-head browser/performance step complete for
573dc412e3: all six cases pass on locked 4-vCPU perf VM, Chrome 149.0.7827.55; load 0.00/0.00/0.00 before and 0.08/0.02/0.01 after. At 60 windows and 8192 initial pending edits, checkpoint mode max completed save 4.0ms / awaited flush 1.6ms; quota-constrained 8192-entry mode max save 12.2ms / flush 2.3ms. Both are below 16ms. Seven measured edits per mode, plus a warmup edit; both cold restores recover Measured 7. The 12.2ms maximum is retained as measured.perf-final.log
Full gate on this corrected head remains pending the currently active merge-base native build. The previous green gate belongs to
cf98d289c.Native merge-base measurement complete:
e157f9b01stripped static x86_64 binary is 21389296 bytes; same scripts/static-cargo recipe as the greencf98d289cgate, byte-identical SHA256 e98f7dfafe827f89fc94d442235f886000a253a0cfbd1213983b864db46b7bf0, delta 0. Its binary is preserved for comparison with corrected head573dc412eafter that head's final gate. No Rust/Cargo changes in this lane.The new final wrapper is queued on the host build slot. New-head browser/perf checks are complete (6 pass, max save 12.2ms / flush 2.3ms). Perf VM has no remaining processes with the job directory as cwd; the owned directory is removed and shared browser installation retained. Production notice captures remain those of
cf98d289c; the only subsequent change is the settled-receipt guard and regression, with no UI component/copy changes.cf98d289c-static-size-comparison.txt
perf-cleanup-final.txt
kayg referenced this issue2026-10-28 17:53:56 +00:00
Resumed verification review on review-sync-p3s at
3b94b1f28d(baseabedd62b06). Retrieved all #250 and parent #243 comments. Tests only; product code unchanged. Continuing independent recovery ordering/notice/subscription review; no deploy.Independent review findings on
3b94b1f28(product unchanged):Six independent controls pass, including ordered offline replay and 20 drop/reconnect cycles with zero retained timers/subscriptions/listeners after disposal. Two new regressions fail as intended. No P0/P1 established.
Integration: sync-journal-r3 overlaps constructor/save/event/resync/pump/dispose and product notice callback; preserve async lifetime/storage gating and held-operation expiry when composing these histories. No merge performed. Existing 3-vs-4 RTT discrepancy remains: D267 and wire contract say four, branch says three.
review-tests.log
findings.txt
Perf-VM relay request for independent review-sync-p3s (#250): please run the existing real-server layout-sync functional qualification with 200 ms shaping on perf-test-01/02/03, recording machine/load. No new benchmark or load generator.
Artifacts after final fast: /home/agent/.cache/codex-scratch/review-sync-p3s/target/x86_64-unknown-linux-musl/release/infraternal and /home/agent/wt-infraternal/review-sync-p3s/apps/tracer/dist. Harness: this branch's apps/tracer/real and playwright.real.config.ts plus packages/protocol/src and tracer/test/evidence-path.ts.
Exact command from apps/tracer on the perf VM (set INFRATERNAL_BIN/TRACER_DIST to copied artifact paths, INFRATERNAL_EVIDENCE_DIR to private evidence, TMPDIR and INFRATERNAL_TEST_STATE_ROOT to a short private runtime root):
TRACER_TEST_PORT=4257 REAL_ONE_WAY_MS=100 /usr/local/bin/perf-run.sh -- bun x playwright test -c playwright.real.config.ts real/layout-sync.spec.ts --grep 'fresh phone restores ten split windows and both devices converge'
Build-host functional run will also be attempted; no build-host performance numbers are evidence. The test-only review changes no native source, dependency or build profile.
Completed/pushed
d8b298839: independent test-only recovery controls. Tracer typecheck, cargo fmt and focused suites pass (32 tests, 260 assertions). Two consecutive gaps keep one outstanding snapshot per recovery; late snapshot cannot replace a newer contiguous delta; two offline edits wait for snapshot then send distinct identities in order; 3s RTT and changing RTT obey virtual budgets; 20 drop/reconnect cycles retain one subscription/listener while live and zero timers/subscriptions/listeners after disposal. Native iterate still compiling on the migrated build VM; browser visible-notice regression is prepared, pending the real binary. No product edits.Completed/pushed
5916174ee: two intentional P3 failing tests for silent pending edits (offline-at-capture; no accepted RTT sample). Focused review output: 6 pass / 2 fail / 87 assertions. Tracer TypeScript check passes. No product edits. Native iterate has reached workspace clippy; real-server visible-notice regression and final fast/static receipts remain pending the build.Iteration finding: CLI compile-out test setup_token_tests::setup_token_command_reads_only_the_expected_socket_response failed at crates/infraternal/src/main.rs:1368 before its assertion: InvalidInput, path must be shorter than SUN_LEN. TMPDIR was the long job evidence/tmp path. Workspace clippy and runtime helper passed. Final gate will use the owned 34-byte alias /home/agent/.cache/codex-scratch/p -> review-sync-p3s/evidence/tmp; no source change or waiver. Full iteration output will be attached with final receipts.
Migrated-host prerequisites repaired: installed repo-pinned cargo-deny 0.20.2 and pinned Chromium 149.0.7827.55 (Playwright revision 1228). Initial real-server run stopped before product execution because the headless browser executable was absent; retry queued.
Literal apps/tracer bun test output (verbatim summary):
557 pass
154 fail
151 errors
81759 expect() calls
Ran 711 tests across 249 files. [36.68s]
151 errors come from loading Playwright .spec.ts with Bun's unit runner; one additional failure was missing dist/chunks.json before build; the remaining two are the intentional P3 review regressions. Production build now succeeds. All .test.ts files are queued separately after build, followed by the requested real-server case and visible-notice regression with REAL_ONE_WAY_MS=100, TRACER_TEST_PORT=4257. No build-host timing claims.
Owned build cleanup removed debug/incremental and stripped completed debug executables, reclaiming 6.3 GiB plus 17,284,168,392 bytes. No shared directory or process touched.
Capacity finding: shared filesystem free space is now 15 GiB (94% used), despite this job reducing its target from 30 GiB to 8.0 GiB after the native iteration. This job's active target is review-sync-p3s/target only. Corrected browser run is queued in build-slot; final two static targets and same-directory baseline comparison still needed. No other job directory or process touched. Orchestrator may need to reclaim idle shared build output before static qualification; I continue the functional work and will remove my target at finish.
Corrected notice browser case remains queued in build-slot with no test output yet. The requested ten-window real-server layout-sync case already passes at REAL_ONE_WAY_MS=100; the added case's first product attempt stopped in its clock harness (host time lagged the installed browser clock), now corrected to sample browser Date.now(). No visible-notice product finding claimed yet.
Additional exact integration risk: sync-journal-r3 at
573dc412eemits refusal state "unconfirmed". The sync-p3s product callback would treat that string as a definitive refusal and show the wrong rollback message; retaining the journal callback instead would ignore the delayed flag and restore error-tone transient notices. Both states must survive composition. No merge performed.Host scheduling observation: /usr/local/bin/build-slot.sh currently iterates three slot files (for i in 1 2 3), while this brief describes two. This job uses one wrapper invocation at a time and does not alter the shared wrapper. Free disk is 13 GiB after this job's target cleanup.
Blocked on shared disk capacity for final static qualification: df now reports 760 MiB available / 100% used, even after removing this job's completed cargo-deny install build output and unused Zig cache. Earlier owned cleanup reclaimed ~23 GiB; current target is ~7 GiB. Other writers consumed space during cleanup. Orchestrator needs to reclaim idle shared output or let active jobs finish cleanup; this job cannot safely complete cold static targets at this capacity and will not touch another job's paths/processes. Corrected single browser check is still queued; functional ten-window/200 ms case already passed. Two P3 notice regressions are pushed at
5916174ee; no product edits or deploy.Continuation start: sync-journal-r3 restored at
573dc412e3; integrating origin/dev63773bb3econ the migrated shared build VM. Retrieved all #250 and parent #243 comments. Retaining the six-case perf receipt and earlier review/hunt tests; sync-p3s is outside this lane. Next: merge resolutions, focused regression tests, sequential pinned static size measurements and final scripts/verify fast. No deploy; only this branch will be pushed.Merged dev
63773bb3eand pushed sync-journal-r3 at660aa612b3. Conflict resolution retains both layout-storage-lock and onboarding-load tests in the frontend gate. Dev pin convergence fixture now checks exactly one dispatched edit plus exact peer coordinates, because pending entries no longer live in the shared cache. Initial fixture failure: 124 pass / 1 fail (legacy cache assertion); corrected run: 125 pass / 0 fail / 9,053 assertions across 16 files, including all earlier journal review and hunt regressions. Tracer typecheck, cargo fmt and diff checks pass. Native production inputs match dev; only the prior receipt-validation test differs. Static measurements and final fast remain.D243 complete at pushed head
660aa612b3. Sequential pinned scripts/static-cargo builds through /usr/local/bin/build-slot.sh produce x86_64 21,386,512 B and aarch64 19,563,608 B; deltas versus dev caps 0 / 0. Both full SHA-256 hashes exactly match the #255 artifact receipt (a981fc26-8d9c-4f89-9b1c-35fc702a8089): the journal lane leaves both native images byte-identical. Caps and docs/PROCESS.md table remain unchanged.static-comparison.json
static-head.log
merge-focused-final.log
Functional Chromium preflight could not launch: chromium_headless_shell-1228 is missing from the migrated VM browser cache; installing the pinned browser under HOME and retrying. No product assertion ran. Timing specs remain skipped on the build host. Final fast follows.
Merged-head functional browser qualification: 4 passed, 2 timing cases skipped, using pinned Chromium 149.0.7827.55. Concurrent first tabs, sign-out vs queued writer, rejected-lock fallback and unavailable-source-listing restoration all pass. The missing headless browser was installed; the first launch failure did not execute product assertions. Final scripts/verify fast is now active.
Perf relay for tonight, head
660aa612b3: no new timing result is claimed. Storage/recovery code and timing harness are unchanged from the earlier573dc412esix-case receipt (max save 12.2 ms / flush 2.3 ms). For a fresh merged-head measurement, the attached archive contains prebuilt browser bundles, two test files and a server-free config. Unpack into the perf job directory within its pinned Playwright 1.61.1 installation, then run from that directory (no checkout or build needed):Please record machine/load and all JOURNAL_REVIEW rows; six cases expected, including both 60-window/8192-entry modes. This job will not contact a perf host.
journal-perf-relay.tar.gz
Final-gate environment finding: my short fixture root /home/agent/.cache/codex-scratch/j3 was a symlink into this job scratch. Eight CLI restart/security tests failed on the placeholder build: PermissionDenied on admitted replacement execution and an explicit alias-versus-canonical path mismatch. This is not a journal source change or a waived gate. Stopped only this job verification process tree (root PID 3214972), replaced the owned alias with a physical mode-0700 directory at the same short path, and restarted full scripts/verify fast on unchanged
660aa612b. The prior interrupted transcript is retained below.verify-fast-symlink-root.log
Second final-gate setup finding on unchanged
660aa612b: daemon::gone_tests::a_vanished_session_gets_a_final_close_and_the_channel_stays_up failed at daemon_gone_tests.rs:119 before its assertion (SUN_LEN). Physical TMPDIR /home/agent/.cache/codex-scratch/j3 is 35 bytes; fixture suffix /g<24 hex>/sessions/<32 hex>.sock adds 73, giving 108 bytes versus the Linux 107-byte bound. The prior /home/kayg recipe was one byte shorter. Created an owned physical mode-0700 /home/agent/.cache/codex-scratch/j root (34 bytes, full socket 107) for the next full run. No source or threshold change and no failure waiver. This run passed clippy, helper/path checks and inode reuse; its workspace test row stopped at agent library 147 pass / 1 setup fail / 20 ignored. The process-per-user/static rows are finishing before the corrected full rerun.Finished continuation on
sync-journal-r3, pushed HEAD660aa612b330490e1adb23e214efa543159a7095; remote head verified and worktree clean. Merged dev63773bb3e(release 9.4, #255, VM fixture fixes and D311). No deploy; issue remains open.sync-p3swas not changed.Merge resolutions are recorded in the merge commit: retain both journal-lock and onboarding-load gate tests; replace the dev pin fixture's obsolete shared-cache pending assertion with exactly one dispatched edit plus exact peer-coordinate convergence. No new product behavior was needed for integration. All earlier journal review/hunt regressions pass: focused 125 tests, 9,053 assertions. Native real-browser functional cases 4/4 pass; the two timing cases are skipped on the build VM. Reviewer checklist completed for locking/lifetime, source coverage and cleanup retries, immutable sent inputs, held-edit resolution, account isolation, module docs and frozen wire/dependency scope.
D243: same pinned recipe, sequential builds through build-slot. x86_64 21,386,512 B, aarch64 19,563,608 B, delta 0 / 0 versus dev caps. Both full hashes match #255's recorded artifacts: byte-identical native images. Caps and PROCESS table remain unchanged. Final-gate artifacts were hashed again before removal.
Final
scripts/verify fast: exit 0, no failure waiver. Environment: sourced ~/infraternal-env.sh; prescribed CARGO_TARGET_DIR; CARGO_BUILD_JOBS=2, RUST_TEST_THREADS=2; development/test debug information and incremental output disabled, release recipe unchanged; physical 34-byte TMPDIR/state root. Earlier symlink and 35-byte socket-path setup failures are preserved in the preceding comments and attachments. Corrected full run passes agent library 148/148 active, server 634/634 active (including recovered-ID receipts and full-place socket cap), and process-per-user 199/199 active.Full gate output is attached verbatim. Exact excerpts:
No new timing claim after the merge: storage/recovery implementation and timing harness are unchanged from
573dc412e. The previous six-case perf result (max save 12.2 ms / flush 2.3 ms) retains that head attribution. A portable prebuilt relay archive and exact perf-run command for tonight are in the preceding comment; this job contacted no perf host.Cleanup complete: no process using the owned target remained; CARGO_TARGET_DIR and both owned physical short fixture directories were deleted. Evidence and prebuilt perf relay remain outside the checkout. No generated evidence was committed.
How to test:
static-comparison.json
static-head.log
merge-focused-final.log
verify-fast-final.log
browser-and-gate-retry.log
verify-fast-35byte-root.log
cleanup-processes.json
cleanup-final.json
Independent round-3 verification started on review-sync-journal-r3, detached target
660aa612b3; merge base with dev63773bb3ec. Tests only; implementation and job worktree remain untouched. Reviewing account locking/sign-out, source inclusion, fallback, held recovery and operation-count complexity; final fast will run on the job head. No deployment.Independent round-3 finding R3-01 — P1: fallback writer recreates account data after peer sign-out.
storage-lifetime.ts:16–25,32–42: unavailable Web Locks selects fallback without creating a lifetime marker. The listener watches only that absent marker (or clear-all), so removing the shared drawing on peer sign-out does not revoke the old tab. After delivering every removal event, its next completed save recreatesinfraternal.canvas.<account>.Failing test pushed in
review-sync-journal-r3at7aef0c626:apps/tracer/src/journal-r3-review.test.ts, testreview r3: unavailable Web Locks cannot let a peer recreate account data after sign-out. Expected key presence false; actual true; 0 pass / 1 fail / 3 assertions. The peer sweep bypasses the local revocation registry, exactly as independent browser contexts do. Implementation unchanged. Initial TypeScript check could not resolve missing installed Bun/Node type definitions; dependencies will be installed before the next check.This is a new P1 writes-after-sign-out interleaving. Under the stated round-three rule, redesign needed. Continuing remaining verification and the exact-job-head full fast gate before final counts.
Independent round-3 finding R3-02 — P1: queued startup creates a new lifetime after peer sign-out, with Web Locks working.
storage-lifetime.ts:19–25,32–39,server-store.ts:54–58: third tab holds account lock; peer sign-out queues first; another signed-in page constructs its store and queues startup second. Sweep removes old marker. Startup then sees absence and creates a new marker. Deliver the asynchronous old-marker removal after startup: it is ignored because the replacement nonce currently matches. A later completed save creates private journal state after sign-out.Failing test pushed in
review-sync-journal-r3at591852c75:apps/tracer/src/journal-r3-startup-review.test.ts,review r3: a startup queued behind peer sign-out cannot create a replacement lifetime. Expected zero private keys, received four: replacement lifetime, order, snapshot, metadata. 0 pass / 1 fail / 2 assertions. Tracer TypeScript check now passes after frozen dependency installation. Implementation unchanged.Current count: 0 P0 / 2 P1 / 0 P2 / 0 P3, requiring redesign needed. Exact-job-head fast continues independently in an owned detached checkout, so intentional review failures cannot contaminate that gate.
Independent review progress at pushed head
40900a1a27: 93 existing focused tests pass / 0 fail / 734 assertions, including earlier journal review and hunt cases. Tracer TypeScript check passes.R3-03 — P2,
storage-journal.ts:183–192: the ordinary 8192-entry append passes a counting control, but after checkpoint retirement throws SecurityError each later single-entry append re-reads every retained source. Two subsequent appends reach a maximum 8195 storage reads, against bounded allowance <=16. Committed failing counting testreview r3: cleanup refusal keeps each later 8192-entry flush bounded; no build-host timing claim. Source retention remains safe in this case; the gap is the unconditional O(1) flush claim.Current findings: 0 P0 / 2 P1 / 1 P2 / 0 P3. Verdict remains redesign needed. Exact-job-head
scripts/verify fastis in native compilation; final transcript follows. No product edits or deployment.findings.md
R3-01.log
R3-02.log
R3-03.log
Pushed
a0ffc82d5: moved all three review files intoapps/tracer/src/canvas/model/, the existing fast frontend test selection, so subsequent implementation gates execute them automatically. Paths now:review-r3-fallback.test.tsreview-r3-startup.test.tsreview-r3-flush.test.tsOnly moves/relative imports changed. Fresh run: 1 pass / 3 intentional fail / 10 assertions. Tracer TypeScript check and diff check pass. Earlier finding commits remain in history. Exact job-head fast remains independent at
660aa612band has passed native tests/advisories; sequential static builds are running. Verdict/counts unchanged: redesign needed, 2 P1 + 1 P2.Independent review verification update: exact job head
660aa612b330490e1adb23e214efa543159a7095passedscripts/verify fast(exit 0). Chromium storage fixtures reproduce R3-01 and R3-02; existing four browser cases pass, as does a holder-closure / unrelated-account control. R3-02 deliberately delays storage listener callbacks; this is a store/lifetime ordering result, not an end-to-end auth-flow claim. Final static-size comparison against merge base is running before handoff.findings.md
verify-fast.log
review-tests-final.log
focused-with-review-final.log
review-check-final.log
browser-functional.log
real-tab-proof.json
Independent round-three verdict: redesign needed. Counts: 0 P0 / 2 P1 / 1 P2 / 0 P3. The round-three stop rule applies because R3-01 and R3-02 are new P1 writes after peer sign-out. Do not continue another patch/review round; return the lifetime/sign-out design to the owner/orchestrator. No implementation changes were made.
Target:
660aa612b330490e1adb23e214efa543159a7095; merge base:63773bb3ecd990b09d3663c1bd32b2cd8e49991f. Review evidence branch:review-sync-journal-r3, heada0ffc82d50cc1460da33b70c75f470cd11e691fc(pushed; tests only).storage-lifetime.ts:18,33,42: without Web Locks the old page has no lifetime marker and ignores removal of the shared drawing. Its next save recreates private account data after peer sign-out. Failing test:review r3: unavailable Web Locks cannot let a peer recreate account data after sign-out. Chromium reproduces one recreated key.storage-lifetime.ts:19–25,35–38: startup queued behind sign-out creates a new marker after the sweep and accepts a delayed removal event because the new nonce matches. A later save recreates four private keys. Failing test:review r3: a startup queued behind peer sign-out cannot create a replacement lifetime. Three-tab Chromium reproduction uses real locks and buffered listener delivery; this verifies the store/lifetime ordering, not an end-to-end authentication-flow claim.storage-journal.ts:181–189: persistent source-cleanup refusal causes each later one-entry flush to re-read all 8,192 retained sources. Failing test:review r3: cleanup refusal keeps each later 8192-entry flush bounded: 8,195 reads versus allowance ≤16. The normal-path 8,192-entry control passes. Counts only; no build-host timing claim.Regression evidence: earlier-round/hunt suite 93 pass / 0 fail / 734 assertions; combined suite 94 pass / 3 intentional failures / 744 assertions. Final strict TypeScript check and diff whitespace check pass. The existing four Chromium storage-lock cases pass; additional holder-closure and independent-account control passes. No private-mode/Safari qualification claim.
scripts/verify fastran in an independent detached checkout of the exact job head, without the review tests, and returned exit 0. Verbatim Rust gate output:Verbatim frontend/protocol totals from the same transcript:
Full transcript and test/browser receipts:
findings.md
verify-fast.log
review-tests-final.log
focused-with-review-final.log
review-check-final.log
browser-functional.log
real-tab-proof.json
Browser reproduction source and runner configuration:
real-tab-review.ts.txt
playwright.review.config.mjs.txt
Review completion receipt: redesign needed, 0 P0 / 2 P1 / 1 P2 / 0 P3. Review head
a0ffc82d50cc1460da33b70c75f470cd11e691fcis clean and matches origin/review-sync-journal-r3. Exact job-headscripts/verify fastpassed (verbatim output and full transcript in the verdict comment above).Fresh stripped x86_64 comparison using the same
scripts/static-cargorecipe: job head and merge base are both 21,386,512 bytes, delta 0 bytes, byte-identical SHA-256c3aba4dc538d0cba60df8dff605aa4c3133ba71b2501a9501836a8992cdfc447. Temporary gate/baseline worktrees, the owned physical fixture roots, and the job target directory have been deleted; review worktree and scratch evidence retained. No implementation fixes or deployment.findings-final.md
static-comparison.json
baseline-static.log
Plan job started on plan-250-journal from origin/dev
63773bb3ec. Reading all #250 and parent #243 history, round-1/2/3 review tests, D243/D259 and snapshot/receipt contracts. Deliverable is docs/plans/250-layout-journal.md only; existing implementation/review branches remain untouched. Comparing whether any durable client journal is needed before recommending a design. No deployment.Design drafted and pushed at
0bdf4ca9eon plan-250-journal (base origin/dev63773bb3e): docs/plans/250-layout-journal.md only. Recommendation: remove durable browser layout/drawing writes, retain per-tab optimistic intents in memory, and let existing server receipts and ordered intents merge changes. Persistent Not saved state explicitly warns that crash/close can lose unconfirmed changes. Two owner questions: accept that loss boundary and cold-offline drawing removal/old-tab cutover. No decision text or product code changed.Findings from source/contracts: D259 is a terminal reconnect budget, not authority to expire uncertain layout operations after 50 ms (journal.ts:110 and server-store.ts:114 on
660aa612b). A snapshot carries no op_ids; matching effects cannot prove identity. Existing auth/sessions.rs logout revokes the browser session and wakes sockets, but instant peer RAM erasure and fencing old JavaScript localStorage writers cannot be promised. Plan makes both limits explicit. Covers all 5 hunt findings, 7 R1 findings, 9 distinct R2 failure sequences (including both omitted-source cases), and 3 R3 findings as separate table rows; compares all four requested shapes. Roughly 500 production lines removed/replaced versus R3; estimate net 360–420 fewer. Full scripts/verify fast is running through build-slot on the docs-only head; no proposed-runtime timing claim.Plan refinement pushed at
2bfb0544e6: condensed to 1,899 words, retained all 24 failure rows (hunt 5; R1 7; R2 9 sequences; R3 3), and made the logout boundary explicit: already-admitted server work may finish, later admission must fail; no local marker can cancel an in-flight call. Docs-only diff against63773bb3everified, whitespace check passes. Full fast began before this prose-only refinement and is still compiling unchanged native inputs; the final checkout/head is clean and pushed. No product code or existing branch changed.Verification completed at pushed head
2bfb0544e6b777301e2f2c33823f1eef2fe46751(docs-only scope). Full fast passed; no product code changed. The run began before the final prose refinement, as noted above; all checked code inputs remained unchanged. Same-recipe merge-base size comparison is running, then cleanup and final handoff.Gate summary, verbatim:
verify-fast.log
gate-summary.txt
scope.json
Plan job complete. Pushed
plan-250-journalat2bfb0544e6b777301e2f2c33823f1eef2fe46751, based on63773bb3ecd990b09d3663c1bd32b2cd8e49991f. Onlydocs/plans/250-layout-journal.mdchanged (171 lines, approximately 1,899 words).Recommendation: remove the durable browser layout/drawing journal, retain pending intents in per-tab memory, and use server receipts as the durable merge point with a persistent Not saved state. The plan compares four shapes, covers all 24 hunt/review failure sequences, estimates net 360–420 fewer production lines versus R3, and specifies functional/browser/performance qualification. Owner questions recommend accepting loss of unconfirmed edits on tab loss and no cold-offline private drawing plus old-tab cutover. Existing product behavior and decisions remain unchanged.
Full fast passed (transcript attached in the preceding verification comment). Same-recipe stripped static x86_64 comparison: branch and merge base both 21,386,512 bytes, delta 0 bytes, SHA-256 identical (
c3aba4dc538d0cba60df8dff605aa4c3133ba71b2501a9501836a8992cdfc447). Branch aarch64: 19,563,608 bytes. No new runtime measurements claimed.Gate summary, verbatim:
verify-fast.log
gate-summary.txt
scope.json
size-comparison.json
baseline-static.log
cleanup.json
Cleanup: own target, detached baseline and fixture root removed; own cache daemon stopped; verification process gone; retained plan worktree clean. No product code, existing branch edits, deployment or issue closure.
Implementation started on layout-per-tab-250, base
2fb412d01(origin/dev, D312). Approved plan2bfb0544ecarried into the worktree. Reading #250/#243 history and review regressions; replacing durable layout/drawing persistence with per-tab memory and server receipt/event confirmation. Will push atomic commits, attach production desktop/phone light/dark captures, and quote final head fast output. No deployment or credentials in evidence.Pushed
664d28138receipt-identity controls and the per-tab store replacement. Before repair: journal controls 2 pass/3 fail; new memory/notice controls 2 pass/25 fail. After: 50 focused tests pass, 667 assertions; tracer check and fmt/diff checks pass. Live capture/flush/reconnect no longer serialize or write layout storage. The original storage-journal/storage-lifetime/layout-storage-lock modules are unmerged on dev and therefore avoided, not deleted. Remaining work: cold offline/identity writers, logout lifetime freeze and shared persistent pending UI, stronger browser/scale controls and final qualification.D312 implementation finding: original-session continuity cannot currently be proved for a suspended tab that misses BroadcastChannel logout, then reconnects after same-account re-login. /auth/session returns user_id/device/fresh (src/auth.ts); device is a caller-provided label and fresh changes on authentication, neither is an original session identity. Connection fresh reconnect uses the ambient cookie after resume refusal. A background check of the same user_id cannot distinguish the replacement session. No queue adoption or private storage writer remains, but that alone cannot satisfy the plan's “revoked session cannot restart a queue under later login” row.
I am continuing functional tests and zero-write controls. Need orchestrator resolution for a server-authenticated original-session binding that does not discard uncertain work or gate D259 recovery. The approved plan forbids a new snapshot field and this job is TypeScript only; I will not claim this row passed from BroadcastChannel tests.
Performance qualification also needs orchestrator relay: shared rules forbid contacting perf hosts from this job. The plan requires locked 4-vCPU runs at 1/10/60 windows and 8192/32768 intents, heap ×3/×10 and D259/D267. No build-host timing will be presented as evidence. Exact runnable artifact will follow after functional browser qualification.
Pushed
f83b82e17(explicit queue backpressure/coalescing) and96e18cf97(legacy retirement ordering). Queue admission now stages placeholder completion/cancellation, owns captured values, and cannot mutate accepted work on refusal. Sent inputs never coalesce. The 32 MiB encoded-byte limit is provisional pending perf-VM heap qualification, not a measured heap bound.Added receipt-before-snapshot and snapshot-before-receipt import tests, refused import, refused cleanup then successful removal. Focused queue/store/journal/P3 run: 34 pass, 0 fail, 188 assertions. TypeScript, fmt and diff checks pass. Source tests for 8192/32768 edits and 8195 stale keys now exercise those actual counts; no build-host timing claim.
Final fast gate and real-browser qualification remain pending. The earlier queued milestone gate was cancelled before starting because source had changed; no green gate is claimed from it. Original-session continuity and perf relay remain as reported in the preceding finding.
Pushed
859e0e01e: reference completion during an in-flight logout now remains in RAM while sends pause. A failed logout no longer strands a completed birth. The new regression failed before the fix; all 22 store tests pass (87 assertions). Global canvas shortcuts respect the inert screen. TypeScript/fmt/diff checks pass.Pushed
78546dfad: prebuilt queue CPU/heap probe, strict compiler checks passing. Please relay the following under the locked 4-vCPU perf VM. This is a measurement artifact, not a checkout or build; the attachment includes the prebuilt probe and test runner. Chromium must already be installed on that VM.Three-tab queue probe bundle
Extract into ~/infraternal-perf/layout-per-tab-250, then:
Return results/layout-queue-perf/matrix.json and the test output, including failed budgets. Matrix: three live tabs, 1/10/60 windows, 8192/32768 retained intents; capture/rebase/reconnect and GC-retained heap, actual three-tab aggregate and per-tab ×10. The bound remains provisional until these measurements. This probe does not establish D259/D267 terminal recovery; the production-browser recovery run still follows the native build.
Full fast is running. Its first compile-out attempt hit a pinned dependency fetch failure; fetching the exact archive into this job's cache produced the required uucode-0.1.0-ZZjBPj96QADXyt5sqwBJUnhaDYs_qBeeKijZvlRa0eqM hash. No dependency/source change. The final gate will need a clean rerun after that failed row.
Correction to the relay command's evidence destination: the harness deliberately rejects output inside its extracted root. Use INFRATERNAL_EVIDENCE_DIR="$PWD/../../../layout-per-tab-250-results" instead of "$PWD/../../results". The returned matrix is then ~/infraternal-perf/layout-per-tab-250-results/layout-queue-perf/matrix.json. Bundle SHA256: e432cca0288a2dab1c62cf6dfb9be14fea9516fc0d47192a70ac26e53aa96fe5.
Pushed
5616935c6: browser specs now observe layout wire messages in runner RAM rather than browser storage. Added delayed three-tab confirmation, explicit tab-loss/cold-offline behavior and failed-logout/late-completion scenarios. Strict tracer and real-spec compiler checks pass; actual browser runs and screenshots are still pending. Worktree is clean.Against D312 base
2fb412d01, native crate sources, Cargo manifests/lockfile, toolchain and static-build recipe are unchanged. Static binary comparison has not yet run. Total current diff: 1290 added / 566 removed (tests and carried plan included); production TypeScript alone is 340 added / 239 removed. Unmerged R3 storage modules were never on dev and therefore are avoided, not claimed as deletions from this base.Pushed
d6838f396: bind logout completion/failure to the initiating local screen generation. Two before-fix regressions failed: a late success forgot replacement work and a late failure thawed the replacement screen. Four controls now pass (16 assertions), including new-lifetime requests and per-lifetime deduplication. Added logout/notification tests to fast. This is a local callback fence, not the unresolved authenticated-session binding.Pushed
215854b32: immediate first gap and slow-snapshot watchdog/offline/disposal controls. Combined P3/logout run: 13 pass, 0 fail, 116 assertions. Strict compiler checks pass. Unavailable/recovered-storage browser scenario also added; runtime qualification remains pending.Current full fast is not green: besides the repaired pinned-fetch failure, unchanged native test streamed_digest_refuses_a_short_file_or_a_changed_stream_before_launch failed at crates/plugins/ssh/tests/install.rs:2703 with SetupRunning (exit 73). No native source changes; no waiver or baseline pass is claimed. The run is now building static release artifacts. I will rerun the affected test and final head gate.
Pushed
81f02a236: production browser controls pass 4/4; bun run check and production build pass. Scenarios cover three tabs, held confirmation with live terminal input, explicit unconfirmed loss/confirmed recovery, warm/cold offline, failed and successful logout, throwing/recovered storage. Focused canvas/privacy/lifecycle unit run: 319 pass, 0 fail, 20244 assertions. Browser plugin unavailable; repository Playwright used. These are Chromium functional checks with iPhone emulation, not actual iPhone/PWA/private-mode qualification.Initial captures below are superseded for spacing: a phone capture exposed the persistent pending and wrapped delayed notices overlapping. A shared-component spacing regression/fix is in progress; owner visual review is still required.
Milestone fast completed: Rust test row FAIL (101), streamed_digest_refuses_a_short_file_or_a_changed_stream_before_launch at install.rs:2703 (SetupRunning 73). Other Rust rows passed, including both static builds: x86_64 21431472 bytes; aarch64 19605896 bytes, exact caps. This is not the final head gate; isolated native retry and final head fast follow. Original-session continuity and perf relay remain unresolved as reported.
Pushed
5b9d2980e: replaced the old offline reload durability test by name in its commit message; the phone now asserts advertised loss and restores four server windows. Pushedef46455fc: R2 held split/close controls inspect decoded intercepted requests, not the upstream observer, and both pipeline/reconnect controls pass.Production-browser finding: phone and concurrent-close tests had no observed pending identities but a visible Not saved notice. A receipt can dismiss a toast before its first requestAnimationFrame; the queued frame then reopened it. The shared toast now fences animation/hide callbacks by message generation. A phone geometry regression also failed before the shared spacing repair (expected nonoverlap true, actual false): the old hard-coded bottom ignored the key tray and wrapped notice height. The repair measures neighboring notice height and retains the shared CSS anchor. No new private chrome.
Fresh production build targeted rerun: 9 passed, including all new loss/storage/logout scenarios, offline closes across restart, both R2 refusal races and ten-window two-device convergence. Full retained functional rerun is now running; timing specs excluded on this build host. Native original-session binding, perf-VM results and actual iPhone/PWA/private-mode qualification remain open.
Pushed
be22c5f6dshared notice fix. Updated production-build captures below supersede the earlier overlapping phone captures. Geometry assertion checks the two visible notices do not intersect; owner visual review is still required. Chromium desktop 1440x900 and iPhone 15 emulation, ink/paper; no actual iPhone/PWA/private-mode claim.Production recovery relay artifact (prebuilt UI, unchanged native executable, existing recovery fixture and runner; no checkout/build required): layout-production-recovery.tgz. SHA256 df910f30818effacbaebbac42a3a6b626c986ef5360c8e3363fe522a59282a9d. Extract into ~/infraternal-perf/layout-production-recovery-250. Chromium, node, Python and openssl must already be installed.
This existing recovery driver requires the orchestrator's isolated root mount namespace with a PRIVATE /run/infraternal; it intentionally refuses an ordinary host invocation. Within that established private fixture environment, for each RESTART_SESSIONS=1/10/60 and RESTART_MODE=daemon/shim/closed, run the following through the perf lock (one case per lock hold):
Set the two matrix variables to the case being run. Both desktop locked and phone unlocked cases run by default. Return complete METRIC lines and failed assertions; use the established port-scoped network driver for a true network RTT profile. REAL_ONE_WAY_MS is message delay and is labeled as such by the harness. This supplements the earlier three-tab 8192/32768 queue/heap probe; it does not itself qualify typing with that queue held. All performance qualification remains pending; no build-host timings are evidence.
Pushed
1543f3fb03. Full retained production-browser run on the final production code: 31 passed / 1 failed, the R2 fixture holding the wrong request sequence. The fixture now selects the confirmed membership, checks the split/close prefix while retaining valid move/resize suffixes, and withholds layout calls only (terminal credit/proof traffic continues). Both R2 variants pass 3 consecutive repetitions: 6 passed / 0 failed. Strict real-spec compiler passes. All four new per-tab browser scenarios pass, including phone notice nonoverlap. Earlier focused unit result: 319 passed / 20244 assertions.The unchanged native streamed_digest_refuses_a_short_file_or_a_changed_stream_before_launch now passes its isolated retry: 1 passed, 0 failed, 94 filtered out. No native fix or waiver. Same-recipe baseline static comparison and full scripts/verify fast are running sequentially through the build slot on this exact head. Worktree clean; no deploy.
Outstanding original-session continuity is still a real unmet requirement (not covered by the local generation/notification controls); no response/binding contract received. Performance artifacts and exact relay commands are above; no performance results received, so the 32 MiB encoded bound remains provisional and D259/D267/heap/task budgets are not qualified. Firefox/WebKit/real iPhone PWA normal/private modes are not qualified by Chromium emulation. These must remain explicit handoff blockers even if fast passes.
Implementation run handoff:
layout-per-tab-250, head1543f3fb034bd37d5797f9474bf327c7eb8dc217, pushed; worktree clean. The approved plan is NOT fully complete. Original-session continuity remains unmet: a suspended tab missing logout notification can still retry under a later login because the current server/handshake exposes no stable original-session binding. Local generation and BroadcastChannel tests do not establish that property. Required perf-VM task/heap/D259/D267 qualification and Firefox/WebKit/real iPhone PWA normal/private qualification are also outstanding. No relay results or session-binding resolution were received. The 32 MiB encoded queue bound remains provisional. Do not treat a green fast gate as proof of those requirements.Implemented: per-tab RAM intents with immutable same-ID retries and receipt/event confirmation; no private layout/drawing/identity cache writer; read-only legacy import retirement after confirmation; immediate pending status through the shared toast; warm offline retention and cold connection screen; revoke-before-forget logout freeze/failure handling and local callback fences. The shared toast also fences animation callbacks and separates wrapped notices above the mobile key tray. No native source, dependency, manifest, toolchain or static recipe changes; frontend changes are TypeScript plus shared toast CSS. The carried plan is unchanged from
2bfb0544e. Unmerged R3 storage modules were avoided, not claimed as deletions from dev.Verification:
scripts/verify fast: exit 0, every Rust row PASS; 563 Bun tests pass, 0 fail (groups 3/2/2/395/105/17/39). Strict real-spec compiler, tracer check and production build pass. Layout/journal/property/privacy/lifecycle tests are included; the focused canvas/privacy run was 319 pass and 20244 assertions.2fb412d01using the same recipe. Baseline, preserved head and final gate artifact SHA256 all159ed851c60e93a2e2960cab0ab101c16e22f6c532b96039bb2b0206d381f194. ARM static size 19605896 bytes, unchanged exact cap. Final size proofProduction screenshots for owner review (supersede the earlier overlapping captures): desktop ink, desktop paper, phone ink, phone paper. Phone is emulated, not an actual iPhone/PWA qualification. Owner visual judgment remains required.
How to test the new bundle:
Author checks: reused shared UI, kept wire/schema/reducers/native dependencies unchanged, verified no live layout storage access at 8192/32768 intents with 8195 leftover keys, retained receipt identity/refusal/duplicate/placeholder/import/teardown controls, reviewed privacy and lifecycle gaps explicitly above. No claim that every failure-table requirement is satisfied while authenticated original-session continuity is unresolved.
Cleanup: job CARGO_TARGET_DIR, detached baseline and owned short fixture root removed; own private sccache stopped. Selected build/browser/fixture PIDs are gone. Cleanup. Source worktree and external evidence retained. No deploy; issue left open.
Diff versus
2fb412d01: 580 removed / 1449 added (tests and 171-line plan included). Production frontend alone: 243 removed / 362 added.Exact-head gate output follows verbatim; command exit status was 0:
Independent verification started: review-layout-per-tab-250 at detached target
1543f3fb03; merge base with origin/dev recorded locally. Tests/evidence only, no implementation changes, push or deploy. Checking D312 failure cases, original-session account isolation, retained review controls and production browser engines.Perf matrix attempt on perf-test-03 (orchestrator, root window agreed with calternal): not run, kit fails before measuring.
Kit
layout-production-recovery.tgz(sha256df910f30…59282a9d, verified). Setup used:unshare --mount --propagation private, with a private tmpfs on/run/infraternal(mode 0700). The driver must run as root: as agent it fails withEACCES readlink /proc/1/ns/mntat perf-restart.js:1411. Root runs take the same two perf locks through/usr/local/bin/perf-lock.sh(becauseperf-run.shonly accepts the agent user), insidesystemd-run --scope --slice=perf.slice.LD_LIBRARY_PATH. Nothing was installed system-wide.Result: all 9 cases (sessions 1/10/60 × daemon/shim/closed) fail the same way right after
restart fixture: CA initialized:That is the
expect.poll(() => new Set(server.bridges.filter(b => !b.closed).flatMap(b => [...b.attached.values()].flat())).size).toBe(visibleSessions)check. The windows reachdata-status="live"(the per-window assertion above it passed), but the fixture's bridge sees no attached session. So the fixture's attach bookkeeping and the head's behaviour disagree. This has to be fixed in the #250 fix round. The kit must pass one case on a build host (in an unprivileged user + mount namespace with a private /run/infraternal, if possible) before it comes back to the perf VM.No numbers were produced, so D259/D267/heap/task qualification stays open. The VM state diff after cleanup was empty, apart from the user-level
~/infraternal-perfwork directory (kept for today's re-run, removed afterwards).Independent finding R250-01 — P2: distinct valid view/column IDs coalesce and lose an accepted edit. journal.ts field() joins identifiers with
/: pairs (a,/b) and (a/,b) collide. Two unsent weight changes show 2/3 optimistically, but confirming all queued requests leaves 1/3 without refusal or disconnect. D312 permits coalescing only the same fields.Failing test committed locally at
06e297ce2on review-layout-per-tab-250: apps/tracer/test/layout-coalescing-review.test.ts,distinct view and column identifiers cannot supersede each other's unsent weights. Output: 0 pass / 1 fail / 4 assertions. Tracer check and diff check pass; implementation unchanged. No push or deploy.coalescing.log
Independent finding R250-02 — P1: original-account layout crosses into a different account after suspended-tab reauthentication. Confirmed against production UI at
1543f3fb0and the real server, SQLite and protocol, not only a local lifetime mock.Sequence: A retains an unconfirmed legacy initialize request; notification is missed, its socket drops and the page freezes. A real /auth/logout succeeds. The same browser then receives a valid unlocked session for distinct account B (existing secondAccount fixture recipe; credentials remain in memory). On resume, the old tab reconnects with B's ambient cookie and retries A's initialize input. B's canonical layout changes from revision 1 to 2 and gains A's appearance reference alongside B's shortcuts reference. This proves different-account server admission, strengthening the implementation's acknowledged same-account continuity gap.
Locations: main.ts:663–672 accepts any nonempty foreground session without comparing user_id; connection.ts:455–476 reconnects with ambient cookies; server-store.ts retains the original queued document. D312/approved plan requires original authenticated lifetime binding across later sign-in.
Failing test committed locally on review-layout-per-tab-250 at
7e4d389200: real/layout-session-binding-review.spec.ts,a suspended original-account import cannot be admitted by a different account session. Strict real-spec compiler and diff check pass. No implementation fixes, push, deploy or issue closure. Current confirmed counts: 0 P0 / 1 P1 / 1 P2 / 0 P3; review continues.session-binding.log
Independent finding R250-03 — P2, inherited: the brief's zero-identity-write requirement still fails. Sign-in and the layout-close hint write tracer.hints.; clearAccountStorage returns true but intentionally retains it. Production Chromium independently observes the account-scoped key after an offline close. This is not a new layout/drawing writer introduced by #250: hints.ts is unchanged and account-storage.ts explicitly documents the retained-hint exception. That exception conflicts with the stronger review requirement and D245 all-account-data cleanup, so it needs explicit owner reconciliation.
Failing tests committed locally at
194a39885on review-layout-per-tab-250: test/layout-identity-review.test.ts (0 pass / 1 fail / 2 assertions), plus real/layout-engines-review.spec.ts all-storage assertions. The browser assertions are soft so subsequent reconnect/reload/logout/Cache/IndexedDB/OPFS checks still execute. Tracer and strict real-spec compiler checks pass. No implementation changes or push.Current confirmed counts: 0 P0 / 1 P1 / 2 P2 / 0 P3. Production WebKit and Firefox functional suites are running with external enrollment adapters; performance qualification remains with the orchestrator and is not inferred from build-host timings.
identity.log
Independent production-engine step: WebKit completed the 32 retained functional tests (timing specs excluded): 23 pass / 9 fail. An additional all-storage/reconnect control also failed on an offline-emulation assumption, making 23 pass / 10 fail across 33 total. No performance conclusions from build-host durations.
Three retained failures are explicit unsupported harness APIs (two CDP diagnostic sessions and mobile mouse.wheel). Other failures involve touch splits, offline reload/reconnect and the intercepted-logout fixture; isolating them before assigning additional product findings. A fresh touch-only control has no network interception. The new all-storage control now uses explicit held layout delivery and socket disconnection, because engine offline emulation can leave an established WebSocket alive.
External-only adapters bootstrap the existing real passkey ceremony in Chromium and transfer the live cookie into the target engine; Firefox disables unsupported isMobile while retaining viewport/touch options. Product and retained test bodies are unchanged. Adapter diff and failure summary attached. Firefox's full matrix is still running. Actual iPhone/PWA/private-mode and perf-VM queue/heap/D259/D267 qualification remain outstanding; the encoded 32 MiB estimate is a P2 qualification blocker, not a claimed measured heap bound or a reproduced budget failure.
webkit-failure-summary.txt
engine-adapters.patch
Independent finding R250-04 — P2: production Playwright WebKit does not split when the enabled touch Split right control is tapped. A dedicated test with no socket routing, delay or offline emulation fails expected 2 windows / actual 1, both on fresh navigation and after reload. The same two controls pass under Chromium and Firefox. This isolates the symptom from the earlier intercepted phone-split fixtures.
Failing test committed locally at
fbd882ca90: real/layout-touch-review.spec.ts, reload=false and reload=true variants. The toolbar/touch code predates this branch; a merge-base UI control will check inherited behavior before the final attribution. No actual iPhone/PWA claim. Strict real-spec compiler and diff checks pass; no product changes.The strengthened all-storage control (
10b0fffcd) holds only layout delivery and explicitly disconnects its socket instead of trusting engine offline flags. It now completes every phase in all three engines, including two devices closing the same window, reconnect, reload and logout. All non-identity checks pass (layout/drawing storage, empty IndexedDB/OPFS, public-only Cache entries); seven soft assertions per engine report the already-recorded retained account hint key, R250-03. It is not seven additional findings.Current reproduced defect counts: 0 P0 / 1 P1 / 3 P2 / 0 P3. Separate P2 qualification blockers remain the unmeasured queue/heap/task/D259/D267 requirements and incomplete real-device/private-mode support evidence. Full retained Chromium rerun and exact-target fast/static comparison are running sequentially.
webkit-controls.log
Independent review production functional matrices completed on the target UI: Chromium 32 pass / 0 fail; Firefox 27 pass / 5 fail; WebKit 23 pass / 9 fail. Timing specifications excluded; no performance claim from build-host runs. Firefox failures comprise unsupported CDP diagnostics, target-engine passkey reenrollment limitations, and an established-WebSocket offline-emulation assumption. WebKit also has unresolved offline/fixture cases; its isolated touch finding is already recorded separately.
The strengthened storage/reconnect/reload/logout control completes every phase in Chromium, Firefox, and WebKit: all layout/drawing, IndexedDB, OPFS, and public-only Cache checks complete without other failures; seven soft identity assertions per engine reproduce the single inherited hint-key finding R250-03. Dedicated touch controls pass 2/2 in Chromium and Firefox, fail 2/2 in WebKit. Exact-head final fast is through native tests and x86 static linkage, now compiling the second architecture. Static-binary and merge-base UI repeats remain pending. Review tests only, current review head
10b0fffcd; no implementation changes, push, or deployment.Independent verification complete. Verdict: do not merge. Reproduced defects: P0=0, P1=1, P2=3, P3=0.
Target
1543f3fb034bd37d5797f9474bf327c7eb8dc217, merge base2fb412d01dcecb81f1f556d18d0350e70644a615. Test-only review branchreview-layout-per-tab-250, headea22fc9ce48beca4ca2d6ce4e8ff5c0edd39f5e4. No implementation fixes, push, deployment, or issue closure.main.ts:663–672,connection.ts:455–476,canvas/model/server-store.ts. A suspended tab misses the notice, real logout revokes account A, and a later valid account B session supplies the reconnect cookie. The original A legacy import is then accepted by B's server session. Shipping static binary + production UI reproduces 3/3: B revision 1 becomes 2 and acquires A's layout reference. Failing testa suspended original-account import cannot be admitted by a different account session(7e4d38920). Server revocation controls passing does not cover this browser lifetime transition.canvas/model/journal.ts:13–15,131–140. Valid view/column pairsa//banda//bhave the same coalescing key; one accepted weight edit disappears after confirmation. Failing testdistinct view and column identifiers cannot supersede each other's unsent weights(06e297ce2), expected weights 2/3, actual 1/3.hints.ts:33–47,account-storage.ts:29–58. Sign-in/close writestracer.hints.<account UUID>and successful logout cleanup leaves it. Deterministic unit failure plus all-storage production controls in all three engines confirm it. This is inherited and requires owner reconciliation of the hint exception with the brief's zero identity writes / D245 requirement. Failing testsign-in and layout-close hints cannot retain the account identifier after sign-out(194a39885); browser control refined in10b0fffcd. Every scenario finishes; the seven soft assertions per engine are the same hint-key finding. Layout/drawing storage, IndexedDB, OPFS, and public-only Cache checks have no other failure.core-screen.ts:938–940,999–1022, exact cause not assigned. Fresh/reloaded production WebKit touch controls both remain at one window after tapping enabled Split right. Shipping binary target UI fails 2/2; merge-base UI fails 2/2; Chromium and Firefox pass 2/2 each. Event traces receive pointer/touch down/up/end but no click. No network interception/offline emulation. Failing teststouching Split right creates a second window without network interception (reload=false/true)(fbd882ca9, event/screenshot evidenceea22fc9ce). These are Linux Playwright results, not actual iPhone/PWA claims.Qualification gaps, separate from reproduced defect counts: the encoded 32 MiB queue estimate is not a qualified heap bound; heap ×3/×10, task budgets, D259/D267 need orchestrator-run perf-VM evidence (
layout-queue-perf.spec.tsand the recovery qualification). No reviewer perf runs made. Actual iPhone/PWA/private-mode and remaining engine harness/offline cases also remain unqualified. No invented missing-artifact or budget failure test.Functional evidence: focused current replacement tests 60 pass / 0 fail / 699 assertions. The two new unit regressions intentionally fail (0 pass / 2 fail / 6 assertions). Retained production matrix: Chromium 32/32 pass, Firefox 27 pass / 5 fail, WebKit 23 pass / 9 fail. Unsupported CDP/mobile-wheel diagnostics, target-engine enrollment limitations, and unresolved fixture/offline assumptions are explicitly separated from confirmed product findings. Earlier R1/R2/R3 source tests were rerun unchanged; removed storage modules/exports prevent loading, so they are not claimed as behavioral passes. Coverage mapping attached.
Exact untouched job head
scripts/verify fast: exit 0. Initial default-root attempt hit the Unix socket fixture path limit; final complete run uses the short fixture root with no source change. Same-recipe stripped static x86 head/base: 21,431,472 / 21,431,472 bytes, delta 0, byte-identical, SHA256159ed851c60e93a2e2960cab0ab101c16e22f6c532b96039bb2b0206d381f194.Verbatim gate summary excerpts (complete output attached; these build-host durations are not performance qualification):
Tracer check and strict review-spec type checks pass. Review checkout is clean. The temporary baseline worktree, Cargo target directory, and owned runtime roots have been removed; owned binary/browser process check found zero remaining. Evidence remains outside the repository; the test patch is attached because this review must not push.
verify-fast-final.log


gate-summary.txt
size-comparison.json
review-tests.patch
findings.md
coverage.md
static-session.log
static-touch.log
base-touch.log
The two images above are target production UI, then merge-base production UI.
Final verdict: do not merge; P0=0, P1=1, P2=3, P3=0; review head
ea22fc9ce4.Fix round 1 started on clean existing worktree /home/agent/wt-infraternal/layout-per-tab-250, branch layout-per-tab-250, base
1543f3fb03. Reading D312, approved failure table, handoff and reviewea22fc9ce; merging unchanged failing review tests first. Scope: session-incarnation admission fence, distinct-field coalescing, identity-free hints/cleanup, shared touch control, perf fixture bookkeeping and rebuilt relay kit. No deployment or dev merge.Merged review
ea22fc9ceunchanged. Reproduced both unit failures before repair (0 pass / 2 fail / 6 assertions).R250-01 repaired in
bf669e4af: tuple-encoded coalescing keys preserve distinct identifier boundaries; only adjacent unsent same-field supersession remains. Focused model suite: 85 pass / 0 fail / 8991 assertions; tracer check passes.R250-03 repaired in
adc01d04a: hints persist anonymous device-wide counters, never an account-derived key/value; accounts share the teaching budget. Sign-out sweeps legacy tracer.hints. keys. Unchanged identity regression plus hint/cleanup controls: 11 pass / 0 fail / 27 assertions; tracer check passes.R250-02 contract implementation follows the supplied decision: Hello capability bit 4 requests an additive Welcome session_incarnation; both layout calls carry it, missing/mismatched values get layout_session_changed before receipt lookup or commit. Server label is domain-separated from credentials and lives only in RAM/wire. No new dependency or database migration. Client-only and account-only binding rejected for suspended-tab and same-account reauthentication cases respectively. Shared touch focus fix and production tests remain in progress.
The build host permits unprivileged user+mount namespaces. /run/infraternal is absent outside the namespace; mounting a private tmpfs on /run first permits creating it and mounting the required private 0700 tmpfs without changing the host. Functional single-case kit qualification will use that layout, with no performance claim from this host.
R250-02 repaired in
0708a2c7b(shared codec/contract vectors) and16ac9d19f(admission/client lifetime). Both codecs retain the legacy Welcome vector and add the negotiated incarnation suffix and layout_session_changed refusal vectors. Server checks precede receipts/commit; client drops the queue without sending on a replacement incarnation. Authenticated HTTP seeds the original lifetime before the first socket, covering the startup cookie-change boundary too. The per-socket label is cached; the admission parser borrows it without constructing another JSON tree.Production Chromium: unchanged different-account suspended-tab regression PASS; added same-account sign-out/sign-in suspended-tab control PASS; unchanged all-storage review control PASS. Real native socket test (including absent/foreign incarnation refusals), touched-crate clippy and strict tracer/real-spec checks PASS. Review's five added files are byte-unchanged from
ea22fc9ce.Mapped-root build-host fixture limitation: the private tmpfs namespace works, but PTY open returns resource_unavailable. Direct unshare --user --map-root-user --mount python3 os.setgroups([0]) returns EPERM; crates/infraternal-shim/src/pty.rs requires that call before spawning the root session. No production privilege change is made to get a local perf result. The original driver's /proc/1/ns/mnt guard independently returns EACCES for mapped root; the repaired guard verifies a private /run/infraternal tmpfs and the single-entry mapped-root uid map when host-init inspection is unavailable. Perf-VM root qualification remains with the orchestrator; no build-host timing is evidence.
R250-04: shared toolbar repair pushed at
8cf7451e3. Production tests pass in Chromium and WebKit: five per engine, including unchanged fresh/reload touch tests and unchanged storage review. Focus stays in the terminal after a touch Tab; a touch Split right creates exactly one additional window. Production screenshots in both themes below; visual judgment is left to independent review.Perf bookkeeping repair
88203b830records explicit and server-restored attachment metadata even in lean mode. The exact attached-session count is unchanged, with an additional per-window live precondition. Three bookkeeping/isolation controls pass; strict TypeScript checks pass. Guardc74a395e2validates private tmpfs plus namespace/mapped-root identity. Build-host setgroups restriction remains as previously recorded; no timing claim.Gate additions
ad793dce2: selected tests are 9 pass / 0 fail / 27 assertions (the commit message incorrectly says sixteen). Retained full layout browser suites and static image comparisons are next.touch-webkit.log
touch-chromium.log
Retained production layout suite on
57ee7ff836: Chromium 40 PASS, 1 skipped (perf-VM-only latency row). Includes every earlier functional review test and both suspended-session cases. The five new review files are byte-unchanged fromea22fc9ce. Protocol adapter correction57ee7ff83only adds incarnation to layout calls, preserving unrelated action schemas.Focused review/codec/lifetime controls: 11 PASS / 0 FAIL / 123 assertions.
WebKit broader functional run is using external enrollment plumbing only. CDP lifecycle/network-diagnostic rows and the mouse.wheel-dependent mobile sync row are Chromium-only and excluded in this engine run; they all passed above in Chromium. Targeted unchanged WebKit touch/storage tests already pass. No build-host timing or heap claim.
full-chromium.log
mapped-root-groups.log
round-one-regressions.log
Production WebKit full selected suite: 29 passed, six failed, one perf-only skip. Five CDP lifecycle/network or mobile mouse.wheel cases excluded, all pass in the full Chromium run (40 pass, one perf-only skip). External enrollment plumbing patch attached; assertions unchanged.
Six-case recheck against prior kit: three pass (phone layout motion, offline close race, all-view removal); three fail (old touch split, offline reload internal navigation error, logout inert assertion).
Same six-case recheck on head: all-view removal passes; five fail (temporary notice already hidden, offline reload internal navigation error, logout inert assertion, connection-status CSS transforms counted as layout motion, offline close race count).
The original terminal-typing failure was not repeated; the temporary-notice assertion fails later instead. Initial startup failure was not repeated. Do not claim the broad WebKit suite is green or all failures are baseline-equivalent.
Targeted unchanged WebKit touch/storage review plus both-theme focus controls: five pass. Same five pass in Chromium. Full Chromium retained suite: 40 pass, one perf-only skip.
Supplemental real daemon restart, current debug executable + production UI, normal UID with transient user service and explicit user-bus address: FUNCTIONAL_RESTART sessions=1 mode=daemon PASS. No timing measurements enabled; exact attachment count and live terminal/queued typing recovery assertions remain intact.
Requested mapped-root namespace case is blocked by host setgroups EPERM, separately recorded; no production privilege workaround. Root perf VM qualification remains open.
full-webkit.log
failure-summary.txt
webkit-prior-recheck.log
webkit-head-recheck.log
engine-enrollment.patch
functional-restart.log
webkit-qualification.txt
Pinned baseline/head static measurements are running sequentially in one build slot. No final gate claim yet.
D243 fresh pinned baseline builds complete on 8by64-compute02, sequentially in one build slot, source origin/dev
2fb412d01d.x86_64-unknown-linux-musl: 21,431,472 bytes; sha256 159ed851c60e93a2e2960cab0ab101c16e22f6c532b96039bb2b0206d381f194.
aarch64-unknown-linux-musl: 19,605,896 bytes. Both match the existing exact caps. Head builds follow with the identical scripts/static-cargo build --locked --release --package infraternal --target recipe and job target directory. Comparison log and both head checksums will be attached after completion.
R250-02 conservative capability-transition control added at
c2fec6e5c: if a queued store has no original incarnation, it must discard its queue before adopting a newly advertised label. An empty store can still adopt its first label. New regression: before 2 pass / 1 fail (one call sent instead of zero); after, with unchanged coalescing/privacy controls, 5 pass / 0 fail / 23 assertions. Tracer compiler check passes. This is TypeScript-only; native input trees are unchanged during the static image comparison.unbound-before.log
unbound-after.log
Fresh head x86_64 static image: 21,434,288 bytes, +2,816 versus origin/dev
2fb412d01. ARM head build remains in progress; both final exact caps will be set after it completes.Author admission-parser audit found that my borrowed-only session-label field rejected valid JSON escapes. Regression fails before the correction (0 pass / 1 fail). Fixed in
42418c0a6with the existing borrowed-or-owned decoder: ordinary JSON labels stay borrowed, escaped labels decode to the same value with owned storage only when needed. Reject a second request tree and rejecting standard JSON string encodings. No new dependency or feature. Two decode/borrow controls, all 14 canvas-plugin unit tests, fmt and touched-crate clippy PASS.escaped-binding-before.log
escaped-binding-after.log
Both native head images are being rebuilt after this correction, sequentially in the build slot. Earlier preliminary head sizes are superseded by those final measurements; baseline images remain 21,431,472 / 19,605,896 bytes.
D243 complete: caps and PROCESS table pushed at
7e2490f07, native source42418c0a6. Fresh pinned static images on 8by64-compute02, one target at a time in the build slot:2fb412d01Both deltas are below 16 KiB. No dependency/feature/release-recipe change; no cap allowance. Final comparison JSON carries all four SHA256s. The initial comparison log also contains superseded preliminary head images; static-final-head.log and the JSON are the post-parser-correction results used for the caps and kit.
final-size-comparison.json
static-comparison.log
static-final-head.log
Final scripts/verify fast is running on
7e2490f07.Final-gate attempt on
7e2490f07hit an environment-path failure: daemon::gone_tests::a_vanished_session_gets_a_final_close_and_the_channel_stays_up calls UnixListener::bind on a 111-byte socket path under the selected TMPDIR; SUN_LEN rejects it. This is not waived as a host-load exception. No source change is needed. A private /home/agent/.cache/l250f alias points into this job’s scratch runtime, reducing that socket path to 97 bytes. The exact original workspace test binary then passes (1 pass / 168 filtered). The full gate will be rerun with the short alias; CARGO_TARGET_DIR remains the required job target.gate-runtime-failure.txt
vanished-short-runtime.log
The short-runtime symlink fixed SUN_LEN but correctly failed eight restart path-trust controls (symlink-parent refusal, canonical executable identity and parent escape checks). No tests or source were changed. Replaced that test configuration with a short real private directory under the scratch root, /home/agent/.cache/codex-scratch/l (job-owned, removed at finish). Original test binary restart controls now pass: 19 passed, 0 failed, 85 filtered. The full gate is being rerun with this direct path; neither failed attempt is waived.
restart-direct-runtime.log
Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 1/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 2/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 3/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 4/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 5/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 6/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 7/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 8/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 9/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 10/11.Final
scripts/verify faston7e2490f075: exit 0. Verbatim transcript 11/11.Implementation handoff: branch
layout-per-tab-250, pushed head7e2490f075fcbb5fd5e429c87d204d38cf220967, clean worktree; no dev merge or deployment.Round-1 fixes: server-enforced authenticated-session incarnation, queued-work discard across account/session changes, distinct-edit coalescing, anonymous hint storage and sign-out cleanup, shared touch-toolbar behavior, and authoritative replay-complete attachment bookkeeping. The exact attachment-count check remains; a live-window precondition was added. The review commits were merged first. All five newly supplied review files are byte-unchanged from
ea22fc9ce; the retained Chromium suite passed 40 tests with one perf-only skip. Commit bodies identify findings and rejected alternatives.Final
scripts/verify fast: exit 0 on this head, no load exceptions. The complete output is quoted verbatim in the preceding eleven numbered comments and attached below. Both final gate artifacts equal the measured artifact hashes. D243 sequential pinned same-recipe comparison against origin/dev2fb412d01dcecb81f1f556d18d0350e70644a615:Final static executable + final production UI: Chromium 7/7 (including suspended tabs across two accounts and sign-out/sign-in as the same account); WebKit 5/5 (unchanged touch/storage reviews, both-theme focus and single activation). Production screenshots below are attached for owner review; no visual-quality judgment is claimed.
Qualification limits remain explicit: the earlier broad external WebKit run was 29 passed / 6 failed / one perf-only skip, with five unsupported CDP/wheel cases excluded (all pass in Chromium). The six-case recheck passed three with the prior kit and one on head. Those failures are documented in the earlier WebKit evidence; this handoff does not claim that the broad suite is green or all failures are baseline-equivalent.
Rebuilt relay kit SHA256:
361fac6387717187b439ba9bcbfadc7f1a87e9bc8965326e47de859ca28a03e7.The artifact contains the final static x86 image, the final gate's production dist, the bundled current
apps/tracer/real/perf-restart.ts, and the same fixture/dependency contents as the earlier kit. Bundle command:bun build apps/tracer/real/perf-restart.ts --target=node --packages=external --outfile=<kit>/apps/tracer/real/perf-restart.js.Supplemental Node run with this final static executable and production UI, normal UID:
FUNCTIONAL_NODE_RESTART sessions=1 mode=daemon PASS, with exact attachment/retained-screen/queued-typing checks intact and no timing measurements enabled.Requested build-host namespace attempt used
unshare --user --map-root-user --mount, private mount propagation and private tmpfs/run/infraternal. It exits 1 before restart because the terminal remainsconnecting. This host deniesos.setgroups([0])with EPERM in that mapped-root namespace;crates/infraternal-shim/src/pty.rs:227requires that operation for its root PTY child. The direct OS proof and actual final-kit attempt are attached. Production privilege handling was not weakened. This is not a passing perf result; the root perf-VM matrix remains for the orchestrator.After extracting the kit on perf-test-03, use the existing root private mount namespace with private tmpfs
/run/infraternaland the suppliedPLAYWRIGHT_BROWSERS_PATH. Do not hide all/run, because the global perf lock may live there. Exact one-case command (retain the orchestrator's matrix variations):Cleanup: required CARGO_TARGET_DIR deleted; temporary baseline worktree and owned runtime directories/alias removed; fixture-owned shims cleaned; no remaining job executable processes. Evidence and kit retained outside the repository. Issue left open.
verify-fast-final.log
layout-production-recovery-fix-250.tgz












perf-kit-sha256.txt
final-chromium.log
final-webkit.log
final-functional-node.log
final-kit-namespace.log
mapped-root-groups.log
cleanup.log
cleanup.log
Independent delta review round 2 starting on local branch review-layout-per-tab-250-r2, detached target/base
7e2490f075. Reviewing R250-01–04 closure, session lifetime boundaries, retained functional tests and broader WebKit failures against dev2fb412d01, pinned static sizes and final fast. Tests/evidence only; no fix, push, deploy or perf-VM run.Perf matrix attempt 2 on perf-test-03 (kit
361fac63…): all 9 cases fail before measuring, so there are no numbers.Same setup as before: real root,
unshare --mount --propagation private, a private tmpfs on/run/infraternal, the driver underperf-lock.sh+systemd-run --scope --slice=perf.slice, user-local Chromium, and the documented env includingRESTART_UNLOCK=0 TRACER_TEST_PORT=4458. Every case stops at the first window:It also failed with the state tmpfs owned by root (mode 0755) instead of uid 1002, so directory ownership isn't the cause. The previous kit (
1543f3fb0) got past this point on the same VM and the same setup, and only failed later, at the attach count. So this head or kit stops the terminal from reachinglivewhen the run is real root in a private mount namespace, which is the configuration the measurement requires (RESTART_MEASURE=1needsREAL_ROOT_NAMESPACE=1).The owner allows real root on the build VMs, so the next #250 round should reproduce this exactly on 8by64 (as root via sudo/netbird root, in the same private mount namespace with a private /run/infraternal, the real driver and real Chromium), instead of the mapped-root namespace that hit
setgroupsEPERM. It then proves one case passes before the kit comes back. This blocks perf qualification, but not functional review.Round-2 review progress on target
7e2490f07: unchanged coalescing/identity review tests plus current incarnation model controls pass 5/5, 23 assertions. All five round-1 added review files are byte-unchanged fromea22fc9ce. New strict real-spec type check passes; raw-wire/restart and resumed-screen probes are prepared.Original old-journal R1/R2/R3 test bytes were also loaded against the replacement implementation: R1 cannot import layoutJournalKeys; R2 cannot import layoutJournalKey; R3 cannot import layoutLifetimeKey/storage-journal. These are removed D312 storage APIs, not behavioral passes. Current replacement failure-table coverage will be rerun. The requirement that those obsolete original sources all pass unchanged cannot literally be claimed.
Fetched origin/dev is
db8c1bdf6, differing from requested browser baseline2fb412d01only in DECISIONS.md (D313); production/native build inputs are identical. Comparison builds remain pinned to2fb412d01.Initial fast attempt failed before tests because shared sccache retained deleted TMPDIR /home/agent/.cache/codex-scratch/l. Private cache socket selected; final run uses a physical 34-byte TMPDIR/state root (one intermediate compilation interrupted before tests to shorten it). No failure waived. No push/deploy/perf-VM runs.
Delta review r2: exact target
7e2490f075passedscripts/verify fast(exit 0). Static sizes reproduce the handoff exactly: x86_64 21,434,576 bytes; aarch64 19,610,728 bytes, both stripped static PIE with no interpreter. All Rust rows pass. Tracer group: 407 pass / 0 fail; connections: 105/0; terminal queries: 17/0; protocol: 39/0. Current failure-table replacements are included. Production session/lifetime and engine comparisons now running; no verdict yet.Verbatim Rust gate output:
Full unmodified gate transcript:
verify-fast-final.log
Delta review r2 finding R250-R2-01 — P2 (new): a fenced old screen silently accepts fresh edits.
At
server-store.ts:64–69, a changed Welcome clears the old queue and permanently sets#revoked.save()returns silently at line 87, whilemain.ts:663–675only checks whether some HTTP session exists and leaves the old screen interactive after same-account replacement.Production
7e2490f07reproduction: suspend an original import, sign out and create a new session for the same account, then resume. The old import is safely dropped. Close a window in the resumed screen: visible console windows become 0 while an authenticated server read remains exactly equal to the pre-close canonical layout (one window). Neither login nor the persistent “Not saved — keep this tab open” status appears. The final assertion fails true/false after 5 seconds. This violates D312’s explicit unconfirmed-work indication. End or recreate the editable screen when fencing its lifetime, preserving the drop of old intents.Failing test committed locally at
2bb788009on review-layout-per-tab-250-r2. Product code unchanged. Strict real-spec TypeScript check passes. The log’s other failure was a reviewer expectation error: rebinding an old operation ID changes its receipt input and is correctly refused; that control is corrected for a rerun. Restart/pending-edit and two-tab cross-account no-retransmission controls passed.session-review.log

Location correction for R250-R2-01 on exact target
7e2490f07: the changed-Welcome block is server-store.ts:59–64, the permanent #revoked assignment is line 62, and the silent save() return is line 82. The main.ts:663–675 reference is unchanged. Evidence and severity remain as reported.Delta review r2 production checks: retained Chromium suite 40 passed / 0 failed / one perf-only skip, with the five round-1 review files unchanged. This includes both suspended-session controls, all-storage/logout checks, both touch controls and both-theme focus/single-activation controls.
Independent raw-wire/restart/two-tab suite: 3/3 pass. Missing, unexpected, empty, null, numeric and stale bindings are refused before existing receipts; caps-0 receives the legacy Welcome and its unbound write is refused. The label never enters URL/localStorage/sessionStorage in the test and is rejected as a cookie. Same-account reauthentication changes it; changing an old receipt's input is correctly refused while a fresh identity succeeds. Server restart retains the label and confirms pending work. With two tabs and a changed account cookie, the old tab sends none of its held operation IDs after the new snapshot.
Native and TypeScript golden vectors pass in the exact-target fast gate. Server admission and client old-intent dropping close R250-02; the separate new P2 editable-lifetime problem remains. Review-only test commit
c15d61b0fis local; no product edits/push/deploy. Dev static builds and paired WebKit qualification are running sequentially; no perf-VM runs.session-wire-final.log
chromium-retained.log
Delta review r2 pinned static comparison reproduced independently, sequentially in one build slot:
2fb412d017e2490f07Recipe:
scripts/auth-toolchain, thenscripts/static-cargo build --locked --release --package infraternal --target <target>for each architecture, same CARGO_TARGET_DIR. Current origin/devdb8c1bdfdiffers from2fb412d01only in DECISIONS.md; native/production inputs match. Both deltas remain below 16 KB.Linkage detail (correcting my earlier “both static PIE” shorthand): x86 is static PIE; aarch64 is static ELF. Both are stripped and have no PT_INTERP. SHA-256s and file/readelf outcomes are in the attached manifest.
The first external WebKit invocation stopped before tests because package.json module metadata was missing in the scratch harness. The metadata is now copied unchanged and test discovery passes; rerun queued/running, no product result counted from the loader errors.
size-comparison.json
baseline-build.log
Delta review r2 continuation (Claude) on branch review-layout-per-tab-250-r2, base
40ff82cf8(previous reviewer's WIP), target7e2490f07. The earlier r2 run stopped at a usage limit. I will re-verify its recorded results (R250-R2-01 P2, the session-binding closure, the static sizes), finish the WebKit touch and broad-WebKit classification against dev2fb412d01, then run finalscripts/verify faston7e2490f07. Tests/evidence only: no product edits, no deploy, no perf-VM runs.Delta review r2: paused for handover (owner request). Review branch
review-layout-per-tab-250-r2pushed at80c2a3578. No verdict yet; current leaning is "merge after fixes".Re-verified independently on exact
7e2490f07scripts/verify fast: exit 0. Tracer 407/0, connections 105/0, terminal queries 17/0, protocol 39/0 (stream differential: zero divergences). Rust table, verbatim:real/layout-*.spec.ts: 43 passed, 2 perf-only skipped, 2 failed (both are reviewer P2 tests). The 5 round-1 review files are byte-identical toea22fc9ce. Earlier review specs changed only in their protocol helper (layout calls are now bound to the Welcome label); their assertions are unchanged.Round-1 findings
Active finding
server-store.ts:59–64,:82;main.tscheckForegroundSession). Failing test:2bb788009.80c2a3578).Broader WebKit failures (same specs, head vs dev production build): no regression
Still unchecked: the account-switch scenario above; a review of the perf-VM "terminal stays connecting under real root" report (the delta touches no shim/PTY code; the harness now asserts
liveearlier; no root here); Hello bit 16 written as a bare literal (P3 candidate).Counts so far: P0 0, P1 0, P2 1, P3 0.
verify-fast-head.log
chromium-head.log
webkit-head.log
webkit-baseline.log
Round-2 review continuation started on
review-layout-per-tab-250-r2, base80c2a3578c62a8ad06b2ae25044f9c190a3474cb, target implementation7e2490f075fcbb5fd5e429c87d204d38cf220967(D312). I’m completing phase A (fast gate, production Chromium, production WebKit), then phase B against dev2fb412d01dcecb81f1f556d18d0350e70644a615(static comparison and WebKit), and reproducing perf-kit361fac6387717187b439ba9bcbfadc7f1a87e9bc8965326e47de859ca28a03e7under real root on compute01. Review/test evidence only; no implementation edits, deployment, or issue closure.R250-R2-02 — P3: the new layout-session Hello capability is an unnamed literal in the client. The contract defines session incarnation as Hello bit 4 (
16) atcontracts/wire.md:780, whileapps/tracer/src/connection.ts:479ORs a bare16nintocaps; adjacent capability bits useRESUME_NEXT,LINK_DOUBT_NOTICESandVIEWER_VISIBILITY(:72–76). This is a maintainability finding, not a reported runtime failure. A named capability constant would keep the new handshake bit identifiable at its use site. Target reviewed:7e2490f075fcbb5fd5e429c87d204d38cf220967.Round-2 review is blocked before the required VM checks. From this runner, SSH to both
claude@infraternal-dev-01-compute01and its FQDN returnsNo route to host; DNS resolves the host to10.69.69.218, and route lookup selectswt0. I have not run builds, browsers, the gate, or the one authorized real-root perf-kit reproduction, and have not started any VM process. This prevents reading the VM's job rules, completing phase A, phase B, or reproducing kit361fac6387717187b439ba9bcbfadc7f1a87e9bc8965326e47de859ca28a03e7. The account-switch probe at WIP head80c2a3578remains unexecuted. Please restore the assigned SSH route or provide the approved plain-SSH path; no alternate or linked environment was used.Update to the previous blocker note: the owner confirmed compute01 is down and approved dev-02 for the remaining work. I’m continuing phase A/B and the real-root perf-kit reproduction on
infraternal-dev-02-std; every heavy command will use~/infraternal-ops/build-slotwith 2 workers (up to 3 where needed). Only the x86 static-size row remains assigned to compute01; I’ll retry its SSH health check every five minutes and report that row pending if the host is still down. No VM checks had run before this update.Correction to blocker comments 70821 and 70839: compute01 is back following the owner’s reboot at 22:53 IST. The earlier route blocker is resolved. Phase A/B browser and paired WebKit work stays on dev-02 as approved; the x86_64 static-size row now runs on compute01 through
~/infraternal-ops/build-slot. The real-root perf-kit repro will run on dev-02 ascuawith sudo only for that repro, also through build-slot, followed by cleanup.The first resumed compute01
scripts/verify fastattempt stopped atscripts/verify: line 108: bun: command not found. The launcher used a nested login shell that discarded the Nix dev-shell PATH. The rerun keeps the Nix PATH and is queued in build-slot; no product finding is based on that setup error.Round-2 progress: review notes are committed and pushed on
review-layout-per-tab-250-r2at2f6f9e332; no implementation files changed.The production head remains at 47 passed / 2 skipped / 2 failed in Chromium; the known P2 is one failure and the account-switch probe has not reached its assertions yet.
bun run checkpassed. The current WebKit/Xvfb attempt fails before assertions withCould not create WPE EGL display: EGL_BAD_PARAMETER.The approved real-root dev-02 run verified kit
361fac6387717187b439ba9bcbfadc7f1a87e9bc8965326e47de859ca28a03e7and initial typing. It did not reproduce a persistent initial “Connecting” state; after server restart it logged twoaddress in useretries and recovered in 1,143.072 ms against 350 ms. The trace points to the perf fixture's parent-only stop/relaunch race. Cleanup verified zero J11 root task processes and restored the host mount.The corrected account-switch rerun and compute01 x86 static comparison remain queued through
build-slot; full-baseline restart-test classification is still pending. I’ll post the final evidence and verdict when those checks finish.Round-2 continuation checkpoint on
review-layout-per-tab-250-r2, head1712a54a2(pushed).Phase A records remain: exact implementation head
7e2490f07passedscripts/verify fast;bun run checkpassed on dev-02. Production Chromium is 47 passed / 2 skipped / 2 failed: the open same-account P2 reproduces, and the corrected cross-account probe timed out before its canonical-state assertions, so it adds no finding. WebKit still fails before app assertions at WPE EGL startup; the paired head/dev classification is in comment 70637.The five full-iterate restart timeouts pass in the focused restart suite on both target and baseline (18/18 each); full baseline classification remains unrun. The real-root kit reproduction on dev-02 reached initial typing, then reported two address-in-use retries and 1,143.072 ms to typable after restart versus a 350 ms budget. The fixture PID-only stop/relaunch race is recorded in the review notes; task process count was zero after cleanup and the host mount was restored.
The fresh x86_64 baseline/target static comparison is queued through compute01
build-slot; log path:~/.cache/codex-scratch/j11-layout-250/evidence/x86-static-comparison.log. Next: finish that row, update the review record, then post the final P0–P3 summary and verdict.Review-branch correction:
review-layout-per-tab-250-r2is now at8bf774543(pushed). I removed the inconclusive account-switch spec after its production run timed out before the canonical-state assertions; the screenshot/log and limitation remain in the notes, and the complete same-account P2 test remains. No implementation files changed. The fresh compute01 x86 static comparison is still queued throughbuild-slot; the existing target artifact is 21,434,576 bytes and matches the exact cap.Round-2 review complete for D312 / #250. Implementation target:
layout-per-tab-250at7e2490f075fcbb5fd5e429c87d204d38cf220967. Review notes are pushed onreview-layout-per-tab-250-r2atf059476bd2fa8f6a87e918750c2a0b253168a7d3; no implementation files were changed. The issue remains open.Findings
ServerCanvasStoreleaves its old screen editable;save()silently drops the edit, the authenticated canonical layout stays unchanged, and the screen does not show D312’s “Not saved” state. This reproduces in the production Chromium test; screenshot and log are attached in comment #70585.apps/tracer/src/connection.ts:479advertises the Hello capability with bare16n, while adjacent bits are named constants. The contract identifies this as Hello bit 4; this is a maintainability finding, with no runtime failure claimed. Evidence is in comment #70820.Phase A — implementation head
The exact-target
scripts/verify fastrun recorded in comment #70582 passed at7e2490f07;bun run checkpassed. The gate also reported tracer 407/0, connections 105/0, terminal queries 17/0, protocol 39/0, and zero stream differential divergences. Its Rust gate table is reproduced verbatim:Production Chromium: 47 passed, 2 skipped, 2 failed. One failure is R250-R2-01. The second-account probe timed out before post-edit canonical-state assertions, so it is inconclusive and is not a separate finding; its exploratory spec was removed from the review branch. Production WebKit on the target fails before app assertions at WPE EGL startup (
Could not create WPE EGL display: EGL_BAD_PARAMETER). The paired head/dev WebKit classification in comment #70637 reports the same broad failures on both builds.scripts/verify iterateon dev-02 reported 98 passed, 5 failed, 1 ignored. The five restart-control timeout tests pass in focusedrestart::testson both target and baseline (18/18 each); the full baseline suite was not rerun, so those full-gate timeouts remain unclassified.Phase B — baseline
2fb412d01Pinned static comparison:
Both deltas are below 16 KiB and both images match the exact ceilings. The fresh x86 static row ran on compute01 through
~/infraternal-ops/build-slotagainst the exact implementation SHA: production trust PASS; stripped static PIE; noPT_INTERP; 21,434,576 bytes against a 21,434,576-byte cap (delta +0); SHA-256d7c66be3346ab16b4e110038c7c9c78718bf0a9f78cee5a204e38a0eea5e6e4b.Real-root perf-kit reproduction
Kit
361fac6387717187b439ba9bcbfadc7f1a87e9bc8965326e47de859ca28a03e7ran as real root on dev-02 throughbuild-slot. It reached initial typing; a persistent initial “Connecting” state was not reproduced. After server restart it logged twoaddress in useretries and took 1,143.072 ms to become typable against a 350 ms budget. The evidence points to a fixture teardown race:apps/tracer/real/server.ts:181–189SIGTERMs and waits for only the detached server PID, then relaunches before the old process group and embedded agent exit; later cleanup kills the process group. The retries align with that race. This does not confirm a product hang; these lifecycle lines are unchanged from baseline. Cleanup recorded zero J11 task processes and restored the host mount; root-owned fixture cache files were removed.Both J11 build targets have been removed from compute01 and dev-02. The review worktree is clean at the pushed head above.
Verdict: Request changes.
Starting J32 fix round 3 on
layout-per-tab-250, baseorigin/dev57491e918. I’m reading D312 and the round-2 findings, then I’ll add failing-first regressions, fix the revoked-store UI state, capability name, restart fixture teardown, and account-switch probe. Verification and browser work will run through compute01build-slot; no product edits yet.J32 contract note (D312, R250-R2-01): D312 says a changed session incarnation drops queued edits without sending and reports lost edits with the existing Not saved state, but it does not define what to do with the old canvas after access changes or what to show when no edit was waiting. This round uses the conservative behavior: dispose the old canvas on any incarnation change and return to passkey sign-in. When unconfirmed edits were present, the sign-in screen will say “Not saved. Changes waiting to save were lost. Sign in again to continue.” An account change with no pending edit will say “Your sign-in changed. Sign in again to continue.” This prevents silently discarded edits and keeps the replacement account's layout out of the old screen. Please clarify revocation behavior in D312.
J32 RED progress on layout-per-tab-250 at
c7a9450af(rebased from origin/dev57491e918):Next: run the deterministic RED browser checks, implement each finding, then run Chromium/WebKit attempts and scripts/verify fast.
Crash-safety checkpoint for J32: WIP regression commits
900c3c7d8,9752ecc8aand06ff2c003are pushed on layout-per-tab-250. They cover the revoked-session store/screen, the D312 Hello capability contract, and deterministic process-group restart teardown; baseline failures have been observed for the store callback and the missing capability name. The account-switch spec now reaches both canonical reads before its final UI assertion. Production fixes remain local until the baseline real-server/browser run is captured. compute01 baseline build log: ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/baseline-build.log.J32 progress on layout-per-tab-250: the focused baseline unit regressions are red as expected: replacement and unbound sessions miss the lost-edit callback, a changed lifetime adopts the replacement snapshot, and the D312 capability test cannot import its named constant. Evidence: ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-red.log. The deterministic real-server process-group RED test is queued through compute01 build-slot; all four slots remain occupied, so I have not changed the fixture before observing its baseline result. Next: capture that failure, then fix and run production Chromium plus the fast gate.
J32 crash-safety checkpoint: WIP source snapshot
390583d3ais pushed on layout-per-tab-250. It ends the old canvas on session-incarnation change with a plain sign-in/lost-edit message and names the D312 Hello bit 4 capability. Fresh baseline unit RED evidence is in ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-red.log. The deterministic process-group and production Chromium RED commands remain queued through compute01 build-slot; neither has started. Next: capture both baseline failures, fix and verify the fixture, build the production tracer, then run Chromium/WebKit and scripts/verify fast. The WIP will be split into atomic passing commits before handoff.J32 queue update at pushed WIP head
390583d3a: the deterministic process-group RED command has waited more than 30 minutes in compute01 build-slot; the production Chromium RED command is queued behind it. The host still has four active build/browser slots in use. No test result is claimed for either command. The baseline unit RED output remains at ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-red.log; I will continue with the assigned VM and report the test evidence when it starts.J32 queue update at WIP head
390583d3a: the process-group RED command has waited over one hour on compute01; the Chromium boundary RED command has waited about 48 minutes. All four build-slot lock files still have live owners. Neither command has started, and no result is claimed. Unit RED evidence remains available at ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-red.log. I will continue once the assigned VM releases a slot.J32 adjustment: after more than one hour without a compute01 slot, I canceled the queued process-only run and moved that architecture-neutral fixture check to dev-02's native ARM64 build through its build-slot. It does not request a browser fixture. The production Chromium suite and remain assigned to compute01. The dev-02 build log will be under ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/dev02-baseline-build.log.
Correction to the previous J32 adjustment note: the fixture-only spec is apps/tracer/real/server-process-group.spec.ts. Its queued compute01 run has been canceled, and the same baseline spec is now building and will run on dev-02 through build-slot with a native ARM64 server. This test does not launch Chromium. The production Chromium suite and scripts/verify fast remain assigned to compute01. No test result is claimed yet.
Dev-02 build status: the first nix develop attempt failed before compilation because its working directory was outside the checkout's flake; no test ran and no binary was produced. The retry now starts from the J32 worktree root and is compiling the native ARM64 infraternal binary before building the tracer bundle. Evidence: ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/dev02-baseline-build-r2.log.
Dev-02 fixture-test update: the native ARM64 binary and tracer bundle built successfully. The target directory was removed after copying the binary to external evidence, but free disk is now 11 GB, below build-slot's 25 GB minimum, so no test ran there. The process-group RED spec is queued again on compute01 using its existing production binary. The only browser suite and fast gate remain on compute01. The native binary receipt is in ~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/infraternal-arm64-baseline (SHA-256 7468e0d9c3714d501804f8985e9c75cf0444e17188505a21880bedfc67cae50b).
J32 progress at
af7019c56275onlayout-per-tab-250:~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-red.log.build-slot; neither has started, so no result is claimed.Next: capture those baseline results, then finish and verify the fixture teardown and browser probes. Queue logs remain under
~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/on compute01.J32 fix-round continuation on layout-per-tab-250 at
fa60db2ce, based on57491e918.D312 says authenticated peers stop on revocation and unconfirmed edits may be lost, but it does not specify the old screen after access is revoked. I’m following the conservative behavior: freeze/end that screen and show “Not saved. Changes waiting to save were lost. Sign in again to continue.” for pending edits. This is the decision for kayg to review.
The P2/P3 regression commits and UI behavior are in progress; real-server account-switch coverage reads both accounts’ canonical layouts before its final screen assertions.
The first process-group RED attempt did not reach the restart assertion: the test used the unset default INFRATERNAL_BIN path target/x86_64-unknown-linux-musl/release/infraternal, while this compute01 baseline build is at target/release/infraternal. It left no matching fixture processes. I stopped the queued run and am relaunching with the explicit baseline binary and a persistent evidence log through build-slot.
J32 fixture-fix checkpoint on
layout-per-tab-250@c54baae52, based on57491e918.The local WIP fix now signals the detached server-owned process group and waits up to 5 seconds for exit before relaunch; if the group remains, it sends SIGKILL and waits one more second before refusing to relaunch. Disposal uses the same bounded teardown. Session shims stay isolated in their own groups.
The corrected baseline real-server regression is still waiting in compute01's build-slot queue. Its evidence log is
/home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/process-group-red-rerun.log; it uses the existingtarget/release/infraternalbinary andTRACER_TEST_PORT=47822. The earlier real-root kit result (two address-in-use retries, 1,143.072 ms against 350 ms) is the current failing evidence. Next: capture the deterministic RED result, verify the fixture fix, run production Chromium/WebKit attempts, the real-root measurement andscripts/verify fast.J32 fixture checkpoint on
layout-per-tab-250@5be962e76, based on57491e918.The deterministic baseline regression now asserts the old detached group is gone before relaunch, and its delayed child is spawned only for the server launch (the fixture also invokes the binary for setup). The process-group teardown fix is pushed as WIP. The previously observed real-root result remains two address-in-use retries and 1,143.072 ms against a 350 ms budget.
The corrected RED run has waited since 00:12 UTC in compute01
build-slot, still at queue position 11 with all four slots occupied. It has not started, so there is no regression-test result yet. Its log is/home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/process-group-red-rerun.log. I’m keeping it against the unmodified fixture until a slot opens; then I’ll sync the pushed branch and run the GREEN checks.Correction to the queue note above: I cancelled the first corrected-baseline launch before it acquired a slot. It used Playwright's default config from the repository root; the real-server regression must run from
apps/tracerwithplaywright.real.config.tsso no proxy occupies its server port. The valid baseline run is now queued on compute01 at position 11; log:/home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/process-group-red-rerun2.log. The fixture source remains unchanged on that VM.J32 fixture regression evidence on layout-per-tab-250 @
23031d8d9. The first queued compute01 attempt was canceled before execution because its environment variable was misspelled. The corrected deterministic test ran on dev-02 ARM64 against baseline binary SHA-256 7468e0d9c3714d501804f8985e9c75cf0444e17188505a21880bedfc67cae50b and the pre-fix fixture at390583d3a6: RED as expected, failing because the old process group still existed when restart attempted the replacement. After copying the fixture fix (server.ts SHA-256 f55bb32f877ac6ce51c459b3507d05d9196d86b9a7a6fa2b2eca8120a6f0a68e), the same test passed 1/1 in 6.6 s. Test source SHA-256 2340004ba1ef40541fae3e8862c10b40344343bb55079ed14f6aca7d670ee3a6. Logs: /home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/process-group-red-arm64-baseline.log and process-group-green-arm64.log on dev-02. Next: compute01 Chromium and fast gate, then rerun the real-root restart measurement.R250-R2-01 unit RED/GREEN update on layout-per-tab-250 @
23031d8d9. The revoked store deliberately clears its in-memory canonical document before ending the old screen, so the regression now asserts an empty document after revocation instead of preserving the old snapshot. Against the pre-fix server-store atc7a9450af, the corrected test is RED: original-lifetime reconnect passes, while replacement/unbound sessions do not report the loss and the no-pending case adopts the replacement snapshot (1 pass, 4 fail, 15 assertions). With the fix, the focused unit set is GREEN: 6 pass, 0 fail, 21 assertions, including the named bit-4 capability test. Evidence logs on dev-02: /home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-red-adjusted.log and unit-green.log. The production Chromium same-account and account-switch checks are queued after the compute01 fast build.R250-R2-01 test refinement on layout-per-tab-250 @
d951dc19a: the revocation unit control now also attempts a save after the session boundary and verifies the cleared store stays empty with no pending work or server calls. Against the pre-fix server-store atc7a9450af, the revised suite is RED (1 pass, 4 fail, 15 assertions). With the current store and named capability, it is GREEN (6 pass, 0 fail, 26 assertions). Logs on dev-02: unit-red-revoked-save.log and unit-green-revoked-save.log. This supplements the production same-account and account-switch browser checks still pending on compute01.J32 fix-round 3 continuation on
layout-per-tab-250@d951dc19a, based on the requestedorigin/dev57491e918. The worktree is clean. The queued duplicate canvas-unit command was stopped before it acquired a build slot; the currentscripts/verify fastrun remains active on compute01 through build-slot, with output at~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/fast.log.Failing-first evidence already recorded: revoked-store/capability RED→GREEN in
unit-red-revoked-save.log/unit-green-revoked-save.log; deterministic fixture process-group RED→GREEN on dev-02 inprocess-group-red-arm64-baseline.log/process-group-green-arm64.log. Current code ends the old screen after incarnation change and names the Hello capability. Production Chromium/account-switch verification, the WebKit Nix/software-GL attempt, and final atomic commit split remain next; no new pass is claimed here.J32 continuation at
d951dc19aon layout-per-tab-250, based on57491e918. The assigned compute01 fast gate is still active through build-slot and is compiling the production Rust dependency graph; current log is /home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/fast.log. The source-level RED/GREEN evidence and fixture regression remain recorded above. Next: exact-head fast completion, production Chromium, conclusive account-switch assertions, WebKit startup attempt, then final commit split and push.J32 gate finding: the first exact-head scripts/verify fast run exited 1 at the replay_size_review test adopted_raw_and_snapshot_replays_keep_pre_resize_geometry. The panic is test-fixture path validation before binary behavior: agent_fixture.py reports its AF_UNIX socket path is 109 bytes after fallback. Both scripts/agent_fixture.py and replay_size_review.rs have identical Git blobs on requested baseline
57491e918andd951dc19a. The fixture supports INFRATERAL_AGENT_TEST_STATE_ROOT; I will rerun fast with a short private root and retain the first transcript for classification. Current transcript: /home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/fast.log. Chromium remains queued behind compute01 build-slot.Correction to the gate classification note: the override is spelled INFRATERNAL_AGENT_TEST_STATE_ROOT (with the n after r). I also loaded the baseline
57491e918version of agent_fixture.py and reproduced the same 109-byte AF_UNIX path ValueError before binary startup, confirming this failure on the requested base.J32 progress at
d951dc19aonlayout-per-tab-250, based on the requested57491e918:scripts/verify fastrun exited 1 atadopted_raw_and_snapshot_replays_keep_pre_resize_geometry, before binary behavior: the shared fixture produced a 109-byte AF_UNIX socket path. The same failure reproduces on57491e918, with identical fixture/test blobs. The retry will setINFRATERNAL_AGENT_TEST_STATE_ROOT=/tmp/j32inside the Nix shell.build-slotat positions 3 and 6. Neither has started assertions or produced WebKit startup evidence yet. Logs:~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/chromium.logandwebkit-nix.log.I will run the corrected full gate after the browser work to avoid rebuilding its target concurrently, then post the exact results and split the WIP tail into three atomic commits.
J32 runner correction at
d951dc19aonlayout-per-tab-250:The first Chromium process acquired a compute01 slot, then exited 127 before the production build or any assertions because its nested
bash -lcdropped the Nix shell'sbunpath (bash: line 7: bun: command not found). I preserved that transcript aschromium-login-shell.log/chromium-login-shell.exit; it is a runner setup failure, not a test result. The runner now usesbash -cand is queued again throughbuild-slot(chromium.log).The initial WebKit runner used the same nested-login-shell pattern. It was stopped by its own PIDs while still waiting for a slot; no WebKit assertion ran. I will start the corrected Nix/software-GL attempt after Chromium completes.
J32 Chromium runtime update at
d951dc19a:The shell-corrected production run built the tracer, but Playwright could not launch Chromium because the VM image lacks
libnspr4.so(alsolibnss3.so, ATK, X11 extensions, GBM, XKB, ALSA and AT-SPI). Both session-boundary tests exited before browser launch; no product assertions ran and no screenshot was produced. I added the matching Nix runtime closure, includingat-spi2-core; the preflightlddcheck now reportsChromium Playwright runtime dependencies: PASS.The corrected Chromium suite is queued through
build-slot; its output will be at~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/chromium-libs.log. It runs the same-account boundary, cross-account canonical reads, existing incarnation probe and process-group regression separately so later specs still run if an earlier assertion fails.J32b start — continuing
layout-per-tab-250atd951dc19aaf3e577cc9fc9fbc71fcfba4157c9d4(clean worktree). Requested base:origin/devf4b61cd6c3f52078cad7573c5367d7a6e75f84f0. The carried branch is currently based on57491e918; I’m rebasing it onto the requested tip before continuing.Round 2 left P2 R250-R2-01 (revoked screen accepts edits that saves drop) and P3 R250-R2-02 (bare Hello capability
16n). I’m also completing the fixture process-group teardown fix (twoaddress in useretries; 1,143.072 ms against 350 ms) and making the account-switch probe conclusive.D312 specifies what to do when a Welcome changes the session incarnation, but does not specify the still-open screen after access revocation. I’ll follow the requested conservative behavior: stop accepting edits and show the plain unsaved/lost-edit state, for kayg’s review.
Next: complete targeted and production-browser regressions, try WebKit with Nix software GL, run
scripts/verify fast, report the D243 size delta, and post the per-finding evidence. No deployment.J32b rebase/checkpoint on
layout-per-tab-250atf04e4631a4bcfae169a01edb2714cfa22c4ec494, rebased ontoorigin/devf4b61cd6c3f52078cad7573c5367d7a6e75f84f0and pushed with lease. The compute01 job worktree is synced, andnix develop -c truesucceeds there.After the rebase, the focused tracer unit tests pass: 28 pass, 0 fail, 113 expectations, including revoked-store edit rejection and the named D312 Hello bit. Log:
/home/claude/.cache/codex-scratch/j32-layout-per-tab-250/evidence/unit-green-after-rebase.log. The earlier failing-first evidence for the capability and revoked store, and process-group regression, is recorded in comments 248, 261–263; the production revocation reproduction is in the round-2 summary at comment 243.Next through compute01
build-slot: runscripts/verify fast, build the production tracer, verify the same-account revocation and both-account canonical reads in Chromium, and run the deterministic process-group test. Then try WebKit with Nix software GL. No new browser or final-gate result is claimed yet.J32b resumed on branch layout-per-tab-250 at
f04e4631a4, based on origin/devf4b61cd6c3. I am continuing the round-2 fixes and evidence: revoked-tab behavior, named Hello capability, fixture process-group teardown, account-switch probe, Chromium/WebKit status, and scripts/verify fast. Existing carried worktree is clean; focused checks are recorded as green, with real-server/browser gates pending.J32b progress on
layout-per-tab-250(current treef04e4631a), rebased ontoorigin/devf4b61cd6c.D312 does not specify what a tab should show after its authenticated session is replaced. The implementation takes the conservative option: revoke the old layout store, ignore subsequent edits/events, end the old canvas, and display a plain sign-in message that reports pending edits lost when applicable. Please decide whether this is the desired revocation state; the choice is not recorded in D312.
R250-R2-01's same-account and cross-account production browser checks, R250-R2-02's named Hello capability, and the fixture process-group teardown regression are present on this tree. Focused Chromium runs and
scripts/verify fastare in progress on compute01. Gate log:~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/fast-after-rebase.log.J32b progress: the exact-head Rust checks passed through tests, clippy, cargo-deny, native fixtures and gate architecture.
scripts/verify faststopped at the old static-size ceiling check: x86_64 21,448,272 bytes (+4,448) and aarch64 19,635,240 bytes (+6,000). The orchestrator brief says D243 caps are removed; I am not changing their values. The log is~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/fast-after-rebase.log.The TypeScript verification stage is running through
build-sloton compute01 and is queued behind other jobs (position 10 at 08:36 IST). Next: finish TS verification, rerun the exact-head production Chromium specs, try headless WPE with Nix Mesa/llvmpipe, split the WIP commits atomically, push with lease, attach evidence and clean the target directory.J32b history update: branch
layout-per-tab-250is now pushed with lease atbd670c8861b3265097d3010e6fe3f0b87d1338f6as three atomic commits. The resulting source tree1a2f2077ac953014c09deefeb2adaa290c621fe2is identical to the tree used by the fast-gate run, whose only reported Rust failure was the removed D243 size cap.TypeScript verification is still waiting in compute01's build-slot queue. After it completes, I will run the production Chromium specs and the headless Nix Mesa/llvmpipe WebKit attempt, then attach evidence and clean the target directory.
J32b progress on
layout-per-tab-250atbd670c8861b3265097d3010e6fe3f0b87d1338f6, based onorigin/devf4b61cd6c3f52078cad7573c5367d7a6e75f84f0.TypeScript verification completed with 39 pass, 0 fail, and 59,970 expectations. The first Chromium run reached Playwright, but three specs failed before app assertions because the runner forced all server fixtures onto port 4481 with two workers; the session-incarnation spec passed. I corrected the runner to use distinct ports 4791–4794 and queued the production rerun.
The first WebKit attempt exited 127 before browser launch: its helper used a stale Nix libwpe path and put Nix glibc on the host loader path. The retry resolves the matching Nix package outputs dynamically, excludes glibc from LD_LIBRARY_PATH, and enables llvmpipe/surfaceless EGL.
Both corrected browser runs are waiting in compute01's build-slot queue. No result is claimed for them yet. Logs are in
~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/:ts-and-chromium-port-collision.log,chromium-rerun.log,webkit-initial-runtime-failure.log, andwebkit-software-gl-retry.log. Next: finish the browser runs, attach exact-head evidence, then remove the VM target directory.J32b queue checkpoint on
layout-per-tab-250@bd670c8861b3265097d3010e6fe3f0b87d1338f6.The corrected headless WebKit software-GL retry has waited about 30 minutes in compute01
build-slot(last reported queue position 8); the corrected Chromium rerun has waited about 30 minutes (last reported position 9). Neither has started its test command or produced browser assertions. The earlier Chromium EADDRINUSE and WebKit loader failures were runner setup issues and remain classified separately in the prior checkpoint.Logs:
~/.cache/codex-scratch/j32-layout-per-tab-250/evidence/webkit-software-gl-retry.logandchromium-rerun.log. I’ll continue when compute01 admits the queued runs, then attach final exact-head evidence and remove the VM target directory.J32b exact-head evidence is attached below.
J32b browser progress on
layout-per-tab-250@bd670c8861b3265097d3010e6fe3f0b87d1338f6.Exact-head TypeScript verification passed 39 tests / 0 failures / 59,970 expectations. The corrected production Chromium rerun passed all four scenarios: same-account revocation, account switch, session incarnation, and fixture process-group teardown. The account-switch test read both accounts' canonical layouts, observed no layout apply after the edit attempt, and required the old screen to end. The unit log, gate transcript, browser log, and three production screenshots are attached in comment 72785.
The corrected headless WebKit attempt uses Nix WebKitGTK/Mesa/libwpe, llvmpipe, surfaceless EGL, and an LD_LIBRARY_PATH excluding Nix glibc/libuv. A Node preflight passes. It remains queued through compute01
build-slotat position 12, waiting about 85 minutes; no WebKit assertions have run. If it reaches WPE EGL startup and still fails there, I’ll record the runtime failure in a separate issue and link it here.J32b round 3 (last) —
layout-per-tab-250@bd670c8861b3265097d3010e6fe3f0b87d1338f6, rebased onorigin/devf4b61cd6c3f52078cad7573c5367d7a6e75f84f0.16nHello capability with namedLAYOUT_SESSION_INCARNATION; the bit-4 unit test passes.canvas.layout.applywas sent after the cookie switched. It passes; the earlier probe was inconclusive.The corrected production Chromium rerun passed all 4 focused cases (same-account revocation, account switch, session incarnation, process-group teardown). The TypeScript suite passed 39 tests with 0 failures and 59,970 expectations. The fast log, Chromium log, unit log, and desktop/mobile screenshots are attached in comment 72785.
WebKit still could not reach app assertions after the headless Nix WebKit + Mesa llvmpipe retry. Playwright stopped at launch with missing
libGLESv2.so.2andlibx264.so; the earlier WPE attempt failed withCould not create WPE EGL display: EGL_BAD_PARAMETER. The log is attached to issue #288.scripts/verify fastRust gate summary (verbatim):The remaining row is the existing size check against removed D243 caps. The measured stripped static binaries are x86_64 21,448,272 bytes (+4,448) and aarch64 19,635,240 bytes (+6,000) against those former caps; no cap was changed.
J63 independent review round 3 started. Reviewing layout-per-tab-250 at
bd670c8861against origin/devf4b61cd6c3in isolated detached worktree /home/kayg/Developer/Worktrees/infraternal-j63-review. Scope: R250-R2-01/02, revocation event ordering, fixture process-group wait, conclusive cross-account canonical checks, protocol/store isolation and reconnect incarnation. No code changes; verify fast is not applicable to this review unless needed to prove a finding.J63 independent review round 3 (last): REQUEST CHANGES.
Reviewed
layout-per-tab-250atbd670c8861b3265097d3010e6fe3f0b87d1338f6againstorigin/devf4b61cd6c3f52078cad7573c5367d7a6e75f84f0, D312, the round-2 findings, and fix/evidence comments 72785/72812. One new P2; no P0/P1.R250-R3-01 — P2: the revocation message falsely reports server-confirmed edits as lost.
apps/tracer/src/canvas/model/server-store.ts:66takeslostfrompending, whose getter at line 78 includes entries with successful receipts still awaiting canonical coverage.apps/tracer/src/main.ts:457turns this into “Changes waiting to save were lost.” Reproduction: receive snapshot revision 1; submit a rename; receive its successful{revision: 2}Result, with its ordered event delayed; reconnect after same-account session replacement. The journal retains the confirmed overlay, so revocation callsonSessionRevoked(true)and shows the loss message even though revision 2 is committed and will reappear on sign-in. D312 explicitly preserves confirmed edits. Distinguish receipt-confirmed work from unconfirmed work, and use uncertainty-safe wording for sent work without a receipt; dropping a local queue does not establish that its edits were lost on the server. Keep the current teardown and fencing.Independent deterministic test on
claude@infraternal-dev-02-std, through~/infraternal-ops/build-slot nice -n10 nix develop -c bun test …, fails at the actual revocation callback:Exact-head focused result, verbatim:
The identical finding test passes on requested baseline
f4b61cd6c(1 pass, 0 fail, 3 expectations; the baseline has no revocation-loss callback), and the earlier version of the same probe passes on immediate pre-fix parent49a0f0ef3. This is a message regression, not evidence of actual server data loss. Test and positive late-callback control are committed locally onreview-j63-layout-250at2630e3300556357bdd9990dbddc1edeb58fc1390; no product changes or push.Other review results:
save, event admission and the pump reject revoked work. Clearing the journal advances its epoch, so late receipt/refusal callbacks cannot resurrect entries. The caller ends the old screen, with generation guards also covering revocation during initial load. The independent combined late-operation/late-snapshot/late-receipt/late-edit control passes without relying on disposal. The newly introduced wording issue is finding 1 above.LAYOUT_SESSION_INCARNATION; its bit-4 test passes independently.No product code changed;
scripts/verify fastnot run, as requested for this review. The removed D243 size-cap failure and WebKit #288 are not findings. No browser/build/server remains running. No target directory was created; the temporary VM review worktree was removed. Small evidence logs remain at/home/claude/.cache/codex-scratch/j63-review/evidence/on dev-02 and are attached here.Orchestrator fix for R250-R3-01 (small, message-only), pushed as
852e15b76onlayout-per-tab-250.When access is revoked, the screen now reports what the tab actually knows (
RevokedWork):unsent: an edit never left the tab. Message: "Not saved. Changes waiting to save were lost. Sign in again to continue."unconfirmed: everything was sent, but something has no receipt yet. Message: "Your sign-in changed. Your last changes may not have saved. Sign in again to check."none: every queued edit has a receipt. Message: "Your sign-in changed. Sign in again to continue."An unconfirmed migration counts as
unconfirmed, because its legacy source stays on disk. Teardown and fencing are unchanged.New tests in
session-incarnation.test.ts:{revision: 2}receipt arrives, its event is delayed, then the session is replaced. Expected["none"].["unconfirmed"].Results on dev-02:
scripts/verify fastis running on compute01. I'll post its output here before merging.