New machines switch to the mesh automatically and fast after add-host (D263) #201
Closed
opened 2026-10-13 06:38:32 +00:00 by kayg
·
345 comments
No Branch/Tag specified
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/infraternal#201
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
D263 / D164 / D233: after add-host, a new machine joins our mesh and its infraternal traffic (terminal first) must switch to the mesh path automatically and fast, without breaking the session. Owner: "I wanna see how we auto switch to our mesh really fast since that's part of our add-host thing too".
Measure on dev (perf VM, 150 ms RTT and LAN): time from add-host success → mesh joined (handshake) → first terminal byte over the mesh path, and whether a session open during the switch keeps every byte and its writer. Then make it fast: target the switch within one RTT + 50 ms after the mesh handshake completes, zero lost/duplicated bytes, no visible reconnect. Fallback to the previous path must be just as quick if the mesh path dies.
Baseline before code changes; continuing mesh-autoswitch at base/head
01dcbee940(origin/dev). #201 had no comments or linked parent; worktree clean, no previous job commits. Prior scratch contained partial build logs only.First unchanged-dev perf-VM sample (LAN, n=1 lifecycle; 31 echo samples per path): enable → observed mesh move 1,412.319 ms; observed working mesh → move 920.892 ms; silent cut → terminal echo 202.338 ms. Direct echo p50/p99 0.193/0.455 ms; mesh 0.415/0.629 ms. Same open attachment/input sequence retained all 3,975 expected bytes, without a repaint or close. Two real kernel nodes / one real PTY / 1,000 public inventory records; load start 1.92 0.68 0.47. Shared VM lock held for this sample only.
Limit: existing crates/infraternal-server/src/remote_mesh_namespace_tests.rs observes working peer counters every 200 ms and times status, not first mesh byte. It starts from enrollment, not the real add-host completion event. These are initial transport diagnostics, not D263 acceptance. Collecting 31 independent lifecycles each at LAN and 150 ms RTT before production changes; precise event instrumentation still needed.
Shipping x86_64 musl baseline: 22,576,464 bytes, SHA-256 36aa5c93c96f8701bf919213088892a17d55ce651e913db7d76765c5fe11a93f. Native fixture's PT_INTERP alone was changed to the VM's Debian loader; shipping artifact unchanged.
Initial LAN receipt · artifact provenance
LAN baseline complete: 31 independent lifecycles on unchanged
01dcbee9. Enable → observed mesh move p50/p99 1,369.358/1,558.858 ms; observed working mesh → move 1,093.600/1,135.277 ms; silent cut → terminal echo 202.104/223.750 ms. Exact byte integrity across every lifecycle: 114,384 expected = received. One run failed the old latency assertion after completing integrity/lifecycle; no sample discarded. 150 ms RTT collection continues against the same frozen artifacts.All 31 LAN receipts
Finding: current fixture times a mesh.status RPC and a 200 ms peer-counter poll, not the handshake/first mesh byte. Next step is measurement-only instrumentation: test-feature kernel timestamp precision and test-only authenticated QUIC/output event capture. Production transport/auth behavior and wire contracts stay unchanged in that step. Both WireGuard and QUIC handshake boundaries will be reported explicitly; enrolled-host fixture results will not be called full add-host results.
Completed and pushed
3e3c4a57: opt-in test-support captures kernel handshake nanoseconds; default builds and public JSON omit the new field. No production transport change. Timestamp tests failed before implementation, then 2/2 passed. cargo fmt --all --check and touched Mesh crate release clippy (--features test-support --all-targets, -D warnings) exited 0. Raw gate logs remain in job scratch/evidence.Kernel source confirms the handshake-complete timestamp uses the realtime clock and netlink exports its seconds/nanoseconds timespec: https://git.zx2c4.com/wireguard-linux/tree/drivers/net/wireguard/timers.c and https://git.zx2c4.com/wireguard-linux/tree/drivers/net/wireguard/netlink.c . This avoids treating a polling delay as part of mesh join time. Next: authenticated QUIC/terminal-byte event capture in the proof process.
Unchanged-dev baseline complete (
01dcbee940), 31 independent lifecycles per venue on infraternal-perf-test under the shared lock. Two real kernel nodes / one real PTY / 1,000 public inventory records; load and artifact provenance are in the receipts.Integrity: LAN 114,384 / 114,384 expected/received bytes; 150 ms 392,494 / 392,494. All 62 lifecycles retained the same open attachment and input sequence without terminal close/repaint. No failed sample discarded: LAN sample 16 completed the lifecycle but failed the existing echo-overhead assertion (mesh p99 13.678 ms vs direct 0.454 ms, +2 ms allowed). Every other run passed that fixture's older assertions.
These are transport diagnostics, not exact D263 acceptance or real add-host completion timing: the existing fixture polls working counters every 200 ms and observes a status RPC. The new test-only phase hooks will remove that ambiguity before choosing an optimization. No production behavior has changed.
LAN receipts · 150 ms receipts · summary
Completed and pushed
8eaf62db: test-only, bounded transport milestone recorder and callbacks at mTLS completion, ledger admission, control readiness and nonempty terminal output. It stores public addresses/stream IDs/timestamps only, never terminal contents. Weak ownership retires recording with the proof; a peer filter prevents unrelated tests from contaminating it. The isolation test failed (3 events vs 2 expected), then passed after filtering. Recorder test, cargo fmt --all --check and touched-server release clippy (--features mesh-transport-proof --all-targets, -D warnings) all exited 0. Shipping code excludes the recorder/callbacks.Verified baseline raw logs: 62 lifecycle logs + 62 exit records
Next step in progress: connect the fixture's node to an inherited public timestamp socket, queried only after the first mesh byte has arrived. This reads the node's exact kernel handshake timestamp without a poller during the measured join, and distinguishes it from the server's QUIC handshake.
Completed and pushed
a31a640190: exact node kernel timestamp query after first mesh output, separate WireGuard/mTLS/control/terminal phases, and an explicit add_host_completion_measured=false marker. No shipping transport change. The node's public timing channel now reuses the fd-3 exec wrapper so the surviving shim cannot retain it; the descriptor assertion failed before this correction (node admission stopped), then real LAN and 150 ms lifecycles passed with exact byte integrity. Fmt, timestamp codec test and touched-server clippy passed. Full scripts/verify fast is running at this milestone.LAN precise receipt · 150 ms precise receipt · descriptor red test
Finding from initial precise probes (n=1 each, diagnostic): the initial 150 ms receipt measured node WG handshake → first mesh output 1,025.667 ms; server mTLS → control ready 448.777 ms; control ready → first output 304.274 ms. LAN showed 1,183.163 ms WG → output but only 18.066 ms mTLS → output. The delay is not just the old status poll: authenticated connection establishment and post-handshake round trips dominate. Improving retries alone cannot meet the 200 ms target. Next implementation must preserve the authenticated connection/session while changing its path, or remove the setup round trips without weakening admission/writer checks. Current quinn's public endpoint rebind changes the local socket; remote-address handling needs careful integration. Independent read-only review of the measurement slice is underway.
Still outstanding: production optimization, actual add-host completion boundary, precise 31+ sample before/after acceptance table, at-scale cost, final static size comparison and final fast-gate output. This comment is a continuation checkpoint, not a completion claim.
Independent review of 01dcbee9..a31a6401: instrumentation is suitable for diagnosis; no production transport/auth change or secret logging. Acceptance is still outstanding.
Two P2 measurement findings are being fixed: reject a kernel latest-handshake timestamp outside this fresh-peer lifecycle or after the recorded mTLS event; label the Output hook as authenticated transport arrival before actor delivery. Also reject wall/monotonic clock disagreement. The exact terminal byte comparison remains separate.
Transport finding: at 150 ms RTT, the final single diagnostic sample spends 765.715 ms from server mTLS completion to first mesh Output (439.670 ms control setup + 325.473 ms first Output), before counting the fresh handshake. These are n=1 diagnostics, not acceptance percentiles. Existing standby/primed-writer machinery can remove later attachment costs but cannot alone meet the 200 ms WG-handshake target.
Pinned Quinn 0.11.12 / quinn-proto 0.11.19 has no implemented client remote-address migration operation: preferred-address configuration is advertised/parsed but the client drops packets from a different remote address. A connection-preserving experiment can reuse its existing AsyncUdpSocket / Endpoint::rebind_abstract, rather than choosing a new engine. It needs shared server CID ownership across leased bindings, trusted public-address mappings, preserved datagram/source metadata, authenticated liveness, and explicit handling of immutable connection.local_ip(). This has not been implemented or declared safe. Separate specific-address listeners currently own different CID spaces.
No task completion claim: production switch/fallback optimization, real add-host completion boundary, precise 31-sample before/after, final size comparison and final fast gate remain required.
Completed measurement correction at
3bf5aa3b(pushed mesh-autoswitch). Independent review accepted the bounded fresh-peer / ordering / clock guard and exact transport-arrival label. No production switch implementation yet.Per-step checks: fmt exit 0; release server clippy with mesh-transport-proof/all-targets/-D warnings exit 0. Red compile failure: log. Final release unit gate verbatim (full log):
Precise baseline finding: LAN sample index 20 passed join, exact-byte/fallback checks, and timestamp validation, then mesh disable failed with PluginError::Storage at remote_mesh_namespace_tests.rs:1096. Public receipt. This is an incomplete lifecycle, not an invalid timing. Initial runner stopped as designed. Resume keeps existing samples without rerunning/replacing them and retains lifecycle_complete=false plus exit=101; collect 31 valid phase receipts per venue and report failed lifecycle counts separately. Any absent or invalid phase receipt still stops collection. The Storage cause is not established; do not treat it as a latency assertion or hide it.
The ongoing fast milestone reached static builds: rebuilt x86_64 musl is 22,576,464 bytes, matching the frozen origin/dev base size (delta 0). Existing static gate limit is 16,404,816 bytes, so that row fails even for the baseline-sized artifact. SHA-256 differs (base 36aa5c93c96f8701bf919213088892a17d55ce651e913db7d76765c5fe11a93f; rebuilt 0159d59839813c00815d7f534cabdf3eacf3edd5089acb8b95f4e1302e64a198). Zero size growth does not imply byte-identical artifacts. ARM static build and final summary remain running; the gate is not green.
Precise unchanged-transport baseline complete: 31 independent phase samples per venue on infraternal-perf-test, per-lifecycle perf lock and load recorded. Source
3bf5aa3b; frozen shipping base01dcbee9. Two live nodes, one real PTY, 1000 public inventory rows; not 1000 live connections. All phase guards passed; no sample discarded or replaced. Nearest-rank p99 is the maximum of 31.Output means authenticated transport arrival before actor delivery. Real add-host completion remains unmeasured (explicit false in every receipt); enable is a proxy. No production optimization has occurred, so there is no after column yet. Both switch and fallback miss the nominal 50 ms LAN / 200 ms WAN targets.
61/62 complete lifecycles compared 501,643 expected/received bytes exactly, retaining the attachment and rejecting Close/repaint Snapshot after readiness. LAN index 20 passed switch and exact-byte fallback, then failed mesh disable with Storage; that incomplete lifecycle remains in the timing table. LAN indices 16/28/29 completed but failed the old local-echo overhead assertion: direct/mesh p99 respectively 422/8629 us, 636/12870 us, 1186/5298 us (allowance +2000 us). WAN 31/31 complete, exit 0. All failures retained.
LAN receipts, WAN receipts, summary, artifact provenance, all 186 raw logs/exit records/receipts.
Completed two experimental steps, both pushed to mesh-autoswitch:
fe5b8b70: exact trusted peer-tuple adapter in remote_path_socket.rs, only test support / unit builds. Real UDP test plus all 12 remote transport tests pass; fmt and release all-target clippy pass. Independent review accepted experimental use only. Red typecheck, 12-test gate, clippy.bf9e68d5: real mTLS + exchanged Hello + bound writer continuity test. Plain rebind failed the bounded functional control; exact tuple translation passes frames before mapping, while mapped, and after return. Connection/stream/writer ids and fingerprints remain unchanged; one connection per endpoint; EOF has no trailing bytes. Original sockets are retained in owned Arcs; pinned Quinn replaces current/previous slots with different descriptors on return (review checked this). Same local IP/different ports only. Red control, final test, all-target clippy.These are functional tests, not performance evidence or production migration. No production caller, cross-IP source solution, shared binding lifecycle, kernel mesh continuity or automatic fallback implementation yet. Existing engines/dependencies unchanged; default builds exclude the adapter.
Checkpoint, not job completion: head
bf9e68d546pushed to mesh-autoswitch; worktree clean after removing only the gate-generated Python cache. No deploy. Production transport remains unchanged; D263 switch/fallback targets and real add-host completion measurement are outstanding.Author review against docs/verification/mesh-agent-transport-review.md: existing engines and shipped protocol/auth unchanged; new adapter excluded from default builds; trusted exact tuples supply routing rather than authority; socket lifetimes and bounded recorder/clock checks reviewed; no private material/payload retained by recorder; actual kernel/PTY evidence distinguishes 2 live nodes/1 PTY/1000 inventory from live connection scale. No UI changes. Parser/fuzz surfaces in shipping protocol unchanged. This is measurement/experimental acceptance only.
Final head
scripts/verify fastexit 1. Full verbatim transcript. Summary verbatim:Static output verbatim:
Frozen origin/dev x86_64 binary and final rebuilt x86_64 binary both 22,576,464 bytes: delta 0 (D243 job constraint). ARM base delta not measured. Production trust, x86 key memory initialization, static linkage and CLI checks passed. Full gate remains blocked by the historical size limits; the owner/orchestrator must reconcile the baseline budget or arrange a separate baseline reduction. This job has not waived or changed the gate.
Next authorized implementation needs shared server CID ownership across leased UDP bindings and explicit trusted source-address selection; plain separate endpoints cannot carry the original connection. Proposed additive SPI direction before changing the contract: a generic leased server-binding method carrying trusted destination addresses, with existing listener behavior retained as its default. Declare this through the remote plugin API, never hard-code mesh address ranges in core. Prepare the server binding/source routes before publishing a node preference. Preserve the running QUIC session/streams rather than dropping its daemon future on route change. The test's server-wide rebind is only a one-pair experiment; it is not safe to apply unchanged to a shared production endpoint.
Still required: cross-IP/source and shared-binding tests; implementation/role-off cleanup; silent-cut fallback and scale measurements on the perf VM; precise 31+ before/after per venue; actual add-host completion boundary; existing-session integrity through real mesh routes; final size comparison and head gate. TCP original-path behavior needs explicit qualification, since QUIC rebind cannot preserve a TCP connection. First-byte instrumentation also needs to correlate a retained connection and fresh output rather than requiring a new mesh TLS event. LAN sample 20 disable Storage cause remains unestablished; retain its failed lifecycle evidence.
Continuation r2 started on mesh-autoswitch at
bf9e68d546; branch base01dcbee940. Read all 11 comments; no parent is linked in #201. Retaining the prior 31-sample-per-venue baseline and its failed disable lifecycle. WG handshake → mesh Output p50/p99: LAN 1118.535/1213.078 ms; 150 ms RTT 877.758/1060.016 ms. Silent cut → echo 201.766/202.357 and 355.561/364.734 ms. No production changes yet. Next: isolate Storage on plugin disable, then shared UDP binding/source routing with retained authenticated connection and stage measurements. Perf only on the authorized VM; no deploy; push mesh-autoswitch only.Continuation r3 (Claude backend engineer) started on mesh-autoswitch at
bf9e68d546; branch base01dcbee940. Read all 12 comments. Plan: (1) stage timestamps for WG handshake → first mesh byte and remove waits, (2) production connection-preserving switch, (3) Storage on mesh-disable with a red test first, (4) fallback within one RTT + 50 ms, (5) 31+ samples per venue on the perf VM before/after, (6) stripped static x86_64 size delta vsbf9e68d5. No deploy; push mesh-autoswitch only.Finding + fix (
ac62f7b0, pushed): cause of the LAN sample 20mesh disable→PluginError::Storage.sqlx 0.8.6
begin_with("BEGIN IMMEDIATE")is not cancellation safe: after the connection worker acknowledges the BEGIN it awaits the handle again (SqliteTransactionManager::begin→lock_handle()) before aTransactionexists. A caller dropped there returns the pooled connection inside an open writer transaction with no owner. Disabling mesh aborts its coordination workers at arbitrary awaits (Store::attach/load_uncacheduse BEGIN IMMEDIATE), sopersist_setting's BEGIN on that connection failed:attempted to call begin_with at non-zero transaction depth. On the production file DB the same leak keeps the RESERVED lock (other writers hit the 5 s busy timeout → Storage) and later statements on that connection join an uncommitted transaction. Not mesh-specific: any cancelled BEGIN IMMEDIATE (7 production sites).Fix:
infraternal_plugin_api::storage::begin_immediateruns the begin in its own task; a dropped caller detaches, and the finished Transaction is dropped in the task, which queues sqlx's rollback. All production BEGIN IMMEDIATE sites use it. Rejected: pool acquire/release checks (cannot reset sqlx's depth without closing the connection), sqlx upgrade (0.8.6 is current; path unchanged).Red before fix: plugin-api test (cancel at poll 1) and
mesh/tests/store.rs::attach_cancelled_at_any_await_leaves_the_store_writable(cancel at poll 2 →Err("Mesh storage unavailable.")). Green after; touched-crate clippy-D warningsand tests pass.storage-red.log · mesh-store-red.log · mesh-store-green.log · clippy · tests
Continuation r4 started on mesh-autoswitch at
beccfb5ec5; base01dcbee940. Read all 14 comments; no linked parent. Clean worktree. Reviewing the production in-place routing commits and retained diagnostics; Storage cancellation fixac62f7b0retained. Prior precise 31-sample baseline retained. Remaining acceptance: real mesh switch/fallback, add-host completion boundary, 31+ after samples LAN/150 ms, no size growth, final head gate. No deploy; push mesh-autoswitch only.Review finding on
beccfb5e: production x86_64 musl artifact is 22,605,872 bytes vs frozen base 22,576,464 (+29,408); no-growth acceptance not met. Retained single-sample VM after results: WG handshake -> first mesh Output LAN 13.429 ms, 150 ms RTT 93.624 ms; silent cut -> echo 129.509/720.639 ms. New captured VM lifecycle reproduced WAN fallback 666.555 ms, exact integrity and no reconnect passed. Packet capture shows client returns to the enrolled tuple but server still has outstanding stream data on the lost path; QUIC migration/path validation and loss timers delay recovery. These are diagnostic n=1, not acceptance percentiles.Proposed additive NodeChannel SPI before implementation: lease a trusted alternative peer address (default unsupported). The remote transport binds it to the already admitted connection's exact source port, preserves its logical tuple on both ends, and owns/removes the mapping with the plugin worker. No new protocol, parser, engine or identity authority. Prepare mapping before publishing node configuration. On a physical path change, bounded recent encrypted datagrams are resent immediately on the new path; QUIC packet numbers deduplicate them. Tests must cover exact foreign tuple refusal, lease cleanup, byte bounds and same authenticated connection/writer. Raw UDP routing hints never authorize data. Also shorten/measure the 100 ms liveness probe interval; current LAN fallback cannot meet 50 ms.
Trusted tuple regression: red compile failure before implementation; real UDP test now passes. It exercises mesh tuple normalization, reply source selection, immediate replay of unacknowledged input on return, and mapping removal on lease retirement. No production acceptance claim yet.
SPI refinement: route_peer accepts an exact SocketAddr. The local QUIC source port is reported over the existing authenticated control channel (optional ServerLink.source_port / Mesh identity transport_port); using the enrolled port would be wrong behind NAT. The host supplies the logical admitted tuple. Default capability is unsupported. No identity is learned from UDP. Prepared aliases are bounded to 10,000 admitted peers and replay to 64 datagrams / 64 KiB per peer.
peer-route-red.log
peer-route-green.log
fallback-capture.tar.gz
Completed and pushed
9df7b853: additive default-unsupported route_peer lease, optional ServerLink.source_port, strict legacy status compatibility regression, and contract declaration (D198/D215/D263). Existing callers initially use absent metadata; production route wiring remains uncommitted pending broader tests. Plugin-api library tests 21/21 pass; touched-server release all-target clippy with mesh-transport-proof exits 0; formatting applied before commit.Correction to the previous SPI note: the implementation keeps mesh.identity completely unchanged. Local transport port metadata is read from the existing extensible mesh.status object, avoiding an unknown field in older coordinators' strict identity decoder. No new wire parser or dependency.
Remaining: real-kernel validation of prepared aliases/replay, faster fallback detection, 31+ after lifecycles per venue, real add-host completion boundary, at-scale costs, size reduction vs base (+29,408 bytes at interrupted head), final fast gate.
Completed and pushed
4e4476c1: fix a missed endpoint wakeup when lease installation races a pending receive poll. A deterministic hook installs the lease after the old socket returns Pending but before receiver registration; without the generation recheck, the new socket is stranded until unrelated traffic/timers. Regression failed before the fix and passes after it; all 17 current remote transport tests pass, remote all-target clippy exits 0, formatting applied. No build-host timing used as performance evidence. The peer alias/replay feature remains uncommitted pending kernel proof.lease-wake-red.log
lease-wake-green.log
remote-debug-clippy.log
Completed and pushed atomic steps through
a04bda284c(clean worktree):a196cedaremote-plugin exact peer aliases + bounded encrypted replay + wildcard source fix;af62c02fweak admitted-node route capability (ownership regression red/green);0aa54e5cauthenticated local QUIC port status;1ecc332bcoordinator installs/owns routes before publishing config (ordering/off tests);a04bda28opt-in recorder resolves current physical peer at callback. No deploy.Per-step gates: 18/18 remote transport tests, 42/42 remote-runtime tests (5 ignored), 9/9 infraternal agent path tests (1 perf-only ignored), 6/6 coordinator tests, 1/1 recorder isolation test; touched-server and remote all-target clippy exit 0; fmt applied. Full scripts/verify fast is running on
a04bda28(scratch evidence-r4/verify-fast.log and .exit).VM diagnostic, n=1 at 150 ms RTT with 14 ms RTT jitter, 2 live nodes / 1 real PTY / 1000 public inventory rows: WG handshake -> first mesh Output 77.219 ms, fixture enable -> handshake 4594.848 ms, silent cut -> echo 547.239 ms. All 11,787 expected/received bytes retained, no new mesh TLS handshake, no close/repaint after readiness, full lifecycle complete. Timing assertion failed because fallback exceeds even the older 400 ms bound. This executable was built from the in-progress route work before the final wake/source/ownership fixes; diagnostic only, not final head acceptance. The callback reports the current physical route, not a per-packet provenance guarantee.
Still outstanding: fast server/client fallback (the 100 ms probe plus RTT detection and retransmission misses 200 ms); real add-host completion event/harness (every receipt still says false); final 31+ lifecycles per venue and before/after tables; scale/cost; shipping size reduction (interrupted-head delta +29,408 bytes); final gate and author review. Storage cancellation fix
ac62f7b0retained. No completion claim.peer-diag-150.json
wildcard-red.log
wildcard-green.log
route-ownership-red.log
route-ownership-green.log
Head
a04bda284c: full fast gate exit 1. Agent unit tests: 91 passed, 3 failed, 14 ignored; failures are healthy_mesh_path_is_never_declared_interrupted, component_down_keeps_its_reason_when_the_mesh_also_stops_answering (admission timeout), and winning_quic_drops_the_unfinished_tls_dial_before_control_dispatch (TCP AddrInUse). These are not the #118 exception. Serial path tests had passed; parallel gate exposes remaining faults. x86 static head 22632752 bytes versus frozen base 22576464: +56288 bytes, D243 fails. ARM head 20039472 bytes; frozen ARM comparison remains outstanding. WAN diagnostic fallback 547.239 ms remains over the 200 ms target; real add-host and 31-sample after measurements remain outstanding. Continuing fixes.verify-fast.log
head-size.json
Step
e4a289ce: client GSO replay could exceed the documented 64 KiB bound: [1024,1024,1024,64512] retained 66560 bytes. Regression fails on the previous implementation. Client now reuses the same bounded Recent buffer as the server, evicting until byte and count bounds hold; duplicate replay implementation removed. Remote transport 19/19 tests pass; all-target clippy and fmt pass. Shipping size remeasurement pending.path-replay-red.log
path-replay-green.log
path-replay-clippy.log
Completed
bd32348e: QUIC-winning dial test reserves TCP+UDP jointly; existing gate AddrInUse failure is fixed without serialising tests. Its functional test and agent all-target clippy pass.Completed
3612d5eb: socket-level reply cannot identify QUIC ACK ranges. Previous client cleared all ciphertext on any reply, including a delayed reply that did not acknowledge newer input. The deterministic unrelated-reply regression fails before the fix; the client now retains bounded recent encrypted history, as the server does, and QUIC owns duplicate suppression. Transport 20/20 and agent path 9/9 tests pass (1 timing test ignored). Bounds remain 64 datagrams/64 KiB per connection: <=64 MiB each side at 1000 peers. VM fallback and shipping size remeasurement pending; no new latency claim.proxy-port-green.log
proxy-port-clippy.log
replay-retention-red.log
replay-retention-green.log
replay-retention-path-tests.log
Finding: queued QUIC DATAGRAMs started liveness expiry before successful socket emission. HeldSocket regression blocks UDP writes, then releases them: previous code reports TimedOut while the healthy probe remains queued. Fix records successful mesh sends and starts RTT budget there; remote transport 21/21 and all-target clippy pass. Agent healthy-idle assertion still fails intermittently under shared-host load (also isolated path suite); it passes on the perf VM with the same debug artifact. This is not a #118 waiver. Proposed test placement, consistent with all-timing-on-VM rule: preserve the exact healthy-idle timed assertion as an opt-in VM check and add a default functional control-frame/connection-integrity test. Admission timeouts and component-down Stopped(0) seen in host runs remain under investigation. No timing acceptance claimed.
Completed
4b09cf1c: mesh liveness deadline begins at successful encrypted socket emission, with notification registration before timestamp check. HeldSocket regression red/green; transport 21/21, agent path 9/9 (1 ignored), fmt and all-target clippy pass. Healthy-idle assertion passes on perf VM; host timing failures remain recorded, not waived.Completed
548ad76a: an unanswered quick return proof could destroy the original connection after initial enrolled replies were lost. DroppedReplies fixture reproduces Stopped(0) on the original control stream. Return now keeps a minimum 1 s QUIC recovery proof guard; routing/replay still happen immediately and successful proof has no added delay. Mesh-candidate deadline unchanged. Regression keeps exactly one admitted connection; 9 functional path tests and clippy pass. This is a recovery guard, not a measured fallback-latency claim. Both-UDP-paths-unavailable TCP test passes.Shipping x86 rebuild at
3612d5eb: 22631536 bytes, still +55072 versus frozen base, D243 FAIL. It excludes the later emission/recovery fixes. All performance acceptance, precise byte-path provenance, real add-host and 31-sample after measurements remain outstanding.probe-send-red.log
probe-send-green.log
probe-send-clippy.log
healthy-vm.log
return-loss-red.log
return-loss-green.log
return-loss-path-tests.log
retained-client-size.json
Completed
245e41dc: exact healthy-idle timing assertion remains unchanged, now opt-in on the perf VM; default functional test verifies 64 ordered control replies and one admitted connection. Mesh-enabled agent suite passes 95 tests, 15 ignored; fmt/clippy pass. This does not waive any timing budget or claim fallback acceptance.Completed
2526491e: opt-in real SQLite writer-begin benchmark, 31 batches × 1000 serial minimal transactions, same one database writer as production. API 22 tests pass, benchmark ignored; fmt/clippy pass. Comparing shared-helper boxing as a shipping-size candidate next; Storage cancellation regressions stay intact. No foundational engine/library change.functional-path-agent-tests.log
functional-path-clippy.log
storage-bench-tests.log
Size-candidate baseline at
2526491eon perf VM, release opt=3/LTO, 4 vCPU, lock held only for measurement, load 0.08/0.25/0.66: 31 batches × 1000 real SQLite BEGIN IMMEDIATE+COMMITs, one writer. Batch p50/p99 89.806/127.304 ms; per-write batch-normalised p50/p99 89.805/127.303 µs (not individual-operation percentiles). Testing an erased boxed begin future next to reduce repeated async-state/drop code in callers. Same cancellation-safe spawned task and same engine; extra allocation must be measured before accepting.storage-before-vm.log
Rejected shared-writer boxing candidate; source restored to
2526491e, cancellation-safe Storage fix remains unchanged. Exact x86 shipping comparison on the same head/profile: unboxed 22634896 bytes, boxed 22665360 (+30464). Boxing fails D243 despite VM SQL results showing no observed slowdown: batch-normalised per-write p50/p99 89.805/127.303 µs before versus 84.912/104.589 µs boxed, 31 batches × 1000 writes each; different low-load windows, not a speed-up claim. Unboxed head still +58432 versus frozen base 22576464, so shipping no-growth remains unmet. Next size work targets shared transport data/state rather than changing the Storage recovery mechanism.storage-boxed-vm.log
unboxed-head-size.json
Completed
5193122e: both tuple keys index one bounded slot arena; receive resolves either tuple with one lookup. Stale lease identity cannot remove a recycled slot, conflicts still refuse, last retirement releases slot/cache allocations. 22 transport tests and clippy/fmt pass. Completed970965d4: paired opt-in lookup benchmark, prior layout only in test code. VM 4 vCPU/7 GB, load 2.01/0.94/0.40, 31 interleaved batches × one million receives over 1000 tuples. Initial two-lookup slot candidate was slower and rejected (70.568/87.286 ms prior vs 78.291/95.850 ms slots, low-load window). Final one-lookup comparison: prior p50/p99 50.136/88.545 ms per batch; slots 24.685/28.730 ms. Capacity-weighted element bytes 444416→331776 (-112640 at 1000), excluding allocator/control/RSS/kernel memory.X86 shipping at
5193122e: 22629008 bytes; -5888 versus previous head 22634896, still +52544 versus frozen base 22576464, D243 FAIL. Full fast gate is running at head970965d4(benchmark code is test-only). Production fallback target, precise byte-path witness, real add-host before/after and 31-sample after lifecycle measurements remain unfinished.peer-slots-vm.log
peer-slots-single-lookup-vm.log
peer-slots-tests.log
peer-slots-bench-clippy.log
Next fallback experiment / additive contract proposal: waiting for a failed mesh round trip, then sending input and waiting for an enrolled echo, spends two network round trips. To target cut→first terminal byte within RTT+50 ms, evaluate one bounded encrypted backup copy after 20 ms while a leased mesh path is active, in both directions. Same QUIC connection/packet numbers, no plaintext replay, no new parser or crypto engine. Exact ciphertext duplicates must not change reply routing hints. Queues, timers and socket references retire with their path/lease; no detached task. Measure bandwidth/CPU/memory at 1000 nodes and 31 LAN/WAN lifecycle samples before retaining this trade-off.
Compatibility proposal: additive Hello capability bit 6, REMOTE_PATH_BACKUP. Nodes enable copies only after the admitted server declares it; legacy peers retain current behavior. Existing Link.standby watch already carries authenticated server Hello caps, so no new core proposal/configuration schema is needed. No capability is a substitute for mTLS, ledger admission or writer binding. Shipping no-growth remains a separate hard requirement (+52544 bytes at current measured head). This experiment is not yet implemented or accepted by measurements.
Head
970965d426checkpoint, NOT DONE. Full fast gate fails: CA unlock audit capture expected 4 events, received 0; server result 420 passed/1 failed/46 ignored. The same gate executable passes that test alone (1 test), which is evidence of an intermittent capture failure, not a #118 waiver. Previous commentary that workspace tests passed was premature. Static x86 22629008 (+52544 vs frozen22576464); ARM20036368, frozen ARM baseline still not established. Historical absolute caps also fail. No TS rows run after Rust failure.Current perf-VM debug-profile namespace diagnostic (not release acceptance): 150 ms RTT ±7ms/direction; load start3.76/2.56/1.55. Exactly13355 expected/received bytes, one preserved connection, restart/off complete. WG handshake→first recorded mesh Output78.872ms; cut→echo522.231ms; cut→fallback status675.119ms. The first-byte recorder still resolves current physical route at callback, not exact packet provenance. Receipt says add_host_completion_measured=false. Fallback fails even the old400ms fixture limit and the requested200ms target. No 31-sample after table or real add-host acceptance claimed.
Remaining: implement/evaluate bounded backup-copy proposal recorded above, with duplicate routing-hint protection and lease/off cleanup; establish precise first-byte witness; compose real SSH add-host with automatic mesh and 31+ independent cycles each LAN/WAN before/after; remove remaining shipping growth and establish frozen ARM comparison; resolve full audit gate failure and rerun head fast gate. No deployment. All commits pushed only mesh-autoswitch; worktree clean apart from generated cache removed. Evidence outside repo, source/artifact hashes attached.
verify-fast-milestone.log
current-debug-150.log
current-debug-150.json
current-diagnostic-artifacts.json
audit-isolated.log
Continuation r5 start: mesh-autoswitch head
970965d426; original base01dcbee940. Read all 31 issue comments; no linked parent. Worktree clean. First merging origin/dev for #207 audit isolation and iterate gate. Remaining: RTT+50ms fallback, D243 no growth, exact byte-path witness and real add-host 31-sample LAN/WAN acceptance; final fast gate and break-it handoff. No deploy. Push this branch only.Fallback analysis on inherited
970965d4: remote_server_path.rs probes every 100 ms; remote_quic.rs::path_interrupted waits RTT+50 ms after socket emission, then use_enrolled replays and terminal echo spends another RTT. Thus reducing the probe interval alone cannot meet cut-to-echo <= RTT+50 ms at WAN RTT=150 ms. Evaluating the recorded bounded ciphertext-copy proposal requires duplicate routing-hint protection first, owned timers/queues, capability negotiation, and at-scale VM costs. Current server receive_peer changes the physical reply hint on every raw tuple change, including exact duplicates; a regression will cover that before copies. No new measurement or acceptance claim.Completed/pushed
50a4d1d4: merged origin/dev804ac0e8; coordination keeps compact access plus authenticated source-port discovery. Rust iterate rows all pass, including the isolated CA-unlock audit test. Combined TS gate first failed with 7 unhandled deferred-icon loads because status.test.ts supplied a DOM lacking querySelectorAll. The fixture now supplies it; full TS compiler/tests/build/transfer/protocol gate passes. No production UI change.Completed/pushed
c473e949: real UDP regression proves delayed duplicate ciphertext changed the physical hint and learned reply socket. Added bounded ciphertext-suffix hint filtering and GRO normalization; every datagram still reaches QUIC, whose authentication/dedup owns authority and bytes. Ring bound 256 × 16 bytes per populated peer, 4,096,000 capacity bytes at 1000; freed with lease. 24 functional tests pass, 2 VM benchmarks ignored; fmt/clippy pass.VM, release opt=3/LTO, 31 batches × 20000 fresh/duplicate pairs across 1000 peers, 1352-byte packets: full digest pair-normalised p50/p99 10294/11767 ns (load 0.04/0.14/0.76); suffix 258/427 ns (load 0.04/0.12/0.69). Different low-load windows, not individual-packet percentiles. Rejected full digest. These are adapter costs, not switch/fallback acceptance or 1000 live authenticated connections. Remaining: copies/capability/lifetime tests, shipping no-growth, real add-host and 31-sample switch/fallback acceptance, final fast gate.
merge-iterate.log
merge-ts.log
duplicate-red.log
suffix-green.log
suffix-clippy.log
duplicate-vm.log
suffix-vm.log
Completed/pushed
c7b59882: additive REMOTE_PATH_BACKUP Hello bit 6; API tests 21/21 and clippy/fmt pass. Completed/pushed92a1b93d: explicitly enabled bounded copies, one owned endpoint socket clock, no detached worker or extra payload queue. Existing ciphertext history remains 64 datagrams/64 KiB. The earliest 20ms clock flushes the retained pending batch, including younger entries, so continuous eviction cannot starve copies. Delivery, off/return, timer registration wakeup and eviction regressions pass: 28 functional tests, 2 VM-only benchmarks ignored; clippy/fmt pass. No production client caller enabled at those heads, and no latency/size acceptance claim.Completed/pushed
ce7128af: server advertises native support, retains admitted Hello bits in the existing metadata slot, enables the peer route only after the admitted client opts in. Real mTLS Hello/suffix/legacy tests included in 42 passing remote-runtime tests (5 opt-in proofs ignored); server all-target clippy with mesh-transport-proof passes. No new parser/engine/authority.New real-mTLS regression: enrolled encrypted copies kept authenticated frame counters fresh even with zero mesh replies, so the previous path detector never returned. Red timeout retained. Fixed detector requires both physical mesh arrival after emission and newly authenticated frames; 29 transport tests pass, 2 ignored. Fix not committed yet. Client wiring also exposed exact pre-sent/daemon Hello byte mismatch; consolidating capability serialization. Client path gate still under investigation (parallel 93/95 pass); no waiver. Remaining: client gates/activation, VM end-to-end/performance costs and size reduction, real add-host completion/witness, 31+ acceptance cycles each venue, final fast gate.
copies-tests.log
copies-clippy.log
server-cap-tests.log
server-cap-clippy.log
copy-liveness-red.log
copy-liveness-green.log
Completed/pushed
2a86cfc5: physical mesh arrival plus authenticated freshness; 29 transport tests pass, 2 VM benchmarks ignored, clippy/fmt pass. Completed/pushed3d7841fc: deterministic regression proves the primary UDP port did not reserve its future mesh tuple. Jointly reserve both paths, transfer fixed-port descriptors, jointly reserve TCP/UDP fixtures, and retain the cut socket unread to avoid ICMP. Full mesh+remote client suite passes 96 tests (15 ignored), clippy/fmt pass; no serialisation or widened deadlines.Completed/pushed
554b2105: native client opts into Hello bit 6 and enables copies only after authenticated server support, with the existing standby watch handling retained-mesh startup. Existing probes use 15ms/30ms interval/margin for negotiated peers; legacy 100ms/50ms stays. One shared capability serializer fixes exact pre-sent/daemon greeting equality. No new connection/writer/frame. Completed/pushed9269ade5: additive wire contract documents negotiation, bounded existing history, owned clock, eviction-safe batch and off/liveness invariants.Shipping x86 musl at
554b2105: 22,295,792 bytes, SHA256 e3e790c3d8f53b35988f81605be471f2ae2506b19111844d9090e861f414e4c3. Delta +91,488 vs merged dev ceiling 22,204,304; D243 FAIL. Delta -280,672 vs original frozen base 22,576,464 includes unrelated owner/dev reductions, and is not claimed as this lane meeting no-growth. Native optimized namespace fixture is linking; VM latency remains unmeasured at this head. Next size candidate keeps the cancellation-safe Storage task and tests intact while sharing the caller poll state, without future boxing (previously rejected). Still outstanding: exact first-byte path witness, real add-host completion, 31+ LAN/WAN cycles and scale costs, both shipping size targets, final fast gate.fixture-port-red.log
client-fixture-green.log
client-fixture-clippy.log
Rejected unboxed shared-poll Storage candidate: x86 shipping 22,326,768 bytes versus 22,295,792 at
554b2105(+30,976). API cancellation tests (22 pass/1 ignored) and clippy passed, but D243 fails more badly. Source restored to the unchanged cancellation-safe helper; no candidate commit, no VM speed claim. Return-status publication fix remains in progress separately.Release transport diagnostics on
554b2105, LAN n=31: cut-to-echo p50/p99 26.314/32.319 ms; cut-to-status 34.871/41.281 ms. Recorded WG-handshake-to-output p50/p99 16.508/998.597 ms: target FAIL. Outliers retained: index 6=998.597ms, index14=127.309ms, index16=51.297ms. All 31 exit0 and phase guards pass; still enrolled-host proxy and current-route output observer, not exact per-packet/add-host acceptance. Investigating setup/refusal/retry timing; WAN collection continues without replacement/discard. Headd99a22b0includes the separately gated status-publication fix, which is not in these frozen554b2105artifacts. Final fast gate running.Correction to LAN comment: all 31 phase guards passed, but 28/31 lifecycles completed. Indices 13,15,17 exit101 and lifecycle_complete=false: mesh disable returns Storage at remote_mesh_namespace_tests.rs:1115 after switch/fallback/restart integrity. Earlier "all 31 exit0" wording was premature and incorrect. No failed sample was discarded or replaced. Partial receipts lack final aggregate expected_bytes/received_bytes; do not count them as complete lifecycle receipts. Raw logs and exit records remain in r5/after/0 on the perf VM. Existing cancellation-safe Storage fix is retained, but these failures need investigation. LAN switch p99 target also fails (998.597ms, index6); fallback p99 32.319ms passes.
Head
d99a22b0checkpoint, NOT DONE. Every commit pushed only mesh-autoswitch; no deploy. Final scripts/verify fast exits1: fmt, VT fmt, compile-out, clippy, workspace tests, cargo-deny PASS; static size FAIL. TS rows not reached after Rust failure; separate full TS gate passed at merge50a4d1d4. Full verbatim gate attached.Frozen after artifacts are554b2105 (
9269ade5docs only), optimized fixture plus shipping static binary.d99a22b0publishes returned route before waiting another proof RTT; regression red/green, 97 client tests pass/15 ignored, clippy/fmt pass; not remeasured. Perf VM4vCPU/7GB, per-lifecycle lock/load recorded, 2 live kernel nodes/1PTY/1000 public inventory rows, not1000live connections. n31 each venue, p99=max; all failures retained.Fallback echo targets pass these diagnostics; switch p99 fails. LAN outliers indices6=998.597ms,14=127.309ms,16=51.297ms. Next stage diagnosis should test whether candidate proof expires before WG handshake, then the one-second retry delays mesh data; this is an inference, not established cause.
28/31 complete LAN lifecycles matched54201 expected/received bytes;31/31 WAN matched395140. Original connection/attachment/writer preserved; no close/repaint after readiness. LAN indices13/15/17 passed switch/fallback/restart then mesh-disable failed Storage at remote_mesh_namespace_tests.rs:1115 (exit101), partial receipts. Prior all31exit0 claim was corrected; no sample discarded. Cancellation-safe Storage fix retained; 22API tests pass. Shared unboxed-poll size candidate rejected (+30976B), source restored. Storage cause still unestablished; inspect remaining cancellation/commit/rollback boundaries with a deterministic regression.
Final x86 shipping22295984 vs merged-dev cap22204304: +91680B. ARM19728304 vs19638744: +89560B. D243 FAIL both. Original x86base22576464 comparison=-280480 includes unrelated owner/dev reductions, not matched-dev no-growth. Budgets unchanged.
Remaining: switch tail; Storage regression/fix; both binary reductions; exact per-packet/terminal-byte witness (observer currently resolves physical route at callback); real SSH add-host completion boundary (all receipts false); at-scale copy/probe costs; final-head31+cycles/fast gate, author checklist and break-it handoff only after acceptance. Existing engines/auth/parser unchanged; additive negotiated copies use owned bounded history/clocks and retire on off. No completion/review-ready claim.
Gate summary verbatim:
Static output verbatim:
after-0.json
after-150.json
after-raw.tar.gz
verify-fast.log
return-status-red.log
return-status-green.log
return-status-clippy.log
diagnostic-provenance.json
Continuation r6 start: branch mesh-autoswitch, head/base
d99a22b0; original base01dcbee940, merged dev804ac0e8. Read all 40 comments; no linked parent in issue body/history. Clean worktree. Remaining: switch p99, mesh-disable Storage regression, D243 reductions, precise path witness and real add-host acceptance (31+ LAN/WAN), final fast gate. No deploy; push mesh-autoswitch only. Investigating remaining Storage boundary first; existing fallback diagnostics retained, not substituted for acceptance.Finding: a deterministic cancelled pool checkout reproduces mesh-disable Storage independently of BEGIN/COMMIT: persist_setting INSERT reports SQLite code 1, no such table: plugin_settings. SQLx check_idle_conn holds a Floating connection while ping/before_acquire awaits; cancellation discards it. The namespace proof uses one in-memory connection, so replacement loses its schema. New regression namespace_store_survives_cancelled_pool_checkout fails before and passes with production-style file/WAL storage; original begin_immediate helper unchanged. Temporary SQL-stage diagnostics removed from source.
35 debug-profile namespace diagnostics on the perf VM retained all failures; none reproduced a natural Storage error. Several completed lifecycles failed the older local-echo timing assertion. This establishes a fixture fault, NOT yet the cause of all three historical failures and NOT release timing/add-host acceptance. Next file-backed lifecycle run must check the remaining failure path.
red green VM diagnostic exit ledger
Completed/pushed
6ee2cf80: pending candidate/return proof now owns fresh 15ms ack-eliciting probes instead of waiting after one initial packet. No application replay, additional connection or detached worker. Deterministic real-mTLS candidate regression fails before (only initial probe) and passes after; 98 functional client tests pass, 15 measurements ignored; fmt/all-target clippy pass. The listener/handshake-loss explanation for historical switch outliers remains a hypothesis until release VM remeasurement, not a target-pass claim.red
green clippy
Completed/pushed
d4e9251d: namespace proof uses a fresh file/WAL database; deterministic cancelled-checkout→mesh-disable regression passes at head. Production begin_immediate helper unchanged. Server all-target clippy and fmt passed. Earlier red/green attached; natural historical Storage attribution still unconfirmed.Completed/pushed
6b60fef1: re-read current authenticated proposal after the await when a mesh interruption and component-down edit race. Latest removal now owns MeshDisabled reason/retry/retirement, rather than stale MeshInterrupted. Existing component-down regression failed during the shared-dispatch candidate test; 98 functional client tests now pass, 15 measurements ignored; clippy/fmt pass. No latency claim.Broad opt-in server test run was invalidated by overlapping rebuild of its self-executed test executable: 416 pass/9 fail/48 ignored, eight child execs missing the replaced executable and one derived fail-closed assertion. These are not a load waiver; serial final gate will rerun fresh.
Read-only size analysis identified duplicated remote daemon generic instantiations (raw TcpTls vs QUIC PrefixedControl). In-progress candidate reuses one existing ControlChannel enum borrowing each wrapper, keeps TCP raw, removes unreachable TCP candidate-proof branch. No allocation/codec added to TCP. 98 functional client tests and clippy pass on candidate; shipping rebuild pending, no size claim. Real bootstrap acceptance composition is now being implemented independently in server test files.
Completed size steps, pushed only mesh-autoswitch:
46f0c541shares one borrowed control adapter across QUIC/TCP, avoiding duplicated daemon monomorphization; x86 static 22,249,456 B (-46,528 B vsd99a22b0).b13bc940constructs/pins the authenticated session once and borrows it in both path drivers; 22,249,136 B (-320 B). 0f10156b removes unreachable parked-client greeting/activation state (server compatibility remains); 22,248,432 B (-704 B). Still +44,128 B over dev ceiling 22,204,304 B; D243 not satisfied. fmt, touched-crate clippy and remote-client suites pass (98/99 passed, 15/16 ignored as fixture added). No shipping performance claim from build host. Real SSH add-host fixture now passes one functional lifecycle per LAN/150 ms venue using immutable shipping payload46f0c541; release 31-sample acceptance pending. The target fixture image lacked nft despite production mesh requiring a filter frontend: the runner stages the existing public frontend and library closure into each owned target; this is a fixture prerequisite, not a change to shipping code. Failed prerequisite runs retained. Current size experiment removes redundant endpoint rebind while retaining fresh authenticated probes; not committed until tests and size evidence.Finding: optimized 1000-transport cost fixture failed on infraternal-perf-test under /var/lock/perf-bench.lock. It retains 1000 real mutual-TLS QUIC pairs (not 1000 daemons/kernel nodes). No-copy profile: 1000 false path interruptions; copy-enabled: 994. All 1000 connections remained open. Thus these windows are NOT accepted idle-cost or detector evidence. Investigating fixture route/ACK setup before attributing to production scale. Fixed-order correlated 31x250ms windows per profile and physical adapter send counters retained. Load 0.43 0.46 0.25 1/229 408794. Receipt: copy-cost-1000-failed.json.
Record correction: the last commit body was amended solely to restore actual newlines and the required final Refs line; 0f10156b is now
73db7cf9, same code. Pushed with lease to this branch only. Current code gate receipts: 99 library tests pass, clippy pass. Real SSH functional debug receipts (not performance acceptance): LAN, 150 ms. These explicitly label route hints as diagnostics and the post-fence fresh marker as causal path proof; neither claims earliest unrestricted byte. Added a separate fenced-canary profile to measure first authenticated Output from a fresh readiness PTY with enrolled ingress fenced before creation, including local firewall overhead. Release runs pending.Completed and pushed
4d3b241a: fresh encrypted probes announce physical route changes on the retained socket; redundant endpoint rebind/CID transition removed. Full agent library suite 99 passed, 16 ignored; fmt and touched-crate clippy pass. x86 static 22,245,200 B, -3,232 B vs73db7cf9, still +40,896 B vs dev ceiling.d9e364feadds the 1000-transport physical-copy counter fixture with production path proof before watchdog admission; numerical re-run pending.28e3605cupdates the author transport review with connection/copy/cancellation/path-witness invariants and outstanding acceptance. Test-only real-addhost fixture now sends a fresh fallback marker immediately at cut, measures complete byte delivery separately from status, and labels baseline reconnect observations separately from strict preservation. No final gate/acceptance claim. Correction to the73db7cf9commit alternative text: the historical rejected boxed candidate was shared SQLite BEGIN IMMEDIATE, not the path driver. Current separate experiment erases only the bounded path-proof future; no idle allocation, measured size/VM acceptance required before keeping it.Rejected two path-future erasure experiments; remote_server_path.rs restored to
4d3b241a(no new heap allocation retained). Bounded proof erasure: x86 22,244,720 B vs 22,245,200 B (-480 B), insufficient justification for extra allocation; 99 tests/16 ignored and clippy passed. Connection setup erasure: 22,263,312 B (+18,112 B), D243 worse despite 99 tests/16 ignored and clippy passing. Current lower-heap-risk candidate factors only the existing SQLite task construction into one ordinary non-generic inline(never) function. Public async begin API, first-poll spawn timing, cancellation ownership and rollback stay unchanged; no extra task/future allocation. Correct socket-host feature gates: 22 API tests passed, one VM writer benchmark ignored; clippy passed. Size and VM write cost remain pending. Real add-host receipt compilation/clippy passed after splitting the enlarged JSON macro; functional/31-sample release acceptance still pending.Rejected noinline SQLite task-constructor candidate and restored the original cancellation-safe helper. x86 static 22,278,768 B versus probe-only 22,245,200 B (+33,568 B), so D243 fails more badly. No extra allocation was introduced, and socket-host feature gates passed (22 tests, 1 VM-only benchmark ignored; clippy pass), but size is decisive. Size receipt, tests, clippy. Earlier boxed-connect receipt: 22,263,312 B.
Next concrete reuse: leased TCP listeners currently rebuild the primary listener TLS configuration from startup identity/root clones. Prepared an immutable shared-acceptor patch, not applied. Its new real leased TCP test failed before that refactor because the fixture reused respond(), which publishes Hello/open but no session census; RemoteHosts.online() specifically requires an authenticated census. Corrected only the test to wait for actor registration, perform the real request, then publish and await the protocol census. Baseline re-run pending. Current shipping source remains
28e3605cwith no erasure/Storage candidate retained. Final acceptance and fast gate still outstanding.Completed and pushed
f6def3de(real leased TCP/mTLS Hello/open/uid, lease TCP/UDP retirement, primary still usable, current-ledger revocation) and0e2ce9ab(reuse the primary startup Arc in leased listeners). Same immutable startup identity/root epoch; no new engine/allocation/task. The context now shares ticket/cache state across destinations within that epoch; every accepted connection still checks durable authority. Before refactor the corrected test passed; after refactor all 44 runtime tests passed, 5 opt-in diagnostics ignored. fmt/clippy pass. Initial fixture failure, corrected before, after runtime suite, clippy. Shipping x86 size linking; both D243 ceilings, coherent release VM samples and final fast gate remain outstanding.0e2ce9abx86 shipping 22,243,984 B, -1,216 B versus probe-only; still +39,680 B over dev ceiling. SHA256 ffddb7af30c2d5ce2fcee33779c98ced7083dfe2bd2ebdaf4bfc0b2a6ad53ac9; immutable VM payload /root/infraternal-bench/mesh-autoswitch/r6/0e2ce9ab/infraternal. D243 still fails; no final acceptance claim.Instrumentation finding: current mesh proof callbacks count QUIC TLS/ledger/control only; TCP admission and mux Output have no corresponding hooks. Unchanged callback counts cannot alone exclude a hidden TCP replacement. Extending the cfg-test observer and a real TLS regression before using strict transport-preservation receipts. Existing fenced mesh-byte causality remains separately valid because both enrolled TCP and UDP ingress were fenced; debug functional receipts remain non-timing, not acceptance. A node-before payload paired with the current fixture/server is a client-only comparison, not unchanged-dev full-stack baseline; reviewing a full baseline overlay and will label scope explicitly. Release fixture compilation and both transport witness gates pending.
Perf VM optimized 1,000-transport rerun, current head
0e2ce9abwith fixed bounded authenticated setup: FAIL. All 1,000 QUIC connections remain open, but healthy-path watchdogs exit. No-copy: six proof failures, 754 ramp and nine warmup interruptions, 772 total. Copy-enabled: six proof failures, 833 ramp and 31 warmup interruptions, 991 total. Load before profiles 0.80/1.08/0.89 and 1.02/1.11/0.90; first copy window RSS 1,051,000,832 bytes. Invalid steady cohorts: these windows do not establish valid 1,000-node CPU/traffic cost. No margin changes or failed-watcher restarts. Investigating actual adapter/watchdog behavior before accepting the measurements.Raw receipt
Full-stack before fixture prepared on detached
01dcbee940, now building locally. It uses the same real SSH/bootstrap acceptance harness and test-only QUIC/TCP observations. Five modified production Rust files compare token-identical after excluding cfg(test)/measurement-only test-support blocks; shipping transport/listener/parser statements remain frozen. This is source provenance, not a binary-equivalence claim. A current server plus an old node is not an unchanged full-stack baseline and will not be labeled as one.baseline-source-purity.json
baseline-overlay-provenance.json
baseline-overlay.patch
Rejected inline(never)+boxed ordinary connect factory after its test: 22,261,776 B x86_64, +17,792 B versus
0e2ce9ab, still over D243. 99 infraternal-agent tests passed (16ignored), touched-crate clippy passed. Restored production source; no allocation retained.Size receipt
Closed a test-observation gap: TCP TLS/admission/control and nonempty authenticated Output are now recorded at the existing parser boundary under cfg(test, mesh-transport-proof). No shipping parser or TLS behavior changes. Real leased + primary TCP regression first failed with hooks removed at missing authenticated TCP Output, then passed with hooks restored; final-source regression and bounded Capture test each 1passed/0failed. Initial TLS may include an unused losing dial, so preserved acceptance requires one initial admission/control activation and unchanged TLS/admission/control counts through cut/fallback, with same terminal/writer byte assertions; it does not call every successful unused TLS handshake a replacement. Integration crate gates pending.
tcp-observer-red.log
tcp-observer-green-final.log
tcp-observer-capture.log
Committed/pushed
7d9f7cb3: latch the first successful physical send of a synchronously armed probe. A real mTLS regression proved the former delayed waiter could adopt a later ACK-only send after the response. Four regression cases cover first polling, backpressure, waiter cancellation, replacement generations and path retirement. Borrowed guards add no allocations; RTT/probe margins unchanged. Scoped plugin suite 40passed/0failed/2ignored; all-target clippy, formatting and diff checks passed. Red/full-green verification file explicitly records captured tool outcomes, not a reconstructed raw log. The 1,000-transport rerun is still needed; this regression does not prove every prior watchdog failure had this cause.probe-first-send-verification.json
probe-first-send-clippy.log
Iteration gate first stopped at latency_proxy.py self-test on the shared build host (receipt retained; invalid venue for timing evidence). D256/job rules require timing on the perf VM. The identical script SHA256119b87ef39ee7022c76dbc641a1428dd5eae7254265e6ddeea6697b56a4a8339 passed self-test --samples3 on infraternal-perf-test under flock -w3600 /var/lock/perf-bench.lock.
No threshold, script or gate row was changed/skipped. A temporary process-local PATH dispatcher matches only this checkout’s exact latency_proxy.py self-test --samples3 invocation and executes it there; every other Python invocation goes to the original Python executable. Dispatcher SHA256b573a9b4c33ce8b221510a45195dc5cd5caa72abdee37aa48b1e71a6cd08cbe8. New scripts/verify iterate run: workspace clippy passed; tests now building. This is a venue correction under the explicit job rule, not a #118 waiver or a completed final gate.
iterate-gate.log
iterate-proxy-perf-vm.log
gate-timing-dispatch.txt
Committed/pushed
f5c050e1: opt-in TCP observation closes the invisible replacement gap. Final server feature clippy passed; runtime suite 45passed/0failed/5ignored. The earlier red/green real TCP and recorder receipts are linked above.Acceptance premise correction: AuthorityGuard::admit runs before Hello, so an unused losing TLS dial can also complete its authority lookup without activating a host transport. Preserved acceptance therefore requires exactly one initial ControlReady activation, nonzero observed TLS/authority checks, and unchanged TLS/authority/control counts through fallback, together with the same terminal/writer byte proof. Raw counts remain in receipts. My prior comment requiring one initial authority callback was too strict; shipping behavior is unchanged.
observer-integration-clippy-final.log
observer-integration-runtime-tests.log
Continuation at
f5c050e195. Iteration gate: fmt, VT native fmt, CLI plugin compile-out, clippy and cargo-deny PASS; test FAIL (101): initial authenticated-channel recv timed out at remote_server_path_tests.rs:309, before proposing mesh. Isolated exact test passes; investigating concurrent-test interference, not treating it as a #118 exemption.Current latched-probe candidate fenced real add-host smoke: LAN handshake-to-first authenticated Output 124.995 ms (over 50 ms); fallback complete echo 28.961 ms, same control channel and exact bytes. 150 ms venue failed readiness timeout (11713.375 ms), retained receipt. No acceptance claim. Baseline unrestricted 31-sample series stopped at LAN sample index 11, readiness failure after 1.524 ms; indices 0–10 passed with exact post-join bytes. Retained failure; new diagnostic receipts will record typed failure before rollback.
Candidate binary sizes: x86_64 22,245,424 B, +41,120 B vs merged-dev ceiling 22,204,304; ARM 19,682,160 B, +43,416 B vs ceiling 19,638,744. D243 remains failing. Allocation-free poll-helper experiment next; no compiler or dependency change.
Raw retained add-host failure series and isolated gate reproduction: acceptance-failed-series.tar.gz
initial-admission-repro.log
Gate contention comparison, unchanged exact test binary and actual Cargo crate working directory: parallel run 98 passed / 1 failed / 16 ignored (a different saturation deadline); serial run 99 passed / 0 failed / 16 ignored. Both initial-admission cases pass in the corrected parallel rerun and serial run. This supports fixture contention but does not prove the initial timeout cause. Next unchanged gate uses RUST_TEST_THREADS=1: every test, deadline and assertion retained; no #118 exemption claimed. Earlier gate failure remains recorded.
initial-admission-crate-cwd-serial-repro.log
iterate-perf-venue-gate.log
Fresh optimized post-latch 1000-transport loopback measurement FAIL on infraternal-perf-test: no-copy: 1000 connections remain open, 3 startup proof failures, 600 ramp guard exits, 0 warmup exits, 397 guards remain for both first/last window. Copy profile: 9 proof failures, 627 ramp and 364 warmup exits, 0 guards remain in every measurement window. Load before profiles 1.12/2.22/4.53 and 1.33/2.23/4.51. These invalid cohorts are not healthy 1000-transport CPU/traffic cost evidence. Cumulative authenticated ACK counters cannot prove responsiveness during a guard lifetime.
Diagnostic-only debug real add-host at150ms: readiness failed, zero Output callbacks; exactly1TLS/admission/control callback, no replacement observed. Post-canary before rollback: selected WGpeer sent21768/received14464 bytes and mesh.status server_link Mesh/MeshProven, both bounded publicqueries answered. Does not establish mesh was proven before terminal timeout. Internal open/attach stages next; debug/diagnostic times excluded from acceptance.
Allocation-free ordinary poll_connect size experiment rejected: x86 binary22264720B, +60416 vs merged-dev ceiling. Touched tests13pass0fail2ignored, clippy+fmtPASS, but size grew. Source restored toHEAD; no shipping change committed.
copy-cost-latched.json
current-diagnostic-debug-failure.txt
Source finding (not yet a causal perf result): the first successful PathSocket send can be an ACK-only Transmit buffered before the watcher arms. Quinn ConnectionDriver holds its connection-state mutex across socket.try_send; send_datagram needs the same mutex. An older ACK-only send can therefore precede fresh probe queue admission. For a first-send lead d, the healthy guard needs d + actual RTT + ACK delay + scheduling <= estimated RTT +30ms. At equal RTT and25ms ACKdelay only5ms remains; the15ms probe interval does not bound flush delay. Existing Connection APIs expose queued/constructed frames, not ack-eliciting physical-send completion. Calling connection.stats from the socket would deadlock on that mutex. Building a deterministic real-TLS writable-gate regression before changing this behavior; margins unchanged.
9c8cd75dpushed to mesh-autoswitch: formatting-only normalization of reused SSH bootstrap fixture, separated from instrumentation. No shipping code change. fmt PASS; server mesh-transport-proof all-target clippy PASS; 45 remote-runtime tests,11 bootstrap tests,7 canary tests PASS.acceptance-stage-clippy-final.log
acceptance-helper-runtime-tests.log
acceptance-bootstrap-tests.log
canary-stage-compacted-tests.log
5c86c9ae: reused SSH transport fixture and optional before/after canary hooks;11 bootstrap tests PASS,fmt/clippy PASS.acceptance-stage-clippy-final.log
acceptance-helper-runtime-tests.log
acceptance-bootstrap-tests.log
canary-stage-compacted-tests.log
057c6b20: reused strict terminal Echo decoder and namespace helpers; bounded baseline failure categories preserve failed writers in receipts.45 runtime tests PASS including cancelled SQLite pool checkout,fmt/clippy PASS.acceptance-stage-clippy-final.log
acceptance-helper-runtime-tests.log
acceptance-bootstrap-tests.log
canary-stage-compacted-tests.log
c6f40dee: real SSH acceptance module with fenced first-canary Output and fresh-marker path witnesses, unchanged admission/control counts, exact terminal bytes, typed failure receipts and opt-in internal canary stages. Unit fields1PASS/canary7PASS,fmt/clippy PASS. No release acceptance claim; pending WAN readiness and1000-guard failures remain.acceptance-stage-clippy-final.log
acceptance-helper-runtime-tests.log
acceptance-bootstrap-tests.log
canary-stage-compacted-tests.log
ad350679: immutable pinned-VM runner, fresh owned lifecycle per sample, lock released between samples, typed baseline operation failures retained, strict public host pin and in-memory SSH key. AST/--help PASS. Current diagnostic/31-baseline runs wait for shared lock.acceptance-stage-clippy-final.log
acceptance-helper-runtime-tests.log
acceptance-bootstrap-tests.log
canary-stage-compacted-tests.log
Measurement temporarily blocked on the mandated shared perf-VM lock. Snapshot2026-10-14 07:50UTC: lslocks reports holderPID597982 (flock WRITE) and19waiting flock writers (WRITE*). Own current WAN internal-canary diagnostic, deterministic old-ACK regression and31-sample baseline series are waiting. No otherjob process stopped or lock changed; waiting withflock -w3600. Current holder must finish/release before required release acceptance can run. Independent fmt/clippy/tests, atomic harness commits and static size work continue.
Experimental host orchestration optimization x86 static22112080B,92224B below merged-dev ceiling22204304; SHA256556aa11207a6eb48e39108d65816554088daa560768267a37d53caff1f7f922c. Not yetaccepted/committed: ARM size and performance comparison outstanding.
Continuation on mesh-autoswitch at
ad350679(merged-dev base804ac0e8). Experimental host-orchestration opt-level=s: x86_64 22,112,080B (-92,224 vs ceiling22,204,304), ARM19,568,976B (-69,768 vs ceiling19,638,744); ARM SHA2560afd033b7fcc13eb8f7f5ca54c8ca45c43f9c1f973a28639fa7a113e25c71133. No profile commit/acceptance: performance comparison pending.Both internal-stage diagnostic and baseline31-b runner exited at flock -w3600 before entering the first sample; these are lock timeouts, not terminal-readiness failures. Retained logs; fresh labels internal-stage-debug-b and baseline31-c queued with the same one-hour wait and per-sample lock release. No other process/lock altered. Prior07:50 timestamp was approximate; authoritative lock snapshot is07:55:08UTC in the attached raw receipt.
internal-stage-lock-timeout.log
baseline-series-lock-timeout.log
perf-lock-snapshot.log
Resume on mesh-autoswitch at
ad350679; merged-dev base804ac0e854. Read all 71 comments; no parent linked in body/history. Preserving unfinished agent opt-level=s experiment and deterministic old-ACK/backpressure regression for validation. Build-host disk has 68 GiB available. Remaining acceptance includes healthy 1000-transport watchdogs, switch tail/real add-host WAN readiness, release 31+ samples per venue, both D243 sizes with profile performance evidence, final fast gate. Existing Storage regressions retained. Measurements only on authorized perf VM; no deploy; push mesh-autoswitch only.Completed
ebeb0fd3on mesh-autoswitch: fresh probe expiry now waits until a new ack-eliciting DATAGRAM is constructed and all counted connection transmits have physically succeeded. Receive freshness keeps its original boundary; deadline uses the recorded flush clock, not delayed waiter time. Connection stats stay outside socket callbacks (driver lock); no new task/allocation or larger RTT margin. Deterministic real-mTLS emission regression fails on prior production code (exit101, ACK-only transmit completed probe waiter); final all-feature remote suite42passed/0failed/3ignored, clippy/fmt/diff checks PASS. VM deadline/1000-cohort acceptance still pending, no latency claim. The inherited timed red fixture hit an unrelated incoming-datagram assertion, so it is not used as the causal red evidence; removed that unnecessary assertion while retaining the blocked-probe/deadline proof.resume-old-ack-functional-red.log
resume-plugin-final-tests.log
Optimized
ebeb0fd3VM rerun: 1000 retained mTLS QUIC pairs, all1000 remain open in both profiles; watchdog/cost acceptance FAIL. No-copy321false interruptions plus2initial proof failures (ramp271,warmup18;709→677guards in first/last window); copy986false interruptions plus9proof failures (ramp950,warmup0;40→5guards). Loads before0.05/0.60/0.92 and0.64/0.71/0.95. Invalid cohorts are not accepted idle CPU/traffic/memory evidence. The old-ACK fix addresses the deterministic regression, not every remaining failure. Adding test-support-only typed failure clocks/counters and per-owned-socket UDP drops, observed outside timed windows; no deadline/buffer change or failed guard restart.VM held-probe deadline regression PASS (1test); final release infraternal-agent functional suite99passed/0failed/16ignored. Scope of separate compiler-profile comparison:50simultaneous real PTYs,31finite serial keystrokes per session (1550samples),151owned processes, local fd protocol. opt3→optS: echo p50/p99 0.750/1.707→0.845/2.270ms; Ack0.632/1.366→0.696/1.912ms. Owned-process CPU0.83→1.00s; daemon RSS8352→6060KiB; aggregate RSS675428→666548KiB (sum includes shared pages repeatedly). Separate low-load windows (before0.786/0.737/0.953,after0.593/0.698/0.942), not a speed-up or network/bulk claim. The small latency/CPU cost stays inside D41’s5ms budget and trades for no-growth static images while transport/parser/crypto/VT crates retain opt3; profile commit and fresh head sizes pending.
resume-copy-cost-1000.json
resume-pty-before.json
resume-pty-after.json
f842e766optimized diagnostic: no-copy cohort PASS with1000live guards,0proof/guard failures and0owned UDP drops. Copy profile retains1000connections but984guards fail; server-owned sockets drop7064packets during ramp and10453by warmup/end;1000client sockets drop0.983failure snapshots lack a mesh reply after emission; authenticated enrolled progress is recent. This establishes real fixture-owned receive loss, not only an early clock. Investigating source-change replay: receive_peer takes the64-packet history then try_send re-inserts/re-arms it, permitting repeated old-ciphertext bursts when fresh path hints alternate. Proposed internal bound before changing code: retain original history; each retained ciphertext can be replayed at most once per physical path, with owned copy deadlines and no replay re-insertion. Fresh QUIC transmissions remain eligible; default protocol/engines/authority unchanged. A regression must reproduce the repeated replay first, then remeasure before accepting this choice.Real add-host release smoke: LAN handshake→fenced first Output89.104ms (FAIL50ms), cut→complete echo41.901ms,1TLS/1control unchanged, exact bytes. WAN readiness fails with0Output and1TLS/admission/control. Internal diagnostic isolates WAN open_readiness wait16.912s, attach0.287ms, then Closed after151ms; WG latest handshake before canary return, status Mesh/MeshProven. LAN diagnostic open_readiness76.776ms dominates; input→first actor data1.109ms. These are diagnostic single lifecycles, not acceptance percentiles. Existing before/smoke/failed-stage receipts retained. Launcher a used an obsolete runner copy and rejected the mode flag before any sample; b uses the current repository runner.
Completed steps:
21abc601bounds agent orchestration code size (core transport libraries retain opt-level 3);f842e766adds opt-in probe failure clocks and owned UDP drop receipts;a01dda60bounds ciphertext replay to once per original packet per physical path. The replay regression failed before the fix (40 sends vs 24), then passed including fresh-packet eligibility; remote 43 functional tests and agent 100 functional tests passed, touched-crate clippy/fmt passed.The latest perf-VM 1,000-connection run remains unacceptable: no-copy 680 guard exits/0 proof failures/1,780 server drops; copy 892 guard exits/5 proof failures/13,062 cumulative server drops. This bounds a proven amplifier, but does not establish healthy switching at scale. Receipt: https://git.kayg.org/attachments/8a14a0ba-c9cd-4cfe-94f3-4ed69b344ac9
Full gate at
a01dda60failed: agent_sessions restart_discovery timed out waiting for shim readiness; static sizes shrank and require ratchets. Measured x86_64 22,114,896 B (-89,408 versus ceiling); ARM64 19,572,048 B (-66,696). Production-trust/static-link/key-memory x86 checks passed. Full output: https://git.kayg.org/attachments/5c25c328-6198-44e7-830b-3212cd32a208Additional gate regression reproduced with a scratch copy containing precisely the measured downward ceilings: test-static-size.py still parses "max" although the gate prints "cap:", raising TypeError. Failing receipt: https://git.kayg.org/attachments/9886ff60-2d03-4ddd-a02b-1a1183009ff3 . Fixing that parser and ratcheting measured ceilings next; then investigating shim readiness and remaining network loss. Real add-host 31-sample acceptance is still outstanding; this is not ready for break-it review.
Completed/pushed
0033dd72: static boundary test parses the current cap field. The pre-fix scratch downward-ratchet run failed with TypeError; afterward x86 exact 22,114,896 B and ARM exact 19,572,048 B pass, +1 B fails, -1 B requires ratchet, unsupported target fails.Completed/pushed
3e9ac334: ratcheted both production ceilings and PROCESS table to those measureda01dda60artifacts. No growth allowance introduced. Python syntax, shell syntax, diff check and both shipping boundary tests pass.The full-gate shim readiness failure remains unwaived. A direct rerun of the same all-feature executable with its original state root passed all three agent_sessions tests; an experimental longer fixture root separately failed the existing path-length guard, and was not retained. A fresh full-head gate is still required. Switch/network acceptance remains outstanding.
Finding: remote_server_path.rs waits only for retry backoff after a refused/interrupted mesh attempt, although the retained PathSocket can receive a late mesh reply while enrolled. A new functional regression with a latched physical mesh hint failed against that timer-only behavior: "a latched mesh reply remained hidden behind the retry timer". Receipt: https://git.kayg.org/attachments/1e46524a-3852-4ee1-a3e8-8cab4d9e22bd
Prepared fix registers the mesh notification before checking the receive clock, remembers a per-fallback boundary, and wakes a new ordinary authenticated proof on a fresh hint. The hint never selects the path or admits authority. Regression covers arrival before polling, stale-history rejection, arrival after registration and unchanged enrolled selection. Agent release functional suite 101 passed/16 ignored; remote 43 passed/3 ignored and clippy passed. Agent clippy and shipping-size remeasurement are pending. This identifies a real timer-only delay; attributing the historical switch tail requires the next VM run.
Separate scale investigation: Bindings retains and replays enrolled-path ciphertext when first selecting the mesh, unlike PathSocket which retains only mesh sends for fast fallback. Those already-sent enrolled packets add replay bursts on upgrade. Proposed internal simplification: retain only mesh ciphertext, consume its bounded queue once on enrolled return, and keep ordinary QUIC stream recovery on upgrade. This would remove the new per-packet replay flags and replay-to-mesh machinery while preserving fast fallback. Functional red/green byte/replay tests and VM losses/latencies must verify the change before acceptance. No parser/auth/engine or wire-contract change is proposed.
Completed/pushed
95464742: fresh latched mesh replies wake authenticated candidate proof through retry backoff. Functional regression retains enrolled selection on a raw hint, rejects stale history and covers arrival before/after registration. Agent release 101 passed/0 failed/16 ignored; remote 43 passed/0 failed/3 ignored; touched-crate all-target clippy, cargo fmt and diff checks pass. Shipping size and VM remeasurement are pending, so this does not establish the switch-tail target yet.Next red: eight original mesh packets sent 24 times over repeated route hints, versus 16 for originals plus their single enrolled return. Evidence path: mesh-only-history-red.log (job scratch/evidence). The initial replay-to-mesh code/flags are being replaced by mesh-only retained history and one consumed enrolled replay; a separate cold-enrolled-history regression is running before that change. This must preserve bytes/writers on real TLS sessions and reduce actual VM drops, not only the send count.
Completed/pushed
27d9b081: retain only original mesh ciphertext; consume the queue once on enrolled return; reuse the normal try_send path and remove replay-to-mesh flags/clones. Both legacy and backup peers pass cold-history and repeated-hint regressions. Before: cold packet sent twice instead of once; eight mesh originals produced 24 sends instead of 16. Red receipts: https://git.kayg.org/attachments/e0fe435b-81e6-4a96-8f74-d8e1bb328698 and https://git.kayg.org/attachments/12b95478-c4c5-44cf-b66c-692566ad9e62 . Remote suite 43 passed/3 ignored: https://git.kayg.org/attachments/ef9f5ed4-34ac-4a3c-9020-b43f5c62ec07 .Agent suite 101 passed/16 ignored serially from Cargo package cwd: https://git.kayg.org/attachments/bbcde997-b3d7-49dc-b459-8e9cac642b01 . Both touched/dependent all-target clippy, fmt and diff checks pass. Earlier parallel run failed a TCP spacing timing assertion on the build host; that exact test passed on the perf VM (22,127 us). An initial direct serial rerun used workspace cwd, causing 12 relative fixture-path failures; corrected package cwd passes. Neither failed run is hidden or treated as acceptance.
Current x86 static image: 22,113,264 B, -1,632 vs preceding ratchet and -91,040 vs original dev ceiling. SHA256 f4ea37aed0bf3d5e899536a71162a207ed01272d2b393f83863ba6b9d6a03e9e. ARM build pending. Current release mesh-proof server fixture SHA256 bf49455d45b14be390e47091ed6f11a886ee5b25f1ce363971dc0b69cbba2a61 (only interpreter adapted for VM).
The latest 1,000-transport scale run is waiting for the shared perf lock; fixture SHA256 97de9050d812dd671ed5bb840de52938bc045f6c72b4eddde8d27763ecc98aab. Real SSH acceptance series and full-head fast gate are still outstanding. No review-ready/completion claim. Removed only inactive owned baseline build cache and remote debug incremental cache as build-host space fell; frozen binaries and evidence retained.
Perf acceptance waiting: current scale measurement uses flock -w 3600 as required. The shared lock holder is PID 935095 (flock), elapsed 25m29s at inspection; eight waiters were listed, including this job. I will not stop another job or change the lock. If the holder is stuck, the orchestrator needs to inspect/release its own measurement. Builds/gate work continues while waiting.
Iteration gate at
27d9b081stopped before Rust checks because deploy-review.test.ts alias rollback exceeded 5 seconds on the shared build host. An unchanged rerun passed all three tests; no timeout threshold or exception added. Full-head gate is still required.Head
55c51b2fpushed: exact shipping ceilings now x86 22,113,264 B and ARM 19,570,688 B; both boundary tests pass.Perf scale attempt did not run: flock -w 3600 expired with exit1 and a zero-byte test log. Preserved cost-exit.json under r7/mesh-only on the VM and local evidence/mesh-only-cost-lock-timeout.json. Requeued a fresh attempt under r7/mesh-only-b; no thresholds, socket buffers or resources changed. Acceptance still needs the shared lock. The orchestrator may need to inspect the owning measurement if it is stuck; this job does not kill foreign processes or change the lock.
Full scripts/verify fast is running at
55c51b2f, waiting on the unchanged timing-proxy gate on the perf VM. Verified VM/local latency_proxy.py SHA256 identical: 119b87ef39ee7022c76dbc641a1428dd5eae7254265e6ddeea6697b56a4a8339. Independent Rust section passed clippy; native fmt initially lacked Zig PATH, corrected pinned-toolchain check passes. Independent TypeScript section is still running. No full-gate/acceptance success claim.Perf resource blocker identified from public process metadata only: shared lock holder PID935095, cwd /root/infraternal-bench/ppu-soak/r3long; child PID935320 is timeout. Holder remained unchanged across inspections; at latest inspection its process elapsed1h16m37s. No argv, credentials or other-job source files read, no foreign processes stopped. Scale attempt-a exhausted flock -w3600 without running; attempt-b remains queued. Full fast timing-proxy check also waits for the same lock. An orchestrator-provided measurement slot/release of the owning soak is needed for #201 performance acceptance; this job cannot change that booking.
Independent Rust integration checks continue. Independent TS compiler, UI unit tests, production build and transfer budgets passed; protocol differential timed out because I ran it beside Rust, and its cargo run recontended for the build cache after prebuild. Rerun serially when Rust finishes; no timeout increase or waiver.
New gate finding at
55c51b2f: refreshed cargo-deny database rejects rustls0.23.45 for RUSTSEC-2026-0346/0347/0348/0349; each lists upgrade >=0.23.46. Upstream released0.23.46 today (14Oct09:51), with these security fixes: https://github.com/rustls/rustls/releases/tag/v%2F0.23.46 . Prepared scope: update the existing rustls patch version, preserving the selected engine, ring provider, protocol/features and parser boundaries. No new foundational library/provider is chosen; no advisory exceptions. Re-run TLS/auth suites, license/advisory checks, both sizes and acceptance for the patched artifact.Independent Rust static builds pass exact ceilings, static linkage and production-trust checks; native x86 key-memory check passes. Workspace tests stopped at SSH install linger denial test:47pass/1fail, SetupRunning at install.rs632 after rollback/re-stage; isolated rerun pending. Missing Zig PATH in the partial invocation was corrected with pinned native fmt. Protocol differential rerun after Cargo-lock contention passed37 tests/34463 assertions. Complete fast gate remains queued for its VM timing check, so not green.
Resume mesh-autoswitch at
d0690d75; merged-dev base804ac0e8. Read all84 comments; no parent linked. Existing source/tests retained.Security step
d0690d75: existing rustls engine patched0.23.45→0.23.46 for four new advisories; remote43pass/3ignored, fmt/all-target clippy pass; cargo-deny advisories/bans/licenses/sources all OK. Receipt: https://git.kayg.org/attachments/33a5791b-c1f1-41ab-8ebc-87e2da329dcd . Shipping x86 22114704B (+1440 above latest cap), ARM19571360B (+672). Both remain below original dev, but exact downward ratchets reject this growth: https://git.kayg.org/attachments/a5e7744b-f293-473b-af93-958bd82dde50 . No ceiling raised. A timeout-based retry simplification grew x86 by736B and was reverted.Pre-security algorithm27d9b081 measurement on authorized VM: no-copy1000/1000guards survive,0proof failures,0owned UDP drops. Copy-enabled980guard exits,0proof failures,20guards survive first/last windows,1000connections remain open; server-owned drops7652 after ramp/10296 after warmup,0client drops. Load before profiles0.93/1.54/1.01 and0.74/1.48/1.00 (see raw receipt for authoritative values). Invalid copy cohort is not accepted scale cost evidence. https://git.kayg.org/attachments/dfb9e4fc-8458-4a0e-8606-e9ce992de118
Full fast at55c51b2f exited1 after its VM timing gate exhausted the one-hour shared-lock wait; no final-head green gate. Unchanged isolated SSH linger test passes; serial protocol differential37pass/34463assertions. Remaining size cut, copy-scale failure, true real-addhost standing-session31/venue acceptance and full final fast gate; not ready for break-it review.
d0690d75push confirmed after bounded SSH connection retry; stale multiplex-master hello delayed prior pushes. No host key/credential files changed. Correction to preceding comment: copy-profile load before was0.79/1.49/1.00, not0.74/1.48/1.00; raw receipt is authoritative.Copy-path source finding: Bindings::poll_recv flushes every pending copy across all peers before it polls any receive socket. Each Recent::copies also flushes younger entries; there is no per-endpoint work bound (up to1000×64 retained datagrams in one pass). Preparing a deterministic non-timing regression for this receive starvation, followed by a bounded, fair drain through the existing copy clock. This is a hypothesis for the owned socket drops; it is not yet measured attribution. Copies retain the existing20ms schedule, same ciphertext and no new worker/socket. Performance/size must be remeasured before acceptance.
Completed88a0fb2f: the shared copy drain now limits each receive pass to32 socket-send calls (not32 GSO segments), retains pending ciphertext, and rotates its peer cursor. Client history uses the same bound; no new worker, socket, larger deadline or buffer. Deterministic pre-fix regression emitted100calls in one pass: https://git.kayg.org/attachments/c727c80a-7aed-4e7e-bf18-4dffc1c34f67 . Final remote44passed/0failed/3ignored, all-target clippy, fmt and diff checks pass. Regression also refills a busy first slot while proving every other peer drains. Fresh patched-TLS release fixture/VM cohort and shipping size are pending; no measured drop-reduction claim.
Rejected separate identical-send consolidation: x8622114704B unchanged from
d0690d75; saved0B, restored original source. It does not consume a commit or weaken the exact size gate. Size cut and real31/venue acceptance remain outstanding.88a0fb2fdependent release suite101passed/0failed/16ignored; branch push confirmed. Fresh rustls0.23.46 cost fixture SHA2560679bd18fdced43687c2ce32ced62b028964ad2ccba0de42ebc64f7803e7c8d2 staged on the named VM and queued with flock-w3600. Iterate is also queued for its VM timing proxy. First bounded-copy x86 measurement22114896B, still1632B above latest cap; final warning-free-source shipping rebuild required.Next size experiment: keep one concrete, pinned heap-owned dial future behind the existing connect entry point, rather than embedding its full state into each caller branch. It is used only for initial/replacement dials, never the established-connection mesh switch, probes, copy drain or terminal frame path. Existing endpoint/runtime cancellation and drop ownership must remain unchanged. Reject if tests, shipping size or measured dial costs regress beyond budgets. No provider, transport or new library choice; no ceiling raised. The pending iterate started before this experiment and will be labeled with its actual tested working state, not asserted as a frozen88a0fb2f final gate.
Fresh88a0fb2f/rustls0.23.46 perf-VM measurement FAIL. No-copy1000guards survive,0proof/guard failures,286server drops by end (0at ramp/warmup). Copy-enabled990guard exits (531ramp/459warmup),0proof failures,10guards remain first/last window;1000QUICconnections remain open. Server cumulative drops5571ramp/9300warmup+end, starting286;0client drops. Loads before0.55/1.27/1.22 and0.67/1.28/1.22. Bounded drain is functionally covered but does not solve scale interruptions; no accepted CPU/traffic cost or causal drop-reduction claim. https://git.kayg.org/attachments/2535839f-159d-4542-8cf9-253d2826494a
Pre-security27d9b081 real SSH unrestricted LAN smoke n1: bootstrapDone1666.859ms, kernel join31.230ms beforeDone, causal fresh-marker complete echo1.053ms afterfence; silent cut→complete echo29.742ms/status45.354ms. Exact bytes, same attachment, initial/finalTLS/admission/control all1. Terminal opens afterinitialjoin; unrestricted routehint is not first-packet provenance, so this does not establish handshake→first standing-terminal mesh byte or31-sample acceptance. https://git.kayg.org/attachments/caaca9b1-84c0-4e33-8c1c-51a673927516
WAN lifecycle remains queued between samples; owned150/0 directory does not yet exist (samplehasnotstarted), not a readinessfailure. No foreign process/lock change. Boxed dial experiment14path tests pass/2ignored, all-target agentclippy+fmt pass; shipping compile stillrunning.
27d9b081unrestricted realSSH WAN150ms n1 also PASS exact bytes/sameattachment/1initial+finalTLS/admission/control. Done9757.613ms; observed kernel latesthandshake10314.245ms afterDone; cut→completeecho180.809ms/status335.130ms; postfence freshmarker151.277ms. Kernel snapshot afterjoin is not an initial-handshake stage trace; no exact firststandingmeshbyte or31-sample claim. https://git.kayg.org/attachments/e0e24d21-5032-4a89-8c6d-ee0e2aa7b3aa . WAN success without earlycanaryfence supports investigating that fixture interference separately, not extending productioncanary expiry.Iteration run (started88, then compiled the uncommitted boxed-dial experiment) FAIL only Rusttest vt_credit approximate1; fmt/VTfmt/compileout/clippy/deny PASS. Not a frozenhead88 or finalfast gate. Preparing required detachedorigin/dev baseline: currentref1b23ef31, own scratch/baseline. Baseline correctdefault-feature vt_credit PASS1 (allfeatures first attempt refused missingper-run publicstagingtrust fixture; gate usesdefaultfeatures). No failurewaiver applies; fresh branch rerun is needed.
Boxed dial experiment rejected/reverted: x8622116464B andARM19574336B grow, although14path tests/2ignored andclippy pass. Next source simplification removes Link.proof asyncmutex: dialing exclusively owns creation, run exclusively owns transfer before shared borrowing. No secondconsumer exists. Existing interruptionclassification is folded into its existinghelper to keep run within lintlimit; tests/sizespending.
756db815andbfaeb3acpushed: transfer the exclusively owned dial proof without asyncmutex; keep interruptionclassification in its existinghelper; initial Available meshclassification reuses its completedproof instead of another physical-path lock. Both logicalsteps pass14path tests/2ignored, all-feature all-target clippy/fmt/diff. Final756 shippingx8622114512B (+1248 above currentcap); bfa shipping buildpending, no ceilingchange.ccff9257pushed, opt-in equivalent full128bit lookup benchmark only. On pinned VM,31batches×20000absent/presentpairs over1000peers: byte-array p50/p9920.258/29.783ms; u12821.909/29.754ms, load0.23/0.86/1.93. No integer representation change accepted; it did not improve median. Fixed arrays, worst-case lookup only, not crypto/socket throughput. https://git.kayg.org/attachments/52566b3e-17f1-4525-af99-2099041bd756 . Remote release44pass/4ignored, clippy/fmtPASS.vt_credit detacheddev1b23ef31 PASS1; branchcleanrerunPASS1; no waiver. A shared cache reused baseline plugin-api metadata missing this branch additions; cleaned only that owned package and rebuilt branch to recover. Baseline must use its own target for latercomparisons. No productsourcefix for this orchestrationerror. Fresh full finalfast stillrequired. Adding test-only ramp/warmup physicaltraffic snapshots: failedguards stoptraffic before steadywindows, so thosewindows cannot explain the earlierburst. Shippingbehavior unchanged.
898d47c5pushed: exact caps ratcheted to x8622112976B andARM19569568B (down288/1120 vs latestcaps; down91328/69176 vs originaldev). Static release hashes x8618efe83a003ea76f71b99a7b300d6aba5c465f482ac3eea08f0c533bfa576dc5; ARMada3d937fa47a2c7232a07d9ef9e3b9b14f528002529eee23b818361ab5cc332. Both shipping builds completed; exact/+1/shrink/unsupported boundarychecks pass: https://git.kayg.org/attachments/222c594b-9feb-458f-9aa9-7340c78ab308 . Final fullfast stillpending.d92b2bbapushed: ramptraffic receipts,101agent release tests/16ignored, clippy/fmtPASS. LatestVM diagnostic stillFAIL998copyguards. During834.609ms admission: client30491datagrams (15154enrolled), server42378(36312enrolled); ramp+1swarmup client31112(15556enrolled), server43569(37392enrolled). No-copy230.086ms admission,0guardfailures,3648client/2341serverdatagrams duringramp. Loads2.01/1.87/1.79 and1.99/1.88/1.79; not a healthycopycost result. https://git.kayg.org/attachments/964bfdcb-68a7-45c5-afd3-095d7c60293eAdding separately labeled controlledstanding profile: genuineSSH bootstrap and enrolled PTY open while only this fixtures outermesh is intentionallyheld; then exact node-kernel handshake observed over privatefixtureUnixsocket (avoids adding networkRTT), freshmarker after enrolledingressfence, same retainedEchoattachment throughswitchandcut. Hold andobserver/fenceoverhead remain explicit; no naturalendtoend claim for heldprofile. Natural unrestrictedprofile retained. Newobserverepoch testPASS; latestserverproof clippyPASS; releasefixture compiling; realpilotnotyetmeasured. No SLA/31sample acceptance claim.
Controlled standing pilot n1/venue, fresh shippingx86SHA18efe83a003ea76f71b99a7b300d6aba5c465f482ac3eea08f0c533bfa576dc5, rustls0.23.46. Both realSSH-installed standingPTys preserve exactbytes, sameattachment and1initial/finalTLS/admission/control. Switch deadlineFAIL: nodefirsthandshake→firstfresh authenticatedOutputLAN2403.618ms / WAN1503733.656ms (budgets50/200). Silentcut→completeechoLAN27.919ms/WAN193.523ms PASS for these single samples. IntentionalinitialWGhold explicitlyreported; no naturalendtoend claim, no31-sample percentile claim.
Artifacts attached separately. Hypothesis toverify: exponential mesh retry sleeps (1→2→4→8→30s) can outlive WGhandshake when no native mesh reply reaches the retained enrolled socket; the existing late-reply wake addresses native replies, not an empty kernel keepalive/handshake. The ingress fence also prevents enrolled proof replies during the causal marker window, so stage timestamps must separate retry/proof delay from fixture-induced loss before selecting a fix. No deadline increase or failedsample discard.
ee92acf1adds controlledprofile/counterpipe;094a8897 adds runner mode;ea23a44f adds bounded observerepoch reset. Serverproof all-targetclippy PASS, observer1PASS, releasebootstrap44PASS/5ignored, AST/helpPASS; pilots functionalPASS/SLAFAIL. Fullfast running on currentbranch (startedwith this same testsource before its atomiccommits); not greenyet. Copy-enabled1000cohort remainsFAIL998guards; both exactsizecapsratcheted. Work notcomplete/reviewready.Read-only review of head
094a8897found that Recent::copies ignores individual copy_at deadlines when the shared endpoint clock fires (remote_peer_routes.rs:71). A fresh enrolled copy may therefore precede its mesh original; receive_peer then retains enrolled routing when the mesh original is fingerprint-duplicate. The existing eviction test explicitly permits early copies, so a new deterministic regression is being run before changing the invariant. Review also found synchronous return replay can send 64 retained entries per peer before Quinn receives its batch, outside COPY_BATCH. This may contribute to the failed 998/1000 healthy-copy cohort; it is not yet a measured cause. Standing switch review identified a non-cancellable enrolled proof ahead of retry_path, and the witness ingress fence can amplify reliable-output recovery. No acceptance claim; 31-sample standing acceptance remains outstanding.Completed
f136ca06(pushed mesh-autoswitch): each retained ciphertext backup now respects its own deadline when the shared socket clock fires. The deterministic regression failed before the fix and passes after; eviction/fairness tests make retained entries due rather than permitting premature copies. Remote transport gate: 45 passed, 0 failed, 4 ignored; touched-crate all-target clippy and workspace fmt pass. No workers, buffers or deadlines added.Failing regression
Transport tests
Effect on the failed healthy-copy scale cohort is unmeasured; shipping size must be rechecked after this production change. Standing caller regression is still being made deterministic; no switch latency claim. Previous-head fast gate remains in Rust server tests; head acceptance and31-sample VM measurements are outstanding.
Completed
4b152e69(pushed mesh-autoswitch): after a mesh candidate miss, the existing late mesh hint can interrupt enrolled return proof and reach fresh candidate proof on the same connection. Biased polling selects enrolled before observing a pre-latched hint. The enrolled recovery guard is retained; raw hints do not authenticate peers or activate control.Caller regression red isolates the gate with real mTLS and paused-clock expiry; isolated retry sees the hint but the old caller remains enrolled. Corrected branch: path tests15 passed,0 failed,2 ignored; touched-crate all-target clippy and fmt pass.
Red
Green
Separate characterization0db669c4 confirms64 retained originals cause64 synchronous replay sends on enrolled-first ordering; a duplicate mesh packet conservatively does not reverse routing. All remote tests46 passed,4 ignored.
Characterization gate
Shipping x86 build and refreshed VM fixture are running. Effect on switch latency, size and healthy-copy scale remains unmeasured; no release acceptance claim.
Canceled the owned fast gate started at094a8897 after Rust tests and cargo-deny completed, while its static build waited on the refreshed fixture target lock. Commits
f136ca06and4b152e69 superseded its compiled test snapshot. No final-head gate claim; rerun the complete fast gate after measured fixes stabilize. Stopped only verified own gate PID2437484 and its waiting cargo PID2809603, no compiler child. Transcript retained as partial evidence.Head4b152e69 static x86=22,113,904B, +928B over the ratcheted22,112,976B ceiling (still90,400B below original-dev22,204,304B). This is a failing D243 check; no ceiling increase. SHA2562cd58790dcc0eb2fa795c582d642e6fe5fafb9f2c03a265c818a23259f03cc52. ARM pending. Refreshed release proof fixturebc7064d12ff59c27e6838cba392c1cbd27dcb4a64476a449055f2d09c9a94458:44 bootstrap tests passed,5 ignored. Transferred artifact hashes verified; controlled standing pilot queued under runner per-lifecycle perf lock.
Baseline audit: the existing before01dc-unrestricted-31-a/venue-0.json actually contains11 samples, despite its directory label; terminal_opened_after_initial_mesh_join=true. It is not31-sample standing-session acceptance. Preserve it as partial evidence, separate from the earlier complete31/venue transport baseline.
Partial real-SSH baseline
Head4b152e69 controlled standing pilot,1/venue: WGHS→fresh meshOutput2473.083ms LAN/3420.163ms at150ms RTT (FAIL50/200); cut→completeecho38.082/194.278ms (single-samplePASS), exactbytes,1 TLS/control throughout. Late-hint fix did not remove the principal delay.
LAN
150ms
Separate packet-header-only diagnostic: server sees queued mesh packets+0.220ms after nodeWGHS; no NEW node mesh send until+2199.757ms; server meshreply+2200.131ms; freshOutput+2200.573ms. Headers alone do not prove early packets were duplicates; source/ordering regression makes that mechanism plausible. This locates the large gap before fresh discovery traffic, not authenticated response processing. Source
f1878655adds bounded opt-in observers owned byPID, no payload/netlink capture, explicitly non-acceptance labels.Node headers
Server headers
Diagnostic receipt
ARM static19,570,224B, +656B over ratcheted ceiling19,569,568B; x86+928B. Both size checks fail. Investigating clock-driven retry limited to negotiated backup peers; legacy cooling and both-path-dead TCP recovery retained. Deterministic negotiated-clock regression red reproduced; green fixture timer-wheel dispatch being corrected.
Legacy controlled-standing test-only transplant prepared with typed post-bootstrap terminal failures; five production files remain non-test-token-identical to01dcbee9. Separate legacy build pending.31+sample acceptance/full-head fast gate still outstanding.
Completed60112905 (pushed mesh-autoswitch): negotiated backup peers retry refused mesh candidates on the existing15ms owned probe clock, starting its deadline before enrolled recovery. Reuse retry_path for both reply/timer wake; every candidate still requires fresh authenticated proof. Legacy peers keep enrolled recovery/exponential cooling; the proven-mesh both-paths-dead failure exit still reaches TCP.
Regression: negotiated-clock test fails before fix; clock/late-hint/legacy/path tests17 passed,0 failed,2 ignored. All-target touched-crate clippy and fmt pass. A timer-wheel boundary in the paused-clock test was corrected with1ms virtual dispatch, not a wall-time assertion.
Red
Green
Next size experiment removes a nested mesh-handshake timeout while retaining the same existing helper deadline and owned driver lifecycle. VM latency/outage cost and new size remain unmeasured.
Legacy test-only source provenance PASS for5 production files against01dcbee9, excluding cfg(test)/measurement-only test-support observations (not a binary-equivalence claim). Baseline standing failures stay typed failed samples; separate baseline build pending.
Source comparison
Test-only overlay
Completed4aa9dd8d (pushed mesh-autoswitch): remove the duplicate five-second timeout around the existing quic_connect helper. Same handshake deadline/TimedOut classification, separate control-proof timeout retained, driver/endpoint ownership unchanged. Silent-handshake virtual-clock regression added; all path tests18 passed,0 failed,2 ignored, all-target touched-crate clippy and fmt pass.
Path gate
Rebuilding shipping x86 for combined negotiated-discovery and timeout reuse; size/latency benefit unmeasured. Full iterate launched on4aa9dd8d with functional tests local and unchanged latency-proxy self-test on VM via pinned wrapper. Legacy standing baseline compiling in separatetarget. No completion or review-ready claim.
Resume at
4aa9dd8don mesh-autoswitch (base unchanged). scripts/verify iterate passed, including all Rust and frontend rows; final fast remains pending. Static images after the shared-timeout change: x86_64 22,113,168 B (+192 B over the current ratchet), aarch64 19,569,952 B (+384 B); no ceiling raised. A reviewed credential-borrowing refactor is being gated and measured.Real-SSH controlled-standing pilots, n=1 each (not percentiles or natural-bootstrap acceptance): LAN handshake→first causally witnessed mesh Output 9.263 ms, cut→complete echo 43.548 ms; 150 ms RTT 156.825 / 218.591 ms. One admitted TLS/control connection and exact bytes throughout. WAN fallback FAILS 200 ms.
Packet diagnostic A failed before switch: initial control callbacks=2, expected1; receipt retained, no successful sample substituted. Diagnostic B passed functional integrity/one connection, but switch229.896 ms FAILS; fallback172.864 ms. Diagnostic observers add overhead and these are not ordinary latency acceptance. Barrier clocks bracket nft invocation; they are not the exact Rust cut instant. Frozen full-stack legacy standing 31-per-venue run has started.
venue-0.json
venue-150.json
acceptance.json
Completed
6edbb284(pushed mesh-autoswitch): optional header diagnostics now bracket only the public fixture barrier tags and collect only the disposable node daemon journal. No rule bodies or packet payloads retained. Python AST/help PASS; real-SSH diagnostic B functional PASS (exact bytes, one connection). Diagnostic A initial-control-count failure retained.fallback-diagnostic-a-run.log
discovery-head-iterate.log
The iterate receipt is the earlier
4aa9dd8dmilestone, not the final fast gate. Credential cleanup remains uncommitted pending touched-feature tests/static sizes. The timing-only TCP proxy test passes on the perf VM (21.457ms); build-host failures are not accepted as performance evidence. Attempting the entire native test binary on the VM revealed missing Python test fixtures; that attempt was stopped by its verified owned PID, not counted as a passing suite.Completed
bd27a3dd(pushed mesh-autoswitch): borrow unchanged enrolled trust/name/address/uid metadata and original credentials when no renewed snapshot is present; one validated owned snapshot supplies both transport TLS configurations when renewed. Expired credentials still fail before runtime/socket creation. Scoped independent auth review found no defects; direct renewed-snapshot supervisor-reconnect coverage remains limited.Fmt/all-feature all-target clippy PASS. All-feature functional suite104 passed,0 failed,16 ignored,1 timing-only test filtered; unchanged TCP spacing test separatelyPASS on locked perf VM (21,457us). Earlier build-host timing failures and missing-fixture VM attempt retained; no threshold or failure waiver introduced. Static sizes are pending; caps unchanged.
borrowed-credentials-all-features-functional-tests.log
borrowed-credentials-all-features-clippy.log
borrowed-credentials-proxy-perf-vm.log
New deterministic red: BACKUP published during the initial held mesh proof does not enable enrolled delivery until candidate expiry. Real mTLS/control setup, paused virtual time; fails at the intended fresh-Ping delivery assertion (not a wall-clock budget). Pending proof currently never observes capability publication. This is a demonstrated startup hazard, not a proven explanation of diagnostic A control=2. Fix will observe the existing watch without resetting proof deadline.
Fresh optimized 1000-transport VM measurement with per-entry deadlines and unchanged20ms copy delay: no-copy1000 healthy,0 false interruptions,0 owned drops. Copy1000 proofs all succeed, but996 false interruptions and server UDP drops7228 after ramp /11835 after warmup (clients0). Steady copy cohort invalid; no copy CPU/RSS budget acceptance claimed. VM load before0.79/1.59/1.77, after0.62/1.51/1.74. Next regression targets synchronous64-entry return replay outside the existing32-send shared receive budget. No buffer/resource/budget raised.
Credential borrowing static x86 measured22,112,528B, SHA2567677c548c61e1f79ead0ce30068aab72d2099bcc8181f6fb61856b624a5d311d; -640B vs4aa9dd8d and448B below current cap. ARM and final ratchet pending.
late-backup-red-dispatched.log
copy20-deadlines-cost-1000.json
Completed8033a173 (pushed): replace synchronous64-datagram enrolled-return replay with existing shared32-send copy scheduler, due immediately; consume each return once, retaining legacy/backup-disable recovery and wildcard local source. No new payload queue;64-entry/64KiB bounds unchanged. Both negotiated/legacy regressions RED128vs64 before receive returns; plugin suite48PASS,0FAIL,4ignored, fmt/all-feature all-target clippyPASS. Cost remeasurement pending; earlier copy cohort remainsFAIL.
Completed281c3571 (pushed): dial runtime now delegates now() to the same existing runtime as its timers. Independent virtual-time test RED before change (real62us vs10s advanced), thenPASS. Selected remote32PASS,0FAIL,5ignored. I prematurely recorded clippy before noticing an uncommitted follow-up test exceeded its line limit; the test setup is now shortened and all-feature all-target clippyPASS is attached. No lint suppression added.
The follow-up late-BACKUP regression was rerun with corrected clock and without the watch fix: stillRED at fresh-Ping delivery, establishing the capability-publication failure separately from the clock mismatch. Watch fix retained, full touched functional suite pending.
Borrowed-credential candidate sizes before these follow-ups: x8622,112,528B (448B below current cap), ARM19,569,696B (+128B over cap), SHA256ARM62eaebd5c70608a98b863e5977b4ab4a8d2c1bfbbc1302df5c5c1e1335a8db33. No ceiling raised; final no-growth remains open.
return-replay-budget-red.log
return-replay-budget-green.log
owned-runtime-clock-red.log
owned-clock-and-late-backup-green.log
owned-clock-late-backup-clippy.log
late-backup-correct-clock-red.log
Completed
a0be3233(pushed mesh-autoswitch): pending proof observes the existing authenticated capability watch without restarting its deadline. Final regression uses a public raw marker physically received on enrolled UDP while the mesh candidate stays pending; it bypasses QUIC pacing and makes no authenticated-stream claim. Prior stream-oracle full-suite failure is retained; the revised suite106PASS,0FAIL,16ignored,1 unrelated timing test filtered. Fmt/all-feature all-target clippyPASS.Fair-return optimized1000-transport VM remeasurement remainsFAIL: no-copy1false interruption and218cumulative server drops; copy943false interruptions,5696drops after ramp /6033after warmup (218preexisting,5815additional), clients0; proofs0fail. This is an improvement over earlier996/11835, not healthy-cohort acceptance. The same-cohort capability toggle differs from production, but full retained history is production-reachable; no waiver, history clear, resource or budget increase.
Observer-free4aa WAN control series currently19 completed samples, not31. At n15 switchmax330.637ms failed200ms; fallback192.347..195.659ms passed those15. A backed15ms enrolled discovery gap can send fresh Output only enrolled, which the causal fixture fences; extraQUIC recovery is a possible explanation, not proven cause. A physical-path regression is prepared outside checkout for the next step. Legacy standing baseline currently27LAN samples, WAN pending.
New shipping x86 with fair-return/clock/watch production code22,113,232B (+256B over cap), SHA256a74bd3590a761411e641262f1c8f35549e2028ea077b7b059a5407456fcda827. ARM and new server fixture building; head iterate running. No final fast or no-growth claim.
late-backup-physical-all-functional.log
late-backup-physical-clippy.log
fair-return-cost-1000.json
Frozen01dcbee9 full-stack legacy standing LAN baseline complete:31/31 exact terminal byte integrity. WG handshake→first causal mesh Output p50/p99 621.533/971.523ms; cut→complete echo261.272/619.473ms. Initial/final control counts are1→3 or2→4: no connection-preservation or writer-continuity claim. RealSSH start→success1724.655/2041.853ms. Success→WG handshake5404.726/16033.704ms includes intentional initial mesh hold, so it is NOT natural-bootstrap latency acceptance. WAN31series continuing.
Old4aa9dd8d observer-free WAN control series copy20-standing-wan-31-a stopped at sample20 (21st lifecycle): bootstrap activated2control connections, expected1.20completed timing samples,1failed lifecycle,10not attempted; not a completed31sample acceptance. This matches diagnosticA failure type. No timing or integrity sample substituted, no cause attributed to newwatch fix. Failed receipt attached; rawVM path /root/infraternal-bench/mesh-autoswitch/r9/acceptance/copy20-standing-wan-31-a/150/20/.
An additional legacy fenced-canary pilot WITHOUT initial mesh hold is pending, to keep natural add-host timing separate from the standing-session witness. Newproduction server fixture and ARM build pending; head iterate stillrunning.
venue-0.json
copy20-standing-wan-failed-20.log
Completed4f8b8b508 and5fd7b45bf (pushed mesh-autoswitch). Path proof now requires an observed post-decryption frame-counter change after the observer is created, rather than elapsed silence: a previously latched invalid public Mesh UDP marker could satisfy the old freshness check. This was path classification on an already authenticated connection, not a certificate/packet-auth bypass. Causal regression RED before fix, then107PASS,0FAIL,16ignored,1 unrelated timing test filtered; plugin48PASS,0FAIL,4ignored; fmt/all-feature all-target clippyPASS.
5fd publishes the validated decrypted Hello capability directly through the existing Link: apply backup before waking capability observers, removing the pending-proof watch loop. Scoped independent auth/source review found no defect; the regression drives the same publisher directly, not an independent delayed-Hello wire test. Callback static sizes still building; no size or timing improvement claimed.
Prior a0 full iterate PASS (attached); not final fast. Controlled-standing realSSH pilot n=1 each, exact bytes and one admitted connection: LAN switch7.365ms/fallback42.875ms;150RTT154.924/194.338ms. These precede4f/5fd and include intentional initial Mesh hold: not31-sample or natural-bootstrap acceptance. Separate legacy natural fenced-canary pilots both failed install/readiness; no natural latency acceptance claimed. Legacy standing WAN31 continues. Fresh-copy1000 cohort diagnostic is being added alongside, not in place of, the failing retained-cohort diagnostic.
invalid-mesh-hint-red.log
explicit-path-freshness-agent-tests.log
explicit-path-freshness-plugin-tests.log
fair-return-standing-pilot-lan.json
fair-return-standing-pilot-wan.json
fair-return-late-backup-iterate.log
Completed84b837d7d (pushed): keep the existing retained-cohort1000 diagnostic and add a separately labelled fresh-backup cohort, plus snapshots of existing bounded history fields outside measured windows. Remote plugin48tests and copyfixture2functionaltestsPASS; all-feature all-target clippyPASS on fixture step. I prematurely continued the commit command after full-worktreefmt found one extra blank line in a concurrent uncommitted test; corrected that line and reran fmt+diffcheckPASS. The committed fixture paths themselves were formatted.
First new VM fixture invocations matched ZERO tests: I omitted the remote feature in the optimized build. These empty runs are not measurements or acceptance; logs retained. Correct all-feature optimized fixture rebuilding. No history/buffer/budget waiver.32/8 paired comparison remains pending.
Causal physical gap regression RED: after candidate refusal fresh datagram went only enrolled, leaving no mesh discovery traffic. Uncommitted fix keeps negotiated copies and mesh discovery across fresh proof epochs; focused23PASS,0FAIL,2ignored, including proposal replacement, capability withdrawal, active cancellation, truthful unproven status and bounded both-UDP-dead handoff to existing TCP dial race. Independent scoped source review found no defect. Five-second silence threshold is sampled at proof-epoch boundaries, not an exact5s exit deadline. Final touched gates pending.
Frozen5fd callback x86 image22,111,760B (1216B below current cap), SHA256f955c59bc24915d59c21874d0735e695c151d63f1fd6955e0de43e79e2062d21. ARM still building; no ratchet or final no-growth claim yet.
backed-gap-red.log
backed-gap-path-tests.log
fresh-copy32-vm.log
retained-copy32-vm.log
Completeda81b2a306 (pushed mesh-autoswitch): continuous backed discovery fix and its causal regression/invariant tests. Final agent functional110passed,0failed,17ignored,1unrelated timing test filtered; focused23passed,0failed,2ignored; fmt/diffcheck and all-feature all-target clippyPASS. Independent scoped review found no defect; post-cancellation proposal churn remains source-reviewed rather than an added multi-change regression. VM timing, scale health, ARM size and finalfast remain pending.
backed-gap-final-agent-functional.log
backed-gap-clippy-final.log
Completedd4fbd44e5 (pushed): additive unheld natural-bootstrap post-success witness timestamps. Existing enrolled-ingress fence and drained Echo attachment provide causal first post-fence Output and complete fresh marker. Reportstart/success/kernelHS→witness as upper bounds including fixtureoverhead, NOT exact earliest naturalMeshOutput. Existingcontrolledstanding and failure semantics unchanged. Fmt/runnerAST/serveralltargetclippyPASS; bootstrap22PASS/4ignored. Corresponding test-only frozenlegacyoverlaybuilding separately; newignoredVMreceipts pending.
Correct remote-enabled optimized32 fixture realVM tests FAIL: freshcopy1000 has16initialprooffailures,910falseinterruptions (865ramp/40warmup/5steady),7417serverUDPdrops/0clientdrops; initialserverhistory [0,0,0,0], so failure is not only inherited history. Retainedprofile no-copy689falseinterruptions (688warmup),2335serverdrops; copy23prooffailures/948falseinterruptions,7109cumulative serverdrops. Current load about1.3–1.7on4cores. These unhealthy cohorts are not cost acceptance; paired8candidate queued, notselected.32fixtureSHA1aa51926ea9977e2e9d8fbb35ba52fd8f57891931bb192c592aeb50b6c0d160a.
Frozen5fd callbackARM19,569,184B (-384B cap), SHA68ea41f7a5fe0cd24c4b11c606aedb86277c2fec4c584ec31ba80267a230ea8b. x8622,111,760B (-1216B). These precede continuousdiscoverya81; gapshippingbuild pending and finalratchet notchanged.
natural-witness-clippy.log
natural-witness-bootstrap-tests.log
retained-copy32-remote-cost.json
callback-shipping-sizes.json
fresh-copy32-remote-cost.json
Eight-send candidate not selected; production limit restored to32. Fresh copy:964 false interruptions,0initial proof failures,7414server drops. Retained cohort: no-copy0interruptions/54drops; copy936interruptions/8initialproof failures/6832cumulative server drops. It does not establish a healthy cohort or reliable improvement over32; control health differed. All1000connections remain admitted, clients0drops. Candidate fixtureSHA ce8fb99d797ac5947eb17ce4f963fe59a1ba24b9d0f61c583156c10d8130dd7c.48plugin functional tests and clippyPASS, but VM acceptance FAIL.
Stopped only the verified owned obsolete prototype iterate PID22707 while it waited on the shared VM timing smoke test. It did not run the full Rust gate and is NOT a passing iterate/fast receipt. Source now tests an exact earliest-deadline cache to skip retained entries for idle/future peers;50plugin tests PASS, performance benefit unmeasured. Size refactor reuses one pinned proof and one enrolled-return future without adding allocations; also unmeasured.
Gap32 shipping x86 measured22,117,808B, +4832over current ratchet (+6048vs callback), SHA10308fa6601a9c64de5dc7c4a2be647c6f9ebc1338c852b445725980b8ad02d6. No ceiling raised. Combined size candidate rebuilding. Legacy natural/unheld31 series started; firstLANsample intact, no series percentile claim.
fresh-copy8-remote-cost.json
retained-copy8-remote-cost.json
gap8-prototype-iterate.log
Completed
cb7d9c49e(pushed): allocation-free reuse of the pinned candidate proof and enrolled-return future. Capability updates preserve the proof deadline; only legacy refusal can interrupt return proof on a routing hint. Agent functional suite:110passed,0failed,17ignored,1unrelated timing test filtered. Fmt/diffcheck/all-feature all-target clippy PASS. Combined shipping image with pending deadline-cache candidate is building; no size benefit claimed yet.The uncommitted deadline cache passes50plugin tests and scoped independent review; an initially oversized test-only stack array failed clippy, changed to a heap vector, then clippy and50tests reran PASS. Cached32VM fresh/retained measurements queued. The fresh command also includes a cached_copy_deadline prelude filter, but those tests belong to the plugin binary rather than this infraternal-agent binary, so that prelude matches zero; only the actual fresh-copy filter is the measurement. No VM plugin-unit claim.
Gap8 ARM image (before reuse/cache)19,574,528B,4960B over cap; no ceiling raised. Gap32 realSSH standing pilot queued as an explicitly earlier, oversized snapshot; final acceptance remains outstanding. Baseline standing and natural series still running; no partial series percentile claim.
cached-deadline-gap-reuse-agent-tests.log
cache32-gap-reuse-final-clippy.log
cache32-final-plugin-tests.log
Proposed additive clarification to contracts/wire.md native path-copy section before implementing the remaining copy-first fix (D215/D263, #201). No frame layout, version, capability-bit or authority change.
Causal real-mTLS regression: enrolled backup arrives first, then its identical mesh original is physically received. Duplicate hint filtering correctly retains the enrolled primary route, but the unchanged mesh guard times out despite fresh authenticated progress and both connections remaining open. The final assertion fails; no fixture timeout. Promoting duplicate hints would break genuine enrolled recovery and is rejected.
For negotiated bit6 peers, retain immediate enrolled primary replies and permit a bounded encrypted alternative mesh reply, using the existing admitted tuples, recent history, owned clock,32-send budget and64datagram/64KiB bounds. It carries the same ciphertext, creates no connection/stream/writer or authority, and never promotes a duplicate route hint. Mesh alternatives are due immediately (a second20ms delay would consume the guard margin); ordinary mesh-to-enrolled backups retain individual20ms deadlines. Unknown coordinates remain dormant; retired leases cannot choose a replacement socket; capability withdrawal cancels alternatives while preserving legacy enrolled recovery. Redundant mesh-reply admission must not evict pending enrolled-return work.
Also reconcile the section with already implemented per-entry deadlines and consume-once bounded enrolled replay: younger entries are not copied early, and pending recovery is independent of negotiated backup cancellation. This corrects stale prose, not a decision change. Existing wire parsers and fuzz targets remain unchanged.
Implementation and old/new regressions, healthy1000-transport proof,31-per-venue release acceptance, exact static ceilings and finalfast remain required. No acceptance claim or owner exception requested.
New real-mTLS causal RED confirms copy-first ambiguity can strand the mesh guard: enrolled ciphertext is read first, identical mesh originals are physically delivered, authenticated traffic progresses and both connections stay open, yet the unchanged guard times out. Intended final assertion fails (1test); no setup/deadlock timeout. An initial unqualified --exact filter matched zero; corrected final receipt executes1test. This establishes a defect mechanism, not proof that it explains every1000-cohort failure. Additive bounded-dualreply contract proposal posted separately before implementation.
Cached32 candidate rejected and removed, with patch preserved outside checkout: fresh990guard exits/0proof failures/7873server drops; retained no-copy1000healthy/0drops, copy983guard exits/0proof failures/7174drops. Clients0drops, all1000connections open. Empty initial history in the fresh cohort falsifies full inherited history as a necessary cause. No copy CPU/RSS acceptance claim. Correct cache/unit invariants alone did not restore healthy transport behavior.
Earlier oversized gap32 realSSH standing pilots, n=1each: LAN handshake→first causal MeshOutput7.752ms, cut→echo35.856ms;150RTT157.954/192.543ms. Exact bytes, same writer/attachment, control/TLS1→1. Intentional initial mesh hold; not natural timing or31-sample acceptance. New gap boxing candidate110agent+48plugin functional tests and clippyPASS, static size and transient-future layout measurement pending; not selected yet.
real-copy-first-guard-red.log
fresh-cache32-remote-cost.json
retained-cache32-remote-cost.json
cache32-remote-summary.json
gap32-standing-pilot-lan.json
gap32-standing-pilot-wan.json
Continuation at
7ffba3ea5(pushed): contract/checklist now state the bounded dual-reply proposal and distinguish held standing-session evidence from unheld post-success witness bounds. Implementation and release acceptance remain pending.The real mTLS copy-first regression now uses one reliable QUIC stream with eight exact exchanges. This corrects the earlier application-DATAGRAM oracle (which could itself lose markers). Against HEAD production, all original mesh packets were delivered and enrolled authenticated progress/connections remained live, but the mesh guard failed. The uncommitted bounded dual-reply candidate passes the same stream test. Full edge tests and independent lifecycle review continue; no scale/timing acceptance claimed.
Rejected boxed-discovery prototype: x86 static 22,119,536 B, +6,560 B over the unchanged D243 ceiling; source reverted. Perf VM layout receipt is only a rejected prototype's type-layout arithmetic (1 real TLS connection, not 1000 connections, no RSS comparison). No memory-saving claim.
A helper AI agent mistakenly attempted plain SSH for the optional layout receipt; host-key verification refused. It stopped without changing keys/configuration or inspecting credentials. The upload and measurement were subsequently performed through the required NetBird SSH transport.
Independent review found one queued alternative can leave a stale shared copy timer after mesh-listener lease removal; no further payload can send, but shutdown must cancel unused work. Adding a failing lifecycle assertion and fixing clock reconciliation before committing production.
real-copy-first-stream-barrier-red.log
real-copy-first-stream-barrier-green-attempt.log
boxed-layout-vm.log
cache32-boxed-discovery-static-x86.log
Legacy controlled standing baseline complete: 31 samples per venue, frozen original-dev payload (
01dcbee9, SHA-25636aa5c93c96f8701bf919213088892a17d55ce651e913db7d76765c5fe11a93f), release fixture on the pinned perf VM. Nearest-rank p50/p99; p99 is the maximum of 31.Bootstrap, exact terminal byte integrity, retained original attachment and causal standing Output proof: 31/31 each venue. Control activation count increased by two in every sample; this baseline does not preserve the authenticated connection/writer. TLS/admission callbacks increased by two throughout LAN; WAN increases were two in 11 samples and three in 20. No recorded bootstrap or standing-terminal failures. Switch exceeds 50/200 ms in 20/31 LAN and 31/31 WAN samples; fallback echo exceeds those budgets in all 62.
The fixture deliberately held outer WG through real SSH bootstrap and enrolled PTY readiness, then released it and fenced enrolled ingress before a fresh marker. Hold, observer/fence and input/echo overhead remain included. These are held-standing baseline results, not natural bootstrap latency. Raw frozen provenance says
ordinary_latency_acceptance=truebecause its runner omitted standing mode from that metadata expression; every individual receipt correctly says false. The current runner already fixes this metadata flag. Raw evidence remains unchanged; receipt scope is authoritative.WAN aggregate · Both-venue summary, counts and quantiles · Raw frozen provenance
This report adds no source changes or benchmark runs and does not close #201.
Correctness step committed and pushed:
fc2a9b3c8. Bounded dual replies fix the real mTLS reliable-stream copy-first regression without promoting duplicate route hints, adding an authority, or changing writer/session identity. Lease/peer/capability removal now cancels unused timers while preserving enrolled Return work and live receive registration. All 59 remote-plugin functional tests pass (4 ignored); touched-plugin all-feature/all-target clippy -D warnings, fmt and diff check pass. Agent functional suite passed 110 (timing-only and ignored scale filters explicit in log). This is not release completion.Perf VM transport-cohort results remain FAIL, not accepted CPU/RSS evidence:
Static x86 attribution snapshot (before final timer reconciliation): 22,119,184 B, +6,208 B over unchanged ceiling. Map shows one shipping steer specialization (19,263 B), not duplicated test specializations. An allocation-free borrowed active-session poll interface is being measured as a separate size candidate; no acceptance yet. Native server fixture and iterate are still running. Full fast gate, final sizes and after 31-sample acceptance remain outstanding.
dualreply-plugin-all-tests.log
dualreply-plugin-clippy.log
dualreply-retirement-timer-red.log
dualreply-cap-withdrawal-timer-red.log
dualreply-fresh1000-a.json
dualreply-retained1000-a.json
dualreply-transport-fixture-provenance.json
dualreply-fresh1000-vm.log
dualreply-predyn-static-size.json
D243 follow-up: borrowed active-session poll erasure did not meet the ceiling (22,119,664 B, +6,688 B). Source reverted; no allocated box was added. The preceding 22,119,184 B map snapshot predates final timer fixes, so the 480 B difference is not an isolated causal comparison.
Actual shipping attribution found separate Config/PeerPolicy deserializer families for slice and File readers. Public node restoration currently deserializes directly from File, while incoming configuration already uses slices. Measuring reuse of the existing slice parser after reading the already-open, no-follow/ownership/mode-validated public file. No schema, path, validation, authority, dependency, or engine change; no new file-size limit. Tradeoff to measure: one transient encoded-file buffer during cold restoration, versus streamed reads and duplicated parser code. Keep the previous implementation in test-only comparison, measure real 1000-peer snapshots on the perf VM, and preserve restore/security regressions before selecting a candidate. No size or runtime acceptance claimed yet.
Measurements queued, not discarded or bypassed: shared perf VM lock /run/lock/perf-bench.lock is held by a different benchmark job (flock PID1981650, cwd /root/infraternal-bench/ppu-soak/r4long; observed elapsed12:04). All our queued measurements still use flock -w3600, and no other process was stopped or changed. Legacy natural WAN is currently18 completed/31; dual-reply standing pilot LAN completed, WAN queued; public-state reader/slice comparisons and quiet ACK-only reproductions queued. Builds and functional work continue independently. This is lock contention, not an authorization or SSH failure.
The native ACK-only real-QUIC regression is ready for quiet VM verification. Build-host functional runs include both an earlier isolated pass and subsequent intended guard failures (full plugin59pass/1fail/4ignored); no causal ACK configuration change or timing acceptance is inferred from the loaded build host. Native receive-diagnostic fixture is also built and frozen for per-socket drain/driver attribution.
Public-state slice reuse candidate x86 size:22,072,432B (-40,544B vs existing ceiling); ARM/runtime comparisons pending. This candidate is not yet selected or committed and ceilings remain unchanged. The older iterate invocation completed successfully; it predates final test/candidate changes and is not a branch-head/final fast gate.
Completed instrumentation step committed/pushed
f6db4005e: opt-in test-only receive diagnostics, unchanged production paths and unchanged cohort health checks. Full infraternal-agent functional suite112pass/0fail (15ignored,3explicitfilters for two ignored scale names and the VM-only packet-spacing check), diagnostic4pass, touched-agent clippy/fmt/diffcheck pass. Native observer fixture is built/staged and queued on the shared VM lock; observer CPU is diagnostic only.Public-state slice candidate remains uncommitted/unselected pending runtime/memory comparison. Static unchanged-recipe sizes:
Earlier iterate transcript attached. Its final line verbatim:
That invocation began before subsequent candidate/test changes; it is an older milestone, not a branch-head or final fast gate. Final fast, accepted healthy1000transport cost, real after31-per-venue results and completed mesh-disable proof remain outstanding. The later ACK-only functional regression is RED in source; a test-only negotiated5ms ACK-delay comparison is staged without production configuration or interval changes. Quiet native comparison pending, so no ACK cause/profile selected.
receive-diagnostic-agent-full-tests.log
receive-diagnostic-agent-clippy.log
receive-diagnostic-vm-artifact.json
slice-restore-shipping-sizes.json
cold-restore-independent-review.txt
public-restore-mesh-tests.log
public-restore-mesh-clippy.log
dualreply-iterate.log
Starting the authorized test-only profiler alignment slice on
mesh-autoswitch, basef6db4005ea7ca3ae49d548c385d3c1f9016a7378(#201). Two atomic commits: enable the existing nix time API for infraternal agent tests, then bracket the opt-in receive diagnostic Instant origin with safe CLOCK_MONOTONIC reads. Default runs and production transport settings remain unchanged. ACK comparison and cold-restore prototype changes stay unstaged. Functional/clippy gates and native fixture receipts will follow; root owns the branch-wide final gate.Atomic test dependency step committed and pushed as
dc3ef54c6(Refs #201). Onlycrates/infraternal-agent/Cargo.tomlchanged: the existing nix time feature is enabled as a dev-dependency, for safe CLOCK_MONOTONIC alignment. Rejected unsafe FFI and another dependency. No production transport policy changed.Scoped gates (verbatim terminal results):
Raw receipts:
evidence/clock-time-feature-{fmt,clippy,tests}.log. The diagnostic anchor is the next separate atomic step. Root retains the branch-wide final fast gate; this scoped receipt is not a claim of final branch acceptance.Atomic opt-in diagnostic anchor committed and pushed as
9c8187d97(Refs #201); only the receive diagnostic module changed. It brackets the existing Instant origin with safe CLOCK_MONOTONIC reads, records nullable absolute nanosecond bounds in the sidecar, and rejects conversion/overflow/missing/reversed bounds. Default runs read no extra clocks. No production transport setting, watchdog interval, margin or acceptance criterion changed.Scoped gates (verbatim terminal results):
Raw receipts:
evidence/clock-anchor-{fmt,clippy,tests}.log. Native release test fixture rebuild has started in the existing r7-shipping target with one build job; immutable artifact/hash receipt follows. No VM run started. ACK and cold-restore prototypes remain unstaged. Root owns the final branch-wide fast gate; these scoped gates do not establish scale acceptance.Anchored diagnostic release fixture frozen from the authorized #201 slice (
dc3ef54c6,9c8187d97).Native build receipt (verbatim):
Immutable copy:
/home/kayg/.cache/codex-scratch/mesh-autoswitch/r9/receive-diagnostic-monotonic-fixtureSHA256:
510af24ec2df7aeded93e6fea7758f2e822014367c887d0fcabd8e71ead52698.Manifest:
r9/clock-anchor-fixture-manifest.json; raw build log:evidence/clock-anchor-native-build.log.The existing ignored fresh-cohort filter uses
INFRATERNAL_MESH_RECEIVE_DIAGNOSTIC=1and the existing external scratch directory. The sidecar clock label is CLOCK_MONOTONIC. Because relative microseconds truncate, boundaryt_usconservatively maps to[before_ns+t_us*1000, after_ns+(t_us+1)*1000). Snapshot boundaries describe a diagnostic ramp envelope including coordinator bookkeeping; CPU is diagnostic only. No VM measurement was launched by this slice, and no new acceptance claim is made. Root owns the shared-VM lock and final branch-wide fast gate.Head
9c8187d97. The opt-in anchored native fixture is frozen (native SHA256 510af24ec2df7aeded93e6fea7758f2e822014367c887d0fcabd8e71ead52698; interpreter-only VM copy b364d037072861aeba3e399a1b7c37868a5da7861e9c7907e93eb03eb9469a73, remote hash checked). Queued one fresh1000 diagnostic with bounded own-PID software CPU sampling: IP-only99Hz, max15s, one shared lock, no permission changes; observer costs are not performance acceptance. Absolute monotonic brackets and conservative microsecond quantization bounds separate the ramp envelope from TLS setup. All helpers/evidence remain outside checkout.Also queued a pre-final LAN kernel namespace lifecycle/mesh-disable proof using the frozen r11 dual-reply server fixture and CLI. It predates the last capability-withdrawal timer edge fix and cold-restore candidate; it cannot serve as matching final release acceptance. Requires completed lifecycle and exact bytes, not the earlier incomplete receipt.
The same other-job lock holder was still present at elapsed41:53. Existing restore, ACK comparison, receive diagnostic, baseline natural WAN and standing WAN jobs remain queued with bounded3600s lock waits. No result skipped or claimed passing; no foreign process stopped.
Baseline sample-count correction: the natural LAN baseline has31 attempts but only30 successful path witnesses because original sample4 is a recorded typed install/readiness failure. To obtain31+ successful timing observations without dropping that failure, queued a predetermined extension of5 fresh LAN baseline attempts with the same frozen legacy fixture/payload and runner. Final report will combine every attempt, keep the original failure in the denominator, show success/failure counts separately, and never relabel the extension as a replacement. Output: r12/acceptance/legacy-natural-lan-extension5-a. Per-lifecycle flock -w3600, no outer lock across the series. No after result or new percentile claimed yet.
Independent ACK-only fixture review: no clear gate-forgery or transport-setting mismatch found. Important oracle scope: the ACK-only completion barrier ends at the eighth server DATAGRAM receipt; it proves guard survival through those receipts, not eight completed ACK cycles or the eighth probe response deadline. At least one authenticated counter change is required on success; raw duplicate datagrams are not that counter. The negotiated5ms comparison also adds ACK-frequency negotiation frames, so even a default-RED/5ms-GREEN result describes the negotiated profile and does not isolate a timer substitution by itself. Real-time guard/DATAGRAM scheduling remains a limitation; quiet VM receipts still pending. These qualifications will accompany results; no production knob selected.
External scheduling block: shared perf lock holder PID1981650 was still present at elapsed1:01:04. Two mandated3600s lock waits expired before their next lifecycle started (runner exit1 from flock, not an application/protocol failure). Legacy natural WAN indices0–18 completed (19 observations); index19 onward not attempted. Pre-final standing pilot LAN complete, WAN index0 not attempted. Raw logs attached below.
Queued resumes only at unstarted legacy WAN indices19–30 and standing WAN index0, with the same immutable runner/artifacts/series and per-lifecycle flock -w3600. Completed samples remain untouched; scheduling timeouts stay recorded. Other previously queued diagnostics/restore comparisons still waiting. Completing acceptance requires the orchestrator/shared perf VM scheduler to provide a lock slot; no foreign process was stopped and no lock was bypassed. Head9c8187d97; task remains incomplete, final fast not claimed.
dualreply-pre-final-standing-pilot-a.log
legacy-natural-31-a.log
Blocked continuation checkpoint, head9c8187d97b9d3e0d037ae020ca0b352c98633eab (remote branch verified). Shared perf lock still held by unrelated PID1981650 at1:05:10; two3600s waits expired. Required external change: orchestrator/shared scheduler provides a perf VM slot. Completed source steps
dc3ef54c6and9c8187d97 are atomic and pushed.Scoped gate output verbatim:
This is the scoped diagnostic gate, not scripts/verify fast. Final branch-head fast remains outstanding.
Unselected cold-restore candidate and ACK regression/comparison remain in the worktree and are backed up in attached patch plus test source. Native/VM artifacts are frozen. Existing bounded queued jobs remain active; resume-state attachment records their session IDs and remaining requirements. No timing acceptance, healthy1000transport cost, final after31-per-venue result, final disable lifecycle or D243 ceiling ratchet claimed. Evidence/scratch retained because work is incomplete. No merge/deploy/issue closure. Attachment helper rejected the raw .rs extension with HTTP422; identical source attached as .rs.txt, no credentials investigated.
resume-state.json
pending-candidates.patch
node_state_restore_tests.rs.txt
clock-anchor-tests.log
clock-anchor-clippy.log
Resuming mesh-autoswitch at head
9c8187d97b, branch mesh-autoswitch. Read #201 history (131 comments; no linked parent in issue body) and current rules including build-slot/perf fairness. Keeping pending public-state slice and ACK-only test changes for validation, not treating them as accepted. Baseline measurements remain those already recorded before production changes. Next: recover queued immutable receipts, measure pending restore/ACK hypotheses under the shared perf lock, finish production acceptance and static ratchet, then final scripts/verify fast. No deployment.Perf scheduling finding at resume: /proc/locks still identifies PID1981650 as holder of /run/lock/perf-bench.lock, elapsed1:08:22; cwd /root/infraternal-bench/ppu-soak/r4long; command requests timeout7500 and a7200-second soak plus600 warmup. /var/lock/perf-bench.owner is absent. This is the same holder recorded in the previous run, exceeding the new30-minute fairness interval. No foreign process or lock changed. Short restore/ACK comparisons will wait for a slot with owner metadata; functional work continues. Orchestrator/scheduler needs to release/resegment that foreign hold.
Functional continuation: inherited slice candidate passed36 mesh tests (12ignored); source now separates restore regressions from the unselected production parser change. Original reader plus1000-peer round-trip regression passes1/1 (47filtered); fmt/diffcheck pass. Full touched-mesh test/clippy commit gate is waiting for build-slot, not bypassed.
The frozen r11 server fixture's namespace_store_survives_cancelled_pool_checkout passes1/1 with the correct Cargo cwd/environment and native loader. This proves the cancelled-last-connection fixture regression, not a completed kernel namespace disable lifecycle; that still awaits VM execution. Initial direct invocation of the interpreter-adapted VM fixture failed at loader symbol resolution before tests; explicit local Nix loader ran the exact fixture successfully. No production code changed in this step and no build-host timing is performance evidence.
Quiet VM comparisons complete, same frozen native fixtures, one shared lock; load0.59/0.32/0.30→0.50/0.31/0.30. Public-state restore31samples/mode,1000public peers,1concurrent restore (not1000live daemons): reader p50/p99 89.060/101.799ms; slice1.571/2.403ms. Encoded file209554B; observed buffer capacity209554B, conservative old+replacement capacity bound419108B for this fixture, excluding decoded tree/allocator overhead. Reader process RSS/HWM6016/6028KiB; slice6068/6068KiB. These include fixture setup/allocator retention, not isolated parser RSS. No file-size cap added. Select existing slice deserializer reuse on the same validated descriptor; prior candidate sizes22072432/19516992B remain provisional until final rebuild/ratchet.
Same-binary interleaved ACK-only copy-first comparison: default29fail/2pass; negotiated5ms31pass/0fail. Both preserve15ms interval/30ms margin. Successful negotiated cases confirm ACK_FREQUENCY sent/received1/1. This supports testing that negotiated production profile; it does not isolate a bare ACK timer substitution from negotiation frames. Oracle ends at eighth server DATAGRAM receipt, not eight completed ACK cycles. It establishes no healthy1000transport or31lifecycle acceptance yet. Implementing client-side request using existing Quinn configuration, same packet threshold/reordering behavior, no new engine/dependency/authority or relaxed deadline; full scale and real SSH acceptance remain mandatory.
Raw comparisons
New diagnostic attempt retained: frozen9c fresh1000 fixture exits before ramp at history_before==[0;4]: actual[3,4356,0,0]. No cost/receive JSON exists because the prelude assertion precedes report writing; this is a failed attempt, not an accepted cohort. After alias creation and backup publication, post-handshake enrolled sends can legitimately queue dormant MeshReply entries before any mesh selection. Existing unknown_mesh_reply_waits_for_duplicate_coordinates_and_off_retires_it covers that production behavior. This is distinct from inherited Original/Return history; no guard has been started/reset.
Proposed fixture correction: at this exact fresh-before-mesh boundary require zero armed/Return entries, retain/report entry/byte counts, and cover acceptance of dormant replies plus rejection of armed/recovery entries with a failing unit test first. Physical routes are still enrolled by construction. All admission, proof, ramp, warmup and sampling health checks remain unchanged. No history clearing, socket-buffer or deadline change.
The new ACK-only test with production negotiated5ms also failed its30ms guard on the loaded debug build host; original log retained. It is a physical timing oracle and belongs to the pinned VM, where the frozen optimized5ms profile passed31/31. No build-host timing improvement or blanket test waiver claimed. Existing functional stream test stays in the normal suite; new ACK-only comparison will be explicitly VM-only.
Completed/pushed
f5044f1a3(restore/security tests) then771a44c1e (validated-descriptor slice-parser reuse). Each is one logical step; production parser stayed unchanged in the test commit. Build-slot touched-mesh gate:36passed/0failed/12ignored; fmt and all-feature/all-target clippy -Dwarnings PASS. Original-reader round-trip and malformed/descriptor cases separately passed1/1 each before selecting production reuse. VM runtime/memory tradeoff and provisional sizes are in the preceding comment; no ceiling ratcheted yet. Finalfast/acceptance remain outstanding.ACK/fresh-startup follow-up is uncommitted and gated separately. All62 raw interleaved logs and startup failure remain attached:
ACK comparison31per mode
Fresh1000 prelude failure
restore-regressions-gate.log
fresh-dormant-prelude-red.log
Additional source finding during the startup diagnosis: remote_quic.rs applies PATH_SAFE_UDP_PAYLOAD=1352 only inside
if client; server transport retains the default discovery ceiling. Pinned Quinn transport parameters advertise the endpoint's max_udp_payload_size, independent of the client's discovery upper bound, so a capped client does not constrain server sends. A connection moved below QUIC can retain its learned enrolled-path MTU. This may strand a larger server output on the1380-byte mesh interface until engine black-hole recovery; current first-byte pilots do not prove the long-lived/bulk case. The three dormant4356B entries match3×1452B probes, but that arithmetic is not yet causal proof.Next narrowly scoped regression will use real mTLS/UDP and acknowledged MTU-probe state, requiring both endpoints' discovered payload to remain mesh-safe after ordinary stream delivery. It should fail on the current server configuration before moving the existing discovery ceiling outside the client-only ACK configuration. This reuses the selected engine and bounds both directions; no new transport/engine or relaxed limit. Separate from the ACK profile commit; final size and VM stream/acceptance checks remain required.
Completed/pushed separate steps73bd4bdf1 (negotiated5ms ACK profile + VM-only copy-first regression),692384165 (fresh-prelude dormant-reply oracle),82bb218a (both-direction mesh-safe MTU). Source-only MTU concern is now reproduced: real mTLS/UDP delivered32KiB and acknowledged discovery, then the original server learned1389B >1352B. Moving the existing ceiling outside the client-only branch passes that same test; no new engine or widened detection budget.
Scoped gates: ACK/fresh remote59PASS/6ignored; agent114PASS/17ignored/1explicit unrelated physical TCP-spacing filter. New MTU complete remote suite60PASS/6ignored; strict all-feature/all-target remote clippy and fmt PASS. Agent/remote strict clippy passed before the separately gated MTU step. New fresh-prelude unit RED on recorded[3,4356,0,0], then GREEN without clearing history or changing any cohort health checks. Physical ACK debug-host failure remains retained; optimized quiet-VM comparison remains29/31default failures versus31/31negotiated passes, not scale acceptance.
Native optimized agent/server fixtures are now rebuilding through build-slot; their release measurements, final shipping sizes/ratchet, complete disable lifecycle, matching31-per-venue after tables and finalfast remain outstanding. Baseline WAN resumes only unstarted indices; the separately labelled legacy LAN extension retains its original failed attempt. No deploy/merge/issue closure.
ack-and-fresh-functional-gate.log
server-mtu-ceiling-red.log
server-mtu-ceiling-green.log
Completed/pushed documentation41f545fad and exact downward ratchet1ce6061f2. Unchanged locked shipping recipe, stripped static images:
Optimized agent/server runtime code frozen at82bb218a9;41f changes comments/checklist only. All native/VM pairs and shipping hashes are in the attached manifest. ELF adaptation changes only the native fixtures' interpreter. Release observer-free fresh/retained1000cohorts, TCP spacing and31production-profile ACK-only cases wait under one bounded acquisition; the separate unstarted plugin waiter PID2087500 was verified waiting/no children, then cancelled to combine these short measurements. No sample was attempted/replaced. Legacy unstarted WAN/labelled LAN-extension resumes remain separate.
Prepared after-runner preserves the real SSH lifecycle/assertions but groups each finite venue under a25-minute hard cap, writes owner/start metadata and releases between venues (≤30-minute fairness). Unique disposable fixtures per sample; signal cleanup retains owned container names. Base runner SHA8452d1baa0aa1f984dd82a77d581e7d9cf9a93435a5e414e603d22fce4a65477; scratch runner cd49c4ec82fe87465d7c7dde44f1d53719ce3bff279d4d4b8a6154d32614c640. After31series not started until scale diagnosis completes. Still incomplete: healthy scale, full after31LAN/WAN tables, complete mesh-disable lifecycle, finalfast. No deploy/merge/closure.
r13-artifacts-and-pending.json
final-size-boundaries.log
Perf fairness45-minute wait report (continuing, not giving up). Our release transport waiter is PID2082262. Current owner file:
PID
2082262 45:21
process-per-user ppu-soak-r4-own 2026-10-15T01:37:09.334197+00:00
2026-10-15T04:01:26+02:00
The next listed waiter is the addhost job, not ours; an earlier commentary identification was corrected. No foreign process/lock changed. All release transport proofs, old unstarted baseline cases, finalfast timing prelude, four conditional31-sample after venues, and complete storage-disable proof remain queued. Conditional after/storage workers refuse to measure if the complete fresh/retained1000 and ACK suite fails; all frozen artifact hashes are already attached. Each after venue has a25-minute hard cap and owner metadata, release between venues. No after sample or passing finalfast claimed.
Release82bb result: fresh1000 remainsFAIL,710guard exits/7initial proof failures,1000connections open;679ramp/17warmup and14steady failures, server drops8569ramp/11448final,clients0. Before history[1,35,0,0] is a legitimate dormant reply. Returning entries2056after ramp and624after windows are important: this fixture never intentionally returns any client to enrolled while a profile runs, so these are server replay transitions caused by ordinary backups/fresh route hints, not operator fallback. Narrow production-profile ACK comparison31/31plus MTU/dead-path/stream proofsPASS; they do not establish scale health.
Next causal test: while mesh is selected and negotiated backups are enabled, an enrolled ciphertext whose mesh original was missed currently switches the primary immediately and queues the entire64-packet replay history. A later mesh arrival can repeat that feedback with new history. This is a hypothesis for the overload, not an attribution of all710failures yet.
Proposed bounded internal selection rule before implementation: keep the current mesh primary on the first fresh enrolled hint for backup peers; confirm enrolled selection only after one15ms backed-probe period without any intervening physical mesh arrival, including duplicates. Replies/copies continue with unchanged20ms deadlines; no packet is discarded or authenticated by hints. Legacy peers and lease retirement keep immediate return. Preserve source-IP/socket coherence while deferring a hint; capability/lease removal clears the pending selection. No timer/task, engine, dependency, authority, frame, version or capability bit added. Real return still replays every retained original once, never assumes an emitted backup was acknowledged. Additive contract clarification and deterministic lost-original/confirmation/reset/legacy regressions precede production selection; then fresh/retained1000, real cut/switch budgets and both sizes must be remeasured. This is a candidate, not an accepted fix.
Cancelling only verified owned unstarted after/storage waiters and the pending fast timing-prelude waiter because current transport prerequisites failed. No after lifecycle started, no failed sample is replaced; the partial fast transcript will be retained as interrupted before Rust/TS checks. Finalhead fast will rerun after the candidate is resolved. Foreign processes/locks unchanged.
Complete82bb suite retained: production ACK-only31/31PASS; MTU/dead-mesh/reliable-stream proofs1/1eachPASS; TCP-spacing1/1PASS. Scale remainsFAIL: fresh710guard exits/7proof failures/11448server drops; retained no-copy61exits/0proof failures/1826drops, copy948exits/0proof failures/11205cumulative drops; all1000connections open,clients0drops. No valid copy CPU/RSS acceptance. Raw failures retained.
Benchmark recipe correction before selecting a new routing heuristic: the batch did not request the prescribed CPUWeight10000 scope. A fresh owned NetBird command reports weight100; all ancestor cpu.max values are unlimited, cpuset0–3, throttling counters0. This rules out an observed cgroup quota, not scheduling contention or production overload. It does not establish that priority caused the failures. Repeating the unchanged frozen e4db… fixture under systemd-run --scope -p CPUWeight=10000, with observer-free fresh+retained controls and a separately labelled receive diagnostic. This explicitly labelled repeat does not replace the failed unscoped batch or reset any guards.
The15ms hint-confirmation idea is still only a proposal. It will not be selected based solely on a single-cause hypothesis. Six verified owned waiting flocks (2106168,2109871,2109888,2109905,2109922,2115380) were cancelled before workers started. No after/storage lifecycle ran. Partialfast1ce6061f2 exited143 at its cancelled timing prelude, before Rust/TS; transcript retained, not a failed production-code test or a passing final gate.
While the scoped measurement waits, rebuilding unchanged merged-dev804ac0e8 in an owned detached scratch checkout using the same locked x86 musl recipe and existing comparison target. This will replace the ceiling-only merged-base size comparison with actual artifact sizes; no baseline code change/deploy. Current branch remains1ce6061f2 with only generated Python cache untracked. All source commits pushed.
release-fresh-82bb-a.json
release-retained-82bb-a.json
release-scale-82bb-a.log
fast-1ce6061f.log
Matched merged-base size build completed at
804ac0e854(owned detached checkout, same locked release/musl/auth-toolchain recipe). Actual x86_64 artifact: 22,236,272 B, SHA-256 211e86882d7e00df7c3c50acf830ca467dfb41b37ab5300c19820dc99a4dcf13. Last committed runtime (82bb218a9, docs/ratchet through1ce6061f2): 22,072,496 B, SHA-256 c562cfebae70b7b25bd239df0c280dd1784d1b9c12e30234b7b42fbadda6d14f; actual delta -163,776 B. The previous -131,808 B comparison was against the merged-dev ceiling, not this rebuilt artifact. Build receipt: $HOME/.cache/codex-scratch/mesh-autoswitch/evidence/merged-base804-static-build.log.Routing candidate remains uncommitted/unselected: the first fresh enrolled backup no longer queues 64 retained Return entries in the causal regression (red [64,2048,64,64], prototype [64,2048,64,0]); no packet is withheld from QUIC. Deterministic confirmation/reset/legacy tests and unchanged replay/source/retirement assertions are being checked. Unchanged-code CPUWeight=10000 control remains queued behind the shared perf lock; no after-acceptance claim.
Receipts for the matched-base size and the unselected routing experiment:
The first complete prototype remote gate passed 62 tests / 6 ignored, plus strict all-target clippy and workspace formatting. I then extended the causal test to both wildcard and leased sockets, source-address coherence, and capability withdrawal; that revised gate is queued. Deterministic explicit-clock coverage checks first/early/confirmed hints, duplicates, intervening mesh ingress, and legacy/lease retirement. No perf acceptance or final fast gate is claimed.
Separating independent Storage diagnosis from the failed scale cohort: queued the complete namespace mesh-disable/restart proof at LAN/150ms using the already-frozen
82bb218a9server and shipping artifact (r13/release), in fresh ns14l/ns14w state and r14/storage-82bb-independent-a receipts. Both cases request their own CPUWeight=10000 scope, bounded 300s each, under one short fair perf lock. Existing exact-byte, disable, restart, latency and cleanup assertions remain unchanged. This is explicitly independent diagnosis, not a release/healthy-1000 acceptance claim. The uncommitted routing prototype is not in these artifacts.Prototype revised gates: 62 PASS / 6 normal ignored, workspace fmt and strict all-target clippy PASS. An initial packaging command used a nonexistent agent feature name and exited before building; corrected to the existing remote,mesh features. Optimized candidate fixture now queued behind build-slot; no source selected or percentile claim yet.
The unchanged-code high-priority control has now waited approximately 45 minutes for the shared perf lock. Current owner receipt at 2026-10-15T05:04:54+02:00:
Continuing to wait. No other job/process/lock touched. The independent Storage proof is also queued, in its own bounded run. Revised routing prototype remains uncommitted; no after acceptance claim. Matched candidate fixture is being built with all-features, matching the frozen control fingerprint; the remote,mesh-only compilation is not used as a matched performance candidate.
Unchanged runtime
82bb218a9control completed under the required CPUWeight=10000 scope on infraternal-perf-test, r14/scoped-control-1ce-a. All 1000 initial path proofs PASS in each profile, but steady health FAIL:Fresh load start 0.02/0.69/0.75, end 0.02/0.66/0.75. No accepted healthy-cohort CPU/RSS numbers. Thus correcting the missing scope does not make the earlier failures disappear; it is not a sufficient cause. The separately labeled observer-cost fresh diagnostic has 709 exits and 19,906 server drops; its counts are not mixed into observer-free results. The no-copy failure burst (27 exits window 7, 342 window 8) also needs explanation independently of backup replay.
Complete raw control + diagnostic receipts.
Frozen, unselected confirmation candidate is queued with the exact same all-features/release recipe and unchanged criteria, source
1ce6061f2+ patch SHA b1e394a0ff654e33436ad1915db7e4f3ab06681172ea73e8529fe6b2fef25589. Native fixture SHA 1668e350c00175e057e94a66bbfae0585dc3f07d83675ee5473df117eaded8d3; loader-only VM adaptation 352a66eceb853f0aa67e2a844f62919026f7e8324f73e71e032322f10bb6433e. Agent functional tests 114 PASS / 17 normal ignored / TCP physical timing test excluded for VM. First direct invocation used repository cwd and failed 12 relative fixture paths with ENOENT; rerun from the crate cwd with the owned short state root passed all 114, no code change. Plugin revised gate 62 PASS / 6 ignored, strict clippy/fmt PASS. Candidate is not yet selected, no release acceptance or final fast claim.Unselected confirmation candidate is frozen for reproducibility; no new commit beyond
1ce6061f2.804ac0e85. Ratchet unchanged; candidate FAILS it.Queued fair VM runs, all CPUWeight=10000, each bounded below 30 min: r14/run-confirm15.py (unchanged fresh/retained + separately labeled receive diagnostic), r14/run-plugin-confirm15.py (31 physical guard cases + MTU/dead-mesh/stream), r14/run-storage-independent.py (unchanged committed artifacts, complete LAN/WAN file-backed disable/restart). Each writes a fresh own output directory and retains failures. No source selection/size allowance/health relaxation.
Started scripts/verify iterate on this exact dirty patch (not final fast), log evidence/r14-candidate-iterate.log. Scratch r14/gate-bin wrappers acquire build-slot per cargo/bun command; their inherited flag prevents nested acquisition. The timing-proxy Python shim delegates the unchanged source SHA 119b87ef39ee7022c76dbc641a1428dd5eae7254265e6ddeea6697b56a4a8339 to a bounded CPUWeight=10000 scope on the perf VM, under the shared lock. Dispatcher waits outside build slots and keeps waiting after scheduling timeouts. No gate or artifact accepted merely from a lock wait.
Independent committed-runtime Storage lifecycle diagnosis completed on the perf VM, both exact-one-test invocations exit0 under CPUWeight=10000, private network/mount namespaces and 8 MiB memlock. No Storage error; file/WAL regression and original cancellation-safe immediate transaction retained.
Both keep the existing terminal assertions, including disabled-survives output. WAN cut→fallback status RPC completion is 353.633ms (not the echo boundary; do not present it as ≤200ms). LAN mesh/direct p99 echo 0.800/0.646ms is an internal single-lifecycle timing check, not 31-lifecycle acceptance. Two live nodes, one real PTY, 1000 public inventory entries; not 1000 daemons. Complete raw logs/reports/load/cleanup receipts: independent Storage runs.
Artifacts are the frozen committed
82bb218a9runtime, not the uncommitted routing candidate. These two passes do not establish all three historical failures' cause or repeated release acceptance. Candidate scale + guard runs remain queued; no final-fast/acceptance claim.Priority adjustment: queue an independent real-SSH standing-session latency series on the frozen committed
82bb218a9runtime (r13 server/payload) while the unselected routing experiment and scale diagnosis continue. 31 fresh add-host lifecycles each at LAN/150ms, identical Rust lifecycle/byte/writer/connection assertions and causal enrolled-ingress fence. This measures the actual requested switch target now; it cannot waive failed 1000-transport health or stand in for final-source acceptance. The original healthy-cohort prerequisite in r13/after-prerequisite.py is unchanged.Scratch orchestration r14/diag-standing-runner.py changes only outer provenance/venue scheduling from the frozen r13 runner: explicit independent_latency_diagnosis=true, release_acceptance_complete=false and failed scale prerequisite in metadata; CPUWeight=10000 scope per venue, 1500s hard measurement cap +30s cleanup, release between venues, retry only unacquired lock scheduling status75. It preserves every actual lifecycle failure and never overwrites an attempted sample. Output r14/actual-ssh/diag-standing82bb-31-a. Standing holds initial mesh until the native terminal exists, then records handshake→causal authenticated Output on the same writer/attachment; it does not claim natural first-byte or original-SSH-shell handoff.
Prepared but not run: bounded own-PID IP-only CPU diagnostic under r14/cpu-sampler. Functional parser/phase-anchor selftests PASS without opening clocks/events. If candidate scale still fails, use its exact frozen fixture SHA; sampler has no inheritance, memory/register/stack payload reads or permission changes. Added aggregate guest CPU/steal/pressure snapshots only during its observer window; all diagnostic observer cost, no accepted CPU claim. No further runtime source changes, committed head remains
1ce6061f2.Frozen original natural baseline progress: WAN indices0..26 completed; LAN extension indices0..3 completed (extension1 is retained readiness failure). Regrouped only never-started WAN27..30 and LAN-extension4 into r14/resume-baseline-tail.py, one bounded900s+30s-cleanup fair run, CPUWeight=10000. Exact original fixture/payload/runner/assertions; original
01dcbee9payload SHA verified. Metadata explicitly distinguishes the high-priority resumed tail from earlier original scheduling metadata; do not call the whole historical series scheduling-matched.Cancelled only owned waiting flocks2212069 (WAN27) and2213398 (LAN-extension4): exact original runner/label/index matched, no case directory, no child PID, and /proc/locks arrow confirmed waiting; opened pidfds, revalidated each, then sent SIGTERM via pidfd. The two original resume controllers consequently exit1 from that scheduling cancellation, not a measured lifecycle failure. No actual attempt removed or replaced; no other job touched. New batch queued, output r14/baseline-tail-a; canonical original per-venue summaries update only after each completed original case.
Working size refinement while immutable candidate-A VM artifacts remain queued: store its 15ms confirmation deadline once, instead of subtracting Instants on every enrolled hint. Same first/early/confirmed/duplicate/reset/legacy behavior and same per-peer field width; no additional timer/task, route or timeout change. Frozen b1e394a0 patch/artifacts remain unchanged and independently labeled. Measure actual size before retaining this representation; candidate-A performance is diagnostic, not final-source acceptance.
The ongoing iteration dispatcher has finished its release checks (3 Bun tests PASS) and is now waiting for the VM timing-proxy check, before any Rust source checks. Its later Rust checks will apply to the new working representation, not the frozen b1e394a0 patch; it remains an iteration gate, never final fast. To serialize any host work after the wait, cargo/bun gate shims and the size refinement build acquire the same private per-job mutex outside build-slot; at most one heavy command from this job can occupy a slot. Child invocations reuse their existing held slot. No host-wide slot settings changed.
Frozen confirmation candidate-A comparison has waited approximately45 minutes (still no candidate cohort directory/output). Current perf owner receipt at 2026-10-15T05:55:41+02:00:
Continuing to wait; no other job/process/lock touched. Plugin finite cases, committed-runtime standing31 series and regrouped original baseline tail are separately queued in finite fair runs. The successful independent Storage pair remains the only new complete lifecycle result in this wait window.
Working deadline representation patch SHA6dc382a021482311c29857541f1f0904e12b5b19f7067a80b07666a8e4e3f4f4, evidence/r14-route-deadline.patch. Its fmt/remote62test/clippy/x86-size command is queued behind build-slot and private per-job mutex (evidence/r14-deadline-gates-size-build.log). The ongoing iterate gate has passed release functional checks (Bun3PASS) and waits for the VM timing proxy before Rust; later Rust source checks apply to this working patch, not frozen candidate-A. No new commit, ratchet change, accepted candidate performance or final-fast claim.
Completed size-baseline cache cleanup: verified frozen
804ac0e85x86 artifact SHA211e86882d7e00df7c3c50acf830ca467dfb41b37ab5300c19820dc99a4dcf13 and retained committed/candidate shipping copies, then removed only this job's completed scratch r7-shipping target (no running build used it). Filesystem free46→63GiB. Main target, all frozen binaries, source baseline checkout and issue receipts retained. r7-shipping is no longer an available warm comparison target; do not resume commands against it expecting artifacts. No source/ceiling change.Candidate-A ELF attribution: .text +832B, .gcc_except_table +4B, NOBITS .relro_padding -832B vs committed runtime; actual file +832B. Deadline representation gates/build remain queued (patch6dc382a0). No candidate performance or final-fast result yet.
REJECTED confirmation routing prototype. Proper CPUWeight=10000 / unchanged health criteria / exact frozen b1e394a0 source, both observer-free suites exit101:
All profiles retain1000 live QUIC connections; clients0 UDP drops. Fresh initial load0.00/0.04/0.49. Separately labeled observer-cost fresh case883 exits,12,147 server drops; not mixed into observer-free results. Reducing replay hints did not establish healthy scale, and the new15ms state adds route latency and832B. No accepted cost/speedup claim from failed cohorts.
Its finite VM suite PASSES31/31 ACK-only guard cases plus MTU, genuine-dead-mesh and reliable-stream cases. This reinforces that single-peer proofs are not scale acceptance. Complete candidate cohort/finite-suite raw receipts.
Rejected deadline representation is also retained as source patch6dc382a0; no compilation/measurement claim. Cancelled only its own pending host build-slot process2668602 before any compiler/test began: verified exact command/cwd/private mutex, stopped the parent to prevent a spawn race, confirmed only sleep/flock children, then terminated those owned waiters via pidfds. Tool exit137 is scheduling cancellation, not a failed test. Restored both modified runtime files to committed1ce6061f2 under the private host-build mutex. Tracked worktree now clean; generated scripts/pycache remains untracked. No prototype commit or ceiling change.
Started bounded99Hz own-PID/IP-only profiling of the exact rejected fixture (same SHA352a66ec...), under CPUWeight=10000 and its internal fair lock, output r14/ip-confirm15-a. Captures aggregate guest CPU/steal/pressure snapshots during the observer only; no permission changes, inherited samples, payload/stack/register reads or load generator. Attribution only, not acceptance. It is queued. Real-SSH standing31 committed-runtime series and original baseline tail continue waiting. Ongoing iterate has completed release checks and waits before Rust; its subsequent Rust/TS checks now apply to restored1ce6061f2. Final fast remains required.
Next measured-work plan, after rejecting route confirmation: existing Peers::copies calls Recent::copies_to for every peer each global timer poll; each call reads Instant::now and retains/scans up to64 entries even when no copy is due. At1000 peers this permits64,000 entry visits and1000 clock reads per poll (static code bound, NOT a measured CPU attribution). Failed1000 controls and all negative prototypes remain recorded. No guard/deadline/protocol relaxation.
First logical step: consolidate the three existing Transmit/send branches into one, preserving explicitly bound/leased source selection and wildcard local-destination hints. Existing real UDP source/lease and authenticated stream tests cover it; run fmt, strict all-target clippy and touched tests before an atomic commit. No shipping size claim at this intermediate step.
Then prototype a per-peer cached earliest pending-entry index (one byte in existing peer padding, no new allocation/task/timer), invalidated through a mutable history accessor. Keep every original ciphertext/replay rule and the32-send budget. A shared poll timestamp can avoid per-peer clock reads; old per-entry deadlines still guard physical sends. Cold/invalid cache scans the existing bounded history; an unchanged future/empty queue can skip it. Actual effects require real UDP regression tests, the queued own-PID profile, matched VM micro/1000-transport proofs and both static sizes. Do not adopt solely from this complexity argument. Static release measurement/ratchet at the resulting milestone, final fast on final head. Existing engines unchanged.
Completed/pushed1b5e2a0aa: one physical Transmit/send construction preserves leased/bound source selection and wildcard local hints. Remote60PASS/6normal ignored, workspace fmt and strict all-target clippy PASS. Verbatim touched gate. No intermediate shipping-size claim; final milestone will measure both targets and retain the ratchet.
Original natural baseline tail completed5/5 original pending cases, no replacement. LAN combined36attempts:34 complete integrity witnesses,2 retained readiness failures. WAN31/31 complete integrity witnesses. Conditional percentiles among complete cases (nearest rank; failures have no finite Output latency):
These are unheld real-SSH native-terminal upper-bound witnesses, including fixture readiness/counter/fence/input+echo overhead, not earliest natural mesh-byte latency or original SSH-shell handoff. Old connection preservation is not claimed. Historical scheduling metadata retained; last5cases explicitly CPUWeight=10000. Complete summary/provenance/tail index receipts; original per-case receipts retained in their original VM paths, including failures. Existing controlled-standing baseline is separate.
Current uncommitted step only encapsulates read/mutable peer history access, preparing consistent cache invalidation; no cache yet. Its touched gate is queued. Full iterate still waits before Rust on the VM timing proxy; source changed since initial queue, so its eventual Rust/TS result belongs to the then-current source, not frozen rejected prototypes. Final fast remains required.
Committed-runtime controlled standing LAN series completed31/31 with exact terminal byte integrity and unchanged writer/attachment; every case has1 initial/final TLS, admission and control callback. No reconnect.
WG handshake→first causal standing Output p50/p99=9.839/174.140ms: LAN50ms target FAIL. Case0=174.140ms; second-largest case13=27.119ms. Retain case0; no warm-up exclusion. Cut→complete echo p50/p99=40.503/45.299ms (50ms target PASS); cut→fallback status RPC44.431/53.966ms, separate boundary. SSH start→Done2025.716/7148.499ms.
Current witness includes private handshake observation, local nft fence and input scheduling before the first Output. Cause of case0 is unestablished; do not subtract fixture overhead or assert the production path missed/achieved50ms from this upper bound. Next test-only diagnostic adds separately enabled timestamps for handshake observer completion, fence start/end, marker send start/end and existing Output. It preserves fence placement and byte/control assertions. No early blocking of both paths, packet-size inference, guard relaxation or new transport. Payload remains frozen82bb218a9. WAN series is queued.
Default-feature scripts/verify iterate completed exit0 (all6Rust rows PASS, TypeScript tests37PASS/0FAIL); complete verbatim iteration output. This is NOT final fast. Remote all-features history-access compilation caught3test-only chained field accesses omitted in the getter refactor; updated the2mutating and1read call sites and queued its full touched gate again. Not committed until that passes.
Original baseline last5raw receipts: tail raw.
Continuation status:
fdab0b8c7makes retained history private behind read/mutation accessors;f1279d502adds the real-UDP retained64 mutation/replay regression plus an ignored 1000×64 bookkeeping-only benchmark. Both pushed; touched fmt/strict all-target all-feature clippy and tests passed (61 passed, 7 ignored after test addition). No transport guard was relaxed.The scoped frozen82bb real-SSH standing WAN run stopped at case22, before mesh hold release: bootstrap succeeded (Done 9,973,607 us) but two ControlReady activations were observed. The first transport emitted a readiness terminal byte; a second source port activated control 835,240 us after the first, 16,066 us before Done. Cases0..21 completed; case22 remains a failed attempt,23..30 never ran. The existing one-control assertion is retained; source trace is in progress. This is not a completed31-sample WAN acceptance or an established mesh-switch regression. Raw partial run including failed stage.
An uncommitted copy-index prototype preserves bounded retained ciphertext/deadlines and scans only after history mutation. Functional tests61 pass/7 ignored; strict clippy/fmt pass. Its static x86 image is22,073,328 B:832 B OVER the committed22,072,496 ceiling, so it is not accepted. Exact source/artifact hashes. Matched no-send bookkeeping baseline/candidate and opt-in handshake-observer/fence/send stage diagnosis are queued on the fair perf lock, separately from release acceptance.
Profiler orchestration correction: the earlier nonblocking polling lock waiter timed out without launching a benchmark or collecting samples. It is not performance evidence. Replacement uses blocking flock, retries only unacquired status75, and retains actual test failures. No global perf permissions or host configuration changed.
Frozen82bb standing WAN partial numbers (nearest rank,22 completed of23 attempted; case22 failed before release and8 remain unstarted): WG handshake→causal standing Output p50/p99=157.725/170.297 ms; silent-cut→complete fallback echo=193.400/198.809 ms; separate status RPC=504.057/517.274 ms. Start→Done=9,992.794/10,481.913 ms. All22 completed cases preserve exact bytes, writer/attachment and initial/final TLS/admission/control counts. Conditional completed-case statistics do not turn this interrupted series into31-sample acceptance. Failed case22 and raw completed receipts remain in the partial archive.
Source finding to reproduce separately: remote_link.rs write_transport rejects a terminal Close after the peer Close has removed the native channel; embedded.rs Command::Detach forwards a Close and treats a failed write as host connection loss. A late native detach regression is drafted at scratch/r15/native-close-regression.rs (real mTLS QUIC, stale input rejection and control Ping/Pong). It is NOT applied/red yet and is not an established cause of WAN case22. The pinned protocol-tag fixture build is finishing first so its source identity cannot race the regression edit.
Cache candidate agent fixture is frozen/staged: exact native/VM identities. Current experimental source: patch20dd1e80, including opt-in stage/protocol observer and copy-index cache; shipping size acceptance is still pending/over ceiling as noted previously. Cached scale suite keeps unchanged guards and records every failure. Its own unstarted waiter2498740 was verified by exact argv, empty children and pending flock status, then cancelled with pidfd before requeueing with scope650s < outer timeout700s; no measurement ran or was discarded.
Shared disk dropped to25GB free. Preserved the completed legacy standing native fixture (SHA4e3db3d04b2de446b8a6e857a23b033c0d609adff6a865147a530e56da1dcbce), removed only this job's completed legacy target and rebuildable debug incremental directory, and recovered about12GB. Cleanup receipt. Frozen baseline/candidate binaries and evidence remain. No other job's files or processes were changed.
05:55 UTC: own-PID profiling measurement has waited >45min (queued05:09:55). Owner file:
process-per-user ppu-soak-r5lazy-soak 2026-10-15T05:42:41.886627+00:00; holder PID2403322, our profiler waiter2455938 remains pending with no child. Continuing blocking flock waits/retrying only unacquired75. No measurement has started or failed. Other pending own runs are matched copy-poll baseline/cached, LAN stage observation, cached healthy-scale checks and protocol-tag WAN diagnosis. No other holder was interrupted.Cached candidate native infraternal-agent functional gate:114 passed,0 failed,17 ignored,1 physical spacing test filtered for VM; full output. The controller protocol-tag fixture has strict all-target clippy and release build pass; source/artifact identity, VM27709b5393744d7c3f30979b12ccc1e078c0010e30f0f7baeaf8269e9faafa0d. It is pinned before further edits and is diagnostic only, using the old82bb shipping payload.
The late-detach regression is now applied in remote_runtime_tests.rs; its first functional run is queued via build-slot. Production detach code remains unchanged. The opt-in
ACCEPTANCE_DISABLE_LIFECYCLEreal-SSH fixture check is also applied after the already-timestamped fallback: disable mesh, assert durable enabled=0, and echo fresh bytes on the same writer before final control/TLS counts. No live result yet. HEAD remains pushedf1279d502. Final-source31-sample acceptance, exact size ceilings andverify fastremain outstanding.Pushed49d5e2217c5df8e23dfe0f0895b1f928e9df5315: native terminal detach is idempotent for a retired even writer and retires its channel even if the final Close encounters peer STOP_SENDING. Authority permits, control errors, odd-id refusal and stale-input refusal remain. The two real-mTLS QUIC regressions failed first: both red. This fixes a reproduced writer retirement path; it does not establish WAN bootstrap case22 causation.
Per-commit gates: fmt exit0; strict all-target clippy exit0; transport suite:
After naming/doc cleanup, both final regressions:
These are functional host gates, not performance numbers or final verify fast.
Perf-VM component baseline finally ran: CPUWeight10000 scope,4vCPU,load0.00/0.14/0.39→0.08/0.16/0.39. Production copy-poll bookkeeping with1000×64 retained histories and physical sends forbidden;31 samples/profile,10 polls/sample. Nearest-rank percentiles below are of each sample's mean poll cost, NOT individual-poll latency or transport acceptance:
Raw JSON,test log; fixed profile order and correlated samples, no isolated allocator/RSS claim. Peer inline size256B (container/heap excluded). This substantiates repeated full-history scanning cost; cached comparison and unchanged1000-transport health remain pending.
An additional uncommitted code-generation variant makes cold next_copy scans share one out-of-line body; touched plugin gates/static size build queued. It is not adopted until measured. The earlier cached candidate remained832B over its exact x86 ceiling. Final-source real-SSH31/venue, disable lifecycle, both size ceilings and full verify fast still outstanding. Profiler's first3600s lock wait expired before execution; only unacquired75 was retried, no measurement failure was suppressed.
Frozen pre-fix controller diagnostic (fdab runtime,82bb node payload, stage observer only) stopped at LAN case4:2 ControlReady callbacks before mesh hold release, versus1 at bootstrap Done. The second activation happened between Done and the first unrestricted terminal/echo precondition. Case4 is a failed attempt; cases5..30 never ran. Raw4-complete+failed-case archive,rounded-wall-stage summary.
The4 completed cases preserve bytes/control/writer. WG→first fenced Output6.7–28.6ms; counter observation1.3–3.4ms, local nft fence4.2–6.3ms, enqueue0.018–0.980ms, enqueue completion→Output0.84–21.24ms. This small interrupted diagnosis neither reproduces/explains the earlier174ms outlier nor supplies31-sample acceptance. No overhead was subtracted. This artifact predates49d5e2217 and cannot validate or invalidate its late-detach fix.
The current test source now writes
bootstrap-transport-failure.jsonat the failing post-echo assertion, reusing the bounded public observer snapshot; the existing assertion is unchanged. Future runners must collect that file as well as the Done snapshot, so a late second protocol activation cannot disappear from the receipt. The currently queued frozen protocol-tag WAN fixture still predates this new failure receipt.Uncommitted gate alignment: packet-spacing test has a physical100ms budget and now carries the same perf-only ignore metadata as the existing timing tests. Its threshold/body are unchanged; explicit VM execution is required before committing that metadata. No build-host timing will be used as performance evidence. The final fast workspace gate will retain functional host checks; physical timing proofs stay on the authorized VM.
Cached-A scale result is FAIL, not accepted cost evidence. Exact fixture73e3ee7f/source peer routesbc40f7fd; CPUWeight10000. Fresh:8/1000 proof failures,883 guard exits(751 ramp+132 warmup),12573 server UDP drops; client drops0. Retained no-copy:0 proof failures,1 steady exit,531 server drops; retained copy:7 proof failures,964 exits(774 ramp+190 warmup),9899 cumulative server drops. Diagnostic fresh:0 proof failures,979 exits(744+235),17347 server drops. Each keeps1000 QUIC connections live, but sampled guards shrink to109/999/29/21, so31 windows are not healthy1000-peer CPU/RSS evidence. All31 finite copy-first ACK guard cases plus MTU/dead-mesh/stream tests passed. Raw suite.
Protocol-tag real-SSH WAN diagnosis also stopped at case2(2 complete,1 failed): both callbacks are QuicControlReady,872142us apart; second activation318634us after the readiness Output and before Done. It uses the frozen pre49d5e2217 controller. Raw. Current test-only observers record both protected detach cleanup paths and a snapshot at the late assertion; no guard was weakened.
The committed82bb CPU diagnostic collected1263 own-PID user-IP samples,0 lost,130 in the conservative1.757s ramp envelope. Exact executable SHAe4db6c9e matched before offline symbol attribution. Ramp leaf groups:ConnectionDriver.poll14,RecvState.poll_socket7,Bindings.poll_recv4,Peers.get_mut3; no standalone copy-loop leaf in that small ramp subset. Full observation includes74 copy-loop leaves/1263 samples. These are leaf counts with inlining/thread-scan/observer limits, not timings, accepted CPU cost or causal proof. The benchmark returned101; sampler returned0. The global scheduler observer recorded17.76% steal among the first8/proc/stat counter deltas during the bounded ramp and17.22% across15.11s. This context does not waive the failed guards.
Raw own-PID/clock/scheduler receipts,offline symbols and scheduler deltas. No permissions, global sysctls or other job processes changed. Read-only VM configuration: rmem_default=rmem_max=wmem_default=wmem_max=212992 B.
New uncommitted bounded phasing candidate retains the immediate first probe and its initial authenticated-counter snapshot, then phases later existing ticks by the allocated UDP port. Its first interval is no longer than15ms; the15ms period andRTT+30ms response margin remain. No watcher reset or warmup exclusion. Earlier C functional gates61 plugin/114 infraternal-agent passed, with18 physical-only ignored tests; C2 rebuild preserves the initial coherent counter sampling explicitly. C was frozen before C2 and has not been measured/adopted.
The cold-scan code-generation variantB remains over the ratchet:22,073,584 B(+1088 vs committed ceiling,-162688 vs rebuilt dev804). Exact size/source hashes,61-test/clippy/static-build log. Its no-inline attribute was removed from the worktree; no ceiling was raised. Final healthy-scale and31-per-venue release acceptance remain outstanding.
Matched perf-VM copy-bookkeeping comparison completed in one CPUWeight10000 scope:1000 peers×64 histories;31 samples/profile×10 polls. Numbers are nearest-rank percentiles of ten-poll sample means, not per-packet/transport latency. Physical sends are forbidden. Fixed baseline→cached-B order; load3.94/2.19/2.49 for both.
Peer inline size stays256B; Recent stays40B, excluding container/heap/allocator. Standalone cached-A also completed: dormant2.667/6.770µs, future11.391/26.961µs, invalidated1202.100/1556.231µs at load0.34/1.15/2.24; that separate run is not a matched A-vs-B comparison. Raw comparisons,summary. Strong component benefit does not waive cached-A's failed transport health.
C2(phased later ticks, immediate first probe and coherent initial authenticated snapshot) now passes61 plugin/114 infraternal-agent functional tests plus strict all-target clippy and fmt;18 physical-only ignored tests, no filtered tests. Gate output. Frozen/staged unstripped release fixtureVM9574cc3e938c368ef0554b3ec115712a3548084300dc1e91928d5defb8c1f7cb; only package strip setting changed, optimization/debug-assertion settings unchanged. Exact source/artifact identity. Queued suite first runs the unchanged physical TCP-spacing test, then all three unmodified1000-transport cohorts; any ordinary failure remains a failed result and triggers a separate bounded IP diagnostic. No source adoption or release acceptance claimed yet.
Rechecked #189 through its latest7940d9f comments: original-session/private-image harness has explicit proofs but is still an unfinished pipeline with size/prompt budgets open. Current compatible real-SSH harness remains the measurement path here; original SSH-shell handoff is not claimed.
Starting Linux mesh issue survey in detached worktree at
5abc64bb8f(origin/dev snapshot); current origin/dev is4911b5911f. Read-only research; no code changes planned.Linux mesh issue survey completed; summary issue: #219. Full report: report.md.
Autoswitch findings relevant to #201: the reviewed branch keeps the enrolled logical QUIC address while mapping physical datagrams onto an applied mesh path, with bounded recent ciphertext replay. This is useful for preserving the server control session, but it does not add peer NAT traversal, Linux route/interface monitoring or a relay for mesh traffic. The latest #201 scale receipt remains FAIL: 1,000 QUIC pairs stay open, but sampled health guards fall to 109/999/29/21 across profiles; all 31 finite copy-first ACK guard cases pass, while healthy-scale and 31-per-venue acceptance remain outstanding. The 15 ms phased-probe candidate is unmeasured. Proposed follow-up: add route/link change recovery and direct/relay hysteresis, and require healthy 1,000-connection cohorts before accepting autoswitch scale/cost claims.
The attached report now includes the exact topic-query terms and pagination caveats. Latest version: report.md, attached to #219. This supersedes the earlier report attachment linked in my survey comment; the autoswitch findings and #201 acceptance status are unchanged.
Step
a2b46212: reserve only the physical TCP packet-spacing test for explicit perf-VM execution (D256); threshold/body unchanged. Exact source hash bbf2a7f1; fmt + strict touched-crate clippy pass, agent 114 passed /18 ignored. VM release test: 1 passed, tcp_proxy_spacing_us=20211 under CPUWeight=10000 and the shared lock. VM receipt, per-commit gates.C2 phased-probe experiment FAIL: fresh 9 proof failures, 900 guard exits (826 ramp/67 warm/7 steady); retained no-copy 767 steady exits; retained copy 6 proof failures/939 exits; independent fresh diagnostic 12 proof failures/961 exits. No health waiver or CPU/RSS acceptance. Summary, raw cohorts + exact-artifact IP diagnostic. Ramp IP profile: 149 samples, zero lost; ConnectionDriver 10, RecvState 8, writable poll 5, timer poll 5, copy drain 2. This does not establish one dominant cause. Separate receive diagnostic records 7,188 enrolled +5,921 mesh server socket drops by ramp end; gaps include three >65ms ready-to-next-poll gaps per socket. Observer overhead and VM scheduling remain diagnostic limitations, not excuses.
Frozen post-49d controller/runtime C2 pair now running independent actual-SSH standing-session switch/fallback plus mesh-disable checks (31 requested each LAN/150ms, original one-control assertion unchanged). Artifact/source hashes. x86 static payload 22,074,672 B = +2,176 over frozen ceiling, -161,600 vs matched dev804. Not adopted; no completion claim. This lifecycle run cannot waive failed scale or size gates, and does not claim original SSH-shell handoff.
Frozen post-49d controller + C2 node: 31/31 actual SSH installs, existing standing PTY preserved across mesh switch, silent cut and mesh-disable. One control activation per lifecycle, final TLS/admission/control counts unchanged, same writer/attachment, exact expected terminal sequence. No Storage failure in 31 durable disables. Raw LAN31 with source hashes/provenance.
No overhead subtracted. Done->handshake 5315.571/5543.047 ms includes the intentional initial mesh hold while opening the standing PTY; it is NOT natural add-host join latency. Fallback status is separately 47.251/56.810 ms; the byte-receipt target uses the measured echo boundary. This diagnostic cohort meets the LAN byte switch/fallback targets; it does not waive C2's failed 1000-transport health or +2176-B frozen ceiling excess. WAN31 is queued after releasing the venue lock. Original SSH setup-shell handoff is not claimed. Late native cleanup paths are covered red/green in 49d; these lifecycle successes alone do not prove which cleanup branch executed.
Working source now removes failed port phasing (frozen evidence retained). Measuring direct-deadline cache alternative and an opt-in repeated-ingress component benchmark; no interval/margin/authority change. Initial strict clippy rejected nested Option cache representation; replaced it with an explicit dirty bit + existing optional deadline. Full per-crate gates/release builds await shared build-slot. No final fast/completion claim.
Frozen post-49d+C2 actual SSH/standing cohort COMPLETE: 31 LAN +31 at150ms, all 62 byte-integrity/writer/control/disable checks pass. Expected marker shapes total 224 bytes per lifecycle (13,888 across62), verified by Echo's exact-prefix/equal-length checks; larger-stream and MTU regressions remain separate. No Storage failures in62 durable mesh disables; one control per lifecycle, TLS/admission totals unchanged.
WAN fallback p99 FAILS configured200ms target by3.695ms; do not round/waive it. WAN Start->Done10026.469/10447.900ms; Done->WG5083.473/5283.185 includes deliberate initial mesh hold, NOT natural joining. WAN fallback status349.537/513.352ms is separately labelled. Raw WAN31; LAN raw linked above. These results do not waive C2 scale/size failures; C2 port phasing is removed from working source. Original SSH-shell handoff not claimed.
Internal-copy-timing proposal before implementation: the fallback echo crosses BOTH endpoints, each scheduling its existing enrolled ciphertext copy after20ms. At150ms this consumes40ms of the50ms overhead allowance before PTY/crypto/clock work (consistent with193.7ms median; not a measured stage attribution yet). Measure a10ms per-entry copy-delay candidate, keeping one copy, existing32-send budget, per-entry clocks,15ms probe interval, RTT+30ms response guard, capabilities, identity and wire unchanged. No earlier item may send a younger copy before ITS new deadline. Run the finite copy-first/authenticated-guard matrix, ordinary1000-transport health, actualSSH31/venue and size gates; do not adopt on the latency result alone. Documentation/claimed profile changes only with measured accepted behavior. A shorter delay trades less headroom for the mesh original against more fallback headroom; physical send counts/health must stay explicit. No foundational engine or protocol choice.
Step
3cf1d3bc5(pushed): opt-in production receive-callback benchmark at1000 full histories. Baseline direct-deadline prototype1158.586ns p50/1348.652ns p99 per callback (31 ten-thousand-callback sample means; repeated known ciphertext, no physical IO/authentication). This is component evidence only. Fmt, strict remote/agent clippy, remote61pass/8ignored, agent114pass/18ignored. Gates.Direct-deadline D frozen (phasing removed): source/artifact hashes. x86static22073264B, +768 vs ceiling/-163008 vs matcheddev804. Peer inline256->272B, +16000B at1000 peers; Recent40B/Datagram80B unchanged; heap/allocator excluded. Matched component raw +31 finite authenticated guard passes:
D full health FAIL, not accepted CPU/RSS: fresh0 proof failures/913 guard exits (824ramp/73warm/16steady); retained no-copy0 proof failures/363exits(3warm/360steady); retained copy0 proof failures/980exits(890ramp/90warm). No-copy server3462drops by end; copy11801cumulative, clients0. All raw health failures retained. Read-only cgroup check: system.slice/user.slice cpu.max=max100000 and system.slice nr_throttled=0; no quota was changed. This disproves a possible fixed guest quota explanation, not the failure itself.
E prototype now follows the prior posted10ms-copy proposal, additionally arms dormant mesh replies only when coordinates first become known. New replies already arm at insertion; retirement clears coordinates and queued alternatives. Existing duplicate-coordinate/lease-retirement tests retained. Waiting for shared build-slot for all touched-crate gates, frozen release fixtures and size. No new protocol/authority/guard timing. No finalfast or completion claim.
Step
e5c55314: extend existing scripts/mesh-addhost-acceptance.py withcheck-standing --receipts DIR. Complete31+ raw cases/venue must match aggregates, exit0, preserve causal path/writer/control/TLS/admission proofs and pass disable; nearest-rank p99 checked against configuredRTT+50ms. This checks byte budgets/integrity only, never scale/size/natural-join acceptance. Python compile and workspacefmt pass. Functional controls reject a failed exit and a missing raw case; raw fixtures restored.Frozen20ms-copy release cohort RED specifically:
Unmodified real-cohort budget RED. Ten-ms candidate not yet measured/adopted; heavy gate/build waiting for shared build-slot.
#189 compatibility rechecked at7940d9f6c: original-session harness requires explicit --session-payload in ADDHOST_INSTANT mode and new bootstrap_instant_tests.rs early-terminal actor/bridge path absent on this branch; bootstrap implementation differs by917lines and live fixture310lines. Its latest20 original-session proofs pass but prompt/size gates still fail; no stable compatible drop-in harness exists here. Do not import the other lane's unmerged production changes or claim originalSSH-shell handoff. Current #201 runner reuses the disposable realSSH fixture approach and reports its controlled native standing PTY separately from natural joining.
Coordination for #220 on mesh-relay (base
c87fc0631): no change to PathKind, route_peer, ServerLink or the path selector. First pushed slice1127aa19dadds only bounded peer-socket candidate state in infraternal-mesh. Next pending slice adds optional endpoint: SocketAddr to Mesh public WireGuard Counters, decoded from WGPEER_A_ENDPOINT; missing endpoint remains compatible and multipart dumps preserve prior metadata. Endpoint presence is explicitly not fresh reachability/reflexive proof. It supplies discovery metadata for D196; #201 retains selection/observer leases and connection continuity. No new polling controller or second selector is being introduced. Will post the implementation SHA after the touched-plugin gate and a real source-port-translation/rebind proof. Switch-over marker remains absent; no workspace/static builds started.Additional bounded diagnostic on frozenD (agent VM SHA1bc0c544), CPUWeight10000/shared lock: benchmark101 (health failure retained), observer0,1072 own-PID snapshots. Conservative ramp sample subset spans1.782160s: process1.20 user CPU seconds/1.60 system CPU seconds. This is FAILED-cohort/observer-cost attribution, NOT accepted1000-transport CPU. Kernel work is substantial and absent from earlier user-IP-only profiles; no single user-space hot function was established.
Requested10ms observer cadence had a111.082ms maximum observed gap in that subset. A separate observer gap cannot prove the benchmark's timer delay or distinguish a VM pause from guest/runtime scheduling. Read-only sched_schedstats was0 after the run; raw per-thread counters nevertheless changed, so no completeness claim for runnable-wait attribution. Global counters have their own sample envelope and are not assigned to this process. No sysctl/permission/quota changed. Raw counters, benchmark failure and exact observer source.
Queued a separate own-PID/TID user+kernel IP-only profile at99Hz; no inheritance, registers, stack data or application memory, no permission changes. A refusal is retained, not worked around. Both pure packing/orchestrator self-tests pass. This is diagnosis, not a health waiver.
E candidate remains unbuilt in the shared host build-slot queue (own mutex held by waiting wrapper; all four occupied slots had active Rust build/test descendants at the read-only check, no nested own slot or stale-lock conclusion). Head
e5c553148pushed; E gates log remains empty. Continue automatically when capacity frees. No finalfast/completion claim.Kernel-inclusive own-PID IP diagnostic on frozenD:915 samples,0lost; conservative ramp329 (206kernel/123user). clear_page_erms64, _raw_spin_unlock_irqrestore27; counts are leaf samples, not durations/causal proof. Raw profile + exact helpers, sampled kernel-symbol attribution only. No inheritance, register/stack/application-memory capture or permission changes.
Pinned Quinn0.11.12 RecvState allocates max_udp_payloadmax_receive_segmentsBATCH_SIZE. Defaults14726432=3014656B/endpoint,3014656000B at1000 client endpoints (calculated virtual allocation, NOT RSS). Kernel THP setting isalways. Process-private diagnostic only, unchanged frozenD/guards/cohorts: default snapshot700872KiB RSS/573440KiB AnonHugePages,19proof failures/952guard exits; private THP-disabled175384KiB RSS/0huge,0proof failures/773guard exits. Both FAIL. No global setting changed, no production memory policy chosen. Both complete controls + exact source. More surviving guards generate more traffic: server drops10009default/26855disabled; do not claim better scale health from memory alone.
Proposal before code: reuse PathSocket's constructor for a fresh dedicated outgoing socket, disable Linux UDP_GRO only after the selected runtime wraps it, and advertise one receive segment ONLY when setsockopt succeeds. Preserve the existing reported capacity on failure/other platforms. Keep server GRO and all GSO sends. Use the already-selected safe nix sockopt API (add nix/net only to remote's agent feature, no new library/engine/unsafe code). This targets per-node receive scratch (1472132=47104B calculated), rather than choosing a process-wide memory policy. Keep actual physical-send accounting inside PathSocket via a test-only forwarding Runtime; no cohort reset or missing copy counters. A real32x1200-byte GSO test must preserve every datagram through single-packet receive buffers; the existing MTU/stream/finite-guard and1000-health tests remain required. Current host functional negative control: GRO-on delivers1472bytes with MSG_TRUNC into that proposed small buffer; this is NEW-budget feature coverage, not a claim old production's correctly sized buffers lost data. Benchmark memory/throughput before adopting.
E is still unbuilt after ~1h shared build-slot wait, so this planned socket candidate can be included before its first build with explicit source hashes. No acquired measurement is restarted/relabelled. Head
e5c553148remains pushed; no finalfast/completion claim.Measurement correctness finding: bootstrap_mesh_acceptance_tests.rs standing_first selected any TerminalByte after the fence boundary, unlike the natural witness's explicit stream2 filter. An earlier callback from another writer could understate the fresh Echo's latency. Published C2 switch p50/p99s (and older controlled-switch tables using that selector) are PROVISIONAL, not release acceptance. No evidence is discarded. Full224-byte Echo equality, actual fresh-marker mesh delivery, unchanged control/writer attachment, fallback echo durations and62 successful durable disables remain valid; those use the actual Echo receiver, not the first-event selector. WAN fallback203695us still fails200000us.
Prepared regression fresh_terminal_witness_excludes_another_writer: deterministic synthetic callback times, wrong writer4 earlier than expected2. Current WORKING helper deliberately retains the extracted old predicate for RED; do NOT commit that state. build-copy10-e.sh now calls fix-witness-after-red.py: require the exact4-versus2 test failure, retain external RED/source hash, add writer filter, reuse it for natural witness, then fmt/strict server clippy/probe tests. It refuses unexpected failures. New receipt records actual Echo writer and first-output writer; offline checker requires equality, so old C2 receipts lacking attribution cannot silently qualify.
Socket candidate prepared, not adopted: PathSocket::from_udp uses existing safe nix UDP_GRO API after wrapping a fresh socket and preserves reported capacity on failure. nix/net is added only to remote's agent feature (commit separately after gates). Dedicated single-connection clients report one receive segment on success; server GRO/GSO remain. CountedRuntime keeps actual copy sends below the adapter; scale JSON records client segment bounds. New real segmented32x1200-byte regression exercises both dedicated sockets with1472-byte receive buffers. Functional kernel controls, no timings: GRO-on truncates under the NEW small-buffer contract; GRO-off preserves all32 unique datagrams. This is new-budget coverage, not a claim old correctly sized production buffers lost data. Workspacefmt, diffcheck, metadata feature selection and Python checker syntax pass; Rust compile/clippy/tests remain pending.
Exact50MiB diagnostic now supports both client configurations and directions, preserving original3LAN/1at300ms samples and90s transfer cap. Small-sample tradeoff diagnosis only, no release p99. E watcher adds a separately bounded530s bulk scope, releasing between component/health/bulk runs. Retired only own unstarted artifact waiter649849 after exact-argv/no-children/no-artifact checks; no measurement was acquired/reset. New watcher continues automatically.
Host queue: E has waited since
08:16UTC (now10:02UTC); needs one available orchestrator build-slot. Own wrapper627093 remains a sleeping waiter, not a stale/nested slot; default capacity4. No bypass or changes to other jobs. Current pushed heade5c553148; uncommitted prototypes and gates are resumable via r15/build-copy10-e.sh and evidence/r15-copy10-e-gates-build.log. No finalfast/completion claim.Compatibility check for the corrected before/after plan: frozen dev node
804ac0e854(22,236,272 B; SHA256 211e86882d7e00df7c3c50acf830ca467dfb41b37ab5300c19820dc99a4dcf13) completed one real-SSH standing session per venue with the frozen C2 controller. LAN control/TLS admissions 1→3/1→3; 150 ms control/TLS 1→3/1→4. Both preserved the full 224-byte terminal sequence and completed mesh disable. This is compatibility evidence, not a latency cohort or a pure dev-at-both-ends baseline. New comparisons will use the same corrected controller for dev-node versus candidate-node. Switch timings remain provisional until the writer-specific witness regression passes.Raw compatibility receipts.
The replacement build-slot waiter is alive, still unacquired; the original unstarted wait log is retained. HEAD
e5c553148. No candidate artifacts or acceptance results yet.Build resource blocker, 2026-10-15 ~12:32 CEST: the candidate has not run a compiler check after waiting since ~10:16 CEST. Original unacquired waiter was replaced once at 12:24; replacement still waits through the unchanged four-slot helper. Read-only slot metadata: slot 1 switchover-release since 12:22:37; slot 2 unnamed since 11:45:00; slot 3 unnamed since 11:39:23; slot 4 research-nat since 11:40:12. No other job/process/slot changed. New source remains uncommitted pending its mandatory gates; no E artifacts or acceptance result exists. The bounded perf staging watcher waits for frozen artifacts. Continuing to wait; release of one ordinary host build slot is needed.
Resource blocker cleared ~12:43 CEST: candidate acquired ordinary build slot 4 through the unchanged helper. Remote-plugin strict all-features/all-targets Clippy passed; functional suite: test result: ok. 62 passed; 0 failed; 8 ignored; 0 measured; 0 filtered out; finished in 9.82s. The new dedicated-receive GSO/single-buffer preservation regression passed. Agent/server gates, writer-witness RED→GREEN and frozen release artifacts are next; no performance acceptance claimed from host tests.
Completed and pushed
a909736b3: first-byte witnesses now select the existing terminal attachment writer, and the offline standing checker requires the recorded writer IDs to agree. Natural and standing profiles share the selector.The deterministic regression failed first with another writer (left 4, expected 2), then passed with the filter. Observer tests: 2 passed; runtime tests: 47 passed, 5 ignored; public failure-receipt test: 1 passed. Strict server Clippy and fmt passed. RED log, RED source/command, GREEN gates.
Earlier controlled first-byte switch numbers remain provisional because their selector was not writer-qualified. Full echo integrity/fallback durations and completed disables are unaffected. Release candidate fixtures are building; no new latency, scale-health or size acceptance claimed yet.
Frozen E components/health on perf VM, fair bounded separate scopes, 4 vCPU, load 0.01–0.66 (raw load strings retained): all 31 independent finite authenticated copy-first ACK guard cases passed. Repeated known mesh ingress with 1000 × 64 retained histories: ingress-before: p50/p99 705.1826/965.8850 ns per callback sample mean (31 × 10000); repeated-ingress: p50/p99 100.0597/124.7847 ns per callback sample mean (31 × 10000). This is adapter callback cost, not terminal/physical IO.
Scale acceptance FAILED and is retained: fresh copy-enabled 0 proof failures, 586 ramp + 252 warmup + 0 steady interruptions = 838; server drops 21,305. Retained no-copy profile: 0 proof failures/interruptions/drops. Retained copy-enabled: 0 proof failures, 525 ramp + 155 warmup + 235 steady = 915; server drops 24,421. All 1000 client sockets report max_receive_segments=1, client drops 0. No accepted copy-enabled CPU/RSS cost is claimed. The receive-buffer change and repeated-ingress optimization did not solve copy-enabled scale health.
Telemetry caveat: the pre-existing cost report hard-codes copy_delay_ms=20; E source/manifest pins the real constant to 10. This stale descriptive field will be corrected before final evidence, without changing profiles, guard limits or failure accounting. No failed acquired run is retried as acceptance.
Components, failed health raw, compact counts, exact source/artifacts.
Bulk controls continue; shipping CLI/server fixture still building. Next diagnosis targets the copy-enabled shared server path and observed UDP drops; no guard widening or global kernel setting changes.
Scale follow-up proposal before code changes: Recent::copies_to leaves a successfully copied mesh Original in history with copy_at=None. A subsequent fresh enrolled ingress calls replay(), converting every retained Original into an immediate Return even though the backup already sent that exact ciphertext. Under route competition this can create a second burst up to 64 retained datagrams per peer (64,000 at 1000 peers). The E profiles show large enrolled-return histories and server drops. This is a source-based hypothesis, not yet causal proof of those failures.
Proposed minimal reuse: consume an Original after a successful scheduled enrolled copy through the existing retirement path, as already done for Return/MeshReply. Keep WouldBlock retries and retain originals on other send errors for explicit recovery; uncopied legacy originals stay unchanged. No new task, timer, queue, authority or guard change. Add a failing exact-byte 64-packet regression before the fix, then compare the frozen E fixture with the changed candidate under the same scale profiles. Current E artifacts remain frozen; no source mutation until its full manifest is published.
E bulk controls FAILED in both dedicated single-segment-client 300 ms RTT cases at the unchanged 90 s transfer bound. GRO-on controls completed exact 50 MiB in 74.086158 s host→server / 72.584960 s server→host (one WAN sample each); GRO-off produced three exact LAN samples per direction, but no successful WAN record. These are small-sample diagnostics, not p99 release results. The receive-buffer prototype cannot be accepted as-is. All four controls, including failures.
E full artifacts now frozen: x86 shipping CLI 22,073,456 B, +960 B over the existing 22,072,496 B ceiling, −162,816 B versus dev804 (22,236,272 B). Full source/artifact manifest. No acceptance claimed.
Completed-backup history regression reproduced before the fix: 64 successful enrolled copies were queued again by replay. The minimal retirement change passes the new exact-byte regression and the unsuccessful-copy recovery test; strict remote Clippy passed. The full suite then failed two old assertions specifically requiring completed-copy retention/replay. Those assertions now expect completed copies consumed while preserving all 63 uncopied legacy packets and the 32-send budget; the bounded-batch test still verifies every actual 4096-byte datagram. Recheck queued through build-slot. RED, RED source/command, first suite output including both obsolete assertions.
An additional bounded own-PID 99 Hz IP-only diagnostic retained a failing E cohort; sampler completed with zero lost records, without permission changes or application memory capture. No causal/acceptance claim from sampling. Raw diagnostic.
Completed and pushed
36f55b7e9: successful scheduled enrolled backups now consume their original through the existing retirement path, preventing a second recovery burst on a later enrolled route hint. WouldBlock retries, failed-copy originals and uncopied legacy recovery remain intact. No new queue, timer, selector API or connection. The uncommitted deadline cache, 10 ms copy-delay and socket-buffer prototypes are separate from this commit.Per-commit gates: fmt passed; strict remote all-features/all-targets Clippy passed; test result: ok. 64 passed; 0 failed; 8 ignored; 0 measured; 0 filtered out; finished in 9.91s. Functional GREEN, plus host-daemon integration recheck. New exact-byte 64-copy regression failed before the fix; failed-send/backpressure and existing budget regressions also pass after it.
A one-shot passive slot-release notification woke only this job's sleeping helper; the unchanged helper acquired ordinary slot 3. No other process/lock or slot pool changed. Release F fixtures are building under that slot. Then run 31 independent path guards and unchanged fresh/retained 1000-transport profiles against the frozen E failure. Scale health, bulk regression, size ceiling, real add-host cohorts and final fast gate remain open.
F scale result remains FAILED after completed-backup retirement. All 31 independent finite path guards passed. Fresh copy-enabled: 0 proof failures, 660 ramp + 6 warmup + 177 steady = 843 interruptions; server drops 33,706. Retained no-copy: 0 proof failures, 162 steady interruptions, 2,553 server drops. Retained copy-enabled: 0 proof failures, 621 ramp + 97 warmup + 151 steady = 869 interruptions; 25,085 server drops. Client drops 0. Load strings in raw: ~1.5–1.74, four vCPU. No healthy copy-enabled CPU/RSS result and no claim that retirement solved the scale bottleneck.
Finite guards, failed raw cohorts, counts, pinned source/artifacts.
A separate bounded, fair-lock VM inspection of newly owned UDP sockets found default SO_RCVBUF accounting 212,992 B. An ordinary per-socket request of 212,992 B or more yields 425,984 B; host rmem_max remains 212,992 B. These are capacity limits, not allocated RSS. No host setting changed. Capacity receipt.
Next, before production changes: an explicitly named ignored diagnostic test will compare the four combinations of current/restored client GRO and default/doubled owned shared-server receive capacity. It retains proof concurrency 8, the existing 15 ms/30 ms guard, all startup/warmup/steady failures, 1000 connections and the same 31 windows. Existing ordinary production-profile gates remain unchanged. This tests socket buffering as a cause of the observed drops and whether GRO can be restored after the failing bulk controls. No global setting, new engine/library, permission or guard change.
The four explicit socket trials all failed the unchanged fresh 1000-transport health criterion (full raw matrix retained): GRO off/default capacity 823 interruptions; off/doubled 718; on/doubled 872; on/default 791. Thus neither restoring aggregation nor doubling the owned receive capacity solves it. No production buffer change will be adopted from these results.
Next diagnostic proposal: measure 15/30, 20/25, 25/20 and 30/15 ms probe interval/response-margin combinations on the restored-GRO/default-socket profile. Each keeps the same worst nominal detection envelope of RTT + 45 ms, below the brief's RTT + 50 ms target; it trades more frequent challenges for a larger per-response scheduling margin. This is existing-engine configuration, not a new transport/engine choice. The original ordinary 15/30 scale gates remain unchanged and failed. Only the explicitly labelled ignored diagnostic will accept these parameters. No production cadence changes until measured; any chosen configuration still needs loss/authentication checks and the full 31+ per-venue real switch/fallback acceptance.
This tests probe packet pressure directly instead of raising global socket limits or silently forgiving failed guards.
Corrected-writer real-SSH smoke passed LAN and 150 ms with frozen F node/controller: writer 2→2; exact 224/224 bytes per venue; control 1→1; TLS/admission counts unchanged; durable mesh disable completed, no Storage failure. Single observations only (n=1 each, not p50/p99 acceptance): handshake→writer-qualified mesh Output 6.834/160.650 ms; silent cut→complete fallback echo 23.867/172.102 ms. These are controlled-standing runs with the intentional pre-join hold; no natural unheld-join claim. Full lifecycle smoke receipts.
The first smoke caught a harness omission: the prepared SSH runner had not enabled ACCEPTANCE_DISABLE_LIFECYCLE. Those two cases verified only 169 bytes and explicitly report mesh_disable_checked=false; they are retained separately and never counted as lifecycle acceptance. The replacement runner sets the lifecycle/stage flags and records them in provenance. Earlier incomplete smoke.
Socket matrix G completed; all four failed as already reported. Existing guards, 1000 fresh transports, concurrency 8 and 31 steady windows stayed fixed. All raw matrix failures, artifact metadata, exact uncommitted source patch against 36f55b7e9. Agent strict Clippy, fmt and 114 functional tests passed; 19 timing/diagnostic tests ignored on host.
The next diagnostic-only cadence matrix is queued through build-slot. Production is unchanged, ordinary 15/30 gates remain unchanged and failed; no socket-capacity change adopted. Final 31+ per-venue acceptance, scale health, bulk regression, exact static ceilings and scripts/verify fast remain open.
H diagnostic is still queued through the ordinary four-slot helper. Read-only process metadata rules out a nested build-slot wait in the current holders: slot 4 → bash → Cargo (compiler-wrapper children); slot 2 → bash → Cargo/build script; slot 1 → Python → Cargo/compiler wrapper; slot 3 → serve-loop.sh → Python/Bun plus sleep. The reserved priority slot is switchover-release and the priority list names only that job; untouched. One ordinary slot is occupied by a long-lived preview rather than a build/test command. No process/slot/helper change made. Own one-shot passive release notifier may wake only this job's sleep; the helper still acquires an ordinary shared lock.
A component fingerprint run was caught in queue without its owner-file preflight. Only its exact own pre-exec flock wrapper was retired (no benchmark child existed); its empty wait log is retained. The replacement used the fair lock, bounded scope, source/artifact validation and owner file. It passed: 31 × 20,000 fresh/duplicate pairs over 1000 peers, p50/p99 sample mean 716/1290 ns per pair; bounded fingerprint capacity 4,096,000 B. This is component cost, not terminal/transport acceptance, and does not indicate the dominant scale bottleneck.
Re-read #201 and latest #189. #189 now records
8b90da9ceand pending final acceptance; its original-shell harness still depends on bootstrap_instant_tests/EarlyTerminal and early admitted handover absent on this branch. The current real-SSH/native-PTY harness continues to make no original SSH shell handoff claim. No other lane production code merged.Fingerprint component receipt for the preceding finding: raw benchmark plus ownership/source/artifact metadata, exact measurement wrapper source. 31 × 20,000 pairs; 716/1290 ns per fresh+duplicate pair sample mean at p50/p99. The 128-bit hint fingerprint remains unchanged; this does not establish a dominant bottleneck or justify weakening it. H is still an unacquired ordinary build-slot waiter. No final acceptance/gate claim.
Separate source finding to reproduce before a fix: receive_peer normalizes a known alias to its logical enrolled address, then also stores that logical address in the generic leased-socket reply cache. try_send already uses the admitted Peers route while that alias exists. PeerBinding retirement removes Peers state but not this duplicate generic cache entry, so the next enrolled reply can still select the mesh-bound socket while the shared lease remains for other peers. This also adds a redundant map/mutex update on known fresh ingress.
Planned real-UDP regression: create an admitted logical/mesh alias, receive mesh ingress, retire only that peer while retaining the shared lease, then require the next logical/enrolled reply to come from the primary address. Also cover a generic cache entry learned before alias admission, and ensure unrelated unaliased mesh replies still use their leased source. If reproduced, make Peers the sole reply-route owner for known aliases and clear any generic logical entry at handoff. No selector API, packet authority, parser, queue or engine change. This is independent of the pending diagnostic cadence choices.
Refined the alias-cache test before implementation: a generic entry learned before alias admission can legitimately be the previous path (another leased listener), so clearing it would be wrong. Preserve that pre-alias memo. Known admitted aliases should update only their Peers route, leaving generic learning to unaliased ingress. Then retiring the alias naturally restores either the primary socket or its original generic leased route, without new stored state.
The real-UDP regression will cover both prior-path forms and verify that an unrelated/unaliased mesh reply still learns its leased source. This replaces the earlier tentative clear-at-handoff idea; no public contract/API change. H's cadence matrix has now failed all four candidates, so production 15/30 defaults remain unchanged. Buffer/GRO/cadence tuning will not be used to waive the outstanding scale failure.
Bulk-diagnostic clarification before new test code: transport_diagnostics.rs already documents that the application relay introduces artificial UDP burst loss. E's WAN timeout stats include large QUIC loss/reordering counts (e.g. host-output 12,539 marked lost of 46,349 sent). Those one-sample GRO-on/off outcomes remain failed receipts, but cannot isolate a receive-aggregation throughput effect from this relay.
Add a separate exact-50MiB control in an owned ephemeral Linux network namespace with kernel netem 150 ms each direction (300 ms RTT), three samples per client configuration/direction. Reuse the exact bulk helper, engine settings, flow-control bounds, crypto/MTU and 90 s per-transfer cap; retain the old relay test and its 3 LAN/1 WAN samples unchanged. The new test requires a namespace distinct from the caller before any network setup, and confirms the RTT. No host qdisc/sysctl/permissions change. Use bounded fair-lock scopes and release between configurations; source/artifact and namespace receipts retained. This is complementary diagnostic evidence, not a waiver of the failed scale gates or a release p99 result.
The alias-cache RED already reproduced the wrong alternative source after retirement. Its minimal fix preserves generic routing from before admission and stops known-alias updates from overwriting it; remote functional suite passed. Full J component artifacts are building before this separate test-only addition.
Completed atomic steps, head
bee9fd4a4(pushed):49a6d8776: reserve TCP and UDP together in the capability test fixture. Wider run first failed AddrInUse (113 pass/1 fail); corrected run: 114 pass, 19 ignored.4583482c5: known admitted aliases no longer overwrite the previous reply-route memo. Regression first returned the retired mesh source rather than the primary source; now both primary and previously leased original routes survive retirement. Remote tests: 65 pass, 8 ignored.2c36cc370: arm dormant replies on first mesh coordinates only, while refreshing coordinates on subsequent updates. VM component experiment, 31 samples of 10,000 callbacks: mean callback p50/p99 705.1826/965.8850 -> 100.0597/124.7847 ns. This excludes physical IO and authentication.bee9fd4a4: cache derived copy deadlines until queue mutation. VM component experiment, 31 samples of 10 polls: dormant p50/p99 1647.747/2556.158 -> 3.715/70.753 us/poll; future-deadline 2220.712/3094.357 -> 6.583/45.452; invalidated every poll 1678.488/2413.447 -> 1177.485/1692.906. Peer size 256 -> 272 B (+16,000 B at 1,000 peers); unchanged 32-send budget and round-robin cursor. These are component results, not a scale-health pass.Validation: fmt, strict remote/agent/server clippy; remote 65 pass, agent 114 pass, receipt observers 2 pass, server runtime 47 pass, public receipt 1 pass. No final full-gate claim.
RED retirement receipt: https://git.kayg.org/attachments/b279e4b2-aace-4d40-a1d9-bd2901354b33
Initial wider run: https://git.kayg.org/attachments/f438a8b2-be33-4de6-aa93-3d2e1c45a23e
Corrected functional gates: https://git.kayg.org/attachments/7fb3d695-6516-430e-bc9c-753c771d3f8f
Measurement clarification: earlier “observer-free” scale runs disabled optional ReceiveDiagnostic but retained global physical-send accounting atomics across all 1,000 connections. A paired accounting-on/off control is queued; its effect is not yet known. Production per-path counters/probes remain enabled in both arms. Original failed runs remain evidence. A separate owned-network-namespace kernel-latency bulk control is queued because the older userspace-relay bulk fixture explicitly documents artificial burst loss. Neither control relaxes acceptance guards.
Pushed
706f0ce26(test-only protocol/terminal-retirement observations) and28e021826(real-SSH fixture durable mesh-disable check, exact verified bytes and unadjusted stage/failure observations). Source matches the frozen J2 server validation: strict clippy; 47 runtime, 2 observer, 1 public receipt tests pass. Earlier corrected F smoke proves 224/224 bytes and durable disable once per venue, not 31-sample acceptance: https://git.kayg.org/attachments/d3c4e4c6-06f1-4a4b-afd1-f9c90f749beeScale findings (all diagnostics retain failed guards; no policy change adopted):
Kernel-latency bulk controls are running separately under short per-case locks. Remaining acceptance and final full gate are open.
Next bounded diagnostic, before any production change: scale failures persist without global send counters; they coincide with owned server-socket drops (18,098–41,197 in accounting-off cases) while clients report zero drops. Previous ordinary SO_RCVBUF requests could obtain only 425,984 B per socket under the unchanged VM cap. Measure a socket-local privileged receive-capacity control at requested 1/2 MiB (Linux accounts double), using existing nix RcvBufForce on only this test's two owned server sockets. Keep 1,000 real-mTLS paths, 15/30 probes, copies, ramp/warmup and 31 windows unchanged; label privilege and actual capacities. No sysctl, host networking, permission or resource-limit changes. This tests whether bounded burst absorption is a factor; a larger buffer is not yet a production remedy or a scale-pass claim. GRO bulk controls continue separately.
Completed/pushed dependency prerequisite
dd410157f, dedicated socket constructor/regression2b2607ec3, production outgoing-call-siteb2f98116e, bulk control16399fab4, and explicit scale diagnosticsd7f6e394a. No new library/version/engine. The constructor advertises one receive segment only after the kernel disables GRO; otherwise it preserves the underlying capacity. Server aggregation and send segmentation remain enabled.Allocation arithmetic from the existing endpoint: 1472 B x 32 batch x 64 segments = 3,014,656 B receive scratch, versus 47,104 B at one segment. Difference 2,967,552 B per outgoing host endpoint, 2,967,552,000 B at 1,000 endpoints. This is allocated scratch capacity, not an accepted healthy-cohort resident-memory result.
Kernel300ms bulk control: all four arms pass 3 exact 50MiB transfers, zero application missing/duplicated bytes, same 90s limit. Host->server GRO on 63.239396/63.295377/62.837671 s; off 62.282370/63.125002/62.458962. Server->host on 62.401897/63.115433/61.800310; off 63.060936/62.074112/61.702607. Three samples per arm, not a p99 claim. Each arm used its own ephemeral network namespace and short fair lock; host network/sysctls unchanged. Raw:
The older userspace-relay off-arm timeouts remain recorded, not waived as passes.
M privileged socket-local capacity diagnostic rejected as a production remedy. Requested capacity 0/1MiB/2MiB gave actual kernel accounting 212992/2097152/4194304 B per server socket. GRO-on proof failures 0/49/222, ramp/warmup/steady interruptions 664/299/0,615/2/0,282/0/0; drops 30545/305586/664861. GRO-off proof failures 0/117/390, interruptions 703/10/93,412/5/20,21/0/0; drops 34585/836287/1190094. All client drops zero; all six cohorts FAIL. Larger queues did not restore health; no production buffer change. Raw https://git.kayg.org/attachments/8af59ff2-85b8-4137-9472-427be707f026 ; manifest https://git.kayg.org/attachments/254a2d49-4f01-49a4-8bd1-e75334f75214 ; source https://git.kayg.org/attachments/b7351425-3e1f-454a-8639-fa2e19585d13
Gates: fmt/strict remote and agent clippy, 65 remote functional / 114 infraternal-agent functional pass. Exact segmented-send regression passes (32 x 1200B in single-datagram buffers). Full iteration gate is ongoing, candidate static/controller build ongoing, paired dev804-node real-SSH 31/venue baseline queued. Scale health, final size ratchets and final fast gate remain open.
Next controlled isolation: the ordinary retained no-copy arm uses 100/50ms probes, while the copy arm uses 15/30ms. It therefore does not isolate copying from probe cadence. Add an explicitly labelled no-copy 15/30ms diagnostic using the same 1,000 mutually authenticated transports, fresh post-TLS aliases, proof concurrency 8, ramp/warmup and all 31 health windows; leave ordinary gates unchanged. Compare with copy-enabled at the same cadence before another production policy change. No relaxed deadlines or survivor-only cost claim.
Current full-feature static candidate N sizes: x86 22,073,456 B, ARM 19,517,504 B. Existing branch exact caps are exceeded by 960 / 512 B; x86 remains -162,816 B versus rebuilt merged dev804 artifact. No cap change selected. After the frozen N controller completes, test a smaller representation of the receive override with identical kernel/transport policy, then remeasure both sizes. N's frozen controller is also queued for paired dev804-node 31-sample/venue real-SSH baseline. #189 reread: latest head still
8b90da9ce; its original-shell handoff integration remains absent from this branch, so our native-PTY fixture makes no original SSH shell handoff claim.Pushed
e0c4d776c(10ms individual backup deadlines at both ends, shared 32-send budget unchanged) and14ac5587d(author checklist updated for committed queue/route/receive invariants; final acceptance still explicitly open). Touched-code gates: fmt, strict remote clippy, 65 remote functional pass. Existing corrected F single-sample real-SSH smoke gave fallback 23.867ms LAN / 172.102ms at150ms and224/224 bytes through durable disable; not a31-sample acceptance claim. Earlier C2 fallback WAN p99 203.695ms motivates leaving more scheduling budget.Frozen full-feature candidate N source
e0c4d776c, source hashes on manifest: x86 22,073,456 B (SHA e878d4ef7dd08b7927f6589a0283c7096834404d0332dced27d36ed80714389d), ARM19,517,504 B (84ce59f336566ab3ff8e8f24def21053986a6dc0f3ebf451f4e23e200824e755). Deltas against earlier branch exactcaps +960/+512 B, so exact ratchet still open; x86 versus same-recipe mergeddev804 actual artifact -162,816 B. No cap changed. Current O representation experiment replaces a cached size with the confirmed one-segment override and reuses an owned close-on-exec descriptor clone; runtime aggregation policy stays the same. Functional/release gates queued after workspace tests.Manifest https://git.kayg.org/attachments/7fba0b25-b6de-4afa-a767-81b9dda2c4b1
Build https://git.kayg.org/attachments/2c90b8af-74c6-4648-8560-a8acfd4a7e8e
Paired node baseline is running: same pinned N receipt controller with unchanged mergeddev804 node, 31 real SSH installs per venue. It is explicitly a node-version comparison with an identical current controller, not an unchanged old full-stack baseline. So far24 LAN cases retain byte integrity. Corrected writer-specific witness and durable mesh-disable flag enabled. Standing profile intentionally holds initial mesh until its PTY exists; natural unheld profiles are separate queued measurements. Original SSH-shell handoff is not claimed (#189 still8b90da9ce). Current iteration gate passed fmt/clippy and is running workspace tests; no final fast result yet.
Paired node baseline LAN complete (31 independent real-SSH installs, dev804 node / identical pinned N controller; deliberate initial mesh hold / standing PTY profile):
Size selection before updating the exact ratchets: current owner continuation says “the size ratchet (delta vs your base ≤16 KB)”. I am using that task-specific budget as authorization for target-specific +960 B x86 / +512 B ARM over the earlier #201 exact caps, rather than allocating16KB of spare slack or changing D243 generally. Proposed final exact caps are the measured full-feature sizes22,073,456 /19,517,504 B; the +1-byte and smaller-unratcheted-build refusal tests stay unchanged. Both increases are below1KiB and within the brief's16KB limit. The x86 image is still162,816 B smaller than the same-recipe mergeddev804 actual artifact; ARM is121,240 B below origin/dev's ceiling (ceiling comparison, not a newly rebuilt ARM-base artifact).
The receive override removes2,967,552 B of allocated scratch per outgoing host endpoint by the existing engine's147232segments calculation (~2.97GB at1000), while preserving server GRO/GSO. Kernel300ms bulk controls passed all12 exact50MiB transfers with no byte errors under unchanged90s limits. Failed1000-health cohorts remain failed; this tradeoff is not a healthy-cohort CPU/RSS acceptance claim. The compact-state experiment saved no static bytes on either target; committed24b857327 solely for the simpler confirmed override / owned close-on-exec descriptor representation. Strict remote+agent clippy,65remote/114infraternal-agent tests, and workspace tests pass.
Final O source/artifact manifests and receipts are retained outside the checkout. Before/after actual-SSH31-sample standing and natural profiles are still running/queued; exact size matching and final verify fast will run at the branch head. No general decision edit or deployment.
Paired dev804-node baseline complete:31 independent actual-SSH installs per venue, same pinned N controller, controlled standing PTY / intentional initial mesh hold. All62 lifecycles pass byte integrity, qualified-writer witnesses and durable mesh disable;13888/13888 bytes total. Every baseline control count1->3 (connection preservation not claimed). No recorded operation failures. Nearest-rank p99=max of31; no outlier discarded.
Pushed24b857327 (confirmed receive-override representation, no measured file-size saving) and0efb50c9c (exact measured size caps within latest continuation's≤16KB delta budget; proposed on issue before editing). Current exact sizes22,073,456/19,517,504 B: +960/+512 over prior branch caps; x86-162,816 vs actual mergeddev804. No slack or general decision change. Exact/+1/downward-ratchet tests PASS both targets, unsupported target refused: https://git.kayg.org/attachments/5acf3d02-64ec-4901-b2a7-e1b6f234e486
O pinned source/artifact hashes https://git.kayg.org/attachments/8b11d590-ef37-43a8-a123-c76230969b13
Iteration gate passed (workspace tests, no load waiver); run includes evolving uncommitted O diagnostic sources and is not a final clean-head fast result: https://git.kayg.org/attachments/38a3ef53-d7fb-427a-b2fd-7f1e72b7f230
Verbatim final line: ITERATE gate passed. This is not the final gate: run scripts/verify fast before reporting done.
Candidate31/venue standing + natural before/after cohorts and same-cadence no-copy isolation are queued under fair perf locks. Another measured job owns the VM now; no lock or process interference. Remaining scale-health failure and final fast gate are open.
Same-cadence O control completed, all four cohorts FAIL with zero path-proof failures. GRO on: no-copy ramp/warmup/steady489/143/60, copy664/47/105; GRO off: no-copy467/100/134, copy659/20/74. Server drops17648/37982/17410/43581; client drops0. Native guards stay15/30, all31 windows retained. Global guest CPU context shows substantial steal, but that is context, not causal proof or a waiver. This establishes that copying alone is insufficient to explain the1000-transport health failure.
Raw https://git.kayg.org/attachments/10078b85-4625-43ec-b289-9e2030ac668a ; strict clippy/65remote/114infraternal-agent/build receipts https://git.kayg.org/attachments/b4db906a-3e1a-4127-a0df-9aac6942b64f
Pushedcc962fd8c adds the explicit same-cadence diagnostic; production defaults stay unchanged. Full fast started clean atcc962fd8c; candidate actual-SSH LAN31 nowpasses target: handshake->qualified Output9.127/35.810ms, cut->complete echo23.404/27.311ms,31byte-integrity checks/control1->1. WAN/natural cohorts continue.
Next diagnostic before any production policy change: the1000-pair fixture co-locates all endpoints on a four-worker Tokio runtime, while the shipped host daemon owns a current-thread reactor (crates/infraternal-agent/src/lib.rs:32; remote CLI main.rs:1446). Add an explicitly labelled current-thread / four-worker-default / four-worker-with-frequent-global-and-IO-checks control, using the same selected engine, same4vCPU VM, same1000TLS paths,15/30 guards, all31 windows and physical counters. No production runtime configuration or foundational engine choice; original failed cohorts remain recorded. This tests scheduling/topology sensitivity rather than relabeling a failure as a pass. The full-gate run will be treated as a milestone if diagnostic sources change during it; the final head still needs its own fast result.
Candidate O controlled standing LAN31 complete on fixed N controller: target PASS for qualified mesh Output and complete fallback echo. WG->first retained-writer mesh Output9.127/35.810ms p50/p99; cut->complete fallback echo23.404/27.311ms. Compare paired dev804-node baseline704.635/1151.133ms and269.345/697.888ms. All31 writer-qualified witnesses,6944/6944 bytes, durable mesh-disable checks and unchanged TLS/admission/control counts pass; control1->1 throughout. Status publication43.102/50.307ms is separate from byte-delivery acceptance. Intentional initial mesh hold; not natural joining or original SSH-shell handoff. Raw https://git.kayg.org/attachments/aa133be2-eda6-499b-b05f-40c1e827f15f ; summary https://git.kayg.org/attachments/7a88c856-b68a-465f-846b-7fc3985f6732
Pushedecfb544a7: ignored runtime scheduling control only; production settings unchanged. Strict all-feature/all-target agent clippy;114functional pass,21measurements ignored on build host; release fixture frozen SHA88b39739367e8fc5f6b6087f08fbd8990c9055c5e6d46f987ba3ac4a703b0832. Same1000paths/15-30guards/31windows/counters, no resources/sysctl/permission changes. Measurement queued. Original4-worker failures remain evidence and are not waived.
Full fast restarted from cleanecfb544a7 after committing that diagnostic. Supersededcc962fd8 gate parent was stopped before Rust checks; its already-queued bounded unchanged timing-proxy child is allowed to finish, then only that stopped parent will retire. No stale-head final-gate claim and no duplicate Rust workspace pass. Candidate WAN now22/31 with byte integrity intact; natural before/after series remain queued.
Full candidate standing acceptance PASS,31each venue, checked offline against every raw case and aggregate. WG->qualified mesh Output p50/p99 LAN9.127/35.810ms, WAN157.353/163.663ms; cut->complete fallback echo23.404/27.311 and173.534/188.602ms. All62 checks preserve13888/13888 bytes, writer2, one TLS/admission/control each, and durable mesh disable. Status is separately43.102/50.307ms LAN and492.537/496.551ms WAN, not the first-byte boundary. Controlled hold; natural joining remains separate.
Raw https://git.kayg.org/attachments/20ffa59e-9b18-44ce-b100-8d28daad4610
Summary https://git.kayg.org/attachments/2e7eec9c-e59e-4373-b4ab-1ec77853878a
Verbatim byte-budget checker https://git.kayg.org/attachments/068ba29c-d0ce-49e2-8d28-870cbd7a859e
P runtime controls all FAIL. Current-thread no-copy proof0, ramp/warmup/steady757/54/33; copy proof8,850/10/4. Four-default no-copy0,557/203/140; copy0,478/193/156. Four-frequent-global/IO-checks no-copy0,425/82/137; copy0,576/114/112. Server drops18115/152163/17168/26871/13623/45228; client drops0. Even the current-thread no-copy case failed with about0.93% global guest steal, so high aggregate steal is not a sufficient explanation. This is context, not an accepted CPU-cost result or causal attribution.
Raw https://git.kayg.org/attachments/dac94c90-4c1a-4808-ad7f-5c59863f5fb9
Manifest https://git.kayg.org/attachments/af0e6343-4269-400f-b909-1cc1f5999859
Gates https://git.kayg.org/attachments/9e68d61b-a187-40ef-9b11-c1e8883a3362
Next explicit diagnostic: separate server reactor ownership from the co-located client generator. Use one owned current-thread server reactor and three client workers, still four total workers on the same4vCPU VM, same1000TLS paths/15-30guards/31windows/counters and socket capacities. Register both server sockets inside that reactor, not just move its accepting future. Compare shared-four-worker and split-default/split-client-fair variants. Ordinary controls and production remain unchanged; no foundational engine or resource increase. This isolates role contention, not a waiver of original failures. The helper and its bounded ownership will be gated under build-slot before measurement. Natural before/after cohorts continue, final head gate remains required after any test-only source change.
S separated-reactor controls also FAIL, all zero path-proof failures. Shared-four no-copy ramp/warmup/steady488/77/49, copy646/10/60; split server1+client3 no-copy318/195/28, copy476/161/27; split with frequent client queue/IO checks no-copy405/68/109, copy441/161/79. Server drops13941/34128/11471/39349/8558/40412, clients0. No healthy CPU/RSS claim or production runtime change.
Raw https://git.kayg.org/attachments/9a60b177-acb2-4056-9e1b-f8a3638c6bff
Manifest https://git.kayg.org/attachments/bbc9581e-c62b-4d59-bab1-844883b95a05
Gates https://git.kayg.org/attachments/1cb76042-d60a-47ab-989c-9e1f52e16938
Pushed84ef07449 after fmt/strict clippy/114functional passes. Full gate Ec milestone exited0, all Rust rows and TypeScript pass; final clean-head gate still required. The superseded84ef gate parent is stopped; its existing child may finish before parent retirement, avoiding duplicate remaining work.
One further isolation before selecting a production change: no-copy direct-mesh TLS handshakes with no server peer aliases, compared with the existing alias-enabled no-copy profiles under shared-four and split1+3 runtimes. Same engine,1000mTLS paths, selected GRO-off clients, default server buffers,15/30guards,31windows and physical counters. The direct control starts on the mesh during TLS so it adds no address-migration validation delay; it cannot establish switch acceptance or replace the alias-enabled release gate. It isolates ongoing peer-alias/history bookkeeping. Record direct-handshake/alias flags and keep all failures. No copying/authority/production policy or resource change; no foundational engine selected. Candidate natural series is still running; standing byte budgets are already accepted separately.
Continuation head
84f2d1c3(pushed): direct mesh TLS/no-alias diagnostic, unchanged production O. All four 1000-transport 15/30ms guard cohorts FAIL: shared aliases/direct ramp+warmup+steady interruptions 608/432; split aliases/direct 581/547. No healthy CPU/RSS claim and no guard relaxation. Strict agent clippy, fmt, 114 tests pass, 21 ignored.The
ecfb544amilestone full scripts/verify fast exited 0; raw transcript attached. A later 84ef workspace run failed session_gone::an_ended_or_unknown_session_gets_the_final_close_without_losing_the_socket (423 server tests passed, 1 failed): tests.rs:1731 unexpected WebSocket Close(None). This is not claimed as the #118 timeout exemption; focused rerun queued. Final new-head gate still required.Natural actual-SSH/node-O acceptance: candidate 62/62 successful, 10726/10726 bytes, unchanged TLS/admission/control counts, 62 durable mesh disables. Dev804 natural baseline had 3/31 LAN install/readiness failures (indices 6,12,24), so only 28 successful LAN latency samples; all failures retained and 8 supplemental attempts queued. WAN baseline 31/31 successful. Natural WAN candidate Done-to-WG p50/p99 4780.185/15054.081ms remains a fast-add-host concern; investigating with bounded header-only UDP observations. Natural post-success output is an upper bound including attach/fence overhead, not the earliest mesh byte or original SSH-shell handoff. Standing writer-qualified 31-per-venue acceptance remains PASS, with published receipts.
r15-direct-t-matrix.tar.gz
direct-t-source.patch
r15-direct-t-gates-build.log
final-ecfb544a-fast.log
Head
4cbee0239pushed; author checklist now distinguishes accepted standing delivery, natural conditional timings and failed scale controls. Production O binary unchanged.Natural unheld actual SSH install (same fixed N controller, dev804 vs O node; native PTY, not original SSH shell). Milliseconds p50/p99; baseline LAN n=28 successful out of31 attempts with3 readiness failures; every other cell n=31 successful/31 attempts. Supplemental baseline LAN cohort pending; failed attempts stay in denominator.
Candidate all62 successes:10726 expected/10726 verified bytes,62 writer-qualified witnesses,62 durable mesh disables, TLS/admission/control1→1. Baseline59 successful lifecycles:10207/10207bytes; three failed operations have no terminal witness. Baseline control3→3 at terminal-observation boundaries does not establish preservation through bootstrap.
Natural Output upper bounds include readiness/link waiting, terminal attach, initial echo, counter queries, firewall fence and fresh input/echo. No earliest-natural-byte or SSH-shell-handoff claim. Negative Done→WG means handshake preceded Done (candidate LAN31/31,WAN5/31). WAN pre-handshake tails remain open; a bounded UDP-header-only diagnostic is queued.
The attached 84ef gate log retains the unexpected Close(None) failure; no timeout waiver claimed. A new clean-head full gate is queued.
r15-dev804-natural31-a.tar.gz
r15-candidate-o-natural31-a.tar.gz
r15-natural-before-after31-summary.json
final-84ef0744-fast.log
Natural baseline supplemental LAN cohort completed:7 successes/8 attempts,1 additional install/readiness failure (index1). Combined baseline LAN35 successes/39 attempts,4 failures retained (10.26%); WAN31/31. Candidate31/31 each venue. Every successful lifecycle still has exact byte integrity, writer/path witness and durable mesh disable. Combined baseline LAN expected/verified6055/6055bytes, WAN5363/5363; candidate5363/5363 per venue.
Updated conditional baseline LAN ms p50/p99 (n35, never dropping failures from attempt denominator):Start→Done1704.981/2306.432; signedDone→WG−17.759/−2.670; Done→fencedOutput upperbound112.813/168.412; WG→fencedOutput upperbound136.270/205.765; Start→fencedOutput upperbound1815.857/2442.760; cut→completeEcho234.182/283.250. Other table cells unchanged.
The exact unchanged debug server test binary that failed the workspace run passes the focused session_gone functional rerun (1passed,0failed). No timing claim or exception inferred from this; cargo-focused and full-head fast checks remain queued via build-slot. Header-only natural LAN diagnostic passed its integrity checks; WAN diagnostic pending.
r18-dev804-natural-lan-supplement8-a.tar.gz
r18-natural-complete-summary.json
Natural header-only diagnostic O/N completed both venues with integrity. WAN public UDP lengths show crossed 148-byte initiations and92-byte responses at1792081275.311–.536, no completed node handshake until retry148 at1792081280.473792 and response92 at.549322; kernel handshake1792081280.549497,4.738831s afterDone. Inference: simultaneous first-handshake collision, not a5s application metadata polling timer (coordination is event-driven).
Next diagnostic: fresh disposable-node input rule drops only server-sourced UDP length156 (148-byte payload) before existing admission, leaving node initiation and92-byte responses available. Capture public UDP headers,3samples each venue on the pinned unchanged O/N pair. This intentionally asymmetric fixture is not ordinary acceptance or a production remedy; it tests the collision explanation before considering any production sequencing change. No host networking settings changed.
Pinned O/N header diagnostic confirms the WAN5s gap precedes the completed kernel handshake. Initial crossed148-byte initiations and92-byte responses are followed by a~5s retry. Suppressing only server-sourced148-byte UDP payloads at the disposable node (excluding enrolled QUIC source4930) gives WAN Done→WG−249.015/−264.725/−285.203ms, all3 beforeDone; but LAN gives+4946.983/−23.926/+4990.970ms. All6 retain byte integrity and durable disable. This supports first-handshake ordering/collision as a cause, while showing asymmetric suppression is not a safe general remedy; rejected for production. No production protocol or firewall change made. Initial a fixture failed before measurement due nft closing-brace syntax; receipt retained with corrected b cohort.
Focused cargo invocation using --all-features failed at build setup because that includes staging-trust and requires a per-run public-key file. No credentials inspected or flags weakened. Exact existing workspace binary focused rerun already passed; final scripts/verify fast uses ordinary workspace feature selection. Full current-head gate remains queued at fmt behind normal build-slot users; proxy self-test passed on perf VM.
r18-natural-udp-stage1-a.tar.gz
r18-session-gone-existing-binary.log
r18-asymmetric-handshake3-ab.tar.gz
r18-asymmetric-handshake3-summary.json
r18-asymmetric-handshake.log
r18-session-gone-focused.log
Final-head gate resource block (
4cbee0239): scripts/verify fast is waiting at fmt through the unchanged normal build-slot helper; proxy self-test passed on perf VM. All four normal slots are occupied (slot2 since17:08,slot4 since17:38,slot3 since17:55,slot1 latest18:23 CEST). No reserved-priority spoof, other-process termination or check bypass. Needed from orchestrator: a normal build slot becoming available; waiter remains active. The earlier passing ecfb milestone and failed84ef workspace transcript remain attached, not substituted for the final head. Production acceptance and diagnostic measurements are attached above.Re-read latest #189 comments: addhost-instant advanced from8b90 to pushed
b3fe6181e4. Its final10+10 production-path receipts now prove same early SSH session/writer/native painted rows and59acknowledged bytes. Final fast and image/ARM/prompt targets remain open there.Reviewed git object crates/infraternal-server/src/bootstrap_instant_tests.rs atb3fe: EarlyTerminal attaches writer2 before handover and verifies commands both sides. That production early-shell bridge is absent from this branch; SSH/agent changes span51files versus our head. Its receipts cannot establish mesh provenance or qualify our O binary, and importing only that test cannot add missing production handover. Current #201 receipts therefore retain native-PTY scope explicitly. A combined original-SSH-shell→mesh acceptance requires #189 production integration and a rebuilt combined controller/node pair; not claimed from either independent batch. No other branch changed or merged.
Normal build-slot became available; current-head fast gate passed fmt, VT native fmt, CLI plugin compile-out (25tests) and strict workspace clippy, and is running the full workspace tests. No priority-slot or helper modification. The queued fmt row includes969s host slot waiting; that transcript duration is not performance evidence. One passive notification woke only our own20s sleeper after a real normal-slot release; unchanged build-slot acquired slot4 for the test command. Head remains4cbee0239.
Current full fast is RED: remote_runtime::tests::renewal_checks::renewal_refuses_same_issuer_wrong_key_purpose_or_lifetime_before_publication failed atremote_renewal_checks.rs:169, valid matching leaf publication returned PermissionDenied;423server tests passed,1failed,46ignored. Prior session_gone failure passes this full run. No #118 waiver applies. Identical compiled renewal test passes focused rerun(1passed,0failed). Neither renewal fixture nor production validator differs fromdev804.
The acceptance fixture creates notBefore at a freshly sampled exact wall-clock second, then OpenSSL validates against its own wall-clock read. Suspected second-boundary fixture sensitivity; exact cause is not yet proven. Test-only correction: put the fixture epoch one second in the past in both renewal cases, retaining the exact seven-day durations, strictly newer expiry, all wrong-key/purpose/lifetime refusals, duplicate-publication refusal and the future-notBefore case(+60s remains59s future). Production trust/time validation remains unchanged. Existing full-gate failure is the red receipt before this test-fixture change; rerun strict touched-crate gates and final full fast afterward.
Functional build-host clock coherence reproducer observed SystemTime-style fine clock1792083419 before AND after C time(NULL)=1792083418. No elapsed/latency benchmark claimed. Local linked source openssl-src300.6.1+3.6.3 crypto/x509/x509_vfy.c uses default verification time viaossl_x509_check_cert_time→X509_cmp_time→X509_time_adj_ex→time(&t). Thus a fresh notBefore at fine-clock epoch can be one second future to the verifier. This confirms the independent clock mismatch; original failure did not record an OpenSSL error code, so its attribution remains an inference.
The test-only1s past epoch preserves all authority/key/purpose/exact-duration/strictly-newer-expiry refusals and positive publication checks. Future-notBefore stays59s ahead. Production validator unchanged. Touched server fmt/clippy/renewal tests are waiting in the normal build pool; final full gate will rerun after the atomic fixture commit. Red receipts and focused original-binary pass are attached; no #118 exemption.
final-84f2d1c3-fast.log
r19-renewal-existing-binary.log
r19-c-clock-coherence.json
Documentation step pushed
de7674aa0: author checklist describes proof boundaries and keeps the measured standing verdict on #201 (D262). cargo fmt --all -- --check and git diff --check pass. No production change.One uncommitted test-only correction remains inremote_renewal_checks.rs (both fixture epochs now1s past, negative checks unchanged). Required fmt/strict server clippy/2renewal tests have waited in normal build-slot since~18:54CEST; no normal slot release was observed during the first bounded20min passive notification window. Waiter remains active; no priority authorization is listed for mesh-autoswitch and no priority spoof/bypass occurred. After those pass: atomic fixture commit+push, current-head full scripts/verify fast, attach full transcript, update head-to-O artifact attestation, then stop for review with natural-handshake/scale/original-SSH-shell limits explicit. Earlier full red transcript retained; do not claim final acceptance complete.
Test-fixture correction pushed
93590f0329: both renewal epochs start1s in the past. fmt and strict server clippy(remote,alltargets)PASS; both renewal regressionsPASS(2passed,0failed). All existing negative cases and production validator unchanged; static implementation remains pinned O.The leading Terminated line in the small gate log is the notification waking only this job's20s waiting sleep after a real normal-slot release; the actual gate command exited0. No test/compiler process was terminated. Full scripts/verify fast now running on this clean committed source head; no further edits planned during it. Original red/full and focused-pass receipts plus clock-coherence/source evidence remain attached above.
r19-renewal-fixture-gates.log
Final93590 fast remains queued in preflight. Correction to passive notifier wording: the earlier scratch watcher matched anyslot-*.who release, including the reserved slot. The latest event was reserved-slot turnover, not normal capacity. It only woke our own20s sleep; unchanged build-slot still permits this job only normal1–4, so no priority access or gate bypass occurred. Scratch watcher now filters exactlyslot-1.who throughslot-4.who and records the released slot. Production/repository source unchanged.
Current93590f032 native portion of scripts/verify fast PASS, including both prior failure cases, strict workspace clippy, all workspace tests, cargo-deny, both exact static size ratchets and x86 key-heap runtime. ARM key-heap runtime remains explicitly not executed on this x86 host. Verbatim native summary:
This is not yet a full fast pass: TypeScript compiler is waiting in the normal build pool. Source remains committed/pushed93590f032. The attempted lightweight-wrapper scheduling adjustment was aborted by the process ownership guard when it detected that the gate had already advanced; no gate/test process was killed and scratch wrappers are unchanged.
r19-93590f03-native-milestone.log
Cleanup preparation: preserved the owned historicalr6/baseline instrumentation from01dcbee94064ee15b35d295943a93589c96b268a before any removal. Bundle contains its full tracked-source patch, all3untracked test/runner sources and a SHA256 manifest; archive SHA256a0b16f5f09e1759ac1ed1495ae6bb27cb70ccd05b941ab5582dc98763ed2a5b5. This is historical instrumentation, not the current N/O acceptance source. No temporary worktree or active build target removed yet. Current93590 native gatePASS; TypeScript stage remains queued.
r19-historical-r6-baseline-source.tar.gz
Resource block on final93590f032 fast: native rows allPASS and fresh static artifacts exactly match measured O; TypeScript compiler has waited since~20:02CEST in the unchanged normal build-slot helper. All normal slots remain occupied. Needed from orchestration: normal build capacity for the remaining TypeScript checks. Waiter stays active, with passive notifications restricted to normal1–4; no reserved-slot spoof, other-process interruption or check bypass. Full fast is not yet claimedPASS. Code remains pushed93590f032, no source edits during this run.
Review checkpoint at pushed
93590f0329; no deployment. Complete scripts/verify fast exited0 on this unchanged source head. Standing QUIC delivery budgets are met; full release acceptance remains incomplete because natural first-byte/original-SSH-shell integration and healthy 1000-transport qualification remain open.Perf venue: designated VM 208, 4 vCPU/7 GB, owned namespaces and shared fair lock. Current O node equals both final static binaries byte-for-byte; pinned N controller held identical in before/after node-version comparisons. Source/artifact/load receipts are attached above. p99 is nearest-rank maximum for 31 samples.
Targets 50/200 ms pass for byte delivery. Fallback status publication is separate: candidate p99 LAN 50.307 ms / WAN 496.551 ms, not claimed within the byte budget. Initial mesh joining is intentionally held in the standing profile until the enrolled terminal exists; this is not natural unheld joining.
Natural real-SSH conditional success timings (ms p50/p99). Baseline LAN 35 successes/39 attempts, 4 readiness failures retained; baseline WAN 31/31. Candidate 31/31 each venue, no attempt discarded. Supplemental 8 baseline LAN attempts are additional fresh operations, not replacements for failures.
Negative Done→WG means handshake preceded success. Natural fenced Output includes readiness/link wait, terminal setup, echo, counter queries, firewall fence and fresh marker roundtrip; it is neither the earliest natural mesh byte nor the original SSH shell's handoff. The observed LAN Start→Done median is higher in the candidate cohort; separate cohorts/load windows do not isolate its cause. The WAN 5–15 s post-success pre-handshake tail remains open. Header diagnostics show crossed first initiations/responses and
5 s retry. Diagnostic asymmetric suppression removed WAN delay in 3/3 but caused5 s LAN delays in 2/3, so rejected for production. #189 b3fe early-shell bridge requires combined-source integration before it can qualify this node/controller pair.Session integrity: candidate standing 62/62, 13,888/13,888 bytes; natural 62/62, 10,726/10,726 bytes. Same native attachment and writer 2; TLS/admission/control 1→1 in every candidate lifecycle, no lost/duplicated bytes. All 124 durable mesh-disable checks pass with no Storage failures; cancellation-safe transaction fix and file/WAL regressions retained. Baseline controls differ and do not establish connection preservation.
Size: final full-feature x86_64 = 22,073,456 B vs rebuilt dev804 = 22,236,272 B, delta −162,816 B. ARM = 19,517,504 B; ARM's actual dev804 binary was not rebuilt here. Exact shipping caps pass both targets, including+1 rejection/downward ratchet; current artifacts are+960/+512 B vs prior branch ceilings, within the brief's 16 KiB allowance. ARM is 121,240 B below dev's recorded ceiling, which is not substituted for an actual rebuilt baseline.
Scale limitation: all preserved fast 15/30 ms 1000-transport diagnostic cohorts remain FAIL, including no-copy, different existing runtime ownership and direct mesh TLS without server aliases. These are 1000 loopback transport pairs, not 1000 daemons/kernel mesh nodes. No accepted healthy CPU/RSS claim or guard relaxation. Measurements and rejected controls remain attached. This is a review checkpoint, not complete release acceptance.
Verbatim final gate summary (full unmodified transcript attached):
final-93590f03-fast.log
r19-head-93590-to-measured-o-attestation.json
Standing before/after receipts: before, after, budget check. Natural complete conditional summary: receipts summary.
Cleanup complete after full gate exited 0: all 3 owned temporary worktrees and the private Cargo target removed; historical instrumentation preserved in the attached source bundle. Frozen measured binaries and issue receipts retained. Main worktree clean.
r19-cleanup.json
Independent review started: review-mesh-autoswitch at
93590f0329; dev merge base8e7467c07c. Read #201 history and author report; no linked parent. Reviewing correctness, byte continuity/access retirement, size attribution, production switch-over compatibility and matched scale controls. No implementation changes or pushes.Independent review step R1 (P3), committed
a719cf3dda: contracts/wire.md ordinary encrypted backup deadline says20ms twice; production COPY_DELAY is10ms. Failing test scripts/mesh-contract-review.test.ts: expected10, received20 (0pass/1fail). Raw receipt: mesh-autoswitch-review/evidence/contract-delay-red.log. Implementation unchanged. Build/scale/fast checks remain queued for normal host capacity.Independent static step: verified frozen804 SHA211e8688 and O SHA79003d13; 22236272→22073456B (-162816). ELF deltas: .text -152768, .rodata -8448, .gcc_except_table -5012, .eh_frame +888, .eh_frame_hdr +1528, .data.rel.ro +928, .data +32, .relr.dyn +16; file layout +20B reconciles exact total. Both unwind sections retained/grow; panic=unwind unchanged. Enabled dependency/features match
fa904fdaa(normalized cargo tree); release signature verification code untouched except cancellation-safe SQLite begin. Main source reductions: shared TCP/QUIC daemon adapter, agent orchestration opt-level=s, one public snapshot slice-deserializer family. Raw section/feature receipts: mesh-autoswitch-review/evidence/{size-elf-attribution.json,feature-difference.txt}. No feature-drop finding.Integration check: merge-base with current origin/dev is804ac0e85;
fa904fdaais absent. git merge-tree --write-tree93590f032fa904fdaa refuses content conflicts in contracts/wire.md and apps/tracer/src/canvas/status.test.ts. No integration edits made; no claim of measured gains on the combined source. Rust review/baseline builds and full gate still await normal slots.Independent kernel/PTy lifecycle checks on perf VM208 PASS, exact-one-test each atLAN/150ms under shared lock and CPUWeight10000. Expected/received2435/2435 and11619/11619bytes; switch, silent-cut fallback, same-session restart, durable mesh-disable echo complete; state retired after both cases. Frozen N controller/current O node, hashes recorded; no combined fa904 qualification claimed. Raw summary: https://git.kayg.org/attachments/f6ee82f3-9f86-42f3-8107-52797a8da445 . Size receipt: https://git.kayg.org/attachments/2f5a9b0b-0882-4813-9fd8-b6ab84a73c7e . R1 red: https://git.kayg.org/attachments/b58eb0ae-bba7-4942-8e05-c7011eabaa2f .
Fresh candidate standing31/venue series started;4LAN cases complete so far. Exact production adaptive1000 control is building fromfa904fdaa. Serialized own shared-target work by cancelling only verified unacquired fast/example waiters; no compiler/test stopped. Full fast will follow the review fixture build, on an isolated unchanged93590 source checkout.
Independent review continuation at
a719cf3d(implementation remains93590f032). Fresh LAN standing31/31 PASS: WG→qualified Output p50/p99=13.329/31.034ms; cut→complete echo24.295/30.737ms;6944/6944bytes;31durable disables. WAN underway.Build orchestration: owned requests were waiting in shared sccache RPCs; isolated job-owned UDS + explicit local-disk config completed a cacheable Rust probe. Cancelled only own Cargo descendants, retained old logs, and restarted both source builds in fresh target-isolated/baseline-target-isolated directories through normal build-slot. RUSTC_WRAPPER caching remains enabled; shared cache/config/processes untouched. Fresh targets seed only existing checksum-pinned toolchain/native caches, not workspace Rust artifacts. Both pending source tests and raw controls remain unqualified until actual runs.
Independent D128 flood control committed
fd25eb48: real mTLS/bound writer42,4MiB each direction,64mesh/enrolled route selections, exact EOF bytes and physical mesh arrival. Focused functional host test PASS1 (8388608payload bytes); not throughput evidence. The frozen VM counterpart remains queued.Strict remote clippy caught a test-only65536-byte stack buffer in the separate raw-invalid-UDP test; changed it to heap Vec and restarted scoped gates. No implementation change or production finding from that lint failure. Productionfa904 control binary now built/staged; all independent1000 controls wait behind the shared VM measurement lock. Full fast still outstanding.
Independent R2 P2 committed
7ceb46f7d5: real enrolled mTLS progress plus invalid32-byte packets from the configured mesh tuple keep path_interrupted alive, although that tuple supplies no valid QUIC response. Exact1test on VM208/4CPU fails intended final assertion with Err(Elapsed(())); both connections remain open, enrolled authenticated counters advance. This affects path-health attribution, not app authorization. Test review_invalid_mesh_packets_cannot_borrow_enrolled_authentication; evidence mesh-autoswitch-review/evidence/invalid-mesh-red.log. The red binary predates only test-buffer stack→heap lint cleanup; no assertion/production change; final-source rerun queued.Independent verification resumed on review-mesh-autoswitch at
7ceb46f7d(target93590f032; dev merge base8e7467c07). Read all 232 comments on #201; no linked parent. R1/P3 and R2/P2 tests and fd25 flood control preserved. Recovering matched fa904 production scale controls, standing receipts, D242 checks and final fast gate on unchanged target. No implementation changes, push or deploy. Prior gate log was empty; new invocation queued through normal build-slot.Continuation evidence recovered: independent standing 31/venue PASS, 13,888/13,888 bytes and 62 durable disables. LAN switch p50/p99 13.329/31.034ms, fallback 24.295/30.737ms; WAN 157.668/164.416ms and 173.486/186.374ms. Same writer, TLS/admission/control unchanged; controlled initial mesh hold, not natural join or original SSH-shell handoff. Receipts: https://git.kayg.org/attachments/47324768-57ea-4bb1-aeed-17fbd64d06ed .
Matched direct-path 1000-pair detector control, exact production
fa904fdaaversus target: production adaptive 269 guard exits; target adaptive 26; target fast 804. All retain 1000 connections. Separate low-load windows on 4-vCPU perf VM; these establish failures, not reliable speedup or healthy-cohort cost. Summary: https://git.kayg.org/attachments/4404a6de-d3c0-4a42-af73-4ccd9f2eb615 . Preserving a normal ignored regression for scale health; current-source compile/rerun queued. Natural pre-handshake tail also exists on dev804 (WAN p99 31.771s versus target 15.054s per retained real-SSH receipts); neither limitation is silently accepted.Independent flood control
fd25eb483now PASS on perf VM: exact one test, 8,388,608 payload bytes across 64 path selections, unchanged bound writer and live mTLS connections. R2 raw-input red attached: https://git.kayg.org/attachments/06c535f7-7298-404f-8e07-68375fa78fc0 . Final-source R2 rerun, D242 established-flow proof and unchanged target fast gate still pending normal build capacity. No implementation edits or pushes.Independent review capacity checkpoint: unchanged-target fast gate queued since 21:56 CEST; focused review/example build queued since 21:58. All four normal slots remain occupied. Public process-name inspection at 22:05: slot1 runs a preview loop; slot2 Cargo/sccache; slot3 NetBird; slot4 Cargo/sccache. No rustc process was present in these descendant snapshots. No foreign process, lock, helper or cache was changed. Needed from orchestrator/owning jobs: a normal build slot release. Waiting continues; no gate result claimed. Seven authority/listener controls pass on the perf VM: https://git.kayg.org/attachments/06f46a82-7b5e-4390-8aea-7627dea86c29 . Independent standing raw checks pass62/62: https://git.kayg.org/attachments/e0e0756b-f00f-4609-a0f2-9fa034510cfb . Transport/flood/scale raw: https://git.kayg.org/attachments/f22d88a7-1096-441d-a322-0052f3d918fa .
Independent R3/P2 committed
a843a518: final review test review_fast_mesh_health_at_design_scale fails its intended healthy-path assertion, 829 exits with 1000 TLS connections open. Same short locked perf-VM window: candidate adaptive 0 exits (PASS); exactfa904fdaaproduction adaptive 68 (FAIL). Direct loopback transport pairs, four workers, 31 windows, startup failures retained; no 1000-daemon or accepted healthy-cost claim. Strict remote Clippy and compilation PASS. This is scale-qualification debt, not evidence of a new P1 or application byte loss; ordinary production already fails the topology. R2 final-source rerun also RED as expected; flood control PASS again.New independent native latency gate on shipping O at 300ms RTT: baseline echo p99 331.64ms; other-session flood 307.69ms; same-session flood392.67ms (334ms ceiling), growth61.04ms (16.67ms ceiling): FAIL. No source attribution or waiver. Exact rebuilt dev804 artifact SHA211e8688 copied into our own perf directory and queued against the same unchanged gate. Earlier path-length setup failure retained separately; shorter owned state path is the only orchestration correction.
Raw final-source scale/R2/flood/native receipts: final-controls.tar.gz
Full target fast and kernel established-flow fixture builds remain queued for normal host slots. No implementation changes or pushes.
Native latency comparison complete at 300ms RTT,31 samples/condition on the perf VM. Same unchanged gate, rebuilt dev804 SHA211e8688 versus candidate O SHA79003d13: same-session flood echo p99 478.13ms before /392.67ms after; same-session growth158.12ms /61.04ms against16.67ms. Both FAIL334ms echo ceiling. Baseline/other-session echo p99 320.01/314.63ms before and331.64/307.69ms after: PASS. Separate low-load windows; no causal speedup, new regression, green latency gate or #118 exemption claimed. Existing same-session budget failure remains debt; no new P1 established by this comparison. Raw baseline native-latency-baseline.tar.gz . Candidate raw is in the preceding final-controls attachment. Final review crate functional66PASS/11ignored; strict ClippyPASS. Full unchanged-target fast and kernel established-flow fixture builds still queued for normal slots. Passive notifier touches only our helper sleep processes after actual normal-slot turnover; no reserved slot or shared-helper change.
Explicit production integration check: actual git rebase
fa904fdaain a disposable detached93590 checkout FAILED at1ecc332b0 (Prepare leased peer routes before publishing mesh configuration), conflict crates/plugins/mesh/src/node_coordination.rs. This is additional to merge-tree wire/status-test conflicts. Aborted and removed only that scratch checkout; review branch and job branch untouched. No claim of measured gains on combined source. Kernel public-state fixture is built and staged; coordinator example compiling under normal slot1. Full target fast still waiting for normal capacity.rebase-fa904.log
rebase-status.txt
Independent D242 kernel access-cut matrix PASS12/12: RuleRemoval, DefaultTightening, PeerRetirement; nft and nf_tables; LAN and150ms RTT. Three live kernel nodes,1000 installed public peers per node, existing IPv4/IPv6 TCP streams plus UDP:96 receiver checks. No newly sent disallowed traffic after each receiving kernel cut. Change-to-last-refusal min134.893ms/max428.575ms, single case per combination, not fleet-scale convergence or percentile acceptance. Public-state/WAL fixture and ordinary coordinator from current review source, owned namespaces, separate short fair perf locks, CPUWeight10000. Raw case summaries retain calibrated RTT and load.
kernel-access.tar.gz
kernel-access-summary.json
Full unchanged93590 fast gate is now running its Clippy phase. Review head remains
a843a518b; implementation unchanged.Independently recomputed natural-join raw receipts, preserving every failed operation. Dev804 baseline LAN35/39 successes (4 failures), WAN31/31; candidate31/31 each venue. All successful cases retain exact bytes, writer2 and durable disable; candidate TLS/admission/control counts unchanged. WAN Done-to-WG p50/p99:4.978/31.771s before,4.780/15.054s after. These are raw-receipt checks of the retained cohorts, not a fresh natural cohort or earliest-byte proof. The pre-handshake delay remains open; available comparison does not establish a new P1. Exact fa904 integration and original SSH-shell handoff remain unqualified as already recorded.
natural-raw-independent-check.json
Full target fast has passed workspace tests and cargo-deny; x86 static build underway.
Independent R4/P2 committed
b4a9a5207: fresh unchanged93590 full-feature x86 musl artifact22,077,808B, cap22,073,456B, delta+4352. New real-artifact TypeScript regression fails at expected size assertion; ARM case pending ordinary build. scripts/verify fast therefore has a RED static result, despite the published green head receipt. No compiler/profile/RUSTFLAGS/CFLAGS overrides were present. Production trust, static linkage and native key-heap check pass. ELF section-size delta is entirely.rodata(+4352); text/unwind lengths unchanged. Public compiler/build-path metadata differs; full causal attribution remains open. D205 size alone is not a P1; no cap or implementation changed. Compared with retained rebuilt dev804 artifact, this fresh image is158,464B smaller; the earlier162,816B reduction applies to the verified frozen O image.rebuilt-artifact-red.log
rebuilt-size-difference.json
size-compiler-metadata.json
Full gate continues through ARM; untouched TypeScript suite will be run separately if the Rust static failure prevents its dispatch. Review counts now3P2,1P3, no establishedP0/P1.
Verdict: merge — MERGE (no P0/P1). Counts: 0 P0 / 0 P1 / 3 P2 / 1 P3. Reviewed target93590f03297997370f3672c2bef4e632bc6a183c; evidence branch review-mesh-autoswitch headb4a9a5207448dd2030048e9d1c2bf657527c840e, clean. Five atomic local commits; no implementation fix, push, deploy or issue closure.
Confirmed regressions: R1/P3 contract20ms versus shipping10ms; R2/P2 unexpected mesh-tuple UDP borrows enrolled authenticated progress; R3/P2 fast health fails at1000 transports; R4/P2 fresh x86 static build exceeds its recorded cap. Each has a committed failing Rust/TypeScript test. Findings, complete test patch.
Independent perf-VM evidence: controlled standing31/venue, switch p99 LAN/WAN31.034/164.416ms, fallback30.737/186.374ms;62 preserved writers,13888/13888bytes and62 durable disables. Separate flood control preserves8388608bytes through64path selections. D242 kernel matrix12/12PASS,96receiver checks across IPv4/IPv6 TCP/UDP, both backends and LAN/150ms:134.893–428.575ms change-to-last-refusal. Three live nodes,1000 installed public peers each; not1000live nodes. Raw receipts are attached in preceding comments.
Limits remain explicit. Same-window1000-pair controls: production fa904 adaptive68 exits; candidate adaptive0; candidate fast829, all1000connections open. No accepted fast scale CPU/RSS cost. Natural raw WAN Done-to-WG p99 is31.771s before/15.054s after; earliest natural byte and original SSH-shell handoff remain unproved. Native same-session latency also FAILS on both rebuilt dev804(478.13ms) and candidate(392.67ms),334ms ceiling. These comparisons do not establish a new P1 or waive a failed budget.
Size attribution: verified frozen matched pair22236272→22073456B(-162816), principally shared daemon instantiation, compact host orchestration and public slice-parser reuse. Enabled feature sets retained; unwind sections retained, panic=unwind, signature verification unchanged apart from cancellation-safe storage. Fresh target x8622077808B(+4352cap), ARM19517504B(exactcap). Fresh x86 variation is entirely.rodata; text/unwind section lengths match the frozen image. Full causal attribution remains open(R4).
Production integration FAILS: actual scratch rebase onto
fa904fdaastops at1ecc332b0 in crates/plugins/mesh/src/node_coordination.rs; merge-tree also conflicts in wire.md/status.test.ts. Aborted/removed the scratch checkout. Gains on combined source are unverified; integration needs its own gates.Full scripts/verify fast on unchanged target exits1. Verbatim summary:
Static sizes verbatim:
Static linkage, production trust and native x86 key-heap checks pass. ARM key-heap runtime was not executed on this x86 host. Separate untouched TypeScript compiler/UI/build/transfer/protocol gate exits0; protocol tail verbatim:
Full fast transcript, TS transcript, artifact hashes. Leading Terminated lines only woke this review's waiting helper sleeps; no test/compiler was stopped during the gate.
Seven owned temporary worktree/build/cache directories removed; frozen artifacts and receipts retained. Cleanup. P2/P3 fixes go forward under the owner verdict rule; the full gate remains red.
R5 started on mesh-autoswitch at
93590f0329; merge base804ac0e854; fetched origin/dev40cc9c5be. Read #201 history (242 comments, no linked parent), review findings and contracts. Will rebase preserving dev switch-over, import/fix R1–R4 regressions, then measure combined-source LAN/WAN, design-scale health and natural WG delay. Earlier native same-session latency fails on both base and candidate; no green claim. No merge/deploy.Rebased 137 commits onto dev40cc9c5be; new head76a91b99b includes all four review regressions (R2 imported without the separate unrequested flood fixture). Resolved node_coordination by retaining compact-access negotiation plus route-before-configuration ordering; retained dev route-liveness contract intact. R1 proposal: document measured shipping10ms individual copy delay (not20ms), preserving the existing LAN/WAN backed-path data behavior. No frame/version/authority change. R2 confirmed source cause: connection-wide counters cannot authenticate a physical tuple. Working on path-specific evidence before claiming a fix. All gains require combined-source remeasurement.
R2 proposed additive transport-internal path proof (D215/D263): a fixed bounded request/response outside QUIC framing, authenticated with HMAC-SHA256 using a direction/domain-separated TLS-exported key from the existing enrolled QUIC session. No new engine, credential, connection, stream, writer or admission. Server enables responses only for an admitted live connection's leased exact tuples; response uses the receiving socket and source address, never a learned arbitrary destination. Client accepts only a matching outstanding monotonic challenge, valid response MAC and exact probed tuple. Replies/requests cannot substitute for each other, old/foreign/invalid responses do not advance health; component/lease retirement cancels responder state. Existing encrypted data and backups remain unchanged. Older peers that cannot prove the alternative keep enrolled service rather than allowing ambiguous physical+global-counter proof. Reuse existing OpenSSL/HMAC and zeroize, with the TLS exporter buffer cleared immediately after key import. This addresses per-path authentication without a second TLS connection or suspending backup byte delivery. Tests will cover invalid input, replay, wrong direction/tuple/session and lifecycle retirement before release measurements.
R1 fixed/pushed a0e2ecec7: contract now consistently10ms; imported review test observed RED(expected10/received20), then PASS1/0. Rebase completed on40cc9c5be with dev switch-over behavior retained; branch force-with-lease push contains reviewed source plus all four regression commits. R2 production prototype now uses fixed45-byte session-exporter HMAC challenges, exact physical response tuple and monotonically increasing outstanding sequence; repeated requests keep their original sequence/clock. Enrolled encrypted data/backup paths remain intact. New key/direction/replay/cancellation unit controls are prepared, not yet claimed passing. R4 prototype remaps Rust/C locations and strips only generated native compiler argv metadata, preserving native platform/date fields; native wrapper/metadata controls pass. No size or performance claim until rebuilt. Heavy native fixture build remains queued in normal build-slot; all4 normal slots occupied. No reserved access or helper changes.
R4 normalization step36532fc09 pushed: Rust/C source remapping, fixed upstream native timestamp, generated compiler-description normalization, and real stripped compile controls under two differently named roots. No cap raised; full static size/reproducibility still pending.
Current R2 key/direction/tuple/old-sequence/replay/cancellation unit passes1/0; remote single-crate checkPASS. Latest full touched-crate fmt/clippy/test/release fixture sequence is queued through normal build-slot. A stripped interpreter-adapted debug fixture (SHA5e1435374aea7d8d4f15698a8e193a780158813409acd613cc7bb4d05489b550) is staged on the perf VM for functional suite and importedR2 regression only; no release timing/scale claim from debug. It waits for the shared fair lock.
Responder lifetime refinement: the admitted native connection owns its key lease independently of mesh routing leases. Mesh disable retires alternatives while enrolled proofs remain available; key generation prevents an older drop from deleting a replacement. Both physical directions now require their own MAC response. Code/contract prototype is not yet committed or acceptance-qualified.
Earlier unit/check receipts (before the latest responder-lifetime refinement):
r5-proof-unit.log
r5-remote-isolated-check.log
R2 functional milestone on perf VM208: current debug fixture functional suite67PASS/0FAIL/12ignored; imported invalid-mesh/enrolled-auth regression executes exactly1test andPASS. This is functional evidence, not release latency acceptance.
Retained debug scale diagnostic: original direct control now uses ordinary100ms/adaptive connection health;289/1000 guard exits with1000TLS connections open. New distinct-tuple session-authenticated control785/1000 exits, also1000open. Both FAIL and are unoptimized diagnostic cohorts, not accepted D170 cost/health measurements; no gain or healthy CPU/RSS claim. Release controls remain mandatory. Requested15/30arguments in the original review fixture label do not describe the direct branch's new adaptive operating cadence.
Focused remote library Clippy -Dwarnings and current key proof unitPASS. Full all-target/touched-crate gate and optimized remote fixture still await normal build-slot; all4normal slots occupied at23:45CEST. The scoped script prioritizes remote release fixture before the remaining server/daemon gates once a slot opens. No priority access or skipped final gate.
Functional evidence: VM suite + R2. Source/artifact attestation and latest unit receipt are being attached separately.
Protocol review finding before implementation commit: selected Quinn's default EndpointConfig accepts a greased QUIC fixed bit (quinn-proto0.11.19 config/mod.rs63, packet.rs585). Therefore the draft leading0x00 envelope is not structurally disjoint from all valid short-header traffic. Keep45bytes but use leading0x80 followed by"ip1" and direction0/1. In the QUIC long-header interpretation these are unsupported versions0x69703100 and0x69703101, so valid negotiated QUIC packets cannot share the discriminator. Both tags remain covered by the session MAC. No engine/configuration change or weaker packet checks. Update contract, parser, socket dispatch and robustness target before code commit, then rerun functional/legacy endpoint checks. Earlier receipts retain the earlier discriminator and are not substituted for final evidence.
Compatibility refinement before wiring production: advertise additive native Hello bit7 REMOTE_PATH_PROOF in both regular and parked greetings. A new client must leave a live enrolled QUIC path in place when the server lacks this bit, rather than attempting a proof an older peer cannot answer and forcing a replacement. Responder allocation also requires the admitted client's bit7. Capability withdrawal retires mesh selection. This changes no frame layout/version or authority; add a no-opt-in regression. The older statement that unchanged peers simply remain enrolled needs this explicit negotiation to be true. Current67-pass/initialR2 receipts predate this wiring and remain earlier functional evidence.
R5 additional regression: withdrawing negotiated physical-proof bit 7 while retaining encrypted-backup bit 6 left candidate discovery running. The existing cancellation test now covers this transition: RED (selected Mesh instead of Enrolled), then GREEN after refusing discovery on withdrawal and rechecking capability before publishing simultaneous success. Head remains 5cd28518d; implementation pending its per-module gates.
Failing regression, same test passing. Release scale controls remain queued on the perf VM lock; no new accepted timing or healthy-cohort cost claim.
R5 steps committed and pushed: 9bf93583b authenticates exact physical responses with existing TLS exporter/OpenSSL HMAC, cancellation epochs and admitted-connection responder leases; 299c86523 negotiates bit 7 and cancels withdrawn proof support; 24c3f6aca preserves independent bare-server and distinct-tuple 1000-pair controls. No new connection/engine/task, no terminal replay.
Per-module gates: remote strict all-target Clippy PASS; 68 ordinary tests PASS /13 ignored. Infraternal-agent strict all-target Clippy PASS;116 ordinary tests PASS /21 ignored (including retained-mesh greeting, TCP recovery and withdrawal). Optimized ignored controls, server integration, combined-source real-SSH acceptance, exact size caps and final fast gate remain outstanding. Normal build slot acquired at00:39CEST; server checks running, static and same-recipe dev comparison builds follow sequentially.
Remote Clippy, remote tests, infraternal-agent Clippy, infraternal-agent tests.
R5 steps pushed: 708e5bb91 keeps physical responders with admitted RemoteLink lifetimes and advertises bit7 on ordinary/primed native greetings; b9bb70dbe adds the bounded public-envelope robustness driver. Server strict all-target Clippy PASS; focused runtime43PASS/0FAIL/3ignored. Locked parser-driver compilation PASS (existing remote-only server dead-code warnings retained in log); no instrumented campaign claimed. Static x86 build now running, ARM/controller and exact dev comparison follow in the same sequential build-slot command.
Server Clippy, runtime tests, parser driver. Dev acceptance observer overlay is prepared, not compile-validated yet. Its CLI size build restores all tracked crates/ source to40cc9c5be first, using only the explicitly shared reproducible compiler recipe; observer hooks return only for the test fixture. No candidate transport implementation is imported into the dev controller.
R5 findings / not release acceptance:
First combined-source x86 static build:22,184,912B, +111,456B over this branch's old22,073,456B cap. Early check stopped further architecture/fixture builds; cap unchanged. Current dev40cc publishes22,204,304B, but its actual matched-recipe artifact is not built yet: no delta-vs-dev claim. Fresh exact dev build is queued to separate combined-source cost from proof changes. Size receipt.
Full artifact still contained four absolute build-only engine/module installation strings, so the earlier compiler-header fix alone is insufficient for R4. Pushed a624264eb remaps only these own-target build paths to stable absolute build-only paths; explicit paths/system configuration/algorithms/unwinding remain unchanged. Wrapper regression RED then GREEN on both targets; fresh real reproduction/key-heap/lifecycle still pending. Red, green.
Lock review found first key export held the receive proof mutex while entering the connection, permitting connection→path→proof versus proof→connection ordering. Regression proves that invariant RED before moving export outside the socket locks, GREEN after; no three-thread runtime hang claimed. Fix awaits full remote gates before commit. Red, green. Existing cost fixture now registers its own admitted responders; this is test-only and uncommitted pending gates.
Perf VM optimized ordinary remote68PASS/0FAIL; original frozen R2/scale controls still wait between fair lock holds. They predate this lock-order follow-up and will not qualify the final source without a rebuilt rerun. Follow-up host gates and exact-dev builds queued through build-slot; no finalfast, cap reproduction or gain table claim.
Current pushed head fd03f9edc, rebased onto origin/dev6eaa5c413. The rebase adds only research/vnc/DESIGN.md relative to the previously pushed tree; production source unchanged. Rebased follow-ups: e0d6ef14e exporter lock fix; fd03f9edc admitted cost-fixture responders; 4a707f0c3 native build-directory remap. Tree attestation.
Full touched-crate gates after the lock fix: remote strict all-target Clippy PASS;69 ordinary tests PASS/13ignored. Infraternal-agent strict all-target Clippy PASS;116 ordinary tests PASS/21ignored. Remote Clippy, remote tests, infraternal-agent Clippy, infraternal-agent tests.
Exact dev40cc CLI build is running under the same final compiler recipe; pre-build git diff of all tracked crates/ files is empty. Observer-only fixture overlay returns after this build. Candidate native objects will be explicitly invalidated before rebuilding, because a C compiler script change alone is not a native source dependency. Fresh reproduction, size delta/caps, final optimized controls,31-sample gains/integrity and fast gate remain pending.
R5 measured size control: exact dev40cc22,316,304B; combined candidate fd03f9edc22,185,872B (−130,432B), same final compiler recipe. Branch cap22,073,456 still fails by112,416B; no cap changed. Both compiler paths resolve to the same Rustup1.99.0 compiler/sysroot. The full candidate no longer contains the detected checkout/target directory strings; fresh second-checkout reproduction is still required. Matched size receipt.
Rejected command-line-only entry-crate opt-level=s trial:25,124,856B, substantially larger; repository profile remains unchanged. Trial bytes.
Next measured reuse: use the HMAC-SHA256 implementation already selected by Quinn/rustls-ring for the physical proofs, keeping all45wire bytes and exporter label unchanged. Pushed9a19ebeef exposes that existing provider directly; resolved package versions/checksums and features unchanged. Pushedf7b0db896 pins independent request/response MAC vectors before changing implementation. This adds no foundational engine/provider selection. Package control, feature graph, original vectors.
Uncommitted provider prototype passes vectors/replay/tuple/epoch/mixed-receive controls. Full gates, exact size and optimized VM cost/gain checks precede adoption. Optimized original R2 case PASS on the earlier frozen fixture receipt; not final-source qualification after the lock follow-up. Dev fixture compile found only observer-module visibility errors; corrected in test-only overlay, rebuild running through build-slot.
R5 optimized diagnostic: current-branch original adaptive control,1,000/1,000 live mTLS connections,2/1,000 guard exits (nodes116,118,TimedOut),31x250ms windows with failures retained from admission. VM load before0.27/1.20/0.82, after0.53/1.23/0.84. Frozen remote fixture SHAf0229372; this is before the later exporter lock/provider follow-ups. This FAIL is retained, not restarted or accepted as healthy cost evidence. Raw log, full report. Direct fast-path and distinct-tuple variants continue through fair separate lock holds; independently built exact-dev adaptive control is queued. No final R3 or gain-table claim.
R5 original frozen optimized scale controls complete (pre exporter/provider follow-ups): all1,000TLS connections remain live in each arm. Direct fast-path0exits; independent current-branch adaptive2; distinct-tuple15ms755; distinct-tuple100ms0. Failures retained from admission through31windows; no healthy-cost claim for failed arms. Separately built exact-dev adaptive151exits in a separate load window; no causal speedup claim. Direct, distinct15, distinct100, exact dev.
Uncommitted existing-provider reuse passes70remote and116infraternal-agent ordinary tests plus strict all-target Clippy. Static22,183,984B:−1,888B versus the preceding matched candidate, still over the unchanged cap. Bytes. Rebuilt optimized scale/cost remains pending; no slower cadence/deadline adopted.
Link map attributes53,108B to inherited run_channel symbols and96,013B to remote ones, plus duplicated terminal polling bodies across the public CLI boundary. Measuring erasure of only the public inherited entry future (one allocation/connection, no per-frame allocation); single-crate check passes, full tests/size/runtime cost pending. Attribution, prototype checkpoint.
Exact-dev real add-host LAN smoke passes byte integrity receipt;31/venue baseline cohort running. Candidate existing-provider/pre-entry prototype LAN smoke passes preservation/integrity; WAN pending. These single samples do not qualify final source or any31-sample target. Current pushed headf7b0db896; provider/entry experiments uncommitted, gate remains incomplete.
Combined-source provider prototype (HEAD f7b0db896 + uncommitted fixed-wire provider reuse): distinct-tuple 15 ms authenticated guard exits 674/1000, live TLS 1000/1000; direct control exits 0/1000. Fresh-admission copy cohort exits 561/1000; server UDP drops rise from 635 after ramp to 21,031 after observation. These are failed qualification runs, not accepted steady-state cost figures. Original OpenSSL provider distinct control exited 755/1000 in a different load window; no causal improvement claimed. Testing the existing bounded 20/25, 25/20, 30/15 ms diagnostic profiles without changing production constants or the original review test. Real SSH disable smoke passed both venues with byte integrity; full receipts still being checked.
r5-provider-distinct15.json
r5-provider-fresh-cost.tar.gz
Completed provider-reuse slice: e97fc44d0 (pushed mesh-autoswitch). Same 45-byte SHA-256 wire vectors; no new resolved package/provider. Remote all-target clippy +70 ordinary tests and infraternal agent all-target clippy +116 ordinary tests pass. Matched x86 artifact 22,183,984 B, -1,888 B vs preceding implementation, still +110,528 B over the unchanged branch cap; no size acceptance claimed. Scale remains red (674/1000 distinct guard exits). Buffer-only diagnosis: fresh-copy 1000 transports, default 212,992 B server receive capacity exits561; unprivileged 425,984 B exits414; explicitly privileged diagnostic 2,097,152 B exits0 with0 UDP drops. No host settings or production socket capacity changed. Testing bounded probe timer spreading next.
Fresh combined-source natural join diagnostic (3 samples/venue, no held handshake): LAN handshakes completed 13,741 /107,531 /15,080 us before add-host success. WAN handshakes completed 4,788,145 /4,779,659 /4,804,871 us after success. Header-only receipts show crossed 148-byte initiations and 92-byte responses before success, then a node initiation about 5 s later immediately preceding the first accepted handshake. This is consistent with the first simultaneous handshake exchange losing its state and waiting for retry; header sizes alone do not prove kernel acceptance/rejection. Production WireGuard configuration remains identical to dev, including both-end 25 s keepalives (D228). Earlier unilateral initiation suppression was not robust on LAN, so it remains a rejected diagnostic, not a production fix. The separate ordinary baseline natural-join 31/venue cohort is now running. These diagnostic witnesses include post-fence harness work and are not an earliest-byte latency claim.
r5-natural-handshake-diagnosis.json
r5-candidate-natural-headers.tar.gz
Combined-dev baseline LAN standing cohort is complete (31 samples): handshake → first causally witnessed terminal Output p50/p99 521,004/748,770 us; silent cut → complete echo 299,506/695,133 us; byte integrity31/31, connection preservation0/31. WAN cohort continues. Matched diagnostic size trials: inherited-boundary generation22,184,336 B; server-only opt-level=z21,908,496 B (-275,840 B), infraternal-agent-only z22,175,536 B (-8,800 B). The 22,073,456 B branch cap is unchanged. Reject the allocation prototype; take server-only z into runtime validation, no accepted final artifact yet. Fresh exact-dev native namespace test: LAN lifecycle completes, WAN returns Storage on mesh-disable at remote_mesh_namespace_tests.rs:993 after preserving fallback bytes. This is a failed baseline lifecycle; it does not waive the candidate disable regression or make native latency green.
r5-profile-size-trials.json
r5-baseline-standing-lan31.json
r5-dev40cc-native-150.log
Bounded timer-spreading trial: distinct authenticated review cohort exits15/1000 with1000 TLS connections live (previous provider-only run674; different recorded load windows). Fresh-copy cohort exits23/1000, all after warmup, with2,339 server UDP drops. Both remain failed qualification runs; direct/R2/ordinary/100ms controls pass. Next candidate spends the existing response margin on 5 ms retransmission once that tuple's measured RTT/variation has elapsed. It preserves the original sequence, physical send clock and response deadline; healthy probe cadence stays unchanged. This tests recovery from brief receive-queue loss instead of extending the acceptance deadline. Server-only z compiler profile is pending runtime validation; no final artifact accepted.
r5-spread-distinct15.json
r5-spread-fresh-cost.tar.gz
origin/dev advanced to492e1505a (merge agent-reconnect #225), adding the supervisor/worker ownership fence, TCP silent-failure handling and durable mesh-state fixes. Rebase is required before final acceptance; the6eaa/40cc comparison remains historical evidence, not the final base. Pending compiler/timer prototypes are preserved outside the checkout. Prepared a detached492e measurement checkout: shipping crates match dev exactly (empty production diff), with the same compiler normalization recipe. New baseline artifacts will use distinct dev492 names so the already-running old cohorts cannot read replacement binaries. The active pre-rebase iterate gate is finishing before the main checkout changes.
Rebased onto current dev492e1505a; local headf0e3e67b8 (combined-source validation in progress before force-with-lease push). Preserved dev's supervisor, sole-worker fence, TCP liveness and failed-work backoff accounting. Kept the no-redial refusal regression using the shared endpoint and unchanged admission count. Exact-dev x86 baseline is22,286,704 B, SHA291cdac6390ab9c6e3ebee82bc8ee0ea0f1e472f27cfd9e6559a0eadee8534c7. Current-dev adaptive control exits168/1000 (all1000 TLS live); this remains a failed control, not a waiver. Combined candidate distinct authenticated cohort now exits0/1000 over31x250ms windows; direct and R2 controls pass. Fresh-copy qualification is pending. The new retransmission-clock regression failed before the fix, then all72 ordinary remote tests and strict all-target clippy passed. It verifies that retries return the first successful physical-send clock, not the latest transmit time. Pre-rebase iterate had one auth-socket observed-event failure (453 pass/1 fail/47 ignored in server); standalone head and baseline both pass, so no load waiver is claimed. New-head full fast is still required. Current-dev real-SSH baseline cohorts are running with distinct frozen artifact names.
r5-first-proof-clock-red.log
r5-proof-retry-tests.log
r5-proof-retry-distinct15.json
r5-dev492-adaptive.json
r5-combined-iterate.log
Combined-source finding (local head f0e3e67b8 on dev
492e1505a): bounded retries passed the distinct-path review cohort (0/1,000 exits) but failed fresh admission (21/1,000 exits). That version is rejected. Its pending allowance could shrink while local scheduler lateness decayed; the new version retains the greatest observed allowance until that exact authenticated challenge is answered. Retransmissions retain their first-send clock and nonce.Current functional checks: remote 72 passed / 13 ignored; host daemon 127 passed / 21 ignored; strict clippy passed for both. Current scale controls and the iterate gate timing self-test are queued behind the perf VM lock; no scale qualification yet. The earlier combined-source static artifacts were 21,920,176 B x86 and 19,314,928 B ARM, below unchanged caps, but predate this correction and are not final artifacts. Full fresh-build reproduction remains pending.
Rejected fresh-admission output, current remote checks, current host-daemon checks.
WAN diagnosis follow-up: perf VM kernel is 6.12.111+deb13-amd64. Upstream 6.12 handshake code overwrites initiation state when consuming a peer initiation and accepts a response only while its own state remains CREATED_INITIATION. Session creation clears that handshake state. The retry constant is 5 seconds. This explains a plausible crossed-initiation failure matching all three WAN header receipts (~4.78–4.80 s after success); it remains an inference, not a kernel-state trace.
Primary sources: https://github.com/torvalds/linux/blob/v6.12/drivers/net/wireguard/noise.c#L592-L603, https://github.com/torvalds/linux/blob/v6.12/drivers/net/wireguard/noise.c#L695-L706, https://github.com/torvalds/linux/blob/v6.12/drivers/net/wireguard/messages.h#L36-L40.
No contained production correction established: unilateral initiation suppression was unreliable on LAN, and changing peer keepalive/endpoint startup can defeat the established-traffic-only NAT admission exercised by the fixture. D228's both-end 25-second keepalive stays unchanged. The accepted-handshake-to-terminal switch and the pre-handshake bootstrap delay remain separate measurements; no claim that the WAN bootstrap delay is fixed.
Current combined-source generation remains unqualified: distinct authenticated paths exit 17/1,000; fresh admission exits 1/1,000, all TLS live. Fresh server UDP drops reach 11,081. Keep these failures. Next measurement will reduce ordinary probe pressure according to the measured response allowance while retaining the first-send clock and outstanding maximum allowance; no copy-delay change or privileged socket setting.
r5 distinct result, fresh raw receipts.
All workspace Rust rows and TS functional checks passed, including the previously failing auth-socket case. However, the gate's unacquired perf-lock SSH request was cancelled to release the wait; SSH returned zero and the driver advanced without running the proxy self-test. This entire iterate run is INCOMPLETE despite its printed success line. The local venue dispatcher now requires the actual proxy PASS receipt; functional mocks prove zero exit without that receipt is rejected. Final fast must rerun every required check. Annotated transcript, cancellation record.
Independent R4 build: x86 21,920,240 B, -366,464 B vs exact dev492 (22,286,704 B), below unchanged cap; ARM and full fresh reproduction continue. Final acceptance helper stopped before launching cohorts because R3 is red. Baseline real-SSH cohorts resumed after the perf lock became available. Current local head remains f0e3e67b8; history normalization and atomic commits follow the active compiler work.
Dev advanced to
9e84426c4(size-ratchet #198). Rebased again; local headcd9d37009includes the separate retransmission-clock fix (previously pushed as 776c778d5). Current-base remote compilation is queued through build-slot before the next force-with-lease push. Preserved dev's D272 native AES-GCM fallback wrapper, unused-extractor pruning, cold-plugin profiles and tighter per-target ceilings. Current floors: x86 21,422,096 B; ARM 19,516,992 B. Old dev492 size artifacts are historical, not qualification against these new floors. Combined wrapper regression failed before its expected remapping arguments were updated, then passes both architectures and idempotent native-dispatch removal. No foundational choice changed.All crate-source bytes match the preceding functionally gated snapshot except the pending remote_quic.rs cadence trial. Format check passes. The next trial changes ordinary observation cadence from one quarter to one half of the measured response allowance; the configured deadline, first physical clock, exact authenticated tuple check and 10 ms copy delay stay intact. Rejected cohort traffic was 105,697 UDP datagrams/s across both ends; this is a failed-cohort pressure diagnostic, not accepted healthy cost.
Prepared a detached exact-dev9e baseline, same compiler normalization recipe, empty production crate diff; test observers are frozen separately. Superseded dev492 real-SSH parents stopped only at unacquired lock waits: 25 LAN attempts each, no running lifecycle killed. Natural joining retains 4 readiness failures; standing retains 0 operation failures. No WAN or complete 31-sample claim for these partial cohorts. New dev9e artifacts and cohorts use distinct names.
Partial raw receipts, attempt denominators, owned-process stop receipt, wrapper RED, wrapper GREEN.
Still required: zero-exit scale qualification, current-base real-SSH 31+ samples per venue, unchanged-or-smaller fresh-reproducible shipping artifacts, final fast on branch head. No final acceptance, deployment or merge.
Published mesh-autoswitch head
cd9d370090on origin/dev9e84426c48with an explicit force-with-lease against the previously pushed clock commit. The separate first-send-clock fix is retained. Combined native-wrapper regression and R1 copy-contract check pass; format passes. All production crate-source bytes match the previous functional snapshot except the uncommitted cadence trial. Current-base compilation is queued through build-slot; no current-base full-gate or scale acceptance claimed. Working changes are limited to the cadence trial and server-only z prototype. The prepared dev9e baseline and final acceptance remain pending compilation.Resource block, 03:24 UTC: the normal build-slot helper has waited since about 02:57 UTC without starting compilation; all four normal slots are held by other jobs. Local/pushed head
cd9d37009, current-dev baseline prepared, pending changes remain the cadence and server-z prototypes. Need normal build capacity from orchestrator scheduling; continuing the existing wait, with no reserved-slot or lock bypass. No new scale or acceptance claim. The rejected generation's direct control has completed at 0/1,000; distinct 17 and fresh 1 remain failures.Operational queue snapshot. This is queue state, not build-host performance evidence.
Blocked on perf VM access at 03:30 UTC. NetBird SSH now returns:
Per the lab rule, stopped further VM access. All three own automatic staging helpers were stopped before new uploads/runs; no active own VM SSH process remains. No plain SSH, host-key change or credential investigation. Orchestrator needs to restore NetBird SSH before performance work can resume.
The rejected stable-budget generation's direct and R2 controls passed; functional and 100 ms control entries exited on connection refusal without starting a test, so they are NOT test failures or passes. Distinct 17/1,000 and fresh 1/1,000 remain genuine rejected measurements.
Build-slot became available. Current-dev half-allowance trial passes strict remote clippy and all 72 ordinary tests (13 ignored); optimized fixture builds and exact-dev baseline builds continue locally. No performance qualification is claimed. Current pushed head
cd9d37009; pending prototypes remain uncommitted. Final fast and release acceptance are blocked on the required VM self-test and measurements.Connection refusal, owned staging stopped.
Local work while perf SSH remains refused: exact dev9e production source plus the shared compiler normalization recipe builds to 21,409,296 B x86 and 19,582,312 B ARM. These are the matched size baselines, not performance measurements. Shipping crate diff was empty before the builds. x86 bytes, ARM bytes.
Dropped the uncommitted server-z prototype: measure the existing server s profile against the new dev cuts first. Only remote_quic.rs remains modified in the worktree (half-allowance trial). It passes strict remote clippy and 72 ordinary tests; no scale qualification. Functional output.
Baseline observer compile caught a test-only wiring error: the adaptive review module was registered at the crate root instead of inside remote_quic, so seven private-helper names were unavailable. Corrected the registration scope; optimized remote fixture now compiles. Production files and already-built shipping baselines are unaffected. RED, GREEN compile, corrected measurement-only sources.
Continuing local server-fixture/static reproduction checks, with automatic VM staging disabled. No further VM connection attempted after refusal. Final measurements and fast remain blocked until orchestrator restores access. Pushed head
cd9d370090.Blocked continuation result — not release acceptance.
Pushed head:
cd9d370090, based on current origin/dev9e84426c48. Rebase keeps dev's reconnect supervisor/fence and approved D272 compiler path. The first-physical-send clock correction is a separate committed change. Working tree retains only the unqualified remote_quic.rs half-allowance trial; server-z was discarded. Pending source patch.D243 local static results, same normalized compiler recipe and existing server s profile:
R4 cold x86 build used a distinct checkout and fresh target with identical tracked inputs. Both artifacts are 21,320,176 B and SHA-256 3352faa33004a27eccd28f19281116a736f7a57346db1a13aac81be33664905c: byte-identical. Shipping AES-GCM known vector, native wrapper/metadata tests, production trust and x86 locked/dump-excluded key-memory checks pass. These artifacts include the pending trial; this is local size/reproduction evidence, not a qualified final head or runtime claim. Final cap ratchets wait for qualified source. Reproduction, both sizes, source-input hashes, fixture hashes and functional receipts.
Remote strict clippy and 72 ordinary tests pass (13 ignored). Both baseline measurement fixtures compile and list the intended ignored tests exactly once. Frozen local artifacts under /home/kayg/.cache/codex-scratch/mesh-autoswitch/r5/:
NetBird SSH refusal remains the blocking external condition. No VM connection was attempted after the recorded refusal; automatic staging remains disabled. Needed from the orchestrator: restore NetBird SSH. Then qualify the frozen half-allowance fixtures at 1,000 transports, retain all failures, and run current-base LAN/150 ms real-SSH standing/natural 31+ sample cohorts with byte/writer/disable checks. Guarded staging helper: r5/stage-offline-artifacts-after-qualification.py; it refuses the recorded outage or missing/failed scale results. Do not remove r5/perf-ssh-refused until access restoration is confirmed.
Latest actual rejected scale generation remains 17/1,000 distinct-path exits and 1/1,000 fresh exits; direct and R2 controls passed. The half-allowance version has no VM results yet. No new before/after latency table, native latency pass, 31-sample acceptance or Storage-disable acceptance claimed. Previous reviewed native latency was red on both candidate and base. The WAN pre-handshake delay remains unfixed; earlier source-linked diagnosis and header receipts are above.
scripts/verify fast was attempted on the clean published head. The pending trial was restored byte-for-byte afterward. Exit 1 at the required perf-VM self-test, before Rust gates; the local dispatcher explicitly prevented a retry after refusal. No full-gate pass. Gate receipt, raw transcript.
Verbatim gate output:
No merge, deployment or issue closure. Continue from the retained worktree and issue receipts after lab access is restored.
Resuming r6 on mesh-autoswitch at
cd9d370090, base origin/dev9e84426c48. Read review findings and all issue comments. NetBird SSH restored at 2026-10-16 04:38 UTC; VM reports uptime 2d13h and load 0.00/0.08/0.56. Retained remote_quic.rs cadence trial unchanged. First qualify frozen trial at 1,000 transports (including admission), original R2 and controls; then current-base acceptance, size ratchet and final fast. No merge or deployment.Resuming r7 on mesh-autoswitch at
e0797a97c, base origin/dev9e84426c4. Read review findings and issue history; retained retry-boundary regression/fix. Prior trial distinct 1/1000 and fresh 6/1000 exits remains rejected. Will commit the functionally checked correction, publish saved receipts, then qualify scale, current-base gain cohorts, reproducible caps and final fast. No merge/deploy.Published
e0797a97cand retry-boundary correction1e3cd82ccto mesh-autoswitch. Saved regression RED, 73-test/strict-Clippy GREEN and failed scale receipts now attached below. Prior 1/1000 distinct and 6/1000 admission exits remain rejected. Current correction awaits optimized scale measurement; no latency acceptance claimed.r6-retry-margin-red.log
r6-retry-boundary-gates.log
r5-half-allowance-distinct15.json
r5-half-allowance-fresh-cost.tar.gz
R3 first combined-source results for pushed
1e3cd82cc: distinct authenticated paths 0/1000 guard exits, all 1000 TLS live, 31x250 ms windows; fresh admission 0/1000 exits through ramp/warmup/31 windows, 0 proof failures. Previous rejected trial was 1 distinct / 6 admission exits. Perf VM208, four runtime workers; load distinct 0.56/0.39/0.48 before, 1.19/0.53/0.52 after. Fresh load 0.34/0.34/0.46 before, 0.56/0.39/0.48 after.Fresh cohort cost (both ends and adapter instrumentation): 2.493 CPU cores, 143174 UDP datagrams/s, RSS103903232 B for 1000 pairs; 28118 server UDP drops retained. No daemon idle-cost or physical-mesh scale claim: this is one loopback cohort with 31 correlated windows. Remaining direct/R2/functional/100ms controls are queued. Current-dev real-SSH baseline cohorts have begun; shipping artifacts are building through build-slot. No final gain/cap/full-gate acceptance yet.
r7-retry-boundary-distinct15.json
r7-retry-boundary-fresh-cost.tar.gz
r7-scale-summary.json
Fresh functional verification of pushed
1e3cd82cc: format PASS; strict remote all-feature/all-target Clippy PASS; 73 ordinary tests PASS, 13 perf-only tests ignored. Optimized fixtures staged with source hashes and VM-loader-patched SHA-256 below. Scale/R2 controls run explicitly on the perf VM, not substituted by skipped ordinary tests. Shipping static builds now running through build-slot; baseline cohorts and remaining controls wait for perf capacity.r7-clippy.log
r7-tests.log
r5-remote-retry-boundary-manifest.json
r5-agent-retry-boundary-manifest.json
Combined-source shipping sizes for
1e3cd82cc, matched normalized recipe vs exact dev9e:Both stripped/static, no PT_INTERP; no cap raised. Independent empty-target x86 reproduction, final downward ratchet and full fast are still required. Controller fixture is compiling. Remaining perf controls and real-SSH cohorts wait behind process-per-user ppu-soak-r8c-dev (owner start 05:51:31 UTC); current baseline has one LAN sample/mode, not complete 31-sample evidence. Continue waiting, no new gain table claimed.
r7-shipping-sizes.json
Completed R4: independent checkout at1e3cd82cc and empty comparison target produce byte-identical x86 artifact21320560B, SHA-256 e89834910b0448076c944ca18c5b2348c543aaaecc8c6c82ebabbb5d53fd397d. Pushed07ed99a1f ratchets x8621320560 / ARM19513808 caps downward. Format, shell syntax, both real-artifact cap checks pass. Current-source Storage cancellation regression passes. Full scripts/verify fast queued on clean07ed99a1f; no full-gate pass claimed.
Remaining perf controls and31-sample comparisons wait for capacity. Baselines retained their two completed LAN samples/mode. Replaced only verified unacquired baseline waiters with bounded venue-batch scheduling: lifecycle function source byte-identical (hash proof attached), all completed/failed attempts retained, provenance checks source/payload/sample/profile identity before resumption. Stop starting cases after1100s, each outer case capped600s: maximum1700s hold, release between venues/batches. Owner file retains actual batch start. No measurement clocks adjusted or production code changed. Original runner/provenance are preserved alongside new method receipts.
r7-reproduction-result.json
r7-reproduction-inputs.json
r7-artifact-cap-tests.log
r7-storage-regression.log
Measurement-method supplement: Forgejo rejected standalone Python attachments with HTTP422 after accepting the four reproduction/gate receipts. The unchanged-lifecycle hash proof, bounded-batch source and verified waiter-retirement receipt are bundled here instead. No measurement or production-source change.
r7-cohort-method.tar.gz
Current-dev (
9e84426c4) LAN standing cohort complete, 31 real-SSH lifecycles: WG handshake -> first causal mesh Output p50/p99 324257/430844 us; silent cut -> complete fallback echo 307716/714530 us. Byte integrity and original writer 31/31; TLS/admission and control counts unchanged 0/31 (sample 0: 1 -> 3). This controlled profile holds the first handshake until a terminal exists; its success-to-handshake delay is artificial, not ordinary add-host timing. WAN standing and both natural cohorts remain pending. Manifest and lockfile match dev; only separately recorded compiler normalization and test observers differ.Current-source R2 passes explicitly on the perf VM (invalid mesh UDP cannot borrow enrolled authentication). Fast direct and exact-dev adaptive controls both 0/1000 exits, 1000 TLS live, 31 windows. Distinct/authenticated admission zero-exit receipts were posted earlier. Remaining ordinary/100ms controls still queued; no complete candidate gain table yet.
r7-dev9e-standing-lan31.json
r7-dev9e-standing-lan31-summary.json
r7-retry-boundary-r2.log
r7-dev9e-adaptive-r7.json
r7-retry-boundary-direct.json
New failing scheduling regression before production correction: healthy_path_proofs_keep_the_configured_polling_interval emits52 authenticated challenges in400ms of virtual time, exceeding28 allowed by the configured15ms interval. This is a functional virtual-clock test, not a build-host latency measurement. Initial100ms observation was insufficient to pass the conservative first response window; the400ms regression exercises learned RTT scheduling.
The retry-margin boundary currently clamps a fresh healthy challenge to measured RTT, overriding ordinary cadence. This helps explain the recorded143174datagrams/s and2.493cores for1000pairs. Correct ordinary interval/deadline scheduling while retaining first physical-send clocks, outstanding maximum allowance, exact-tuple authentication and reserved retries; use the existing connection identity only for non-secret phase spreading to avoid new size growth. Wire copy delay remains10ms. Rerun candidate scale/cost/gain qualification on the corrected source; earlier1e3cd82cc results remain historical. Canceled only the still-queued07ed99a1f full gate; no gate output or pass claimed. Baseline batches continue unchanged.
r7-healthy-cadence-red.log
Matched current-dev standing baseline complete: 31 real-SSH lifecycles per venue on perf VM208. Controlled initial handshake is held until a terminal exists; this table measures switching/fallback, not natural joining. All62 preserve measured bytes and the original writer; TLS/admission/control counts change in all62.
Natural LAN has reached31 attempts with at least one readiness failure retained; WAN natural remains pending. Corrected candidate scheduling regression passes in virtual time; full functional/optimized/size qualification is queued through build-slot. No final candidate latency or full-gate pass claimed.
r7-dev9e-standing-wan31.json
r7-dev9e-standing-summary.json
Perf fairness wait report: the older source ordinary-control waiter has been unacquired since06:27:40UTC, now over45minutes. Current owner file:
Continuing the existing waits; no lock bypass or other-job process changes. The corrected cadence fixtures are now staged (74 ordinary tests and strict Clippy pass; x8621320400B / ARM19513440B, both below lowered caps), awaiting scale/cost qualification before committing the trial. All31+31 current-dev standing samples are complete; natural LAN retains1readiness failure among31attempts. Need shared perf capacity for remaining controls, natural WAN and candidate cohorts; no final gain/gate acceptance yet.
Committed and pushed
2f3b874b1after verifying its source bytes match the qualified fixtures. Healthy polling now respects the configured interval; non-secret phase spreading reuses the public UDP port. The virtual-clock regression failed before the fix; 74 ordinary remote tests, strict Clippy and format pass.Perf VM208, four-worker shared runtime, 1000 actual mTLS transport pairs, 31x250ms windows each: distinct and fresh-admission exits both0/1000, all1000 connections live, zero startup proof failures. Compared with
1e3cd82cc:CPU -20.5%, traffic -28.5%; RSS +1748992B observed. These are correlated transport-only windows including both ends/adapter counters, not daemon idle cost or physical-mesh scale. New fresh load before0.86/0.93/0.74, after1.11/0.99/0.76. Stripped artifacts21320400B x86 /19513440B ARM, below both current caps. Independent reproduction is building. Remaining unchanged controls are grouped into one short locked suite; only frozen waiters with no child commands were retired. No final latency/full-gate pass yet.
r7-floor-scale-summary.json
r7-cadence-floor-fresh-cost.tar.gz
r7-cadence-floor-distinct15.json
r7-floor-tests.log
r7-floor-clippy.log
Pushed head28380fc49: final production code2f3b874b1, then exact D243 caps21320400B x86 /19513440B ARM. Independent empty-target x86 build is byte-identical, SHA-256 aa06e92509951c2101f750ba903044a0fbfd5f42340a4a334268e27bb8a43183. Fixture producer predates the commit, so attached source-identity proof verifies its captured patch equals the committed change and every fixture source hash matches head. All corrected controls pass explicitly on VM208: distinct/direct/R2/74ordinary/100ms and fresh admission; scale cohorts retain1000 live TLS connections and0exits across31windows each.
Baseline raw124attempts attached: standing31LAN+31WAN; natural31LAN(1readiness failure)+31WAN(0failures). Natural WAN observed kernel-handshake stamp after success p50/p99 4.770/36.648s; fresh post-fence Output after success6.303/54.309s. The stamp is queried public kernel state, not proof of the earliest handshake, and the fenced marker is an upper bound including fixture work, not the earliest unrestricted Output. Do not confuse these with controlled standing switch bounds. A separate3/venue packet-header experiment suppresses server initiation delivery only inside owned node namespaces while both peers still transmit/NAT-prime; it changes no production settings and is excluded from ordinary latency acceptance.
Candidate standing/natural31/venue and full scripts/verify fast are running/queued on clean28380fc49. No final latency or full-gate acceptance yet; no merge/deploy.
r7-dev9e-acceptance.tar.gz
r7-floor-qualified-source-identity.json
r7-floor-reproduction-result.json
r7-floor-artifact-cap-tests.log
r7-cadence-floor-control-exits.json
LAN candidate acceptance, final source
39a13398a(compiled inputs unchanged from2f3b874b1), pinned perf VM, release, 31 lifecycles per profile. No failed candidate operations; all 62 mesh-disable cases passed.Standing-session p50/p99, microseconds: handshake→first mesh Output dev9e 324257/430844 → candidate 8860/20568; silent cut→complete fallback echo 307716/714530 → 23173/27032. Candidate 31/31 original attachment/writer, 6944/6944 bytes, TLS/admission/control counts unchanged. Status publication is a separate slower boundary: cut→Fallback status 576983/838377; byte recovery must not be described as equally fast status publication.
Natural unheld LAN add-host: 31/31 candidate vs 30 successful + 1 recorded operation failure in 31 dev attempts. Candidate start→success 1938121/2302211 us; success→observed kernel latest handshake -22875/-59 us (already joined before success); success→fresh mesh-proven post-fence Output upper bound 97358/139205 us, including fixture setup/fence overhead. This marker is not a claim about earliest unrestricted mesh byte.
150 ms cohorts and final scripts/verify fast remain running. Next VM cases currently wait behind process-per-user ppu-soak-r8c-shared (lock owner 08:48:20 UTC); no measurement runs outside the shared lock.
r7-candidate-standing-lan31.json
r7-candidate-standing-lan31-summary.json
r7-candidate-natural31-floor-r7-venue-0.json
r7-candidate-natural31-floor-r7-venue-0-summary.json
Final-head first full fast attempt: fmt, VT fmt, CLI compile-out, Clippy, cargo-deny and static builds PASS. Workspace tests stopped at daemon::gone_tests::a_vanished_session_gets_a_final_close_and_the_channel_stays_up: InvalidInput, "path must be shorter than SUN_LEN". Fixture path under the descriptive TMPDIR was 124 bytes; Linux permits 107 excluding NUL. This is not an allowed timeout waiver and is not being waived.
The unchanged exact test passes with TMPDIR=/home/kayg/.cache/codex-scratch/m7, a job-owned symlink to the same mesh-autoswitch/tmp directory (107-byte socket pathname). Both static artifacts match lowered caps exactly: x86 21320400 B, ARM 19513440 B. No source or cap change. Full scripts/verify fast is queued again on head
39a13398awith the short scratch alias; every row must pass.r7-final-verify-fast-first.log
r7-short-socket-temp-retest.log
r7-short-socket-temp-path.txt
Author transport checklist reviewed on
39a13398a: exact physical tuple/session/sequence authentication; key and worker lease retirement; encrypted-copy deduplication and preserved writer/attachment; additive capabilities; bounded history and receive work; no foundational engine change; fixed-envelope parser fuzz target retained. No UI diff against base9e84426c4.Explicit review regression run: R1 contract/code 10 ms agrees, and both real final static artifacts equal the lowered caps: 3 pass, 0 fail, 7 assertions. R2 and direct/distinct/fresh scale receipts were posted separately. Native same-session review latency remains a recorded red result on both historical sides (392.67/478.13 ms, 334 ms ceiling); no narrower lifecycle result or clean fast gate can turn it into a green claim. New native 0/150 checks are still queued, separately from that budget.
r7-final-r1-r4-review-tests.log
Combined-source standing-session acceptance complete: head
39a13398advs pinned dev9e84426c48. Release fixture + shipping payload, 4-vCPU perf VM, 31 samples each venue, nearest-rank percentiles. Initial mesh deliberately held until the original enrolled PTY exists; natural unheld joining remains separate.Candidate 62/62 lifecycle successes; 13888/13888 terminal bytes, original writer 2 and attachment preserved, no replacement TLS/admission/control callbacks, 62/62 durable mesh disables and subsequent terminal echoes passed. The prior dev standing cohort retained bytes but replaced transport callbacks; it is not connection-preserving. Status publication remains slower than byte recovery and does not meet the byte budget.
The first Output witness includes kernel observation, local fence and marker input/echo overhead, with no subtraction. Both stages use the same real SSH installer, shipping daemon and public mesh data plane. Current compiled inputs match
2f3b874b1; later changes are only caps/process text. Natural WAN joining, remaining kernel-handshake diagnostic and final full fast rerun are still running.Offline receipt gate output, verbatim:
r7-candidate-standing-acceptance.tar.gz
r7-final-standing-check.log
r7-final-source-identity.json
Combined-source natural add-host acceptance complete on
39a13398avs dev9e84426c4: release, pinned 4-vCPU VM, 31 attempts per venue. Initial mesh joining is unheld. Percentiles are nearest rank; baseline LAN has one failed readiness operation in 31 attempts, retained in the archive/denominator; its measured latencies are conditional on 30 successful operations. Candidate is 31/31 successful at each venue.Across both profiles and venues: 124/124 lifecycle successes, 24614/24614 bytes verified, writer 2 and the same attachment, unchanged TLS/admission/control counts, 124 durable mesh-disable + subsequent echo checks.
Natural kernel time is the latest handshake observed after normal readiness, not independently proven to be the first handshake. The fresh post-fence Output is a causal mesh-path witness but an upper bound on usability: it includes terminal setup, status/counter RPCs, fence and marker input/echo. It is not earliest unrestricted mesh Output or the original SSH-shell handoff. Controlled standing acceptance separately proves the post-handshake first-byte target.
The WAN join delay remains: median 4.785 s, tail 9.967 s after success in the candidate cohort. The packet-level hypothesis/control remains running; do not describe this as a fast end-to-end handshake. Final full fast rerun and native WAN check remain queued.
r7-candidate-natural-acceptance.tar.gz
r7-candidate-natural31-floor-r7-venue-0-summary.json
r7-candidate-natural31-floor-r7-venue-150-summary.json
WAN handshake investigation, matched final-source controls complete (
39a13398a/ payload aa06e925). Same release fixture, packet observer, real installer and payload in both arms; three fresh cases per venue/arm, all 12 operation/integrity checks passed. Diagnostic counts remain separate from the ordinary 31-sample acceptance cohorts. VM kernel: 6.12.111+deb13-amd64.Unfiltered WAN headers show crossing 148-byte initiations and both 92-byte responses, followed by node initiation retries 5008.677, 5017.204 and 5054.725 ms later; kernel handshake timestamps track the latter exchange. The filtered WAN cases have no initiation retransmission before the observed handshake. Every filtered case dropped exactly one 176-byte IP packet. Both peers still sent outbound, with unchanged endpoints and 25-second keepalives: the test only dropped server→node type-1 delivery after connection tracking inside the owned node namespace. The identical rule makes all three LAN cases wait 5003–5010 ms for the node retry. No production filter or VM/network setting changed.
Interpretation: a symmetric initial kernel handshake/peer-installation interaction explains the five-second steps; this is supported by the direction intervention, not merely a correlation with application attach. Upstream v6.12 keeps one pending handshake state, replaces it when consuming another initiation, and accepts a response only in the locally-created-initiation state; its retry constant is five seconds. That state-level explanation is an inference from the public headers plus upstream code, not a traced assertion about the exact Debian kernel state. References: handshake state, retry constant.
No contained production shortcut is validated: the measured directional workaround fixes WAN by regressing LAN. Keep D228/D242 behavior; a production remedy needs coordinated initial peer sequencing and NAT/rekey coverage. The ordinary 31-case WAN cohort still has a 4.785/9.967 s success→observed-handshake p50/p99; that delay remains open on #201. Natural timestamps observe the latest kernel handshake after readiness, not an independently observed first-handshake event.
Separate native 0/150 ms lifecycle checks both PASS on the combined source, including original-stream byte preservation, restoration and restart. Candidate cut→echo: 22116/163042 us respectively (one cut per lifecycle; main standing table provides 31-sample quantiles). Candidate initial move is 491703/5005237 us: do not confuse startup/mode convergence with the post-handshake byte boundary. Native 300 ms review budget remains historical RED on both sides (392.67/478.13 ms against 334 ms), not waived or relabelled green.
The diagnostic scheduler retired only a verified unacquired waiter (no child or measurement), preserved two completed cases and ran the remaining four in one short bounded hold; sources and ownership receipt are attached. Final full fast has passed its real perf-VM proxy self-test but is still waiting for a host build slot at fmt; no final green claim yet.
r7-candidate-init-delivery-control.tar.gz
r7-candidate-unfiltered-headers-control.tar.gz
r7-handshake-control-packet-analysis.json
r7-handshake-control-method.tar.gz
r7-candidate-native-0.log
r7-candidate-native-150.log
r7-candidate-native-0.tar.gz
r7-candidate-native-150.tar.gz
Final verification is the only pending gate on pushed
39a13398a. All requested perf-VM acceptance/scale controls, native 0/150 lifecycles, static reproduction and matched WAN-cause controls have completed; receipts/tables are above. Full scripts/verify fast rerun passed the real VM proxy self-test and has waited for a host build slot at fmt since about 09:46 UTC (still queued at 10:04 UTC). First full attempt passed all static rows but failed the 124-byte fixture pathname; the unchanged test passes with the documented 107-byte scratch alias. No test waiver.Resource needed from orchestrator scheduling: a build slot for the remaining full gate. Four common slots are occupied (mesh-unreachable-review build + full gate, addhost-instant milestone, link-tooltip baseline build); mesh-autoswitch is absent from the reserved priority list. I am retaining the queued command and continuing to wait, without changing the global scheduler, another job, or gate checks. Snapshot attached (09:55 UTC; holders have since advanced).
r7-host-build-slot-wait.json
Reproduction inputs preserved before cleanup: baseline observer overlay (tracked patch plus all added source), separate frozen-shipping-payload provenance, and final head/qualified-source identity. Integrity rollup confirms 124/124 candidate lifecycles, 24614 exact bytes, writer 2, TLS/admission/control counts 1→1, and 124 durable mesh disables.
Final gate runner retains the unchanged scripts/verify fast command in one actual orchestrator/build-slot. Its identical proxy self-test runs on the perf VM; nonblocking VM lock refusal exits 75 and releases the host slot before any timed self-test/local gate starts, then retries. Descendant Cargo/Bun commands inherit the actual held slot. No cached self-test, skipped check or synthetic pass. Retired waiters were unacquired and their partial receipts remain separate. The active full gate is still queued for its one host slot.
r7-baseline-observer-overlay.tar.gz
r7-baseline-overlay-preservation.json
r7-final-gate-method.tar.gz
r7-final-all-lifecycle-integrity.json
11:05 UTC continuation status, head
39a13398a: no code work or perf acceptance remains pending. All requested before/after, 1000-transport and integrity controls are attached. Current static outputs equal the measured/frozen x86 and ARM payloads byte for byte; no cap drift.Only full scripts/verify fast completion is pending. Its local portion first queued around 09:46 UTC; after preserving and retiring unstarted per-command waiters, the single-slot run has queued since about 10:12 UTC. All four common slots and the reserved slot are locked. mesh-autoswitch remains outside the updated priority list. Needed from orchestrator scheduling: one host slot for the full gate; no global configuration or another job is changed. The real gate command stays queued, with no cached self-test or omitted checks. No final pass or review-ready claim.
r7-host-slot-still-queued.json
r7-current-static-output-identity.json
Gate resource update: common host slots briefly became available at 11:09 UTC while the VM was occupied; seven full-gate attempts exited 75 before the timed proxy acquired its lock. Precise phase: repository/release prechecks ran and passed each time; no timed self-test or Rust/TS phase began. Earlier shorthand "no local checks" was too broad. These are resource refusals, not accepted gate results.
The runner now waits outside host slots using read-only VM lock-ownership hints, avoiding repeated prechecks during a long VM hold. A hint is advisory only: the real timed command must still acquire nonblocking flock and emit its genuine completion metric. No cached timing, check omission or synthetic pass. Current holder: process-per-user ppu-soak-r8e-aged, started 11:09:54 UTC. The former unstarted waiter was retired by verified PID with no current check interrupted; all seven resource-attempt receipts remain. Latest runner and retirement/history are attached.
r7-final-gate-method-v2.tar.gz
Verification orchestration simplified: the prior host/VM handoff added unnecessary retries. Restore the standard whole-command orchestrator/build-slot invocation for unchanged scripts/verify fast, with the actual timed proxy self-test waiting normally on the perf-VM flock. Only unacquired VM lock-wait timeouts repeat inside that same self-test call; prechecks are not repeated. Descendant Cargo/Bun calls inherit the genuinely held host slot. Same four-slot cap, same gate checks, same shipping source/payloads, no cached timing.
The read-only-wait coordinator was retired by verified PID while it had no child and no checks active. Current standard-method runner supersedes the two earlier method attachments; they remain historical orchestration receipts only. Final gate output is still pending.
r7-final-gate-method-standard.tar.gz
Finished R5/R7 scope on pushed
39a13398ad1bfb500ed66ae5250d1ab97d8cf2de, based on dev9e84426c48f3818cebaa99acede14fec1d24ab4e. Fullscripts/verify fastexit 0; every row passed. Stop for the requested break-it review; no merge or deployment performed.Release perf-VM standing-session results; 31 fresh cases per venue per source, nearest-rank p50/p99, milliseconds. Initial mesh joining is intentionally held until the original enrolled PTY exists; all fixture overhead remains included.
All 124 candidate lifecycles across standing + natural profiles/venues passed: 24614/24614 raw terminal bytes, writer 2 and original attachment preserved, TLS/admission/control callbacks 1→1, and 124 durable mesh-disable + subsequent echo checks. No Storage failure. Baseline natural LAN retains one failed readiness operation in 31 attempts; its latency percentiles are conditional on 30 successful operations. Full natural start/success/observed-handshake/fresh-output tables and raw archives are posted above.
Scale qualification: 0/1000 exits for distinct physical paths and 0/1000 for fresh admissions, with no proof/ramp/warmup failures and 31 observation windows. Single-tuple adaptive control also 0/1000. These are co-located authenticated transport pairs, not 1000 daemons/kernel mesh nodes. The selected cadence correction reduces measured both-end CPU 2.493→1.982 cores and datagrams 143174→102338/s against the preceding
1e3cd82cctrial; CPU -20.5%, sends -28.5%. R1 contract/copy delay agrees at 10 ms; R2 invalid-mesh packet regression actually ran and passed; R4 fresh independent x86 build is byte-identical.Final shipping artifacts equal the VM measurement payloads; both static/no PT_INTERP, no hidden size slack. The x86 independent-checkout SHA is aa06e92509951c2101f750ba903044a0fbfd5f42340a4a334268e27bb8a43183.
Remaining limits are explicit: natural WAN success→observed kernel latest handshake is 4.785/9.967 s; success→fresh mesh-proven Output upper bound is 6.154/11.177 s, including readiness/terminal/status/fence overhead. This is not earliest unrestricted Output or independently observed first-handshake timing. Matched controls support a crossed initial handshake/retry interaction: suppressing one initiation direction fixes WAN while regressing LAN by ~5 s, so that workaround is not shipped; D228/D242 behavior remains. Published fallback-state p99 is slower than byte recovery (838.377 ms LAN / 1823.700 ms WAN). The historical native 300 ms review budget remains RED on both sides (392.67/478.13 ms vs 334 ms); current native 0/150 ms lifecycle checks PASS without relabelling that historical result.
Author transport checklist completed. Temporary checkouts, both build targets and fixture scratch removed; job-owned compiler daemon stopped by verified PID; frozen artifacts/evidence retained. Checkout clean and remote branch head verified. Full gate transcript follows verbatim in the next comment and is also attached, with exit/size/cleanup/remote receipts:
r7-final-verify-fast.log
r7-final-verify-fast-exit.json
r7-final-static-artifacts.json
r7-final-cleanup.json
r7-final-remote-head.txt
scripts/verify fastverbatim output on39a13398ad1bfb500ed66ae5250d1ab97d8cf2de; exit 0. SHA-256 of attached original transcript:d081ab20fd50203508b34d72fc95b271211f270e07e51e7400170258642555ad.Delta review started on review-mesh-autoswitch-r2 at target/base
39a13398ad(production base9e84426c4). #201 has no linked parent. Earlier tests a719cf3dd..b4a9a5207 were replayed; all are already incorporated, so duplicate conflicts were resolved to target copies and empty picks skipped. Original review branch and implementation worktree unchanged. Reviewing rebase range-diff, session-bound physical proofs/retry cadence, prior R1–R4, measured scale/byte latency and size. No implementation fixes, push or deployment.Delta source review: rebase range-diff and new physical-proof/cadence commits checked against D215/D263 and additive contracts. Earlier R1 passes; frozen final-artifact R4 tests pass both targets (3 tests/7 assertions including R1). Native compiler/path-normalization controls pass. Size changes retain existing transport/crypto/unwind code; direct ring/zeroize dependencies reuse existing resolved packages. Full fresh target gate and interleaving property test queued through normal build-slot since ~12:15 UTC; no compiler/check started yet. Distinct 1000-path + original R2 and 10 standing lifecycles per venue queued behind shared perf lock; no fresh measurement accepted yet. Needs host/perf scheduling capacity, not implementation permission. No push/deploy.
Independent receipt step (target
39a13398a, no implementation changes): checked all 124 raw candidate case JSON/exit receipts against venue aggregates. All exact bytes 24614/24614, writer2/same attachment, TLS/admission/control1→1 and124 durable mesh disables. Retained artifacts independently reproduce reported matched-dev size deltas: x8621409296→21320400 (-88896B), ARM19582312→19513440 (-68872B); hashes match source manifests, static/no interpreter, unwind sections retained. R1 + both frozen artifact-cap tests3PASS/7assertions. Evidence mesh-autoswitch-review/evidence/r2-{independent-124-raw-check,size-attribution}.json. Fresh gate/property builds and new perf measurements still wait for shared capacity; no fresh pass claimed.R5 P2, committed99147e84f: preserved known slower published fallback status with a real-receipt regression, scripts/mesh-status-review.test.ts.31raw standing lifecycles/venue, dev9e→39a13398a p99: LAN713552→838377us;150ms1009196→1823700us. Intended failures0PASS/2FAIL/504assertions, while byte integrity/writer retained. This is delayed displayed state, not a P1 terminal outage; natural WAN join delay also exists in production (4.770/36.648s versus4.785/9.967s). Evidence:r2-status-{receipts.json,red.log}, independent124-raw-check.json. Implementation unchanged; no push. Fresh gate/property and perf reruns remain queued.
Replay correction: all four earlier failing tests were already present on target, but the earlier passing flood control
fd25eb483was not. Restored its exact test body while preserving target probe regressions, cherry-picked as a2d77e459. It checks8388608 exact bidirectional bytes and64 path selections. This corrects the earlier shorthand that all five commits were incorporated. No runtime source change. Fresh test/gate queues remain active.Fresh independent VM208 controls PASS on hash-checked final-source fixture: original R2 executes1test/PASS; R3 original direct cohort0/1000exits/1000live; distinct authenticated-path cohort0/1000exits/1000live,31windows each, all startup exits retained. Distinct load0.19/0.29/0.77→0.78/0.41/0.80; direct0.71/0.40/0.80→0.60/0.39/0.79. Shared lock/CPUWeight10000,4workers/co-located transport pairs, not1000daemons. Fresh10/venue standing series now running. First latency launch stopped before samples because a copied runner dependency was absent; retained error, staged helper, new labelreview-standing10-r2b. Restored flood commit amended only trailing whitespace to16b0df5ea; format checkPASS. Full gate still queued; focused native build running.
Independent final-source perf rerun complete:10 fresh controlled-standing real-SSH lifecycles per venue,20/20PASS,4480/4480exactbytes, writer2/sameattachment, TLS/admission/control unchanged,20durable disables. Perf VM208/4vCPU/shared lock/CPUWeight10000. p50/max ms: LAN handshake→mesh Output7.601/9.655, cut→fallback echo23.016/24.033;150ms RTT157.261/160.337 and173.746/176.433. All byte budgets50/200msPASS; no subtraction of fixture overhead. Published fallback status remains slow: LAN521.401/777.051ms, WAN1155.650/1655.783ms. Original R2PASS and both1000direct/distinct controls0exits/1000live. Earlier124rawlifecycles independently checked. Implementation unchanged. Only focused property/flood compilation and full fresh fast gate remain; native crypto dependency still compiling, full gate queued.
Review head ae85945b9: committed interleaving property corpus (512seeds×3tuples=1536cases), restored flood16b0df5ea, status regression99147e84f. Runtime source remains target39a13398a. Property/flood execution still pending native OpenSSL dependency compilation (focused build entered normal slot12:26UTC). Full scripts/verify fast entered normal slot~12:53UTC, initial release/artifact checksPASS; real timed proxy self-test waits on the perf-VM lock before Rust/TS. Final artifact precheck at current headPASS88paths. No final gate claim. Fresh native300ms comparison of dev9e/target also queued under shared VM lock; all specifically requested scale/20standing reruns alreadyPASS. No push/deploy; continue from mesh-autoswitch-review/evidence/r2-{property,verify-fast,native-controller}.log.
Focused final-source verification PASS at review ae85945b9: interleaving property executes exactly1test (512seed schedules×3tuples=1536cases); restored flood executes exactly1test,8388608exactbytes/64pathselections/writer42. The preceding Cargo --exact short-name invocation compiled correctly but matched0tests; that empty run is retained and is not acceptance. Actual qualified executions are r2-property-executed.log and r2-flood.log. Format/diff checksPASS. Runtime remains39a13398a. Full gate initial checksPASS; currently waits at real perf-VM proxy check. Native300 comparison queued. No push/deploy.
Fresh native300ms gate,31samples/condition/source on VM208, both exit1: current-dev9e PTYecho p99 baseline324.29/other-flood338.01/same-flood359.86ms; target325.48/298.62/380.12ms. Same-flood growth35.57→54.64ms against16.67ms; ceiling334ms fails both. Target same-session tail is20.26ms worse in this pair, while medians are lower; no causal attribution/waiver. Reverse source-order repeat queued with identical fixtures/payloads/thresholds. New boundary under investigation: admitted physical-proof registry after a QUIC-validated source-port rebind; test awaiting compilation, not yet a confirmed finding. First fullfast also records an own review-test fmt failure (missed earlier nonzero check); formatting corrected, property commit amended toc6259cbdf. Retain failed transcript and rerun complete fast after current build. No implementation change/push/deploy.
R6/P2 committed
f4a6354d5: proof responder stops after QUIC accepts a new UDP source port on the original authenticated connection. Initial proofPASS; exact datagrams pass both ways after rebind, accepted.remote_address equals newsource and both original connections remain open; final physical-proof assertion fails Err(Elapsed(())),0PASS/1FAIL. VM208/sharedlock/CPUWeight10000,load0.11/0.90/0.87. Location remote_bindings.rs177 (admission-time tuple key); test remote_path_proof::review::review_admitted_proof_survives_validated_source_rebind in normal test module. No implementation fix; P2 availability, not demonstrated app byte loss/auth bypass. Evidence r2-rebind-{red.log,manifest.json}. Native300 reverse repeat: target416.70ms/prod474.67ms same-floodPTYp99, versus first prod359.86/target380.12; both sources fail334ms/isolation growth. Difference reverses, no causal newP1 established. Full fast first attempt remains running/static builds; known own-test fmt failure is corrected and full gate will rerun. No push/deploy.Fresh independent static rebuild checkpoint, review-mesh-autoswitch-r2 at
f4a6354d5, implementation39a13398a:R1 PASS (10 ms contract); R2 PASS (original authentication regression plus 1,536 interleaved seed/path cases); R3 PASS (direct and distinct cohorts each 0 exits/1,000 live); R4 PASS (fresh x86/ARM builds byte-identical to the retained candidate, 21,320,400 / 19,513,440 B, deltas -88,896 / -68,872 B). Artifact/contract controls: 3 pass, 0 fail, 7 assertions.
The first full fast run had only my test-formatting failure; that error was fixed, all other Rust/static rows passed. Final full run on
f4a6354d5is waiting for the required perf-VM network self-test lock; no completed final-gate claim yet. The mesh #201 author checklist was checked against delta receipts; natural joining, status publication and native latency limits remain separately qualified.MERGE — delta review complete. 0 P0, 0 P1, 2 P2, 0 P3. Implementation
39a13398a; review-mesh-autoswitch-r2 headf4a6354d58. Review tests only; clean worktree, no implementation fixes, push or deploy.Earlier findings: R1 PASS (10 ms contract); R2 PASS (original authentication test plus 1,536 seeded interleaved path cases); R3 PASS (fresh direct and distinct cohorts each 0 exits/1,000 live); R4 PASS (fresh x86/ARM static builds exactly reproduce retained hashes and sizes). Restored flood test PASS: 8,388,608 exact bytes, 64 route selections, writer 42.
Two open P2s, committed failing reproducers:
99147e84f: slower fallback status publication. Production→target p99 from 31 raw cases/venue: LAN 713,552→838,377 us; 150 ms RTT 1,009,196→1,823,700 us. scripts/mesh-status-review.test.ts: 0 pass/2 intended failures/504 assertions with real receipts. Byte recovery remains separate and succeeds.f4a6354d5: remote_bindings.rs:177 retains the admitted responder under its initial source tuple. Same live authenticated connection exchanges exact datagrams both ways after source-port change, but physical proof stops answering. review_admitted_proof_survives_validated_source_rebind: 0 pass/1 intended failure on VM208. Responder availability gap; no demonstrated data loss or authentication bypass. Measured source hash matches committed test.Fresh VM208 measurements under shared flock/owner file/CPUWeight10000: 10/10 LAN and 10/10 at 150 ms RTT, 4,480 exact bytes, same writer/attachment, 20 durable disables. Switch p50/max 7.601/9.655 ms and 157.261/160.337 ms; fallback echo 23.016/24.033 ms and 173.746/176.433 ms. All original 124 raw lifecycles independently checked: 24,614 exact bytes and 124 disables. Original fresh-admission cohort also checked: 1,000 proofs/live, zero proof/ramp/warmup/window exits.
Size: x86 21,409,296→21,320,400 B (−88,896); ARM 19,582,312→19,513,440 B (−68,872). Fresh hashes match retained artifacts, unwind/linkage/key-heap checks pass; the principal cut is text, no needed engine/feature removed. Review tests add no shipping bytes.
Known natural WAN witness 4.785/9.967 s versus production 4.770/36.648 s is pre-existing delay, not an established new P1. Native 300 ms budget remains RED on both sources: production→target p99 359.86→380.12 ms; reverse order target→production 416.70→474.67 ms (334 ms ceiling). Ordering reverses; no causal new P1 established and no budget waiver. No earliest-natural-mesh-byte or original SSH-shell-handoff claim.
Final scripts/verify fast PASS at the review head. The first run's sole failure was my test formatting, corrected before this full rerun; failed transcript retained. Exact final summary output follows; the full verbatim transcript is attached. Dedicated P2 reproducers remain intentionally red outside the general gate.
r2-verify-fast-final.log
r2-final-exit.json
r2-final-source-attestation.json
Own local build target/private cache and owned perf directories/helper removed; raw receipts retained. Cleanup receipts attached. Issue remains open.
Starting night-release on branch night-release at base
650472b987. Integrating reviewed mesh-unreachableefd114292with mesh-autoswitch; preserving both storage behaviours and canonical static build coverage. Release artifacts and signed staging proof only on authorized perf VM fixtures; push night-release only, no deploy. Prior issue histories retained in night-release/evidence; neither issue links a parent.Finding/blocker: required /home/kayg/.cache/codex-scratch/reconnect-release/artifacts does not exist (ls exit 2). Retained reconnect-release/result.md and log.txt record
492e1505ax86 SHA256 23fc5e92df83e49da2d351cc19fbf24983475eebc2ce98c961e03bd29254b2c6 (22,299,504 B), ARM f47a8b8396034686e91cc0ae862dc73a21be086a522ab925135669bf53b9806a (19,726,904 B), but no bytes. Exact production-binary service-upgrade proof and old-web tree comparison require those retained artifacts from the orchestrator; a source rebuild will not be substituted. Merge/new-build/independent staging qualification continues. No production or owner-host access.Web source comparison against live
1bcb7a87b: production changes are confined to apps/tracer/src/canvas/mesh/api.ts and packages/protocol/src/mesh.ts; four other changed paths are Mesh tests. packages/ui has zero changed paths. Build metadata embeds new commit and wall-clock date, so rebuilt bundle hashes also change for metadata. Old web artifact tree is absent; this is a source comparison, not a byte-for-byte artifact comparison.web-source-diff.txt
missing-production-artifacts.txt
Merged source preliminary x86 static build: 21,300,016 B; SHA256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606. Below #201 exact cap 21,320,400 B by 20,384 B and below #229 cap 21,395,600 B by 95,584 B. No profile/engine/feature change or ceiling increase. This is marker-0 build; final production-marker-1 artifacts will be frozen separately. ARM, plugin/API qualification, staging and final fast remain pending.
Both merged-source stripped static builds fit below BOTH parent caps. x86 21,300,016 B (SHA256 236e8496260f6de12d93ecd0e0ce1f920716e1029b6a3db0ecdf1bccfe055606); ARM 19,500,376 B (SHA256 0e300cec45579a7949567ffb26e014c5be7f4027add438095dddc66f54675577). Against stricter #201 ceilings: -20,384 / -13,064 B. Against #229 ceilings: -95,584 / -64,392 B. Exact caps in rust.sh and PROCESS now ratchet down to those measured bytes. Exact / +1 / shrink / unsupported-target boundary controls all PASS. These are preliminary marker-0 artifacts; marker-1 frozen release bytes, matched merge-base comparison and staging remain pending. Plugin/API focused gate is active, serially after the architecture builds.
Merged static controls PASS: both artifacts have no checkout, target, registry or absolute compiler-wrapper prefixes (2 tests / 10 assertions). Exact caps pass both targets; +1-byte, unratcheted smaller artifact and unsupported-target cases refuse. Functional artifact checks only; test-runner durations are not performance evidence. Caps remain 21,300,016 / 19,500,376 B. Focused GNU storage/compiler qualification continues before merge commit; no staging or final-gate pass yet.
static-boundaries.log
static-paths.log
Focused merged-source gate: plugin Mesh all-feature strict Clippy PASS; plugin suite 65 passed / 0 failed / 52 lab-only ignored. Server Mesh API: 34 passed / 0 failed / 2 lab scale tests ignored (492 unrelated tests filtered). Includes healthy owner/scoped inventory around malformed rows, one bad-row diagnostic across clones, repeated-error counting/recovery, real SQLite admission categories and coordinator clone counts. The coordinator uses cancellation-safe begin_immediate with typed diagnostics after conflict resolution. TypeScript and production-marked frozen artifacts are next; no final fast/staging claim.
Integration committed:
cb25dae3b983f28623d5f3af97d7f154f4946fd0merges reviewedefd114292into650472b98. One Rust remapping driver remains; compiler coverage from both sides is retained. Typed Mesh storage errors and cancellation-safe writer ownership coexist.Focused gates: fmt, strict touched-crate Clippy, 65 plugin tests, 34 server Mesh API tests, tracer check/build/transfer and compiler/size controls pass. Matched recipe at merge base: x86 21,267,152 → 21,300,016 B (+32,864); ARM 19,464,288 → 19,500,376 B (+36,088). Both exact caps are below BOTH parents. Growth above 16 KiB retains scoped healthy-host inventory, typed storage diagnostics and bounded repeated-error/recovery reporting; no foundational engine, optimization profile or dependency version changed. Section deltas separate executable code and metadata. Final release-head build/runtime/head gate still pending.
Matched artifacts · section deltas · matched baseline build
Completed separate test step
3b2183eaa: opt-in bounded repeat mode retains one standing PTY/writer for ten mesh/fallback cycles. Each cycle fences enrolled ingress for fresh mesh Output, checks the entire cumulative stream and unchanged TLS/admission/control counts, then retains its receipt. Existing initial path boundaries and durable component-disable checks remain.Per-commit checks:
cargo fmt --all --checkPASS; strict server Clippy (remote,mesh-transport-proof, all targets) PASS; bounded failure-receipt test 1 passed, 0 failed. Real cycle execution is still pending, not claimed from this compile check. Final-head artifacts are rebuilding. Staging source equals this head except the previously reviewed #225 one-way exact-release test adaptation.Final-head release artifacts frozen from
3b2183eaac8a9d8cc52a922dfbbbaecf895fd854using pinned Zig and the exact Nix make/perl Cargo recipe, marker 1. Frozen web install/build/transfer budget PASS; 256 web files, 3,362,009 identity bytes. Both native bytes equal the pre-test-commit candidates and remain below both parents' caps./home/kayg/.cache/codex-scratch/night-release/artifacts/infraternal-x86_64-musl: 21,300,016 B, SHA2565562e6d7d952926c81d1689f1a4a792aef9185a04320ff4576980f3c19e01c6a/home/kayg/.cache/codex-scratch/night-release/artifacts/infraternal-aarch64-musl: 19,500,376 B, SHA2564f4357515e328042a9bdfff367270497f4c58902f25a6530a9abb8814a300400/home/kayg/.cache/codex-scratch/night-release/artifacts/web-dist.tar: 3,573,760 B, SHA256a82f1a25756a0ee47dd7524e53b06a17bb94356e595b1c4c00cac920738feaab/home/kayg/.cache/codex-scratch/night-release/artifacts/web-SHA256SUMS: 22,410 B, SHA2566b08a5c2413f1b18a5b2addeeb10bac593c95fab8ee67ec9e7aabcca3c9a3dc9Artifact manifest · build output
Browser/signed-staging qualification is starting on the locked perf VM with these retained bytes. Original reconnect-release artifact directory and known prior VM service copies are absent; exact production-binary remote upgrade and full old-web artifact comparison remain blocked. Production unchanged. Final fast gate still pending.
#229 real-browser qualification PASS, 5/5, exact frozen x86 and web artifacts at head
3b2183eaa. Cold CA-locked inventory loads and recovers after unlock without reload (restart_after_unlock=false); healthy inventory survives a malformed name; missing storage retries after repair; disabled Mesh recovers after re-enable; subscription saturation returns a bounded error while calls work. Perf VMinfraternal-perf-test, locked CPUWeight=10000 scope, load 1.37/1.50/1.28 before and 2.07/1.65/1.33 after. Functional assertions, no visual-quality verdict.5-test transcript



Restart browser groups, signed handoff, repeated standing-terminal cycles and final fast gate remain pending. Original production artifact restoration still required for the production-binary remote upgrade.
Browser milestone PASS at head
3b2183eaa, no retry/skip: 17/17 real cases (Mesh 5, restart 2, liveness 4, three-window isolation 6) plus 5/5 mock cases. Exact frozen x86 SHA5562e6d7 and frozen web archive SHAa82f1a25; port4493. Every group ran on perf VM under a separate bounded CPUWeight10000 lock scope, with machine/load in its log. Browser cleanup audit: owned executable processes0 before/after, no owned active units/containers. Raw production-build screenshots retained; no self-assigned visual quality verdict.Restart 2 · liveness 4 · three-window 6
browser-results.json
browser-mock.log
browser-cleanup-audit.json
browser-raw.tar.gz
Signed staging, repeated standing-terminal path cycles and final head fast gate are pending. Exact old-production-binary remote upgrade and old web artifact comparison remain blocked on the absent reconnect-release artifacts. No deployment.
Signed staging milestone PASS at final head
3b2183eaaon perf VM under one bounded lock scope. Standard unchanged proof: 12 signed handoffs preserve 128/128 real PTY shim IDs and echoes; real passkey ceremony/CA state preserved and unlocked; 6 reattached retained-session sockets responsive after idle. Unusable-image refusal, automatic rollback, backup integrity (10,000 nodes / 50 users), local polling and negative-control test PASS.Supplemental one-way signed handoff into EXACT normal-trust frozen x86 SHA
5562e6d7d952926c81d1689f1a4a792aef9185a04320ff4576980f3c19e01c6a, with actual ARM/web files staged: 128/128 shim IDs and echoes, CA unlocked, 6 reattached sockets responsive. Three selected ignored tests ran 1/0 each. Original pre-handoff TCP sockets are not claimed to survive replacement. ARM staged/hash-checked, not executed on x86 VM. Exact phase is one-way; rollback is covered separately by the standard proof. Temporary authority exists only in sender/test images, not the retained release artifact.VM load before1.54/1.24/1.19, after standard tests1.50/1.27/1.21. Exact handoff listener-gap bracket28.713855ms, one trial rather than a percentile or availability guarantee. Lock released; unique deploy/r75319416 directory absent. Native 0/150ms path/lifecycle checks are now queued; final fast gate still pending. Exact old-production remote upgrade still blocked on missing original492e artifact.
staging-proof.txt
staging-source-provenance.json
native-fixture-provenance.json
Native release qualification PASS at head
3b2183eaa, both configured RTT venues0/150ms on perf VM in separate bounded locked CPUWeight10000 scopes.native-cycles-summary.json
native-results.json
native-0.log
native-150.log
native-receipts.tar.gz
final-source-provenance.json
All available staging checks now pass. Full final-head fast gate is running (timing calibration delegated to perf VM). Exact old-production remote upgrade and complete old-web artifact comparison remain blocked on absent original reconnect-release artifacts. No deployment.
Resource cleanup milestone at head
3b2183eaa: perf VM owned-process audit0 before/after; all four exact owned container names absent; browser ports4493/4494 closed; browser/native/a0/a150/s fixture directories absent. The signed proof separately removed deploy/r75319416. Only this job’s calibration source remains for possible gate rerun, no running process. Public proof-driver sources (11 files + SHA list) attached for reproduction; no runtime database or signing material in that archive.vm-cleanup-audit.json
vm-fixture-cleanup.txt
proof-drivers.tar.gz
proof-drivers.json
Head fast gate continues on host after successful perf-VM calibration. Original production artifact restoration is still required before the old-binary upgrade/comparison can be verified. No production deployment.
DO NOT DEPLOY. Required original production artifacts are absent: exact492e1505a remote-service upgrade and complete production-artifact byte comparison remain unverified. The orchestrator must restore the original native release bundle (x86 SHA23fc5e92df83e49da2d351cc19fbf24983475eebc2ce98c961e03bd29254b2c6) and the production web reference tree. No source rebuild substituted; no production deployment or owner-fixture access.
Head
3b2183eaac8a9d8cc52a922dfbbbaecf895fd854pushed tonight-release, independently verified at origin. Mergecb25dae3bpreserves both storage behaviours and one Rust remapping boundary; separate test commit3b2183eaaadds the bounded same-writer repeat mode. Checkout clean./home/kayg/.cache/codex-scratch/night-release/artifacts/infraternal-x86_64-musl5562e6d7d952926c81d1689f1a4a792aef9185a04320ff4576980f3c19e01c6a/home/kayg/.cache/codex-scratch/night-release/artifacts/infraternal-aarch64-musl4f4357515e328042a9bdfff367270497f4c58902f25a6530a9abb8814a300400/home/kayg/.cache/codex-scratch/night-release/artifacts/web-dist.tara82f1a25756a0ee47dd7524e53b06a17bb94356e595b1c4c00cac920738feaabWeb directory:
/home/kayg/.cache/codex-scratch/night-release/artifacts/web/;256/256 files matchweb-SHA256SUMS. All retained archive/native hashes match after the gate. Shipping marker1 artifacts above were used for staging; ordinary gate marker0 files have different hashes with identical sizes and were removed with the build target./home/kayg/.cache/codex-scratch/reconnect-release/artifactsremains absent.Matched merge-base size deltas with the same recipe:+32,864 B x86 /+36,088 B ARM. Both exact merged caps21,300,016/19,500,376 B are below BOTH parent caps. Above16KiB cost is justified by scoped healthy-row retention and bounded storage diagnostics/recovery; no engine, optimization profile or dependency version changed. Attribution and matched artifacts are attached earlier.
Web source versus live1bcb7a87b: shipping changes only
apps/tracer/src/canvas/mesh/api.tsandpackages/protocol/src/mesh.ts; four Mesh test files also differ;packages/uiunchanged. Build commit/date metadata changes too. This source comparison is not a complete old-artifact byte diff.Final
scripts/verify faston this head: exit0. Workspace1173 passed/0 failed/163 lab-only ignored; tracer/UI210 passed/0 failed; metadata2/0; protocol37/0. No host-load exception used. Timing calibration ran on the locked perf VM. Verbatim Rust summary and final protocol-test summary (full raw transcript attached):Cleanup: host/VM owned-process audits empty; all4 exact native containers absent; browser ports closed; entire own VM job directory removed. Own target, baseline/snapshot sources, test/runtime scratch and temporary bundles removed. Release/comparison artifacts and evidence retained outside the checkout. Author checklist completed. Issues remain open.
final-fast.log
gate-summary-verbatim.txt
gate-artifact-identity.json
final-workspace-test-counts.json
final-artifact-hash-check.txt
author-review.json
vm-final-audit.json
vm-final-cleanup.txt
host-final-process-audit.json
local-cleanup.json
final-web-file-check.txt
pushed-head.txt
Continuation start: branch night-release, base/head
3b2183eaac(unchanged). Read #229 and #201 history; neither issue links a parent. Run ONLY the remaining enrolled remote production-to-release service restart proof on isolated perf-VM fixtures, no rebuild or deployment. Production x86 SHA256 23fc5e92df83e49da2d351cc19fbf24983475eebc2ce98c961e03bd29254b2c6 (22299504 B); retained night-release x86 SHA256 5562e6d7d952926c81d1689f1a4a792aef9185a04320ff4576980f3c19e01c6a. Reuse a prebuilt enrollment/browser-terminal test controller, with isolated service orchestration; retain shim identity and byte-stream evidence.Continuation milestone, unchanged head
3b2183eaa: original production SHA23fc5e92 -> exact frozen SHA5562e6d7 completed one systemctl restart. Daemon1800232 ->1800251; same shim1800240 and session socket5fa6075f29c3448b90e8bcdb08d2c9b8.sock across replacement. Browser sequence assertions received614 bytes, lost0/duplicated0; remote reconnect assertion passed. Overall test FAILED only its cleanup assertion: retained older test controller ignores the newer configurable repository/home roots. Authenticated socket cleanup removed our retained shim and exact generated state; ps -p1800240 empty. First setup-only attempt also resolved a mount command canonicalization issue, before any service existed. Corrected private mount aliases now keep every generated path inside this job and provide the compiled cleanup-helper path. Clean rerun queued; no rebuild or artifact/source change. Not a final qualification yet.First test · First public receipt
Cleanup correction confirmed on unchanged artifacts: real enrollment/browser-terminal test PASS (1 passed,0 failed), restart output614 contiguous bytes/lost0/duplicated0, and no retained shim. Extra observer wrapper still returned1 because it sampled the post-restart shim after the fast successful test had already removed it. This is an observation race, not a product assertion failure. Final receipt run now pauses only the disposable server/test controller during process identity capture, resumes it for the actual reconnect/byte assertions, then requires normal cleanup. No rebuild, no source/artifact change.
Passing functional test · Observer race receipt
DEPLOY OK — the sole remaining production-binary remote upgrade check PASS on unchanged head
3b2183eaac. No rebuild, source change, commit, push or production deployment in this continuation.Perf VM infraternal-perf-test, exclusive perf-bench lock and CPUWeight10000 scope. Enrolled original production
492e1505a, opened one real remote PTY through an authenticated browser-protocol socket, stopped only its supervising daemon, switched the installed binary, then invoked systemctl restart exactly once. No automatic restarts (NRestarts0 before/after). Daemon1811267 ->1811286; original shim1811274 and original session socket2c94408584f744baaaa4d32d0e48950e.sock survive both the stopped interval and replacement. Browser receives614 contiguous output bytes including output produced during the stopped interval: lost0/duplicated0, session kept, reconnect PASS. Real test1 passed/0 failed; observer exit0/errors0; normal fixture cleanup PASS. Foreign-owner denial assertion also passes.A brief SIGSTOP of only the disposable server/test controller brackets the new daemon/process snapshot, followed by SIGCONT before reconnect and byte assertions. This prevents successful test cleanup racing the observer; it is functional evidence, not a latency claim. Prebuilt controller reused unchanged (SHA2560da48e0b67048a607cb66a603ecb0e56965b7e2f6524f144fc13d8ed526443ab); fixture home/repository aliases exist only in a private mount namespace; private loopback network namespace. Load before/after1.03/1.37/1.43. Earlier setup/observation failures and passing functional results remain attached above; both harness issues resolved in this final run.
Artifacts remain identical:
Cleanup: every recorded PID absent; own executable and command-process audits empty; service LoadState=not-found/ActiveState=inactive; own VM job directory removed; first attempt's exact generated state removed. Private namespaces released. Temporary input archive deleted; public receipts and release artifacts retained. Author release checklist complete; checkout clean. Issues remain open.
The earlier full scripts/verify fast on this exact head remains exit0; per continuation instructions it was not rerun because that would rebuild. Verbatim saved output (full transcript https://git.kayg.org/attachments/4107018f-fb87-4165-a832-ef41737172c8):
production-upgrade-test.log
production-upgrade-receipt.json
production-upgrade-provenance.json
production-upgrade-drivers.tar.gz
upgrade-artifact-identity.json
production-upgrade-cleanup.json
Deployed to production 2026-10-17 03:43 IST: release
3b2183eaa(dev = #201 mesh-autoswitch + #229 mesh-unreachable). x86 5562e6d7…, ARM 4f435751…, web from the same build. MainPID 41798, /healthz 200. Backup /var/backups/infraternal/20261017-001332-3b2183eaa…. release-prod-live and dev =3b2183eaa. CA locked by the restart (outcome="cold"); owner to unlock.Start mesh-autoswitch-p2 fix-forward on branch mesh-autoswitch-p2, base/head
2a5302cc7(origin/dev); production comparison3b2183eaa. Read #201 history; no linked parent. Preserve and cherry-pick R5/R6 failing tests before implementation. Scope: prompt fallback status, same-session proof after source-port rebind, natural WAN diagnosis with LAN guard, matched D243 size and final fast gate. Perf VM only; push this branch only; no deploy.Fix-forward finding: remote_bindings.rs is byte-identical between production
3b2183eaaand starting dev2a5302cc7(SHA256 87d13521ec7def53857512a353f1de1b88a93be2763a1783e0f017c8d9ab18cc). Its responder map records only admission-time tuples. The WIP repair follows the original live connection's validated remote address and original exporter MAC; it caches the moved entry and retires it by lease identity. No re-admission or new key.Status choice for measurement: keep the adaptive mesh-health guard intact; independently prove enrolled backup delivery and publish that observed delivery path while mesh discovery continues. Publication must not cancel/restart the guard. Mesh and recovery challenges use disjoint sequence parity, preserving the existing opaque u64 envelope and physical-source checks. Enrolled responses cannot authenticate mesh. Additive contract clarification proposed here before editing contracts/wire.md: an admitted responder follows validated source changes, and displayed delivery may change before conservative failure detection. No engine/dependency/profile change. New controls cover cross-path replies, publication/guard independence and the enabled observer at 1000 transport pairs. No measured improvement claimed yet.
Immutable baseline builds and the original R6 reproducer are retained outside the checkout. VM reproducer is waiting on the shared lock; no sample has run or been replaced.
Original R6 reproduced on the perf VM from immutable starting-dev sources (remote runtime matches production
3b2183eaa): initial proof succeeds, exact authenticated datagrams pass both ways after source-port change, original connections remain live, and the final proof assertion fails with Err(Elapsed(())). 0 passed / 1 failed, exit 101. Only the GNU fixture interpreter was adapted to the VM loader; shipping sources unchanged. rebind-red.logFirst focused release type check of the WIP remote plugin passed. Runtime validation of the repair, publication latency, scale and sizes remain pending. Retained the review's exact 8,388,608-byte / 64-selection flood control as well as its failing tests.
Review-test replay pushed to mesh-autoswitch-p2 at
bb7330784(66236cd48R5,e876fb63dproperty dependency,bb7330784R6). No production fix committed yet. The original R6 red receipt is attached above; first remote release type check passed. Immutable baseline native CLI is built; optimized server fixture and static comparison are still building through the normal slot. WIP source and exact resume state follow; no latency, scale, size or completion claim.wip-runtime.patch
checkpoint.txt
Completed first functional control, not release acceptance: proven_backup_delivery_is_visible_before_mesh_health_expires executes exactly one test, 1 passed / 0 failed. It requires fallback delivery publication while the original mesh guard remains alive, then a valid mesh response restores delivery on the original live connections. Original R6 still has the separate retained red receipt; repaired R6/perf controls are next on the VM. Optimized comparison server fixture is built; baseline static build continues.
recovery-functional.log
native-fixtures.json
Continuation (takeover) on mesh-autoswitch-p2: head
bb7330784(origin/dev base2a5302cc7), 8 uncommitted WIP files from the previous worker. Earlier controls-a VM run failed in the harness (missing /usr/bin/time), and controls-b was interrupted before its first case. Neither produced product evidence. Plan: keep the sound WIP, plus two refinements found on review. (a) The R6 responder repair scanned every admission on each unknown-tuple proof; it now looks only at same-IP admissions, and each moved key indexes its cached tuple, so retirement is O(1). (b) R5 recovery publication could flip to Fallback on a healthy mesh that is merely slower than the enrolled path by more than 10 ms; it now publishes only once the mesh challenge is overdue against that path's own smoothed RTT and variation. Failing tests come first, then the fixes, then perf-VM cut→status numbers for base and head (LAN and 150 ms).Pushed mesh-autoswitch-p2 at
c9b2cfda8(on top ofbb7330784):12d6d6e7cfix(remote): the admitted responder follows the live connection's QUIC-validated source change (R6). It needs the original exporter MAC, so no new key or identity is ever created. Only same-IP admissions are examined (ordered map), and moved responders are indexed for O(1) retirement.c5c0afe9dfeat(remote): an independent enrolled recovery challenge publishes Fallback delivery before the guard expires (R5). It publishes only once the mesh challenge is overdue against the mesh path's own RTT+4·var (at least 10 ms). Finding: the WIP version (publish on the first enrolled answer) flipped a healthy 40 ms mesh to Fallback. The new test slower_healthy_mesh_keeps_published_delivery was RED on the WIP ("a healthy 40ms mesh published enrolled delivery") and is GREEN after the gate.9e8367dd5feat(agent): the steer loop re-reports on each delivery change without restarting the guard.65d93ea0ftest(remote): restores the 8 MiB flood control and adds the backup-observer 1000-pair cohort.c9b2cfda8test(server): the namespace lifecycle now measures cut→status concurrently with cut→echo, so status can no longer hide behind the echo.Touched-crate gates PASS: fmt; strict clippy for remote, agent and server (mesh-transport-proof); remote lib 79 passed / 0 failed / 16 ignored; agent server_path 28 passed / 0 failed / 2 ignored. Base/head fixtures are building for the perf VM (namespace lifecycle cut→status/echo at LAN and 150 ms, interleaved base/head, plus R6/flood/scale controls). No latency claim yet.
flap-red.log
gate-touched.log
Perf VM (infraternal-perf-test, bounded lock per lifecycle, CPUWeight=10000) at head
8e5bb3c69(the static node is the c9b2cfda8-era build, and the server fixture includes the8e5bb3c69floor fix). Base = starting dev2a5302cc7, whose runtime equals prod3b2183eaa, with the same concurrent cut→status/cut→echo test change. 31 interleaved base/head namespace lifecycles per venue.R5 is NOT fixed yet. The head median improves at LAN only; tails and WAN are unchanged. Nonzero exits come only from the existing post-metric assertion "local mesh echo p99 exceeded direct by 2ms" (base 14, head 10 at LAN). Every metric line was emitted.
Finding: the delay is node-side publication, not the status RPC. A split-instrumented head fixture (15 LAN runs) records when the answering mesh.status call was issued versus when it returned. Each call costs only 11–24 ms during the cut, but the first call that sees Fallback is issued between 0.03 and 526 ms after the cut, spread roughly uniformly. So the node publishes Fallback late in both base and head. The recovery publication (and base's guard) is not firing at about cut+RTT in this lifecycle. A diagnostic fixture that logs node-side recovery/guard events is queued behind the host load throttle.
Controls on the same VM, head binary: R6 rebind PASS; alias+rebind index retirement PASS; interleaved property corpus PASS; 8 MiB flood PASS; invalid-mesh/enrolled-auth PASS; slower-healthy-mesh PASS. The early-delivery test failed 1 of 4 (race with the guard's unmeasured allowance), fixed in
8e5bb3c69and then 3/3 plus the full probe module PASS on the VM. 1000-pair scale: head direct/distinct/backup guard exits 0/0/0 (base direct 0, base distinct 2). CPU over the 7.79 s window: distinct 15.45 CPU-s, backup observer 16.65 CPU-s (+7.8%), RSS about 105–127 MB for both ends of 1000 pairs.ns-base-head-31.txt
split-0.txt
controls-c-summary.txt
controls-c-driver.log
PAUSED on the coordinator's order (host load). No build, test or perf run is active or queued for this job. Pushed head
543f5dd9eon mesh-autoswitch-p2.New since the last comment:
4f00cff7e: D243. The ordered admission index cost 16,320 B. With the same recipe, the static x86_64 binary is 21,342,928 B with it and 21,326,608 B without. Starting dev is 21,300,016 B, so the head is +26,592 B: above 16 KiB, below 64 KiB, and not yet justified or reduced. Strict clippy and the remote suite for this commit are pending (throttle). Its source compiled in the static build and in the server-fixture build.543f5dd9e: the namespace test now records STATUS_SPLIT (issue vs return of the answering status call).Resume plan: build the diagnostic fixture, then 15 LAN runs to locate the late publication; fix it with a failing test first; consider a server-visible push of link changes for WAN parity; re-measure base/head 31 per venue; run strict clippy/tests for
4f00cff7e; settle the D243 delta; then scripts/verify fast.split-150.txt
diag.patch
Closed by the orchestrator during the 2026-10-28 handover (#254), on the owner's request to close finished issues. Evidence: Mesh auto-switch deployed 2026-10-17 as
3b2183eaa, which is in dev and prod. Reopen if anything is still missing.kayg referenced this issue2026-10-28 17:53:55 +00:00