kayg/infraternal
Watch
1
0
Fork
You've already forked infraternal
 0
Code Issues 356 Pull requests Projects Releases Packages Wiki Activity Actions

D291: enroll new hosts through the existing add-host SSH connection #233

New issue
Open
opened 2026-10-16 15:55:26 +00:00 by kayg · 68 comments
kayg commented 2026-10-16 15:55:26 +00:00
Owner
Copy link

Parent/context: #230. Branch join-over-ssh, base 41a3a81abc. Implement D291 using the authenticated add-host SSH session and a private remote endpoint, retaining enrollment authentication and moving to the D215 mesh path. Cover plain SSH and NetBird SSH, rootless/rootful, forwarding refusal and interrupted join/retry. Acceptance: perf VM LAN/150ms before/after p50/p95, at most one added SSH round trip, make-before-break handoff, seven failure cases at least 20 repetitions each, matching static size comparison and scripts/verify fast. No deploy or merge; push only join-over-ssh.

Parent/context: #230. Branch join-over-ssh, base 41a3a81abceaa4ea7457d84ee05297ca2244efcc. Implement D291 using the authenticated add-host SSH session and a private remote endpoint, retaining enrollment authentication and moving to the D215 mesh path. Cover plain SSH and NetBird SSH, rootless/rootful, forwarding refusal and interrupted join/retry. Acceptance: perf VM LAN/150ms before/after p50/p95, at most one added SSH round trip, make-before-break handoff, seven failure cases at least 20 repetitions each, matching static size comparison and scripts/verify fast. No deploy or merge; push only join-over-ssh.
auth-socket-baseline-same-window.log
341 B
lifecycle-publication-before.log
1.7 KiB
primitives-gate.log
8.4 KiB
real-forward-unix.log
185 B
real-forward-disabled.log
185 B
handoff-before.log
1.5 KiB
handoff-after.log
5.9 KiB
cli-gate.log
2 KiB
admission-before.log
1.1 KiB
server-candidate.patch
44 KiB
admission-after.log
3.3 KiB
baseline-build.log
9.5 KiB
unix-parent-before.log
887 B
unix-parent-after.log
3.6 KiB
vm-baseline-smoke-runtime.log
2.4 KiB
vm-baseline-lan.log
12 KiB
reused-http-gate.log
22 KiB
vm-baseline-lan-cohort.log
40 KiB
vm-baseline-150-cohort.log
40 KiB
vm-candidate-port-blocked-smoke-after.log
2.6 KiB
automatic-forward-before.log
1.4 KiB
automatic-forward-after.log
1.1 KiB
vm-candidate-forward-delegated.log
2.5 KiB
peer-order-before.log
2.1 KiB
peer-order-after.log
6.9 KiB
mesh-ready-gate.log
5.3 KiB
current-http-symbols.txt
5.9 KiB
vm-netbird-smoke.log
2.5 KiB
bridge-gate.log
21 KiB
vm-ready-handoff-smoke.log
2.5 KiB
vm-ssh-drop-smoke.log
3.8 KiB
vm-rootless-prerequisite.log
2.7 KiB
codec-contained-gate.log
14 KiB
vm-443-prerequisite.log
2.9 KiB
plain-contained-gate.log
16 KiB
vm-443-cleanup-diagnostic.log
6.7 KiB
vm-443-file-metadata.log
1.1 KiB
vm-forwarding-disabled-cohort.log
35 KiB
plugin-storage-unit.log
1,015 B
vm-ca-locked-cohort.log
25 KiB
private-route-before.log
1.5 KiB
plain-contained-after.log
1.6 KiB
parked-bootstrap-before.log
1.2 KiB
parked-bootstrap-after.log
569 B
primed-origin-after.log
673 B
tunnel-cancel-before.log
1.2 KiB
tunnel-cancel-after.log
563 B
vm-owned-data-cleanup.log
2.7 KiB
private-route-after.log
568 B
source-checkpoint.patch
63 KiB
touched-functional-gates.log
19 KiB
production-controller-final.log
27 KiB
vm-release-baseline-lan.log
40 KiB
vm-release-baseline-150.log
40 KiB
final-x86-static.log
443 B
vm-release-final-blocked-smoke.log
2.5 KiB
pty-order-before.log
1.1 KiB
pty-order-after.log
505 B
vm-release-prior-handoff-control.log
2.5 KiB
vm-ordered-handoff-smoke.log
2.6 KiB
vm-after-facts-interruption-smoke.log
4.3 KiB
full-bootstrap-activation-before.log
569 B
tcp-primed-before.log
686 B
serial-transport-gates.log
5.3 KiB
vm-primed-link-diagnostic.log
2.8 KiB
vm-admission-trace.log
2.9 KiB
late-priming-before.log
1.2 KiB
seed-boundary-before.log
1 KiB
late-priming-before.log
1.2 KiB
late-priming-after.log
564 B
seed-boundary-after.log
200 B
vm-queue-handoff-smoke.log
2.9 KiB
queue-handoff-gates.log
11 KiB
queue-x86-size.json
327 B
queue-arm-size.json
268 B
queue-static-checks.log
640 B
vm-server-restart-smoke.log
7.4 KiB
restart-lease-before.log
1.1 KiB
restart-lease-after.log
516 B
vm-queue-release-150-smoke.log
2.7 KiB
restart-recovery-gates-short-root.log
9.7 KiB
vm-server-restart-after.log
4.3 KiB
restart-runtime-cleanup-gates.log
5.5 KiB
interrupted-record-label-gates.log
707 B
vm-ssh-drop-recovery-smoke.log
4.9 KiB
vm-slow-link-before.log
2.7 KiB
vm-server-restart-matrix.log
65 KiB
recovery-x86-size.json
414 B
recovery-arm-size.json
388 B
recovery-static-checks.log
640 B
vm-ssh-drop-matrix.log
82 KiB
vm-ssh-drop-matrix-summary.json
149 B
packet-offload-before.log
1.1 KiB
packet-offload-after.log
1.2 KiB
vm-final-after-lan.log
43 KiB
recovery-release-and-iterate.log
305 KiB
vm-final-150-and-loss-smoke.log
48 KiB
vm-final-timing-summary.json
976 B
final-verify-fast.log
304 KiB
vm-final-reliability-matrix.log
262 KiB
normalized-size-comparison.json
1.2 KiB
lifecycle-routing-before.log
1.4 KiB
lifecycle-after-gates.log
9.7 KiB
lifecycle-publication-after.log
4.5 KiB
lifecycle-routing-after.log
928 B
vm-unavailable-paths-matrix.log
98 KiB
retired-x86-size.json
445 B
retired-arm-size.json
383 B
retired-static-checks.log
640 B
retired-normalized-size.json
1.1 KiB
lifecycle-release-builds.log
2.5 KiB
storage-handoff-before.log
1.3 KiB
storage-handoff-after.log
807 B
storage-handoff-gates.log
2.5 KiB
storage-handoff-after.log
807 B
retired-final-verify-fast.log
278 KiB
continuous-static-checks.log
640 B
continuous-normalized-size.json
1.1 KiB
continuous-final-verify-fast.log
305 KiB
continuous-x86-size.json
480 B
continuous-arm-size.json
407 B
continuous-release-and-gate.log
10 KiB
restart-observer-before.log
1 KiB
restart-observer-after.log
100 B
vm-continuous-final-matrix.log
416 KiB
vm-continuous-recovery-matrix.log
152 KiB
continuous-final-summary.json
12 KiB
summarize-final-cohorts.py.txt
4.5 KiB
observer-final-verify-fast.log
259 KiB
auth-socket-head-isolated.log
259 B
auth-socket-head-same-window.log
337 B
auth-baseline-build.log
20 KiB
observer-normalized-size.json
1.1 KiB
vm-job-cleanup.log
246 B
capability-port-before.log
1.8 KiB
capability-port-after.log
783 B
capability-port-after-gates.log
2.2 KiB
port-final-verify-fast.log
305 KiB
final-x86-size.json
350 B
final-arm-size.json
352 B
final-normalized-size.json
1.6 KiB
local-cleanup-final.json
357 B
primed-origin-before.log
1.3 KiB
observer-final-verify-fast-retry.log
160 KiB
kayg commented 2026-10-16 15:57:08 +00:00
Author
Owner
Copy link

Starting #233 on join-over-ssh, base 41a3a81abc (origin/dev). Read #230 body and all 48 comments; no further parent linked. #230's final repaired head is 80a81d313, not in this base. Read CLAUDE.md, DECISIONS.md, ARCHITECTURE.md and PROCESS.md. Tracing the separate pinned HTTPS enrollment and authenticated daemon startup paths before changes. No owner fixture access, deploy or merge.

Starting #233 on join-over-ssh, base 41a3a81abceaa4ea7457d84ee05297ca2244efcc (origin/dev). Read #230 body and all 48 comments; no further parent linked. #230's final repaired head is 80a81d313, not in this base. Read CLAUDE.md, DECISIONS.md, ARCHITECTURE.md and PROCESS.md. Tracing the separate pinned HTTPS enrollment and authenticated daemon startup paths before changes. No owner fixture access, deploy or merge.
kayg commented 2026-10-16 15:58:00 +00:00
Author
Owner
Copy link

Findings before implementation: bootstrap.rs:~665 invokes a separate pinned HTTPS enrollment command; install.rs:241 then starts a daemon whose remote_server_path.rs initially dials the advertised UDP/TCP endpoint. Tunneling only HTTPS leaves readiness dependent on the blocked port. D215 already has authenticated path proof and replacement, which must be reused for the bootstrap handoff. #230 final repair is on 80a81d313 and will be incorporated as its existing atomic dependency commits before retry acceptance.

Contract proposal (D291, additive): the existing authenticated SSH master may open only attempt-scoped reverse forwards to server-owned local enrollment/agent endpoints. Ambient configuration forwarding stays disabled. Prefer an owned private remote Unix socket; servers supporting only TCP forwarding require a loopback-only ephemeral listener, with existing TLS name/pin and certificate admission unchanged. Refusal must explicitly name SSH reverse forwarding; never silently return to a direct 7443 join. Forward lifetime is bounded by the same SSH attempt; partial registration and cancellation retire with that connection. The enrollment capability still travels only on private stdin; no additional trust derives from the SSH stream.

Compatibility evidence: NetBird documents remote TCP port forwarding, and its current server registers tcpip-forward/cancel-tcpip-forward; Unix streamlocal support must not be assumed. Sources: https://docs.netbird.io/manage/peers/ssh and https://github.com/netbirdio/netbird/blob/main/client/ssh/server/server.go . Functional real-server tests will determine support/refusal behavior rather than treating documentation as acceptance.

Findings before implementation: bootstrap.rs:~665 invokes a separate pinned HTTPS enrollment command; install.rs:241 then starts a daemon whose remote_server_path.rs initially dials the advertised UDP/TCP endpoint. Tunneling only HTTPS leaves readiness dependent on the blocked port. D215 already has authenticated path proof and replacement, which must be reused for the bootstrap handoff. #230 final repair is on 80a81d313 and will be incorporated as its existing atomic dependency commits before retry acceptance. Contract proposal (D291, additive): the existing authenticated SSH master may open only attempt-scoped reverse forwards to server-owned local enrollment/agent endpoints. Ambient configuration forwarding stays disabled. Prefer an owned private remote Unix socket; servers supporting only TCP forwarding require a loopback-only ephemeral listener, with existing TLS name/pin and certificate admission unchanged. Refusal must explicitly name SSH reverse forwarding; never silently return to a direct 7443 join. Forward lifetime is bounded by the same SSH attempt; partial registration and cancellation retire with that connection. The enrollment capability still travels only on private stdin; no additional trust derives from the SSH stream. Compatibility evidence: NetBird documents remote TCP port forwarding, and its current server registers tcpip-forward/cancel-tcpip-forward; Unix streamlocal support must not be assumed. Sources: https://docs.netbird.io/manage/peers/ssh and https://github.com/netbirdio/netbird/blob/main/client/ssh/server/server.go . Functional real-server tests will determine support/refusal behavior rather than treating documentation as acceptance.
kayg commented 2026-10-16 16:08:25 +00:00
Author
Owner
Copy link

Completed API slice da75d60fa and TLS slice 13a67c5ec; both pushed to join-over-ssh. API fmt, strict all-target Clippy and tests PASS. TLS strict all-feature/all-target Clippy + full remote plugin suite: 22 passed, 0 failed. Real TLS checks prove a Unix byte relay preserves client certificate/name verification, a wrong server name is refused, and an anonymous peer offering infraternal/1 cannot become an enrolled channel. No new dependency or crate feature. Initial default-feature-only run did not exercise transport; it was replaced by the full all-feature gate. Evidence: /home/kayg/.cache/codex-scratch/join-over-ssh/evidence/primitives-gate.log (TLS passed; subsequent SSH lint found a function-length diagnostic, being corrected). No join/performance acceptance claimed yet.

#230 dependency imported as its 19 existing atomic commits, ending at 21610561a on this branch (source equivalent to repaired 80a81d313 plus this branch's decision-only base). Its original gates/VM receipts are recorded on #230; final branch gate will include it again.

Completed API slice da75d60fa and TLS slice 13a67c5ec; both pushed to join-over-ssh. API fmt, strict all-target Clippy and tests PASS. TLS strict all-feature/all-target Clippy + full remote plugin suite: 22 passed, 0 failed. Real TLS checks prove a Unix byte relay preserves client certificate/name verification, a wrong server name is refused, and an anonymous peer offering infraternal/1 cannot become an enrolled channel. No new dependency or crate feature. Initial default-feature-only run did not exercise transport; it was replaced by the full all-feature gate. Evidence: /home/kayg/.cache/codex-scratch/join-over-ssh/evidence/primitives-gate.log (TLS passed; subsequent SSH lint found a function-length diagnostic, being corrected). No join/performance acceptance claimed yet. #230 dependency imported as its 19 existing atomic commits, ending at 21610561a on this branch (source equivalent to repaired 80a81d313 plus this branch's decision-only base). Its original gates/VM receipts are recorded on #230; final branch gate will include it again.
kayg commented 2026-10-16 16:12:17 +00:00
Author
Owner
Copy link

primitives-gate.log
real-forward-unix.log
real-forward-disabled.log

Reverse-forward slice 9db7a606f pushed. Strict all-target SSH Clippy and full SSH tests PASS (61 installation cases included). Real synthetic SSH fixture: private Unix byte relay PASS; forwarding disabled PASS with the fixed reverse-forward refusal message; both retain the original SSH connection and refuse lost-master reconnect. Initial fixture directory discovery failed because its longer scratch prefix triggered the existing socket-length fallback; reruns under the shorter job scratch path pass. Counts here are functional, not host timing evidence. TCP compatibility-only fixture, production join wiring and mesh handoff are still pending.

[primitives-gate.log](https://git.kayg.org/attachments/354c8e23-a86e-4785-b4d4-ea0a09094811) [real-forward-unix.log](https://git.kayg.org/attachments/fbb7f59b-d4af-43e8-bb48-4afd6ea3b8c8) [real-forward-disabled.log](https://git.kayg.org/attachments/1d40cfb8-cb7f-4d3e-9aae-b91c6f55c8d2) Reverse-forward slice 9db7a606f pushed. Strict all-target SSH Clippy and full SSH tests PASS (61 installation cases included). Real synthetic SSH fixture: private Unix byte relay PASS; forwarding disabled PASS with the fixed reverse-forward refusal message; both retain the original SSH connection and refuse lost-master reconnect. Initial fixture directory discovery failed because its longer scratch prefix triggered the existing socket-length fallback; reruns under the shorter job scratch path pass. Counts here are functional, not host timing evidence. TCP compatibility-only fixture, production join wiring and mesh handoff are still pending.
kayg commented 2026-10-16 16:22:08 +00:00
Author
Owner
Copy link

Finding: existing remote_server_path::replacement accepts TLS plus Ping/Pong proof and transition drops the old link before the replacement server Hello is verified. New real TCP/TLS→QUIC regression withholds that Hello, then sends Ping over the bootstrap channel: fails with UnexpectedEof (0 passed, 1 failed). This does not meet the owner make-before-break requirement. Keep the original dispatch live through replacement application greeting verification, retain the negotiated decoder, and avoid sending Hello twice. Fix is in progress; no handoff acceptance claimed.
handoff-before.log

Finding: existing remote_server_path::replacement accepts TLS plus Ping/Pong proof and transition drops the old link before the replacement server Hello is verified. New real TCP/TLS→QUIC regression withholds that Hello, then sends Ping over the bootstrap channel: fails with UnexpectedEof (0 passed, 1 failed). This does not meet the owner make-before-break requirement. Keep the original dispatch live through replacement application greeting verification, retain the negotiated decoder, and avoid sending Hello twice. Fix is in progress; no handoff acceptance claimed. [handoff-before.log](https://git.kayg.org/attachments/267123de-9b1c-45f1-861a-1b42b0569154)
kayg commented 2026-10-16 16:26:07 +00:00
Author
Owner
Copy link

Installation slice d95e4d92a pushed: 62 real-filesystem installer tests pass, including two-command tunnel enrollment/install/start and private runtime retirement on commit and rollback. No new remote setup/cleanup command was added.

Host link slice 8f628a287 pushed: fixed the before-proven handoff gap. The original bootstrap dispatch remains usable until the mesh server Hello is verified; negotiated decoder/greeting state is retained and Hello is not repeated. Strict remote+mesh all-target Clippy and 33 active remote-link tests PASS, 3 existing measurement opt-ins ignored. New delayed-Hello regression fails before and passes after; no benchmark claim from build-host test timings.
handoff-after.log

Installation slice d95e4d92a pushed: 62 real-filesystem installer tests pass, including two-command tunnel enrollment/install/start and private runtime retirement on commit and rollback. No new remote setup/cleanup command was added. Host link slice 8f628a287 pushed: fixed the before-proven handoff gap. The original bootstrap dispatch remains usable until the mesh server Hello is verified; negotiated decoder/greeting state is retained and Hello is not repeated. Strict remote+mesh all-target Clippy and 33 active remote-link tests PASS, 3 existing measurement opt-ins ignored. New delayed-Hello regression fails before and passes after; no benchmark claim from build-host test timings. [handoff-after.log](https://git.kayg.org/attachments/f3de3c52-8633-414e-baa7-6e95899d65f6)
kayg commented 2026-10-16 16:39:34 +00:00
Author
Owner
Copy link

Dependency finding on this base: D228's Linux rootless userspace mesh and D196's TCP/TLS 443 mesh fallback are not implemented here. crates/plugins/mesh/src/node_actions.rs::Owner owns only node::Node; node.rs::up unconditionally creates/configures the kernel WireGuard interface. remote_dial_runtime.rs wraps ordinary Tokio UDP sockets only. crates/plugins/relay/src/lib.rs explicitly declares CLI actions before relay runtime exists; no TCP/TLS 443 mesh implementation is present. The inspected mesh-macos-node branch selects userspace only under cfg(target_os="macos"), not Linux. A working first SSH enrollment alone cannot turn these into an ongoing rootless/UDP-blocked mesh path.

Needed from orchestration: integrate/assign the decided D228 Linux userspace path and D196 443 runtime before claiming those end-to-end acceptance rows. This job is not choosing a new foundational engine or silently adding its binary cost. Continuing independently with kernel-mesh/rootful native acceptance, explicit no-path diagnostics, failure cleanup, size comparison and final gates. D291 is not claimed complete while the required paths are absent.

The perf VM already has a preexisting infraternal0 interface and infraternal_mesh table; neither was changed. Native tests use dedicated controller/target containers and their own network namespaces rather than adopting shared host state. Existing Debian systemd fixture image is available. Baseline static build is now running through this job's one build slot; no build or timing measurement runs on the VM host outside its lock.

Dependency finding on this base: D228's Linux rootless userspace mesh and D196's TCP/TLS 443 mesh fallback are not implemented here. crates/plugins/mesh/src/node_actions.rs::Owner owns only node::Node; node.rs::up unconditionally creates/configures the kernel WireGuard interface. remote_dial_runtime.rs wraps ordinary Tokio UDP sockets only. crates/plugins/relay/src/lib.rs explicitly declares CLI actions before relay runtime exists; no TCP/TLS 443 mesh implementation is present. The inspected mesh-macos-node branch selects userspace only under cfg(target_os="macos"), not Linux. A working first SSH enrollment alone cannot turn these into an ongoing rootless/UDP-blocked mesh path. Needed from orchestration: integrate/assign the decided D228 Linux userspace path and D196 443 runtime before claiming those end-to-end acceptance rows. This job is not choosing a new foundational engine or silently adding its binary cost. Continuing independently with kernel-mesh/rootful native acceptance, explicit no-path diagnostics, failure cleanup, size comparison and final gates. D291 is not claimed complete while the required paths are absent. The perf VM already has a preexisting infraternal0 interface and infraternal_mesh table; neither was changed. Native tests use dedicated controller/target containers and their own network namespaces rather than adopting shared host state. Existing Debian systemd fixture image is available. Baseline static build is now running through this job's one build slot; no build or timing measurement runs on the VM host outside its lock.
kayg commented 2026-10-16 16:42:25 +00:00
Author
Owner
Copy link

Enrollment shape f8186a1db and CLI route adapter 0884f9e21 pushed. The optional bootstrap field is omitted for legacy invitations; durable endpoint/name/issuer remain unchanged. HTTPS uses the private route before DNS and retains certificate verification. Shared Unix parents and a wildcard-bound TCP listener reached through loopback are refused. Strict touched-crate Clippy and 11 remote CLI tests PASS, including real kernel listener checks. Earlier test invocations with a source-filename filter selected zero CLI tests; the accepted gate uses remote::tests and contains all 11 results.

Current uncommitted work: server-owned attempt listener, SSH controller wiring and isolated native fixture. The source base's missing Linux userspace/443 mesh paths remain recorded separately; no rootless/443 acceptance or final completion claimed.
cli-gate.log

Enrollment shape f8186a1db and CLI route adapter 0884f9e21 pushed. The optional bootstrap field is omitted for legacy invitations; durable endpoint/name/issuer remain unchanged. HTTPS uses the private route before DNS and retains certificate verification. Shared Unix parents and a wildcard-bound TCP listener reached through loopback are refused. Strict touched-crate Clippy and 11 remote CLI tests PASS, including real kernel listener checks. Earlier test invocations with a source-filename filter selected zero CLI tests; the accepted gate uses remote::tests and contains all 11 results. Current uncommitted work: server-owned attempt listener, SSH controller wiring and isolated native fixture. The source base's missing Linux userspace/443 mesh paths remain recorded separately; no rootless/443 acceptance or final completion claimed. [cli-gate.log](https://git.kayg.org/attachments/ab1a8f39-8b13-4c3f-bd49-03bc084fbec3)
kayg commented 2026-10-16 16:47:59 +00:00
Author
Owner
Copy link

Scale finding in the candidate before acceptance: private join routers lack a browser source IP, so cloning the issuer's per-source slots made all concurrent SSH attempts share its anonymous-source quota. New eight-body admission test fails (0 passed, 1 failed): independent attempts are refused before their bodies are polled. Fix keeps one source quota per private listener while retaining the issuer's shared global signing semaphore. Public enrollment admission stays unchanged. Regression is rerunning before the server slice is committed.
admission-before.log

Scale finding in the candidate before acceptance: private join routers lack a browser source IP, so cloning the issuer's per-source slots made all concurrent SSH attempts share its anonymous-source quota. New eight-body admission test fails (0 passed, 1 failed): independent attempts are refused before their bodies are polled. Fix keeps one source quota per private listener while retaining the issuer's shared global signing semaphore. Public enrollment admission stays unchanged. Regression is rerunning before the server slice is committed. [admission-before.log](https://git.kayg.org/attachments/be78a585-0ff2-4ea5-a1ff-83eaad8179ae)
kayg commented 2026-10-16 17:03:21 +00:00
Author
Owner
Copy link

Checkpoint, pushed head 0884f9e21: API, TLS socket support, reverse forwarding, private runtimes/command saving, gap-free host handoff, optional invitation shape and CLI adapter are committed. Server activation/controller hooks, private listener/admission fixes, contract and isolated native acceptance harness remain uncommitted; attached source patch includes tracked and new files so the issue alone can restore them. Not a final gated result.

Baseline static build completed from exact merge base 41a3a81ab with the shipping recipe, one shared job target: 21,422,096 bytes. Artifact: /home/kayg/.cache/codex-scratch/join-over-ssh/artifacts/baseline-infraternal. Original source archive: scratch/baseline. The new eight-attempt admission regression now PASS after failing before the quota fix; shared global signing capacity is retained. Added a real idle-HTTPS lifetime check to the server gate.

Current single wrapped command queued since 18:55 CEST: fmt; strict Clippy for server/host/CLI; server bootstrap + remote HTTP tests with remote enabled; candidate x86_64 static build/copy. Output: scratch/evidence/server-candidate-gate.log. Next: fix gate findings, commit controller and server slices, copy the fresh GNU server-test executable plus statics and source harness to /root/infraternal-bench/join-over-ssh on the named VM. Run only owned controller/target containers under the perf lock, max 30-minute segments; never touch its preexisting mesh interface/table. scripts/join-over-ssh/acceptance.py reuses the existing Debian systemd image and memory-only SSH fixture; supports clean, blocked port, forwarding refusal, TCP-only compatibility, CA lock, netem RTT/loss. Before/after p50/p95, full failure matrix, size delta/caps and final fast are still pending. Linux rootless/443 runtime dependency remains unresolved as recorded above; no completion claimed.
server-candidate.patch
admission-after.log
baseline-build.log

Checkpoint, pushed head 0884f9e21: API, TLS socket support, reverse forwarding, private runtimes/command saving, gap-free host handoff, optional invitation shape and CLI adapter are committed. Server activation/controller hooks, private listener/admission fixes, contract and isolated native acceptance harness remain uncommitted; attached source patch includes tracked and new files so the issue alone can restore them. Not a final gated result. Baseline static build completed from exact merge base 41a3a81ab with the shipping recipe, one shared job target: 21,422,096 bytes. Artifact: /home/kayg/.cache/codex-scratch/join-over-ssh/artifacts/baseline-infraternal. Original source archive: scratch/baseline. The new eight-attempt admission regression now PASS after failing before the quota fix; shared global signing capacity is retained. Added a real idle-HTTPS lifetime check to the server gate. Current single wrapped command queued since 18:55 CEST: fmt; strict Clippy for server/host/CLI; server bootstrap + remote HTTP tests with remote enabled; candidate x86_64 static build/copy. Output: scratch/evidence/server-candidate-gate.log. Next: fix gate findings, commit controller and server slices, copy the fresh GNU server-test executable plus statics and source harness to /root/infraternal-bench/join-over-ssh on the named VM. Run only owned controller/target containers under the perf lock, max 30-minute segments; never touch its preexisting mesh interface/table. scripts/join-over-ssh/acceptance.py reuses the existing Debian systemd image and memory-only SSH fixture; supports clean, blocked port, forwarding refusal, TCP-only compatibility, CA lock, netem RTT/loss. Before/after p50/p95, full failure matrix, size delta/caps and final fast are still pending. Linux rootless/443 runtime dependency remains unresolved as recorded above; no completion claimed. [server-candidate.patch](https://git.kayg.org/attachments/0adb493d-b94f-4fdb-89e5-9b2436c3a378) [admission-after.log](https://git.kayg.org/attachments/8e384bed-6ba4-4b49-b928-8b9c72cf5900) [baseline-build.log](https://git.kayg.org/attachments/95d8a544-7688-4fa0-957c-e3945575867e)
kayg commented 2026-10-16 17:38:19 +00:00
Author
Owner
Copy link

Pushed 8c616fba1: strengthened the shared-parent test, which exposed that the CLI path check was missing. The corrected test failed before the fix; strict CLI Clippy and all 11 CLI remote tests pass afterward. Earlier passing coverage did not establish this invariant.

Before, after.

Perf-VM fixture smoke now passes a real rootful direct baseline join (1/1), exact merge-base static binary 21,422,096 bytes. The isolated controller/target containers and their two init PIDs were removed. This is fixture validation, not the requested p50/p95 dataset. Raw smoke.

Initial candidate static size is 21,704,304 bytes (+282,208 over merge base), unacceptable under D243. No cap was raised. A transport-code reuse change is under test and measurement; acceptance remains pending. Linux rootless mesh and TCP/TLS 443 runtime prerequisites remain absent as recorded above.

Pushed 8c616fba1: strengthened the shared-parent test, which exposed that the CLI path check was missing. The corrected test failed before the fix; strict CLI Clippy and all 11 CLI remote tests pass afterward. Earlier passing coverage did not establish this invariant. [Before](https://git.kayg.org/attachments/08bd1a49-3ca9-4e3f-92ef-4cd3017ab900), [after](https://git.kayg.org/attachments/7743b292-e8d0-4854-882e-49834def5ad1). Perf-VM fixture smoke now passes a real rootful direct baseline join (1/1), exact merge-base static binary 21,422,096 bytes. The isolated controller/target containers and their two init PIDs were removed. This is fixture validation, not the requested p50/p95 dataset. [Raw smoke](https://git.kayg.org/attachments/7371c76e-58c0-418f-aa9a-48ae00bfb32a). Initial candidate static size is 21,704,304 bytes (+282,208 over merge base), unacceptable under D243. No cap was raised. A transport-code reuse change is under test and measurement; acceptance remains pending. Linux rootless mesh and TCP/TLS 443 runtime prerequisites remain absent as recorded above.
kayg commented 2026-10-16 17:40:35 +00:00
Author
Owner
Copy link

Baseline LAN cohort encountered a real readiness failure on sample 5: four preceding direct rootful joins passed, then terminal readiness failed and rollback retained its receipt. Exact merge-base binary 41a3a81ab, 21,422,096 bytes; isolated perf VM containers, initial sample-5 load 0.81/0.57/0.64. Controller uses the new harness with the legacy direct transport selected. All fixture PIDs, containers and network were removed. Raw run.

The harness previously stopped the cohort on retained state; it now records each failed sample and continues, so failed baseline samples remain in the denominator. No successful-only timing claim will be made. Candidate handoff regression is covered separately; actual candidate matrix is still pending.

Baseline LAN cohort encountered a real readiness failure on sample 5: four preceding direct rootful joins passed, then terminal readiness failed and rollback retained its receipt. Exact merge-base binary 41a3a81ab, 21,422,096 bytes; isolated perf VM containers, initial sample-5 load 0.81/0.57/0.64. Controller uses the new harness with the legacy direct transport selected. All fixture PIDs, containers and network were removed. [Raw run](https://git.kayg.org/attachments/274b1efd-8047-4612-bc00-bd855a853664). The harness previously stopped the cohort on retained state; it now records each failed sample and continues, so failed baseline samples remain in the denominator. No successful-only timing claim will be made. Candidate handoff regression is covered separately; actual candidate matrix is still pending.
kayg commented 2026-10-16 17:44:58 +00:00
Author
Owner
Copy link

Pushed b2f087ada: SSH orchestration now owns the private listener through authenticated readiness and mesh handoff; capability plaintext is erased after stdin. Strict SSH/remote Clippy, 23 remote TLS tests and the complete non-opt-in SSH suite pass. Plugin gate; the same transcript later contains a server acceptor API compile error in an uncommitted size experiment, now corrected and being rechecked.

Baseline LAN formal cohort: 20/20 passed, rootful OpenSSH, check-to-Done p50=1522.0 ms, nearest-rank p95=1667 ms. Exact artifacts, machine/load and cleanup PIDs are in raw cohort. Initial pilot failure remains recorded above; it is not folded into or removed from this separate cohort. Baseline 150 ms cohort is running. No candidate performance claim yet.

Pushed b2f087ada: SSH orchestration now owns the private listener through authenticated readiness and mesh handoff; capability plaintext is erased after stdin. Strict SSH/remote Clippy, 23 remote TLS tests and the complete non-opt-in SSH suite pass. [Plugin gate](https://git.kayg.org/attachments/aebf979c-d4ef-4312-a0b9-910605aa83e5); the same transcript later contains a server acceptor API compile error in an uncommitted size experiment, now corrected and being rechecked. Baseline LAN formal cohort: 20/20 passed, rootful OpenSSH, check-to-Done p50=1522.0 ms, nearest-rank p95=1667 ms. Exact artifacts, machine/load and cleanup PIDs are in [raw cohort](https://git.kayg.org/attachments/8bf3e2bc-b745-4802-8ea8-49afc65ba01d). Initial pilot failure remains recorded above; it is not folded into or removed from this separate cohort. Baseline 150 ms cohort is running. No candidate performance claim yet.
kayg commented 2026-10-16 17:48:45 +00:00
Author
Owner
Copy link

Baseline formal 150 ms RTT cohort completed 20/20 rootful direct OpenSSH joins on the perf VM, 0% configured loss. Check-to-Done p50=9747.5 ms, nearest-rank p95=9872 ms. Same exact merge-base static binary and controller as LAN. Raw cohort, machine/load and cleanup PIDs.

Candidate blocked-port smoke initially stopped before joining because the fixture nft chain needed a delimiter; corrected and rerunning. No product success or failure is inferred from that setup error. Candidate static size remains over budget pending the HTTP code reuse build; caps unchanged.

Baseline formal 150 ms RTT cohort completed 20/20 rootful direct OpenSSH joins on the perf VM, 0% configured loss. Check-to-Done p50=9747.5 ms, nearest-rank p95=9872 ms. Same exact merge-base static binary and controller as LAN. [Raw cohort, machine/load and cleanup PIDs](https://git.kayg.org/attachments/a603eba0-04e1-4138-9fa3-0fd9b6b4cbe6). Candidate blocked-port smoke initially stopped before joining because the fixture nft chain needed a delimiter; corrected and rerunning. No product success or failure is inferred from that setup error. Candidate static size remains over budget pending the HTTP code reuse build; caps unchanged.
kayg commented 2026-10-16 17:51:09 +00:00
Author
Owner
Copy link

Blocked-port native smoke reached a production adapter omission: AutoAuth::Pending did not delegate reverse_forward to its selected authenticated connection. The tunnel failed before enrollment with the default unsupported message, despite a forwarding-capable OpenSSH server. Native failure.

Added a focused regression before the fix: it failed with the same unsupported message. Failing regression. The wrapper now delegates only after the read-only check has selected a connection; pre-auth forwarding refuses without requesting credentials. All eight authentication-order tests pass afterward, proving no additional opens or credential requests. Strict Clippy and refreshed native artifacts are pending.

Blocked-port native smoke reached a production adapter omission: AutoAuth::Pending did not delegate reverse_forward to its selected authenticated connection. The tunnel failed before enrollment with the default unsupported message, despite a forwarding-capable OpenSSH server. [Native failure](https://git.kayg.org/attachments/b840bd01-31cc-4008-bc4a-f5401c0c3168). Added a focused regression before the fix: it failed with the same unsupported message. [Failing regression](https://git.kayg.org/attachments/16654408-47f9-4874-8c5b-e4ae92ae616e). The wrapper now delegates only after the read-only check has selected a connection; pre-auth forwarding refuses without requesting credentials. All eight authentication-order tests pass afterward, proving no additional opens or credential requests. Strict Clippy and refreshed native artifacts are pending.
kayg commented 2026-10-16 17:57:07 +00:00
Author
Owner
Copy link

Pushed be6fa8a9e: contracts/ssh-bootstrap.md now specifies the additive private route, exact-session forwarding, unchanged HTTPS/mTLS authority, source quotas, command-count bound and make-before-break completion. Documentation diff check passes. These are invariants, not acceptance claims.

The second size experiment remains unacceptable: stripped x86_64 21,703,088 bytes (+280,992 over merge base). Reusing HTTPS service types did not recover the growth; no cap changed. The next release retains symbols outside the checkout so the growth can be attributed rather than guessed. Strict server Clippy, 21 bootstrap tests and eight HTTP tests pass for the in-process listener; refreshed forwarding/native tests are pending.

Pushed be6fa8a9e: contracts/ssh-bootstrap.md now specifies the additive private route, exact-session forwarding, unchanged HTTPS/mTLS authority, source quotas, command-count bound and make-before-break completion. Documentation diff check passes. These are invariants, not acceptance claims. The second size experiment remains unacceptable: stripped x86_64 21,703,088 bytes (+280,992 over merge base). Reusing HTTPS service types did not recover the growth; no cap changed. The next release retains symbols outside the checkout so the growth can be attributed rather than guessed. Strict server Clippy, 21 bootstrap tests and eight HTTP tests pass for the in-process listener; refreshed forwarding/native tests are pending.
kayg commented 2026-10-16 17:58:28 +00:00
Author
Owner
Copy link

Pushed 42034b301: automatic authentication delegates forwarding only to its selected authenticated session. Strict SSH Clippy passes; all eight authentication-order tests pass, including the regression that failed above. The test proves zero extra opens and no additional credential requests for the forward. Native artifacts are being refreshed before repeating the real blocked-port case.

Pushed 42034b301: automatic authentication delegates forwarding only to its selected authenticated session. Strict SSH Clippy passes; [all eight authentication-order tests pass](https://git.kayg.org/attachments/3f8bcb45-32c1-4b3c-bce3-92e8998af704), including the regression that failed above. The test proves zero extra opens and no additional credential requests for the forward. Native artifacts are being refreshed before repeating the real blocked-port case.
kayg commented 2026-10-16 18:02:11 +00:00
Author
Owner
Copy link

Refreshed native controller passes the real rootful OpenSSH join with direct TCP/UDP agent port 4930 blocked: private reverse forward 8.152 ms, HTTPS join 113.152 ms, authenticated terminal proof, mesh replacement and clean cleanup (1/1). The installed static binary still predates the controller-only forwarding fix; artifact hashes are recorded. Raw smoke.

Speed is not accepted: check-to-Done 7,892 ms, including 6,104.290 ms in mesh_handoff. Source inspection shows node workers can advertise a new server path before the server worker has applied that node peer. A held-server-configuration ordering regression is being run to test this hypothesis; no kernel retry cause is claimed as proven yet. Candidate cohorts remain pending.

Refreshed native controller passes the real rootful OpenSSH join with direct TCP/UDP agent port 4930 blocked: private reverse forward 8.152 ms, HTTPS join 113.152 ms, authenticated terminal proof, mesh replacement and clean cleanup (1/1). The installed static binary still predates the controller-only forwarding fix; artifact hashes are recorded. [Raw smoke](https://git.kayg.org/attachments/eb1b7cbc-2a04-43d0-99e7-b142e390517e). Speed is not accepted: check-to-Done 7,892 ms, including 6,104.290 ms in mesh_handoff. Source inspection shows node workers can advertise a new server path before the server worker has applied that node peer. A held-server-configuration ordering regression is being run to test this hypothesis; no kernel retry cause is claimed as proven yet. Candidate cohorts remain pending.
kayg commented 2026-10-16 18:08:22 +00:00
Author
Owner
Copy link

Pushed e69a1cabf: first modern node configuration now waits for its peer to be acknowledged by the server and for the advertised server listener to be bound. One event-driven watch is shared per mesh service; scoped worker ownership prevents stale teardown from clearing a successor. Existing configured nodes do not acquire this first-connection barrier.

Ordering regression fails before, six coordination tests pass after. Subsequent strict all-feature mesh Clippy and all eight library tests pass, including successor ownership; complete gate receipt will be attached when the fixture build finishes. This proves ordering, not the cause or elimination of the observed 6.1-second native wait; real timing must be repeated. No new dependency or feature was added.

Pushed e69a1cabf: first modern node configuration now waits for its peer to be acknowledged by the server and for the advertised server listener to be bound. One event-driven watch is shared per mesh service; scoped worker ownership prevents stale teardown from clearing a successor. Existing configured nodes do not acquire this first-connection barrier. [Ordering regression fails before](https://git.kayg.org/attachments/5c098937-b64e-43d0-a111-1d19bdcdb102), [six coordination tests pass after](https://git.kayg.org/attachments/79122042-3a39-4bc8-9e7c-02780958afac). Subsequent strict all-feature mesh Clippy and all eight library tests pass, including successor ownership; complete gate receipt will be attached when the fixture build finishes. This proves ordering, not the cause or elimination of the observed 6.1-second native wait; real timing must be repeated. No new dependency or feature was added.
kayg commented 2026-10-16 18:10:24 +00:00
Author
Owner
Copy link

Complete mesh readiness gate and fixture build: strict all-feature mesh Clippy, 8/8 library tests; the pinned NetBird SSH counterpart also passes gofmt, go vet and static compilation. A NetBird blocked-port smoke is queued. The build-host Docker socket required sudo -n; no credentials were inspected. The non-running extraction container was removed by its exact ID.

The symbol-retaining candidate is 21,704,424 stripped bytes, still over budget. Codec symbols show duplicated TLS HTTP codec instances across the CLI/server release-profile boundary. A single boxed server-startup future is under size measurement to confine its concrete type; no per-request allocation was added. Caps unchanged. Full final acceptance is not yet met.

Complete [mesh readiness gate and fixture build](https://git.kayg.org/attachments/df6a2a0c-1915-4548-b52f-4f8ffd94bdef): strict all-feature mesh Clippy, 8/8 library tests; the pinned NetBird SSH counterpart also passes gofmt, go vet and static compilation. A NetBird blocked-port smoke is queued. The build-host Docker socket required sudo -n; no credentials were inspected. The non-running extraction container was removed by its exact ID. The symbol-retaining candidate is 21,704,424 stripped bytes, still over budget. [Codec symbols](https://git.kayg.org/attachments/cd3684be-d914-412d-a62e-f8e5cd4a957e) show duplicated TLS HTTP codec instances across the CLI/server release-profile boundary. A single boxed server-startup future is under size measurement to confine its concrete type; no per-request allocation was added. Caps unchanged. Full final acceptance is not yet met.
kayg commented 2026-10-16 18:11:15 +00:00
Author
Owner
Copy link

Pushed 5ad6ae8a9: the pinned disposable NetBird SSH counterpart now has an explicit remote-forwarding opt-in, default off. gofmt/vet/static build pass. Rootful NetBird SSH blocked-port smoke passes 1/1 through the loopback TCP compatibility route, including terminal proof, mesh replacement and fixture cleanup. The smoke used the pre-readiness-guard artifacts and retained the ~6-second handoff delay; no optimized timing claim yet. Full matrix and rootless/443 prerequisites remain outstanding.

Pushed 5ad6ae8a9: the pinned disposable NetBird SSH counterpart now has an explicit remote-forwarding opt-in, default off. gofmt/vet/static build pass. [Rootful NetBird SSH blocked-port smoke](https://git.kayg.org/attachments/d835eda8-b495-4e42-b199-73b23873d8d0) passes 1/1 through the loopback TCP compatibility route, including terminal proof, mesh replacement and fixture cleanup. The smoke used the pre-readiness-guard artifacts and retained the ~6-second handoff delay; no optimized timing claim yet. Full matrix and rootless/443 prerequisites remain outstanding.
kayg commented 2026-10-16 18:13:34 +00:00
Author
Owner
Copy link

Pushed cfae9116b: private Unix routing retains the concrete TCP/TLS codec via an attempt-owned opaque relay. The temporary listener accepts only its initiating exact local connection and is gone before return; lease drop closes the Unix relay. The TCP compatibility path uses no relay. The first static comparison saved 15,616 bytes, but total candidate size is still over budget and not accepted. Strict remote/CLI Clippy, the 23-test remote gate and the 11-test CLI gate pass. Native smoke proves both Unix and TCP first-connection routes; repeated performance/reliability acceptance remains pending.

Pushed cfae9116b: private Unix routing retains the concrete TCP/TLS codec via an attempt-owned opaque relay. The temporary listener accepts only its initiating exact local connection and is gone before return; lease drop closes the Unix relay. The TCP compatibility path uses no relay. The first static comparison saved 15,616 bytes, but total candidate size is still over budget and not accepted. Strict remote/CLI Clippy, the 23-test remote gate and the 11-test CLI gate pass. Native smoke proves both Unix and TCP first-connection routes; repeated performance/reliability acceptance remains pending.
kayg commented 2026-10-16 18:17:16 +00:00
Author
Owner
Copy link

Pushed bdae1e5ac: attempt-owned in-process HTTPS/mTLS listener, exact host certificate admission, unchanged durable identity/endpoint, private invitation routing and mesh-confirmed completion. Its two real TLS/binding regressions are included. Pushed d07e0b287 separately: eight concurrent private enrollment sources share the global budget without sharing one anonymous source bucket; the earlier failing admission regression remains attached above.

Strict server Clippy, 22 bootstrap tests, 8 HTTP tests and the real TLS socket check pass in the current combined source. Full static/size gate and repeated native acceptance remain pending. Refreshed controller smoke is running to remeasure the mesh-ordering fix. Linux rootless mesh and TCP/TLS 443 runtime remain external prerequisites; neither is claimed implemented.

Pushed bdae1e5ac: attempt-owned in-process HTTPS/mTLS listener, exact host certificate admission, unchanged durable identity/endpoint, private invitation routing and mesh-confirmed completion. Its two real TLS/binding regressions are included. Pushed d07e0b287 separately: eight concurrent private enrollment sources share the global budget without sharing one anonymous source bucket; the earlier failing admission regression remains attached above. Strict server Clippy, 22 bootstrap tests, 8 HTTP tests and the real TLS socket check pass in the current combined source. Full static/size gate and repeated native acceptance remain pending. Refreshed controller smoke is running to remeasure the mesh-ordering fix. Linux rootless mesh and TCP/TLS 443 runtime remain external prerequisites; neither is claimed implemented.
kayg commented 2026-10-16 18:18:42 +00:00
Author
Owner
Copy link

Fresh controller with e69a1cabf passes the real blocked-port smoke and reduces measured mesh_handoff from 6,104.290 ms to 155.436 ms; check-to-Done falls from 7,892 ms to 1,605 ms. After ordering fix. Single samples only; final p50/p95 cohorts remain pending. The installed agent artifact is unchanged between these two controller runs.

SSH interruption smoke passes: the exact recorded SSH master PID is terminated during the first join command; that attempt fails, the ordinary retry finishes the earlier setup, then joins successfully. The ledger ends with one new live host plus the prior host, zero live unconsumed tokens and zero attempts. No rollback receipt remains. Raw retry and cleanup. One smoke does not satisfy the required 20 repetitions.

Fresh controller with e69a1cabf passes the real blocked-port smoke and reduces measured mesh_handoff from 6,104.290 ms to 155.436 ms; check-to-Done falls from 7,892 ms to 1,605 ms. [After ordering fix](https://git.kayg.org/attachments/c05206f1-fd00-4699-8332-6edac5070e70). Single samples only; final p50/p95 cohorts remain pending. The installed agent artifact is unchanged between these two controller runs. SSH interruption smoke passes: the exact recorded SSH master PID is terminated during the first join command; that attempt fails, the ordinary retry finishes the earlier setup, then joins successfully. The ledger ends with one new live host plus the prior host, zero live unconsumed tokens and zero attempts. No rollback receipt remains. [Raw retry and cleanup](https://git.kayg.org/attachments/5cd54857-93b0-4028-881d-256252870914). One smoke does not satisfy the required 20 repetitions.
kayg commented 2026-10-16 18:20:07 +00:00
Author
Owner
Copy link

History correction: a zero-context partial stage placed the fixture-only direct_join initializer at the wrong source line in the last two pushed commits. The working source used by all passing checks was correct. Repaired only those two commits, without changing working source, and pushed with an exact expected-head lease to preserve the atomic buildable-history rule.

Listener implementation bdae1e5ac is now 7650347878. Admission regression d07e0b287 is now 4771dd3bbc. All earlier SHAs remain unchanged. The committed initializer is now immediately inside the fixture RemoteBackend construction.

History correction: a zero-context partial stage placed the fixture-only direct_join initializer at the wrong source line in the last two pushed commits. The working source used by all passing checks was correct. Repaired only those two commits, without changing working source, and pushed with an exact expected-head lease to preserve the atomic buildable-history rule. Listener implementation bdae1e5ac is now 7650347878e8297e7865a7f1eeac8f43351b789c. Admission regression d07e0b287 is now 4771dd3bbc6d75b05af851d3032fdecbfb51c22c. All earlier SHAs remain unchanged. The committed initializer is now immediately inside the fixture RemoteBackend construction.
kayg commented 2026-10-16 18:23:24 +00:00
Author
Owner
Copy link

Rootless diagnostic confirms first connection through SSH: join, service start, authenticated facts and terminal proof all pass. Ongoing mesh never becomes available; after the bounded handoff wait it reports that enrollment succeeded through SSH but mesh could not be established, rolls back successfully and leaves prior authority intact with zero live invitations or attempts. This is an expected-failure diagnostic, not rootless acceptance. Raw diagnostic.

Containing the HTTPS startup future saves 128,320 stripped bytes: candidate 21,576,104, still +154,008 against merge base and unacceptable. The plain HTTP startup future has the same profile-boundary duplication; its equivalent containment is being measured. Only one allocation at startup is introduced, outside request dispatch. No cap was raised. Completed server/size gate.

Rootless diagnostic confirms first connection through SSH: join, service start, authenticated facts and terminal proof all pass. Ongoing mesh never becomes available; after the bounded handoff wait it reports that enrollment succeeded through SSH but mesh could not be established, rolls back successfully and leaves prior authority intact with zero live invitations or attempts. This is an expected-failure diagnostic, not rootless acceptance. [Raw diagnostic](https://git.kayg.org/attachments/8fc3992e-e523-4fda-86a2-1796f88da491). Containing the HTTPS startup future saves 128,320 stripped bytes: candidate 21,576,104, still +154,008 against merge base and unacceptable. The plain HTTP startup future has the same profile-boundary duplication; its equivalent containment is being measured. Only one allocation at startup is introduced, outside request dispatch. No cap was raised. [Completed server/size gate](https://git.kayg.org/attachments/70ab7ce6-d5ff-4709-acb2-fa0aaf90e819).
kayg commented 2026-10-16 18:28:24 +00:00
Author
Owner
Copy link

UDP-blocked diagnostic: SSH join, authenticated facts and terminal proof succeed, then mesh handoff is unavailable and produces the explicit message as expected from the missing 443 runtime. Rollback itself fails and retains its receipt, so the diagnostic is FAILED; fixture processes/network still cleanly removed. Raw diagnostic. Adding fixed numeric rollback status and public socket/filesystem metadata to isolate the cleanup failure. The missing transport does not excuse incomplete cleanup.

Latest functional gate passes strict server Clippy and the targeted bootstrap/TLS checks but fails saturated_native_control_queue_does_not_hide_primary_terminal_output in the broader runtime group (46 passed, 1 failed, 3 ignored). It is not one of the allowed load exceptions and is not accepted as green. Running the exact case again; no baseline or causality claim yet. Static build in that run did not execute because the test failed.

UDP-blocked diagnostic: SSH join, authenticated facts and terminal proof succeed, then mesh handoff is unavailable and produces the explicit message as expected from the missing 443 runtime. Rollback itself fails and retains its receipt, so the diagnostic is FAILED; fixture processes/network still cleanly removed. [Raw diagnostic](https://git.kayg.org/attachments/29a2f0e1-e994-4721-89ea-b330507cc966). Adding fixed numeric rollback status and public socket/filesystem metadata to isolate the cleanup failure. The missing transport does not excuse incomplete cleanup. [Latest functional gate](https://git.kayg.org/attachments/357382ab-68bf-49db-9662-ea8c98e1d513) passes strict server Clippy and the targeted bootstrap/TLS checks but fails saturated_native_control_queue_does_not_hide_primary_terminal_output in the broader runtime group (46 passed, 1 failed, 3 ignored). It is not one of the allowed load exceptions and is not accepted as green. Running the exact case again; no baseline or causality claim yet. Static build in that run did not execute because the test failed.
kayg commented 2026-10-16 18:36:53 +00:00
Author
Owner
Copy link

Cleanup diagnosis: service MainPID=0, ActiveState=inactive, SubState=dead, with no remaining /var/lib/infraternal socket in /proc/net/unix, but the receipt remains. Public process/socket metadata. During the same owned fixture, filesystem metadata shows mesh-public.json and mesh-routes.json in /var/lib/infraternal/sessions. finish_fresh correctly refuses these unjournaled runtime files.

Proposed/following fix: fresh installed plugin state selects the installation data directory, which existing transaction rollback already owns. Existing runtime-side state retains its location for rollback compatibility; no migration or broad deletion is introduced. The choice stays inside the mesh plugin. Other node roles remain available if its state coordinate is inaccessible. A pure storage-selection regression passes; strict full-feature checks and real UDP-blocked cleanup recheck are pending.

Cleanup diagnosis: service MainPID=0, ActiveState=inactive, SubState=dead, with no remaining /var/lib/infraternal socket in /proc/net/unix, but the receipt remains. [Public process/socket metadata](https://git.kayg.org/attachments/0d934e7d-025d-40f8-86ca-d1e6d81640c7). During the same owned fixture, [filesystem metadata](https://git.kayg.org/attachments/73fdd92f-3676-4a42-a0ea-f258e8c8ed8a) shows mesh-public.json and mesh-routes.json in /var/lib/infraternal/sessions. finish_fresh correctly refuses these unjournaled runtime files. Proposed/following fix: fresh installed plugin state selects the installation data directory, which existing transaction rollback already owns. Existing runtime-side state retains its location for rollback compatibility; no migration or broad deletion is introduced. The choice stays inside the mesh plugin. Other node roles remain available if its state coordinate is inaccessible. A pure storage-selection regression passes; strict full-feature checks and real UDP-blocked cleanup recheck are pending.
kayg commented 2026-10-16 18:43:03 +00:00
Author
Owner
Copy link

Forwarding-disabled cohort passes 20/20: every run reports the explicit reverse-forwarding refusal, retires the new invitation/host authority, removes attempt state and proves fixture cleanup. Raw cohort. This pre-start restriction does not execute the uploaded older agent artifact; exact hashes are recorded. Locked-CA 20-run cohort is next.

Storage root unit passes: fresh state selects data; existing runtime state stays in place; existing data wins without moving or deleting legacy files. Full-feature Clippy and real cleanup recheck remain pending.

Strengthened the SSH interruption fixture: it will now terminate the recorded master only after authenticated host facts prove the newly installed service and first channel are live. The earlier one-run 10 ms join-command pilot remains useful but does not validate this stronger case; it will be repeated at least 20 times.

Forwarding-disabled cohort passes 20/20: every run reports the explicit reverse-forwarding refusal, retires the new invitation/host authority, removes attempt state and proves fixture cleanup. [Raw cohort](https://git.kayg.org/attachments/ad23ea87-1b8c-40a0-9825-c967f15c2fec). This pre-start restriction does not execute the uploaded older agent artifact; exact hashes are recorded. Locked-CA 20-run cohort is next. [Storage root unit](https://git.kayg.org/attachments/a255f285-23e0-45a7-8b51-eeffb9c8f7a7) passes: fresh state selects data; existing runtime state stays in place; existing data wins without moving or deleting legacy files. Full-feature Clippy and real cleanup recheck remain pending. Strengthened the SSH interruption fixture: it will now terminate the recorded master only after authenticated host facts prove the newly installed service and first channel are live. The earlier one-run 10 ms join-command pilot remains useful but does not validate this stronger case; it will be repeated at least 20 times.
kayg commented 2026-10-16 18:49:21 +00:00
Author
Owner
Copy link

Locked-CA cohort passes 20/20, before upload, preserving the prior host and leaving zero live new invitations or attempts. Raw cohort.

First-route regression: an unavailable private SSH endpoint still caused first_connection to dial the ordinary agent listener. A real-TLS regression fails before the fix (before); the private-route error now propagates to bounded retry instead. Existing mesh proposals and manual legacy joins keep their existing selection. The focused test passes after; full touched-crate gates are pending.

Measurement qualification: current native controller artifacts were built with the test profile; installed binaries are static release builds. Existing cohort timings are diagnostic relative measurements, not final production-profile acceptance. Final before/after cohorts will use a release-optimized controller and explicit artifact hashes. No speed acceptance is claimed yet.

Locked-CA cohort passes 20/20, before upload, preserving the prior host and leaving zero live new invitations or attempts. [Raw cohort](https://git.kayg.org/attachments/a2c7d004-f373-4561-b5dd-7ca40dc32d07). First-route regression: an unavailable private SSH endpoint still caused first_connection to dial the ordinary agent listener. A real-TLS regression fails before the fix ([before](https://git.kayg.org/attachments/ba5abbf9-1e61-4684-8655-2d5e690c2187)); the private-route error now propagates to bounded retry instead. Existing mesh proposals and manual legacy joins keep their existing selection. The focused test passes after; full touched-crate gates are pending. Measurement qualification: current native controller artifacts were built with the test profile; installed binaries are static release builds. Existing cohort timings are diagnostic relative measurements, not final production-profile acceptance. Final before/after cohorts will use a release-optimized controller and explicit artifact hashes. No speed acceptance is claimed yet.
kayg commented 2026-10-16 19:08:55 +00:00
Author
Owner
Copy link

Size experiment now fits the existing x86_64 ceiling without raising it: 21,306,216 stripped bytes, -115,880 against the original merge-base build. Both HTTP startup futures retain their concrete codecs inside the server crate, with one allocation per listener startup. Gate/size receipt. This artifact predates the latest first-route and priming fixes; final normal-build sizes for both targets remain pending. The previously failed control-queue case passes the exact rerun; full final gate is still required.

Continuity correction: the original held-greeting test did not model ordinary server admission retiring its previous actor. With that behavior modeled, it fails (before). Bootstrap now reuses the existing primed standby greeting/activation path, verifies the parked greeting before old-link retirement, and shares its implementation with warm probes; no new wire protocol. The strengthened client regression passes (after).

A real server regression also found the bootstrap-origin flag was retained after primed activation (before); it now changes only after activation and passes (after). Full touched-crate gates and a release-profile native controller are being built before final matrix/timing runs.

Size experiment now fits the existing x86_64 ceiling without raising it: 21,306,216 stripped bytes, -115,880 against the original merge-base build. Both HTTP startup futures retain their concrete codecs inside the server crate, with one allocation per listener startup. [Gate/size receipt](https://git.kayg.org/attachments/80c65aee-13d9-4a25-882e-4b9cc91b339c). This artifact predates the latest first-route and priming fixes; final normal-build sizes for both targets remain pending. The previously failed control-queue case passes the exact rerun; full final gate is still required. Continuity correction: the original held-greeting test did not model ordinary server admission retiring its previous actor. With that behavior modeled, it fails ([before](https://git.kayg.org/attachments/7229dd8a-dadd-4da6-afc2-4860e34d1c02)). Bootstrap now reuses the existing primed standby greeting/activation path, verifies the parked greeting before old-link retirement, and shares its implementation with warm probes; no new wire protocol. The strengthened client regression passes ([after](https://git.kayg.org/attachments/aa71c325-6b1c-42b4-be95-1393e3290cff)). A real server regression also found the bootstrap-origin flag was retained after primed activation ([before](https://git.kayg.org/attachments/82cc58a7-e5b7-4ef9-bc5e-95a4ccc03ecf)); it now changes only after activation and passes ([after](https://git.kayg.org/attachments/41fd72f6-27f9-4cb8-9b2d-804a9a7500a9)). Full touched-crate gates and a release-profile native controller are being built before final matrix/timing runs.
kayg commented 2026-10-16 19:31:03 +00:00
Author
Owner
Copy link

Additional lifecycle regression: an accepted but unfinished TLS handshake outlived its dropped private listener until the handshake deadline (before). Attempt cancellation now also covers the acceptor future, independently of HTTP connection watchers, and the real-socket regression passes (after). Idle accepted HTTPS remains covered separately. Full touched-crate Clippy/tests and release-profile controller build are queued.

A fresh diagnostic run now uses the 21,306,216-byte agent artifact to recheck the owned data-directory rollback fix with UDP blocked. This is an expected no-path diagnostic while the 443 runtime is absent; it must nevertheless remove its setup state.

Additional lifecycle regression: an accepted but unfinished TLS handshake outlived its dropped private listener until the handshake deadline ([before](https://git.kayg.org/attachments/e83ae923-0a80-4806-ad64-c0e40752e097)). Attempt cancellation now also covers the acceptor future, independently of HTTP connection watchers, and the real-socket regression passes ([after](https://git.kayg.org/attachments/a4872815-dc7a-4461-a896-bad572f4cd4f)). Idle accepted HTTPS remains covered separately. Full touched-crate Clippy/tests and release-profile controller build are queued. A fresh diagnostic run now uses the 21,306,216-byte agent artifact to recheck the owned data-directory rollback fix with UDP blocked. This is an expected no-path diagnostic while the 443 runtime is absent; it must nevertheless remove its setup state.
kayg commented 2026-10-16 19:33:06 +00:00
Author
Owner
Copy link

Owned-data rollback fix passes the real rootful UDP-blocked diagnostic: join/facts/terminal succeed; no ongoing mesh path is available; the explicit message is emitted; rollback succeeds; no receipt remains; prior host preserved, zero live invitations and attempts. Raw recheck. This proves clean refusal, not the still-missing 443 transport.

The first-route failure regression also passes after propagating private-route errors (after). Current uncommitted source checkpoint is attached separately; atomic commits await their complete touched-crate gates.

Owned-data rollback fix passes the real rootful UDP-blocked diagnostic: join/facts/terminal succeed; no ongoing mesh path is available; the explicit message is emitted; rollback succeeds; no receipt remains; prior host preserved, zero live invitations and attempts. [Raw recheck](https://git.kayg.org/attachments/ab3cfbc2-a0e7-4bf4-9167-c23bc9d8a554). This proves clean refusal, not the still-missing 443 transport. The first-route failure regression also passes after propagating private-route errors ([after](https://git.kayg.org/attachments/13aeb059-5e7a-4435-850a-b916f35e7599)). Current uncommitted source checkpoint is attached separately; atomic commits await their complete touched-crate gates.
kayg commented 2026-10-16 19:36:20 +00:00
Author
Owner
Copy link

Continuation checkpoint: branch join-over-ssh, pushed head 4771dd3bbc. Pending source and acceptance harness. Subsequent unfinished-handshake cancellation is described and attached above. Working-tree code remains uncommitted until the complete touched-crate gate passes.

The combined gate/release-controller build is waiting for the shared build slot. Final work still required: release-profile before/after LAN/150ms cohorts; 20-run final-artifact matrix, including the strengthened SSH interruption and server restart; final normal-build x86_64/aarch64 sizes and ceiling ratchets; scripts/verify iterate and scripts/verify fast. Linux userspace/rootless mesh and TCP/TLS 443 runtime remain missing prerequisites; the current branch does not implement either. No deploy or merge.

Continuation checkpoint: branch join-over-ssh, pushed head 4771dd3bbc6d75b05af851d3032fdecbfb51c22c. [Pending source and acceptance harness](https://git.kayg.org/attachments/b9a04468-f466-475c-ac2a-86266ee0ba44). Subsequent unfinished-handshake cancellation is described and attached above. Working-tree code remains uncommitted until the complete touched-crate gate passes. The combined gate/release-controller build is waiting for the shared build slot. Final work still required: release-profile before/after LAN/150ms cohorts; 20-run final-artifact matrix, including the strengthened SSH interruption and server restart; final normal-build x86_64/aarch64 sizes and ceiling ratchets; scripts/verify iterate and scripts/verify fast. Linux userspace/rootless mesh and TCP/TLS 443 runtime remain missing prerequisites; the current branch does not implement either. No deploy or merge.
kayg commented 2026-10-16 20:00:16 +00:00
Author
Owner
Copy link

Reviewer checklist checkpoint (docs/ssh-bootstrap-evidence.md): attempt ownership remains in the SSH plugin/backend; private-route errors no longer introduce a public-port dependency; initial handoff uses the existing primed activation protocol; canceled acceptors close unfinished TLS; fresh mesh public state is inside journaled installation data. No dependencies or protocol frames added. Focused before/after regressions are linked above. Combined strict touched-crate gates and the release controller have been queued for more than 30 minutes behind the shared build slots; no gate result or final acceptance is claimed. Full native server-restart matrix, final release timing cohorts, both exact static ceilings and final verify fast remain pending. Linux rootless mesh and mesh TCP/TLS 443 remain missing runtime prerequisites.

Reviewer checklist checkpoint (docs/ssh-bootstrap-evidence.md): attempt ownership remains in the SSH plugin/backend; private-route errors no longer introduce a public-port dependency; initial handoff uses the existing primed activation protocol; canceled acceptors close unfinished TLS; fresh mesh public state is inside journaled installation data. No dependencies or protocol frames added. Focused before/after regressions are linked above. Combined strict touched-crate gates and the release controller have been queued for more than 30 minutes behind the shared build slots; no gate result or final acceptance is claimed. Full native server-restart matrix, final release timing cohorts, both exact static ceilings and final verify fast remain pending. Linux rootless mesh and mesh TCP/TLS 443 remain missing runtime prerequisites.
kayg commented 2026-10-16 20:03:39 +00:00
Author
Owner
Copy link

Pushed e3d52bcd0: D291 contract now specifies parked greeting verification and explicit activation, bounded completion if SSH independently disappears, and no silent legacy-port dial when the private route fails. cargo fmt --all --check and git diff --check pass. The short-lived preceding documentation SHA was amended only to preserve real paragraph breaks in its commit body; source unchanged. Combined implementation gate has acquired a slot: strict agent/mesh and server Clippy pass; tests/release controller remain running.

Pushed e3d52bcd0: D291 contract now specifies parked greeting verification and explicit activation, bounded completion if SSH independently disappears, and no silent legacy-port dial when the private route fails. cargo fmt --all --check and git diff --check pass. The short-lived preceding documentation SHA was amended only to preserve real paragraph breaks in its commit body; source unchanged. Combined implementation gate has acquired a slot: strict agent/mesh and server Clippy pass; tests/release controller remain running.
kayg commented 2026-10-16 20:06:39 +00:00
Author
Owner
Copy link

Atomic steps pushed after formatting and strict touched-crate Clippy:

  • aa15da96b: fresh mesh state uses journaled installation data; existing runtime state stays in place. Mesh library 40 passed, 12 opt-in checks ignored; real UDP-blocked cleanup recheck linked above passes.
  • 71ee42826: failed private first route propagates to bounded retry instead of dialing the public agent listener. Real TLS regression fails before, passes after; agent path group 34 passed, 3 opt-in checks ignored.
  • 94f1f0a24: existing primed standby/activation retains the first link through replacement readiness; server clears bootstrap origin only after activation. Strengthened continuity and activation regressions pass; remote runtime 39 passed, 3 opt-in checks ignored.
  • 80480463f: listener drop cancels unfinished TLS admission. Real unfinished-handshake and accepted-idle-HTTPS checks pass; bootstrap group 23 passed, 2 opt-in checks ignored.
  • 8010359e7: contain HTTP/HTTPS startup futures inside the server crate to reuse codecs across release profiles. Prior size experiment 21,306,216 bytes (-115,880 vs original base); final full-code normal builds still pending.

Completed touched-crate functional gates. The same slot is now compiling the release-optimized native controller. No final fast-gate or complete native acceptance claim.

Atomic steps pushed after formatting and strict touched-crate Clippy: - aa15da96b: fresh mesh state uses journaled installation data; existing runtime state stays in place. Mesh library 40 passed, 12 opt-in checks ignored; real UDP-blocked cleanup recheck linked above passes. - 71ee42826: failed private first route propagates to bounded retry instead of dialing the public agent listener. Real TLS regression fails before, passes after; agent path group 34 passed, 3 opt-in checks ignored. - 94f1f0a24: existing primed standby/activation retains the first link through replacement readiness; server clears bootstrap origin only after activation. Strengthened continuity and activation regressions pass; remote runtime 39 passed, 3 opt-in checks ignored. - 80480463f: listener drop cancels unfinished TLS admission. Real unfinished-handshake and accepted-idle-HTTPS checks pass; bootstrap group 23 passed, 2 opt-in checks ignored. - 8010359e7: contain HTTP/HTTPS startup futures inside the server crate to reuse codecs across release profiles. Prior size experiment 21,306,216 bytes (-115,880 vs original base); final full-code normal builds still pending. [Completed touched-crate functional gates](https://git.kayg.org/attachments/918e7c3a-fb95-4e7b-8e51-a5731aab3801). The same slot is now compiling the release-optimized native controller. No final fast-gate or complete native acceptance claim.
kayg commented 2026-10-16 20:27:34 +00:00
Author
Owner
Copy link

Release-controller build completed successfully after the strict functional gates: complete output. Controller SHA256 6d05f6b6a760308c401a2e7add44035b5cda10fd330aa5ff30d97014f100ada8.

Final-profile before measurement, original merge-base static agent ed277078b043873512ac04cdbd5433ceecca533789108705665cbec899bbde65: LAN 20/20 joins, check-to-Done p50 1754 ms, p95 1924 ms. Every sample retained the prior host, ended with zero live unconsumed invitations and zero attempts, and proved owned container/network/PID cleanup. Perf VM 4 CPU, kernel 6.12.111+deb13-amd64; per-sample load and fixed image are in raw cohort. The 150 ms cohort is queued next. Final static candidate and aarch64 build plus verify iterate are running serially in one build slot; no final acceptance yet.

Release-controller build completed successfully after the strict functional gates: [complete output](https://git.kayg.org/attachments/ff9ac0d6-450d-4d7d-bf4f-7d543f62b87b). Controller SHA256 6d05f6b6a760308c401a2e7add44035b5cda10fd330aa5ff30d97014f100ada8. Final-profile before measurement, original merge-base static agent ed277078b043873512ac04cdbd5433ceecca533789108705665cbec899bbde65: LAN 20/20 joins, check-to-Done p50 1754 ms, p95 1924 ms. Every sample retained the prior host, ended with zero live unconsumed invitations and zero attempts, and proved owned container/network/PID cleanup. Perf VM 4 CPU, kernel 6.12.111+deb13-amd64; per-sample load and fixed image are in [raw cohort](https://git.kayg.org/attachments/06991fbb-78a7-44ff-b8ca-475bd310fc0f). The 150 ms cohort is queued next. Final static candidate and aarch64 build plus verify iterate are running serially in one build slot; no final acceptance yet.
kayg commented 2026-10-16 20:36:18 +00:00
Author
Owner
Copy link

Final-profile before measurement at 150 ms RTT, 0% loss: 20/20 joins; check-to-Done p50 9953 ms, p95 10088 ms. Same release controller and original-base static agent hashes as the LAN cohort. Every run preserves prior authority and ends with no live new invitation/attempt and full fixture cleanup. Raw 150 ms cohort.

Final normal-recipe x86_64 static build is 21,307,216 bytes, versus original base 21,422,096: -114,880 bytes. It is 25,248 below the existing ceiling; no allowance was added. SHA256 98e387b1e18870bc50882d426840b96f75059b62bc49fbfb8bbc8edf15588dba; static PIE, no PT_INTERP. Artifact receipt. Arm64 is still building, followed by verify iterate. Final-artifact blocked-port smoke is queued for the shared perf lock. Native server-restart matrix and the rootless/443 runtime prerequisites remain unresolved; full acceptance is not claimed.

Final-profile before measurement at 150 ms RTT, 0% loss: 20/20 joins; check-to-Done p50 9953 ms, p95 10088 ms. Same release controller and original-base static agent hashes as the LAN cohort. Every run preserves prior authority and ends with no live new invitation/attempt and full fixture cleanup. [Raw 150 ms cohort](https://git.kayg.org/attachments/7e404e7b-4ce7-4747-9886-33360046f1b2). Final normal-recipe x86_64 static build is 21,307,216 bytes, versus original base 21,422,096: -114,880 bytes. It is 25,248 below the existing ceiling; no allowance was added. SHA256 98e387b1e18870bc50882d426840b96f75059b62bc49fbfb8bbc8edf15588dba; static PIE, no PT_INTERP. [Artifact receipt](https://git.kayg.org/attachments/c57b76d1-473b-4fcb-bfe3-b218174bf35b). Arm64 is still building, followed by verify iterate. Final-artifact blocked-port smoke is queued for the shared perf lock. Native server-restart matrix and the rootless/443 runtime prerequisites remain unresolved; full acceptance is not claimed.
kayg commented 2026-10-16 20:45:40 +00:00
Author
Owner
Copy link

Final static candidate blocked-port smoke succeeds functionally but fails the speed bar: check-to-Done 11929 ms, with 10289.620 ms in mesh_handoff; join 122.065 ms, facts 61.535 ms, readiness 45.807 ms. Raw final-artifact smoke. Not accepted as fast. Comparing the previous non-primed client with the same optimized controller. A suspected race is the temporary readiness writer opening on the initial TCP link before primed activation; existing native standby fixtures begin from QUIC. Proposed contract refinement: prove/activate the ongoing path before opening the readiness PTY, so terminal proof exercises the final path and does not become a retained legacy writer during priming. Adding a red ordering regression before changing the pipeline; no mechanism claim until measured recheck. Arm64 normal build is 19,523,832 bytes (784 below the existing ceiling); verify iterate is running.

Final static candidate blocked-port smoke succeeds functionally but fails the speed bar: check-to-Done 11929 ms, with 10289.620 ms in mesh_handoff; join 122.065 ms, facts 61.535 ms, readiness 45.807 ms. [Raw final-artifact smoke](https://git.kayg.org/attachments/468eb9b0-8445-4609-abfe-7f695244ea4f). Not accepted as fast. Comparing the previous non-primed client with the same optimized controller. A suspected race is the temporary readiness writer opening on the initial TCP link before primed activation; existing native standby fixtures begin from QUIC. Proposed contract refinement: prove/activate the ongoing path before opening the readiness PTY, so terminal proof exercises the final path and does not become a retained legacy writer during priming. Adding a red ordering regression before changing the pipeline; no mechanism claim until measured recheck. Arm64 normal build is 19,523,832 bytes (784 below the existing ceiling); verify iterate is running.
kayg commented 2026-10-16 20:53:09 +00:00
Author
Owner
Copy link

Ongoing-path/readiness ordering regression is red before (before): the real installation-script fixture fails with "Readiness PTY opened before ongoing-path proof." Moving tunnel_ready ahead of terminal_canary makes the focused case green (after). This also makes the readiness PTY exercise the final proved path. Changes remain uncommitted until strict SSH checks and the optimized native recheck; the hypothesis for the 10-second stall is not yet confirmed. Current verify iterate is still running and is not the final gate.

Ongoing-path/readiness ordering regression is red before ([before](https://git.kayg.org/attachments/759cbf9d-b6e2-4beb-a136-2576da84f4f1)): the real installation-script fixture fails with "Readiness PTY opened before ongoing-path proof." Moving tunnel_ready ahead of terminal_canary makes the focused case green ([after](https://git.kayg.org/attachments/65830ba2-a417-4f29-9ec0-c4c2d170605a)). This also makes the readiness PTY exercise the final proved path. Changes remain uncommitted until strict SSH checks and the optimized native recheck; the hypothesis for the 10-second stall is not yet confirmed. Current verify iterate is still running and is not the final gate.
kayg commented 2026-10-16 20:57:31 +00:00
Author
Owner
Copy link

Pushed 1ea55304c: prove the ongoing path before opening the temporary readiness PTY; 4d120e1f6 separately documents that ordering. Formatting, strict SSH Clippy and all 63 installation tests pass. The focused red/green receipts are linked above. The first verify iterate run had exactly one failure, that new ordering regression compiled before the fix; fmt, VT native fmt, plugin compile-out, Clippy and cargo-deny all passed. A fresh canonical iterate run follows the corrected release builds.

With the same optimized controller, the previous non-primed client control completes the real blocked-port join in 1696 ms with mesh_handoff 158.381 ms, versus the new primed client smoke 11929 ms / 10289.620 ms. Control receipt. This narrows the regression to the new transition, without yet proving the readiness-writer hypothesis. Rebuilding the corrected optimized controller and both static targets. The after-facts SSH interruption diagnostic is also queued using the prior controller; no final repeated-matrix claim.

Pushed 1ea55304c: prove the ongoing path before opening the temporary readiness PTY; 4d120e1f6 separately documents that ordering. Formatting, strict SSH Clippy and all 63 installation tests pass. The focused red/green receipts are linked above. The first verify iterate run had exactly one failure, that new ordering regression compiled before the fix; fmt, VT native fmt, plugin compile-out, Clippy and cargo-deny all passed. A fresh canonical iterate run follows the corrected release builds. With the same optimized controller, the previous non-primed client control completes the real blocked-port join in 1696 ms with mesh_handoff 158.381 ms, versus the new primed client smoke 11929 ms / 10289.620 ms. [Control receipt](https://git.kayg.org/attachments/0ec38c6d-c0b5-4de9-9cd6-d2ac631c1eb7). This narrows the regression to the new transition, without yet proving the readiness-writer hypothesis. Rebuilding the corrected optimized controller and both static targets. The after-facts SSH interruption diagnostic is also queued using the prior controller; no final repeated-matrix claim.
kayg commented 2026-10-16 21:28:12 +00:00
Author
Owner
Copy link

The ordered release-controller smoke still has mesh_handoff 10386.670 ms / total 12368 ms: raw. The readiness-writer hypothesis did not explain the stall. The ordering remains useful because readiness now proves the final path, but no speed fix is claimed.

Strengthened client regression completes greeting, activation and a replacement control exchange (pass). A new real TLS server regression promotes TCP to native QUIC and dispatches a control call (pass). The stall remains in runtime/plugin integration. Added content-free native link notices and test-only admission phase tracing to isolate it.

After-facts SSH interruption passes 1/1 with exact-ledger cleanup and ordinary retry (raw). It takes 188.25 seconds overall: first attempt reaches the old 180-second limit, successful retry reports 8200 ms from its own Check event. The harness now records whole-controller elapsed time separately so that reset cannot disguise retry duration. This is not the 20-run final matrix.

Second iterate: all workspace tests pass; Clippy alone rejects a new fixture import after statements, now moved to module scope. Fresh strict touched-crate gates are queued. The process-restart fixture compiles; it retains public SQLite and kernel state, supplies generated CA material only through checked private pipes/stdin, enumerates/stops the actual controller processes, then starts a fresh controller. It has not run yet. Full final fast gate and all final cohorts remain pending.

The ordered release-controller smoke still has mesh_handoff 10386.670 ms / total 12368 ms: [raw](https://git.kayg.org/attachments/0cfd4bd4-4641-4196-8ce4-0540c39048df). The readiness-writer hypothesis did not explain the stall. The ordering remains useful because readiness now proves the final path, but no speed fix is claimed. Strengthened client regression completes greeting, activation and a replacement control exchange ([pass](https://git.kayg.org/attachments/1ce044e5-34a5-40c0-94fa-3da28037a497)). A new real TLS server regression promotes TCP to native QUIC and dispatches a control call ([pass](https://git.kayg.org/attachments/f7331087-8dee-4dbf-ab37-0e79549c2314)). The stall remains in runtime/plugin integration. Added content-free native link notices and test-only admission phase tracing to isolate it. After-facts SSH interruption passes 1/1 with exact-ledger cleanup and ordinary retry ([raw](https://git.kayg.org/attachments/c33fe63c-ea98-4dd6-aed2-987b164ef79f)). It takes 188.25 seconds overall: first attempt reaches the old 180-second limit, successful retry reports 8200 ms from its own Check event. The harness now records whole-controller elapsed time separately so that reset cannot disguise retry duration. This is not the 20-run final matrix. Second iterate: all workspace tests pass; Clippy alone rejects a new fixture import after statements, now moved to module scope. Fresh strict touched-crate gates are queued. The process-restart fixture compiles; it retains public SQLite and kernel state, supplies generated CA material only through checked private pipes/stdin, enumerates/stops the actual controller processes, then starts a fresh controller. It has not run yet. Full final fast gate and all final cohorts remain pending.
kayg commented 2026-10-16 22:08:47 +00:00
Author
Owner
Copy link

Committed and pushed 9129f6018 (client activation continuity regression), 1ff74cc45 (real TCP-to-native server regression), and 4596c1b32 (native SSH fixture hooks). Serial remote-runtime group: 40 passed, 3 ignored; strict agent/server clippy passed. The unchanged 100 ms native-output assertion had failed under concurrent group execution; the serial run passed without changing that assertion.

Native admission tracing located the ~10 s handoff stall: registry admission begins, but the actor never begins candidate preparation. Primary EOF leaves the actor waiting on its transport receive queue, while the candidate was queued as an ordinary command. A new regression closes the real TCP connection and waits until the actor observes closure before presenting the verified mesh candidate. Before: fails with proved TCP replacement remained parked: Elapsed(()). After routing candidates through the existing bounded transport queue: passes. Full touched-area checks are running; cancellation regressions also cover dropped candidates and retained admission waiters. No wire format or dependency change.

The restart fixture's first private-pipe functional check exposed a test-only key-export error: the existing TLS key is P-256, not the Ed25519 issuer. It now preserves the existing TLS DER format; generated private material remains only in guarded process memory and anonymous pipes. Native restart execution remains pending.

Committed and pushed 9129f6018 (client activation continuity regression), 1ff74cc45 (real TCP-to-native server regression), and 4596c1b32 (native SSH fixture hooks). Serial remote-runtime group: 40 passed, 3 ignored; strict agent/server clippy passed. The unchanged 100 ms native-output assertion had failed under concurrent group execution; the serial run passed without changing that assertion. Native admission tracing located the ~10 s handoff stall: registry admission begins, but the actor never begins candidate preparation. Primary EOF leaves the actor waiting on its transport receive queue, while the candidate was queued as an ordinary command. A new regression closes the real TCP connection and waits until the actor observes closure before presenting the verified mesh candidate. Before: fails with `proved TCP replacement remained parked: Elapsed(())`. After routing candidates through the existing bounded transport queue: passes. Full touched-area checks are running; cancellation regressions also cover dropped candidates and retained admission waiters. No wire format or dependency change. The restart fixture's first private-pipe functional check exposed a test-only key-export error: the existing TLS key is P-256, not the Ed25519 issuer. It now preserves the existing TLS DER format; generated private material remains only in guarded process memory and anonymous pipes. Native restart execution remains pending.
kayg commented 2026-10-16 22:11:58 +00:00
Author
Owner
Copy link

Pushed 19bd30885: admit verified mesh candidates through the existing bounded transport queue after primary EOF; preserve five-second preparation deadline, candidate classification and cancellation cleanup. Strict server clippy passes; embedded 54 passed/3 ignored, serial remote runtime 41 passed/3 ignored, unchanged assertions. Gate output. Regression before, after.

Corrected restart identity pipe check passes locally with the existing P-256 TLS key encoded as DER. Private output stayed in memory; owned container init PID 3611491 gone, container removed. Functional receipt. Native queue-fix diagnostic is waiting for the perf lock; no new latency claim yet.

Pushed 19bd30885: admit verified mesh candidates through the existing bounded transport queue after primary EOF; preserve five-second preparation deadline, candidate classification and cancellation cleanup. Strict server clippy passes; embedded 54 passed/3 ignored, serial remote runtime 41 passed/3 ignored, unchanged assertions. [Gate output](https://git.kayg.org/attachments/edb144f7-f6e0-4f12-b5e5-a6b3d5206070). Regression [before](https://git.kayg.org/attachments/5daa5be5-71e5-4571-8319-b1e52bc01c84), [after](https://git.kayg.org/attachments/827666a1-90cb-4ac9-9fbe-4f4468c86ee5). Corrected restart identity pipe check passes locally with the existing P-256 TLS key encoded as DER. Private output stayed in memory; owned container init PID 3611491 gone, container removed. [Functional receipt](https://git.kayg.org/attachments/03e24c44-b485-4cd2-8ba4-1fcf3bddf082). Native queue-fix diagnostic is waiting for the perf lock; no new latency claim yet.
kayg commented 2026-10-16 22:16:36 +00:00
Author
Owner
Copy link

Native blocked-agent-port diagnostic after 19bd30885 passes 1/1: mesh_handoff 209.066 ms, Check-to-Done 2336 ms, whole controller 2487.182 ms. Admission trace now reaches actor_connect_begin, actor_connect_ready, activation and metadata publication. The prior diagnostic was ~10.4 s in handoff; the queue correction removes that stall. This debug-controller sample is functional diagnosis, not final release timing acceptance. Same static daemon 98e387b1e18870bc50882d426840b96f75059b62bc49fbfb8bbc8edf15588dba; diagnostic controller 1e40548290d04d09109b4af69dd470db2fee3fae750adff69adc79b0cbae66b1. Raw sample. Prior host preserved; zero live invitations/attempts; both owned container PIDs gone, network removed.

Pushed c184d7350 separately: opt-in test-only fixed admission phase labels, compiled out of shipping builds. Actual process-restart smoke queued; both normal static builds and optimized controller rebuilding serially in one build slot.

Native blocked-agent-port diagnostic after 19bd30885 passes 1/1: mesh_handoff 209.066 ms, Check-to-Done 2336 ms, whole controller 2487.182 ms. Admission trace now reaches actor_connect_begin, actor_connect_ready, activation and metadata publication. The prior diagnostic was ~10.4 s in handoff; the queue correction removes that stall. This debug-controller sample is functional diagnosis, not final release timing acceptance. Same static daemon 98e387b1e18870bc50882d426840b96f75059b62bc49fbfb8bbc8edf15588dba; diagnostic controller 1e40548290d04d09109b4af69dd470db2fee3fae750adff69adc79b0cbae66b1. [Raw sample](https://git.kayg.org/attachments/db7c573d-5fc2-4225-8a82-87e58196bc50). Prior host preserved; zero live invitations/attempts; both owned container PIDs gone, network removed. Pushed c184d7350 separately: opt-in test-only fixed admission phase labels, compiled out of shipping builds. Actual process-restart smoke queued; both normal static builds and optimized controller rebuilding serially in one build slot.
kayg commented 2026-10-16 22:22:06 +00:00
Author
Owner
Copy link

Pushed efd8edad3: exact normal static ceilings ratcheted down under D243/D264, no allowance. x86_64 21,300,944 bytes, original merge base 21,422,096, delta -121,152; arm64 19,516,856 bytes. Both stripped static, no PT_INTERP; production-trust and exact-size checks pass. x86 receipt, arm receipt, checks.

Optimized native controller compilation continues in the same build slot. Actual process-restart diagnostic remains queued for the shared perf lock. No final full-gate or repeated-matrix claim.

Pushed efd8edad3: exact normal static ceilings ratcheted down under D243/D264, no allowance. x86_64 21,300,944 bytes, original merge base 21,422,096, delta -121,152; arm64 19,516,856 bytes. Both stripped static, no PT_INTERP; production-trust and exact-size checks pass. [x86 receipt](https://git.kayg.org/attachments/91e73073-53dd-48b6-af71-c5e52b00dfa5), [arm receipt](https://git.kayg.org/attachments/c4348c58-ea5e-4cac-8b0b-13128cb004ea), [checks](https://git.kayg.org/attachments/8435b090-aa24-4f36-a927-ce8678da745b). Optimized native controller compilation continues in the same build slot. Actual process-restart diagnostic remains queued for the shared perf lock. No final full-gate or repeated-matrix claim.
kayg commented 2026-10-16 22:27:11 +00:00
Author
Owner
Copy link

Actual server-process restart fails 1/1: all recorded controller namespace PIDs stopped (exit 137), fresh process retires unfinished bootstrap authority, then authenticated SSH check returns exit 73 / "Setup is already running on this host." Prior host remains active, zero live invitations/attempts. Failing native test. Both owned container PIDs gone and network removed. Root cause: the host's intentional 180-second lease and running installed daemon survive server loss; the existing earlier-attempt inspector admits only already-stopped staging leftovers. Waiting out the lease alone cannot remove the live daemon.

Proposed additive internal contract refinement before implementation: distinguish an interrupted activated fresh journal from stopped leftovers. Report only bounded public host/transaction/directory coordinates, without stopping anything during inspection. Only the existing durable failed-host ownership check (same server, owner, authenticated trust lookup, account and privilege mode; revoked unfinished authority) may authorize that journal's original rollback. An unowned/live attempt still returns SetupRunning and cannot be taken over through a Start fresh confirmation. Revalidate the exact private journal and published service/launcher ownership immediately before mutation; retain the existing manager stop, lease retirement and rollback receipts. Unknown, upgraded or changed layouts remain untouched. Adding real-file regression before changes; no final acceptance claim.

Actual server-process restart fails 1/1: all recorded controller namespace PIDs stopped (exit 137), fresh process retires unfinished bootstrap authority, then authenticated SSH check returns exit 73 / "Setup is already running on this host." Prior host remains active, zero live invitations/attempts. [Failing native test](https://git.kayg.org/attachments/2aa893b4-d24c-4f73-a682-4a22edb196ec). Both owned container PIDs gone and network removed. Root cause: the host's intentional 180-second lease and running installed daemon survive server loss; the existing earlier-attempt inspector admits only already-stopped staging leftovers. Waiting out the lease alone cannot remove the live daemon. Proposed additive internal contract refinement before implementation: distinguish an interrupted activated *fresh* journal from stopped leftovers. Report only bounded public host/transaction/directory coordinates, without stopping anything during inspection. Only the existing durable failed-host ownership check (same server, owner, authenticated trust lookup, account and privilege mode; revoked unfinished authority) may authorize that journal's original rollback. An unowned/live attempt still returns SetupRunning and cannot be taken over through a Start fresh confirmation. Revalidate the exact private journal and published service/launcher ownership immediately before mutation; retain the existing manager stop, lease retirement and rollback receipts. Unknown, upgraded or changed layouts remain untouched. Adding real-file regression before changes; no final acceptance claim.
kayg commented 2026-10-16 22:43:46 +00:00
Author
Owner
Copy link

Atomic steps pushed:

  • b7c720d82 shares existing attempt cleanup generation, including login-owned reverse-forward runtime retirement.
  • fcc43bb69 maps local test runtimes inside each job fixture.
  • 923b17e09 adds read-only interrupted-fresh observation and durable failed-host-authorized original rollback. Foreign units/data, post-observation service changes and unowned active attempts are refused. Public Start fresh cannot take over that state.
  • 00c3873a8 documents the additive contract proposed above.
  • c50cb7a11 adds actual controller-process restart fixture; generated identities stay in private pipes and memory.
  • dc9f1ade0 adds the owned native VM acceptance harness.

Strict SSH/server clippy passes. Installation 69/69, ownership shell 30/30, bootstrap 23 passed/3 opt-in ignored. Combined gate, runtime cleanup gate, final interrupted record/API refusal check. The initial broad run used a scratch path whose resulting Unix socket was 108 bytes; removing the unnecessary /t suffix keeps every file under this job and the exact tests pass. No assertion was relaxed.

Actual server restart now passes 1/1 (after): recorded controller processes killed with 137, public ledger/kernel retained, fresh process retires unfinished authority, automatic exact owned rollback, fresh enrollment/mesh/terminal proof. Prior host preserved; zero new live invitations/attempts, no receipts or forwarding runtimes, both owned PIDs gone. Whole diagnostic controller 20.66 s; not release speed acceptance. Real-file before and after.

Strong after-facts SSH drop passes 1/1 (raw): first attempt reports bounded ongoing-path failure, ordinary retry succeeds, whole diagnostic controller 34.861 s versus the earlier 188.25 s. Final successful-attempt Check-to-Done 2351 ms is recorded separately. Full cleanup proven.

Optimized 150 ms smoke passes 1/1 (raw): Done 10759 ms. This single sample does not establish the one-RTT speed bar against the 9953 ms before median. 150 ms / 1% loss smoke passes 1/1 with full cleanup (raw), Done 13716 ms. No final cohort claim.

Final normal static builds, optimized native controller and verify iterate queued serially in one build slot. Running the 20-case actual process-restart functional cohort with the verified diagnostic controller while those compile; release speed cohorts remain separate. Rootless Linux mesh and TCP/TLS 443 runtime prerequisites remain unavailable in this base.

Atomic steps pushed: - b7c720d82 shares existing attempt cleanup generation, including login-owned reverse-forward runtime retirement. - fcc43bb69 maps local test runtimes inside each job fixture. - 923b17e09 adds read-only interrupted-fresh observation and durable failed-host-authorized original rollback. Foreign units/data, post-observation service changes and unowned active attempts are refused. Public Start fresh cannot take over that state. - 00c3873a8 documents the additive contract proposed above. - c50cb7a11 adds actual controller-process restart fixture; generated identities stay in private pipes and memory. - dc9f1ade0 adds the owned native VM acceptance harness. Strict SSH/server clippy passes. Installation 69/69, ownership shell 30/30, bootstrap 23 passed/3 opt-in ignored. [Combined gate](https://git.kayg.org/attachments/409b7933-5d4c-44a7-ba7d-4079f4037002), [runtime cleanup gate](https://git.kayg.org/attachments/35dfe57a-c7b1-4623-9156-d122e96da81e), [final interrupted record/API refusal check](https://git.kayg.org/attachments/10635717-9088-4e09-b8ff-f01ba3d1b15e). The initial broad run used a scratch path whose resulting Unix socket was 108 bytes; removing the unnecessary /t suffix keeps every file under this job and the exact tests pass. No assertion was relaxed. Actual server restart now passes 1/1 ([after](https://git.kayg.org/attachments/11f0246f-71f0-45c0-9667-b557e2254c26)): recorded controller processes killed with 137, public ledger/kernel retained, fresh process retires unfinished authority, automatic exact owned rollback, fresh enrollment/mesh/terminal proof. Prior host preserved; zero new live invitations/attempts, no receipts or forwarding runtimes, both owned PIDs gone. Whole diagnostic controller 20.66 s; not release speed acceptance. Real-file [before](https://git.kayg.org/attachments/a2db7a10-af70-49e5-9f80-a29e7fb6e67f) and [after](https://git.kayg.org/attachments/d436c63b-e879-40ed-82c5-c5e524fa0f1f). Strong after-facts SSH drop passes 1/1 ([raw](https://git.kayg.org/attachments/f9b0cf11-73e1-4372-b643-b5ca1d31d190)): first attempt reports bounded ongoing-path failure, ordinary retry succeeds, whole diagnostic controller 34.861 s versus the earlier 188.25 s. Final successful-attempt Check-to-Done 2351 ms is recorded separately. Full cleanup proven. Optimized 150 ms smoke passes 1/1 ([raw](https://git.kayg.org/attachments/88003cf1-196e-492a-91e4-903dfcdaff66)): Done 10759 ms. This single sample does not establish the one-RTT speed bar against the 9953 ms before median. 150 ms / 1% loss smoke passes 1/1 with full cleanup ([raw](https://git.kayg.org/attachments/beab1750-3a90-4a2b-909f-687808efb729)), Done 13716 ms. No final cohort claim. Final normal static builds, optimized native controller and verify iterate queued serially in one build slot. Running the 20-case actual process-restart functional cohort with the verified diagnostic controller while those compile; release speed cohorts remain separate. Rootless Linux mesh and TCP/TLS 443 runtime prerequisites remain unavailable in this base.
kayg commented 2026-10-16 22:49:56 +00:00
Author
Owner
Copy link

Actual server-process restart functional matrix: 20/20 pass, zero failures. Every sample kills the recorded controller processes after authenticated facts (137), retains public SQLite/kernel state, starts a fresh controller, retires unfinished authority, performs owned rollback and completes fresh SSH enrollment, mesh proof and a real terminal. Each retains the prior host, ends with zero live invitations/attempts, no leftover receipts/forwarding runtimes, removes its network and both containers, and proves their PIDs gone. Raw 20-case cohort.

Diagnostic controller db1c52a734ca2933fa63dfa92310a63736f7f9bce2481ac822a6c51c5b1a2fb3; daemon 4395470974d7207c7f9eef2b8cc05c09f8f25c7b371d7b38bdfab0a81c78d5bc. Whole-controller p50 7090.0415 ms / p95 20914.468 ms; final fresh-process Check-to-Done p50 2868 ms / p95 16037 ms recorded separately. Functional diagnostic cohort, not release speed acceptance. Per-sample VM load is in the raw receipt. Four five-case SSH-drop segments are queued, releasing the perf lock between segments. Final optimized builds are compiling.

Actual server-process restart functional matrix: 20/20 pass, zero failures. Every sample kills the recorded controller processes after authenticated facts (137), retains public SQLite/kernel state, starts a fresh controller, retires unfinished authority, performs owned rollback and completes fresh SSH enrollment, mesh proof and a real terminal. Each retains the prior host, ends with zero live invitations/attempts, no leftover receipts/forwarding runtimes, removes its network and both containers, and proves their PIDs gone. [Raw 20-case cohort](https://git.kayg.org/attachments/bf917c91-53d0-4f0e-ae1e-548286362bc4). Diagnostic controller db1c52a734ca2933fa63dfa92310a63736f7f9bce2481ac822a6c51c5b1a2fb3; daemon 4395470974d7207c7f9eef2b8cc05c09f8f25c7b371d7b38bdfab0a81c78d5bc. Whole-controller p50 7090.0415 ms / p95 20914.468 ms; final fresh-process Check-to-Done p50 2868 ms / p95 16037 ms recorded separately. Functional diagnostic cohort, not release speed acceptance. Per-sample VM load is in the raw receipt. Four five-case SSH-drop segments are queued, releasing the perf lock between segments. Final optimized builds are compiling.
kayg commented 2026-10-16 23:00:15 +00:00
Author
Owner
Copy link

Pushed e16d0860d: final exact D243/D264 ceilings, zero slack. x86_64 21,302,672 bytes, SHA256 d711215f20f790fdcdbd12b738c81b9f7401efed485377154517b388538cc9bb, delta -119,424 versus original merge base. Arm64 19,522,152 bytes, SHA256 f3fbeb4e33385a3f392b7604fe10e479b8d20641a34f082a1ef814b8042e5e18. Both stripped static without PT_INTERP; production-trust and exact-size checks pass. x86, arm, checks.

Restart recovery adds 1,728 bytes x86 / 5,296 arm over the queue-fix milestone; its justification is the real restart failure becoming 20/20 successful owned recoveries. Both final sizes remain below the ceilings inherited by this branch. No dependencies/features added. Optimized controller build continues, then canonical iterate. SSH-drop functional matrix has completed two five-case segments: 10/10 pass, each with full cleanup; two segments remain.

Pushed e16d0860d: final exact D243/D264 ceilings, zero slack. x86_64 21,302,672 bytes, SHA256 d711215f20f790fdcdbd12b738c81b9f7401efed485377154517b388538cc9bb, delta -119,424 versus original merge base. Arm64 19,522,152 bytes, SHA256 f3fbeb4e33385a3f392b7604fe10e479b8d20641a34f082a1ef814b8042e5e18. Both stripped static without PT_INTERP; production-trust and exact-size checks pass. [x86](https://git.kayg.org/attachments/889e92ff-d740-4f58-9b25-b1edd47e4315), [arm](https://git.kayg.org/attachments/326c9f86-e400-403e-a911-aabad0312674), [checks](https://git.kayg.org/attachments/20265eb0-3af0-49f5-8e18-1b928ed90988). Restart recovery adds 1,728 bytes x86 / 5,296 arm over the queue-fix milestone; its justification is the real restart failure becoming 20/20 successful owned recoveries. Both final sizes remain below the ceilings inherited by this branch. No dependencies/features added. Optimized controller build continues, then canonical iterate. SSH-drop functional matrix has completed two five-case segments: 10/10 pass, each with full cleanup; two segments remain.
kayg commented 2026-10-16 23:18:08 +00:00
Author
Owner
Copy link

Strong SSH-drop functional matrix complete: 20/20 pass across four five-case locked segments, releasing between segments. Every case kills the exact SSH master after authenticated facts, reaches the bounded ongoing-path failure, and ordinary fresh-SSH retry succeeds. Prior host preserved; zero live invitations/attempts and no leftover receipt/forwarding runtime; both owned container PIDs gone and network removed for every case. Whole-controller p50 34153.577 ms / p95 34967.661 ms. These include the failed first attempt; retry-local Done is reported separately. Raw cohort, aggregate. Same diagnostic controller/daemon hashes as the restart cohort above.

Packet-loss fixture finding: a new owned-container assertion fails before its fix because TSO and GSO are on (before); both probe PIDs gone. The kernel applies netem loss before GSO segmentation (primary source, matching 6.12.111). The earlier 1% smoke is therefore coalesced-buffer loss diagnosis, not final per-packet acceptance. Fix disables TSO/GSO/GRO only in both owned loss-test namespaces, verifies them, and gives the segmented binary transfer a bounded queue without incidental limit loss. Recheck is queued. Added per-namespace TCP congestion metadata, actual qdisc/drop receipts and harness SHA; the existing no-loss LAN/150 shaping is unchanged.

Optimized native controller 00fab374c090fc6601a8daf7956d5b912cea72c3c4963af351b04b51d9a32bcc is built/shipped. Canonical iterate passed Clippy and is running workspace tests. Final release LAN-after cohort queued; no final speed/full-gate claim yet.

Strong SSH-drop functional matrix complete: 20/20 pass across four five-case locked segments, releasing between segments. Every case kills the exact SSH master after authenticated facts, reaches the bounded ongoing-path failure, and ordinary fresh-SSH retry succeeds. Prior host preserved; zero live invitations/attempts and no leftover receipt/forwarding runtime; both owned container PIDs gone and network removed for every case. Whole-controller p50 34153.577 ms / p95 34967.661 ms. These include the failed first attempt; retry-local Done is reported separately. [Raw cohort](https://git.kayg.org/attachments/1e0fa6cc-8a30-4470-978e-678a215c167b), [aggregate](https://git.kayg.org/attachments/a805870a-8f0e-437c-ac5b-9d39626a0e23). Same diagnostic controller/daemon hashes as the restart cohort above. Packet-loss fixture finding: a new owned-container assertion fails before its fix because TSO and GSO are on ([before](https://git.kayg.org/attachments/ebb3aaed-86c3-43ff-957a-4ce7127d160e)); both probe PIDs gone. The kernel applies netem loss before GSO segmentation ([primary source, matching 6.12.111](https://raw.githubusercontent.com/gregkh/linux/v6.12.111/net/sched/sch_netem.c)). The earlier 1% smoke is therefore coalesced-buffer loss diagnosis, not final per-packet acceptance. Fix disables TSO/GSO/GRO only in both owned loss-test namespaces, verifies them, and gives the segmented binary transfer a bounded queue without incidental limit loss. Recheck is queued. Added per-namespace TCP congestion metadata, actual qdisc/drop receipts and harness SHA; the existing no-loss LAN/150 shaping is unchanged. Optimized native controller 00fab374c090fc6601a8daf7956d5b912cea72c3c4963af351b04b51d9a32bcc is built/shipped. Canonical iterate passed Clippy and is running workspace tests. Final release LAN-after cohort queued; no final speed/full-gate claim yet.
kayg commented 2026-10-16 23:40:18 +00:00
Author
Owner
Copy link

Final optimized release timing cohorts, 20/20 each, all ledger/receipt/network/container/PID cleanup checks pass. Static daemon d711215f20f790fdcdbd12b738c81b9f7401efed485377154517b388538cc9bb; optimized controller 00fab374c090fc6601a8daf7956d5b912cea72c3c4963af351b04b51d9a32bcc; harness 4e860db2533c3e067329990c8799fe6cd1f2b74e37d972265515d9275ed40740. Fixed image/kernel and per-sample VM load in raw receipts; both namespaces use BBR.

Check-to-Done Before p50/p95 ms After p50/p95 ms
LAN 1754 / 1924 1855 / 2031
150 ms, no loss 9953 / 10088 10693 / 10769

The conservative first-link checkpoint is authenticated facts complete (Connect event, emitted after connected_facts and before tunnel_ready/terminal proof). Its before→after p50/p95: LAN 1664/1814 → 1538/1741 ms; 150 ms 9256/9358 → 8922/9003 ms. Thus the tunnelled first connection is faster at this checkpoint; total completion is +101 ms LAN / +740 ms at 150 ms because it now waits for proved mesh activation. Do not read the first-link numbers as the whole join. SSH operation counts remain net zero for successful Unix forwarding, at most one additional command round trip for the TCP fallback (existing real-file command-count regression).

Final LAN raw, 150 ms raw and packet-loss smoke, recomputed checkpoints. Before raw cohorts are linked above.

Per-packet 150 ms/1% smoke passes with offloads disabled in both owned namespaces, exact 0.075 s one-way / 0.01 loss qdisc options, 17 outgoing and 136 incoming drops, zero overlimits. Done 13708 ms; full cleanup. Offload/configuration recheck. Pushed 8ada1d190 for the fixture correction. The 20-case final blocked-port/TCP-only/forward-denied/CA-locked/NetBird/loss cohorts are queued, each releasing the lock between cohorts.

Canonical iterate passed: ITERATE gate passed. This is not the final gate: run scripts/verify fast before reporting done. Full iterate/build output. Full fast is running on 8ada1d190.

Latest origin/dev 3b2183eaa has no new rootless Linux or 443 runtime prerequisite. It does add approved static path normalization; preparing a controlled baseline/head size comparison with that same recipe in scratch, without changing shipping code or merging. The earlier ordinary-recipe sizes remain exact artifact sizes, but the normalized comparison will remove checkout-path strings from the code delta.

Final optimized release timing cohorts, 20/20 each, all ledger/receipt/network/container/PID cleanup checks pass. Static daemon d711215f20f790fdcdbd12b738c81b9f7401efed485377154517b388538cc9bb; optimized controller 00fab374c090fc6601a8daf7956d5b912cea72c3c4963af351b04b51d9a32bcc; harness 4e860db2533c3e067329990c8799fe6cd1f2b74e37d972265515d9275ed40740. Fixed image/kernel and per-sample VM load in raw receipts; both namespaces use BBR. | Check-to-Done | Before p50/p95 ms | After p50/p95 ms | |---|---:|---:| | LAN | 1754 / 1924 | 1855 / 2031 | | 150 ms, no loss | 9953 / 10088 | 10693 / 10769 | The conservative first-link checkpoint is authenticated facts complete (`Connect` event, emitted after connected_facts and before tunnel_ready/terminal proof). Its before→after p50/p95: LAN 1664/1814 → 1538/1741 ms; 150 ms 9256/9358 → 8922/9003 ms. Thus the tunnelled first connection is faster at this checkpoint; total completion is +101 ms LAN / +740 ms at 150 ms because it now waits for proved mesh activation. Do not read the first-link numbers as the whole join. SSH operation counts remain net zero for successful Unix forwarding, at most one additional command round trip for the TCP fallback (existing real-file command-count regression). [Final LAN raw](https://git.kayg.org/attachments/f069905c-a026-43c2-8e97-00e00718430d), [150 ms raw and packet-loss smoke](https://git.kayg.org/attachments/2cf041f4-9224-4ab9-a56c-e5e81247d3e4), [recomputed checkpoints](https://git.kayg.org/attachments/9214c7b4-bf90-4434-84e8-d1ae50b25237). Before raw cohorts are linked above. Per-packet 150 ms/1% smoke passes with offloads disabled in both owned namespaces, exact 0.075 s one-way / 0.01 loss qdisc options, 17 outgoing and 136 incoming drops, zero overlimits. Done 13708 ms; full cleanup. [Offload/configuration recheck](https://git.kayg.org/attachments/dcae4331-8c2c-489b-86be-6dc416868c36). Pushed 8ada1d190 for the fixture correction. The 20-case final blocked-port/TCP-only/forward-denied/CA-locked/NetBird/loss cohorts are queued, each releasing the lock between cohorts. Canonical iterate passed: `ITERATE gate passed. This is not the final gate: run scripts/verify fast before reporting done.` [Full iterate/build output](https://git.kayg.org/attachments/99ea6cd0-d63b-4d6f-8d64-c9bdd30f2c6c). Full fast is running on 8ada1d190. Latest origin/dev 3b2183eaa has no new rootless Linux or 443 runtime prerequisite. It does add approved static path normalization; preparing a controlled baseline/head size comparison with that same recipe in scratch, without changing shipping code or merging. The earlier ordinary-recipe sizes remain exact artifact sizes, but the normalized comparison will remove checkout-path strings from the code delta.
kayg commented 2026-10-16 23:45:29 +00:00
Author
Owner
Copy link

Full scripts/verify fast on committed head 8ada1d1909b63162aae58a6b2c064129f3907f25: exit 0. All Rust rows PASS; TypeScript/compiler/bundle/oracle checks and tests pass. Verbatim Rust summary:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| test                      | PASS         |
| cargo-deny                | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

Verbatim final test result:

 37 pass
 0 fail
 34463 expect() calls
Ran 37 tests across 10 files. [2.41s]

Complete gate output, verbatim. Both static builds pass exact ceilings, production trust, linkage and key-heap checks. Post-gate x86 artifact SHA256 remains d711215f20f790fdcdbd12b738c81b9f7401efed485377154517b388538cc9bb, identical to all final release timing cohorts. No load-failure exemption used. RUST_TEST_THREADS=1, unique TRACER_TEST_PORT=4286, one slot-wrapped run.

Final release blocked-agent-port cohort also passes 20/20; Check-to-Done p50 1885 ms / p95 2006 ms, all ledger and exact fixture cleanup checks pass. Remaining cohorts release between runs. Controlled normalized size comparison is queued with one separate release comparison folder; shipping source untouched.

Full `scripts/verify fast` on committed head `8ada1d1909b63162aae58a6b2c064129f3907f25`: exit 0. All Rust rows PASS; TypeScript/compiler/bundle/oracle checks and tests pass. Verbatim Rust summary: ```text +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | test | PASS | | cargo-deny | PASS | | static builds | PASS | +---------------------------+--------------+ ``` Verbatim final test result: ```text 37 pass 0 fail 34463 expect() calls Ran 37 tests across 10 files. [2.41s] ``` [Complete gate output, verbatim](https://git.kayg.org/attachments/f17d4b17-d651-4eca-b9cf-4f527a1d8bff). Both static builds pass exact ceilings, production trust, linkage and key-heap checks. Post-gate x86 artifact SHA256 remains d711215f20f790fdcdbd12b738c81b9f7401efed485377154517b388538cc9bb, identical to all final release timing cohorts. No load-failure exemption used. `RUST_TEST_THREADS=1`, unique `TRACER_TEST_PORT=4286`, one slot-wrapped run. Final release blocked-agent-port cohort also passes 20/20; Check-to-Done p50 1885 ms / p95 2006 ms, all ledger and exact fixture cleanup checks pass. Remaining cohorts release between runs. Controlled normalized size comparison is queued with one separate release comparison folder; shipping source untouched.
kayg commented 2026-10-17 00:11:30 +00:00
Author
Owner
Copy link

Follow-up finding on 8ada1d190: StoredIdentity.bootstrap survives certificate publication and every daemon restart (crates/infraternal/src/remote.rs:460, 505); run_bootstrap reuses the same hint after a proved mesh link (crates/infraternal-agent/src/remote_server_path.rs:706). If the mesh preference is subsequently removed and the attempt-local SSH socket is gone, the reconnect owner can keep trying that obsolete socket.

Additive D291 contract proposal: retire the hint only after an authenticated mesh transport becomes active; serialize durable retirement with existing certificate publication, preserving the host key and certificate. Failed first SSH connections still refuse public-port fallback. In-process retirement survives watchdog restarts; durable retirement survives daemon restart and renewal cannot restore the old hint. No new network trust or routing engine. Adding a restart/publication regression before the fix.

Follow-up finding on 8ada1d190: StoredIdentity.bootstrap survives certificate publication and every daemon restart (crates/infraternal/src/remote.rs:460, 505); run_bootstrap reuses the same hint after a proved mesh link (crates/infraternal-agent/src/remote_server_path.rs:706). If the mesh preference is subsequently removed and the attempt-local SSH socket is gone, the reconnect owner can keep trying that obsolete socket. Additive D291 contract proposal: retire the hint only after an authenticated mesh transport becomes active; serialize durable retirement with existing certificate publication, preserving the host key and certificate. Failed first SSH connections still refuse public-port fallback. In-process retirement survives watchdog restarts; durable retirement survives daemon restart and renewal cannot restore the old hint. No new network trust or routing engine. Adding a restart/publication regression before the fix.
kayg commented 2026-10-17 00:16:59 +00:00
Author
Owner
Copy link

8ada1d190 reliability milestone: blocked agent port 20/20; Unix forwarding disabled with allocated private TCP fallback 20/20; all SSH forwarding disabled 20/20 clear refusals; locked CA 20/20 clear refusals; native NetBird SSH with blocked agent port 20/20. Slow link (150 ms RTT, 1% per-packet loss each direction, offloads verified off on both owned veth endpoints) 20/20: Check-to-Done p50 13,220.5 ms / p95 13,726 ms. Every ledger and exact process/container/network cleanup check passes. Complete cohort receipts. Actual SSH-drop and actual server-process restart cohorts previously attached are 20/20 each.

Controlled same-recipe x86 static comparison: base 41a3a81abc 21,356,048 bytes; head 8ada1d190 21,240,720 bytes; delta −115,328 bytes. Both stripped static PIE. Same normalization recipe from dev 3b2183e applied only to detached comparison snapshots, not shipping source. Exact refs, recipe paths, sizes and hashes. Includes prerequisite #230 work; not a D291-only delta.

New lifecycle regressions demonstrated before fixing: daemon restart/publication failure, real mutually authenticated reconnect failure. Fix and crate gates in progress; the above remain a precisely identified milestone. Rootless ongoing mesh and outer TCP/TLS 443 remain absent runtime paths, not successful joins.

8ada1d190 reliability milestone: blocked agent port 20/20; Unix forwarding disabled with allocated private TCP fallback 20/20; all SSH forwarding disabled 20/20 clear refusals; locked CA 20/20 clear refusals; native NetBird SSH with blocked agent port 20/20. Slow link (150 ms RTT, 1% per-packet loss each direction, offloads verified off on both owned veth endpoints) 20/20: Check-to-Done p50 13,220.5 ms / p95 13,726 ms. Every ledger and exact process/container/network cleanup check passes. [Complete cohort receipts](https://git.kayg.org/attachments/53fc78cd-53dd-4807-b7ad-938f465b2969). Actual SSH-drop and actual server-process restart cohorts previously attached are 20/20 each. Controlled same-recipe x86 static comparison: base 41a3a81abcea 21,356,048 bytes; head 8ada1d190 21,240,720 bytes; delta −115,328 bytes. Both stripped static PIE. Same normalization recipe from dev 3b2183e applied only to detached comparison snapshots, not shipping source. [Exact refs, recipe paths, sizes and hashes](https://git.kayg.org/attachments/a2134db8-4740-4db0-86c4-7e82b0e0f5f6). Includes prerequisite #230 work; not a D291-only delta. New lifecycle regressions demonstrated before fixing: [daemon restart/publication failure](https://git.kayg.org/attachments/f230674d-2e9e-4c82-82f9-b5a59ae72f69), [real mutually authenticated reconnect failure](https://git.kayg.org/attachments/9db89703-1b3e-4e6d-8208-0f04c09eabd9). Fix and crate gates in progress; the above remain a precisely identified milestone. Rootless ongoing mesh and outer TCP/TLS 443 remain absent runtime paths, not successful joins.
kayg commented 2026-10-17 00:25:25 +00:00
Author
Owner
Copy link

Completed and pushed atomic lifecycle steps: d97a3f4e0 adds serialized durable retirement and shared retry-owner state; 35bb64df0 reuses the private atomic identity writer, retaining retirement across daemon restart and certificate renewal; 49f18fdd4 consumes the hint on authenticated mesh selection while keeping first private failures closed; 9114eed4e records the additive contract proposed above. No new marker, SSH command, key, dependency or transport engine.

Fmt and strict Clippy pass for the touched plugin, CLI and infraternal agent. Retirement tests 3/3, CLI identity tests 11/11, functional path tests 15/15 (one explicitly perf-VM-only timing test not run on this host). Complete targeted gates. Both earlier failing regressions now pass: restart plus renewal, real TLS/QUIC reconnect. Static artifacts/controller rebuild queued through the host slot; final exact ceilings, measurements and full fast gate will identify the updated head.

Completed and pushed atomic lifecycle steps: d97a3f4e0 adds serialized durable retirement and shared retry-owner state; 35bb64df0 reuses the private atomic identity writer, retaining retirement across daemon restart and certificate renewal; 49f18fdd4 consumes the hint on authenticated mesh selection while keeping first private failures closed; 9114eed4e records the additive contract proposed above. No new marker, SSH command, key, dependency or transport engine. Fmt and strict Clippy pass for the touched plugin, CLI and infraternal agent. Retirement tests 3/3, CLI identity tests 11/11, functional path tests 15/15 (one explicitly perf-VM-only timing test not run on this host). [Complete targeted gates](https://git.kayg.org/attachments/c917e0f0-20d9-430a-bfcb-e5db6c032b0d). Both earlier failing regressions now pass: [restart plus renewal](https://git.kayg.org/attachments/1cdcd252-ef52-4fac-8f62-7d4fe4b48e8f), [real TLS/QUIC reconnect](https://git.kayg.org/attachments/3f0de2cb-0115-4465-8879-bea95887a9e4). Static artifacts/controller rebuild queued through the host slot; final exact ceilings, measurements and full fast gate will identify the updated head.
kayg commented 2026-10-17 00:33:49 +00:00
Author
Owner
Copy link

Missing-path matrix complete on the 8ada1d190 milestone: UDP blocked 20/20 expected refusals, rootless Linux 20/20 expected refusals, zero completed joins in either cohort. Each reaches enrollment/authenticated facts through SSH, then fails the unavailable ongoing mesh path; all cases leave the prior host intact, zero attempts/live unused invitations, no setup/runtime receipts, and remove their exact owned processes/containers/network. All 40 cases and cleanup receipts. These are diagnostics, not successful-join acceptance.

Acceptance remains blocked on runtime work from the owner/orchestrator: Linux mesh is the kernel/netlink backend (crates/plugins/mesh/src/wireguard.rs), with no Linux rootless engine, and the ongoing transport still uses identity.endpoint rather than an outer 443 mesh route (crates/infraternal-agent/src/remote_server_path.rs:589). Latest dev 3b2183eaa also lacks these. This job selects no foundational engine and does not claim D196/rootless transport implementation. No approval request: SSH/bootstrap fixes continue through final artifacts/gates.

Missing-path matrix complete on the 8ada1d190 milestone: UDP blocked 20/20 expected refusals, rootless Linux 20/20 expected refusals, zero completed joins in either cohort. Each reaches enrollment/authenticated facts through SSH, then fails the unavailable ongoing mesh path; all cases leave the prior host intact, zero attempts/live unused invitations, no setup/runtime receipts, and remove their exact owned processes/containers/network. [All 40 cases and cleanup receipts](https://git.kayg.org/attachments/8e0dc0aa-7140-40f6-ad8b-f028b17a5408). These are diagnostics, not successful-join acceptance. Acceptance remains blocked on runtime work from the owner/orchestrator: Linux mesh is the kernel/netlink backend (crates/plugins/mesh/src/wireguard.rs), with no Linux rootless engine, and the ongoing transport still uses identity.endpoint rather than an outer 443 mesh route (crates/infraternal-agent/src/remote_server_path.rs:589). Latest dev 3b2183eaa also lacks these. This job selects no foundational engine and does not claim D196/rootless transport implementation. No approval request: SSH/bootstrap fixes continue through final artifacts/gates.
kayg commented 2026-10-17 00:45:56 +00:00
Author
Owner
Copy link

Pushed 1371201e1: exact shipping ceilings after the lifecycle fix. x86_64 musl: 21,303,632 bytes, +960 versus the previous milestone, −118,464 versus the original merge-base artifact. The 960-byte cost fixes the demonstrated obsolete-route reconnect/restart failures. x86 receipt and SHA256. arm64 musl: 19,519,080 bytes, −3,072 versus the previous milestone. arm64 receipt and SHA256. Both are stripped/static with no PT_INTERP and pass production trust; no size allowance. Exact checks, verbatim. Fmt and shell syntax pass.

The optimized controller and normalized same-recipe comparison are building serially in the same host slot. A prepared runner will verify copied artifact hashes and repeat final 20-case timing/reliability cohorts on the perf VM. Full fast will run on this committed head after the build slot is released. No further source changes pending.

Pushed 1371201e1: exact shipping ceilings after the lifecycle fix. x86_64 musl: 21,303,632 bytes, +960 versus the previous milestone, −118,464 versus the original merge-base artifact. The 960-byte cost fixes the demonstrated obsolete-route reconnect/restart failures. [x86 receipt and SHA256](https://git.kayg.org/attachments/615a8df8-a218-44a0-83a3-6ada6d858960). arm64 musl: 19,519,080 bytes, −3,072 versus the previous milestone. [arm64 receipt and SHA256](https://git.kayg.org/attachments/457c58a5-e323-4434-8ce9-f241e7ddf8ed). Both are stripped/static with no PT_INTERP and pass production trust; no size allowance. [Exact checks, verbatim](https://git.kayg.org/attachments/60f5d33d-0faa-4f43-858c-c8cc3ed08768). Fmt and shell syntax pass. The optimized controller and normalized same-recipe comparison are building serially in the same host slot. A prepared runner will verify copied artifact hashes and repeat final 20-case timing/reliability cohorts on the perf VM. Full fast will run on this committed head after the build slot is released. No further source changes pending.
kayg commented 2026-10-17 00:54:40 +00:00
Author
Owner
Copy link

Final controlled D243 comparison at 1371201e1: merge base 41a3a81abc is 21,356,048 bytes, head is 21,241,680 bytes, delta −114,368 bytes. Both stripped static x86_64 PIE with no PT_INTERP, identical normalized recipe overlaid from dev 3b2183eaa onto separate archived snapshots. Refs, exact recipe, hashes and sizes; all four rebuilds, verbatim. Shipping source/recipe unchanged. Removed the completed comparison build cache; retained its small public artifacts and attached receipts.

Final updated-artifact LAN cohort is 20/20: Check-to-Done p50 1,783 ms / p95 2,003 ms, controller elapsed p50 1,876.144 / p95 2,095.941 ms. All exact cleanup checks pass. Current shipping daemon 04b1b09d49f5c70afd8745ae6bbe82f8b2114ec96069ca93b80c098d06f45ff1; optimized controller a6d3af7c0a1441bd6941552ceb1c2f8ed388a23357f800c416daa6e55e1a8050. Final full fast queued on committed 1371201e1; remaining cohorts wait/release the shared perf lock normally. Full final receipts will be attached after collection.

Final controlled D243 comparison at 1371201e1: merge base 41a3a81abcea is 21,356,048 bytes, head is 21,241,680 bytes, delta −114,368 bytes. Both stripped static x86_64 PIE with no PT_INTERP, identical normalized recipe overlaid from dev 3b2183eaa onto separate archived snapshots. [Refs, exact recipe, hashes and sizes](https://git.kayg.org/attachments/fde3fa8b-e76e-405d-a40f-30862213aee1); [all four rebuilds, verbatim](https://git.kayg.org/attachments/e128c0c8-15ab-42be-ae84-5bd9f81dfe00). Shipping source/recipe unchanged. Removed the completed comparison build cache; retained its small public artifacts and attached receipts. Final updated-artifact LAN cohort is 20/20: Check-to-Done p50 1,783 ms / p95 2,003 ms, controller elapsed p50 1,876.144 / p95 2,095.941 ms. All exact cleanup checks pass. Current shipping daemon 04b1b09d49f5c70afd8745ae6bbe82f8b2114ec96069ca93b80c098d06f45ff1; optimized controller a6d3af7c0a1441bd6941552ceb1c2f8ed388a23357f800c416daa6e55e1a8050. Final full fast queued on committed 1371201e1; remaining cohorts wait/release the shared perf lock normally. Full final receipts will be attached after collection.
kayg commented 2026-10-17 01:05:15 +00:00
Author
Owner
Copy link

Review finding on 1371201e1: my initial durable-retirement hook runs after the original SSH-backed channel is dropped and before the mesh dispatcher activates (remote_server_path.rs:730). A delayed publisher regression fails with UnexpectedEof when the original peer attempts a Ping during publication. This violates the D291 continuity bar even though ordinary joins pass.

Correction: reuse the existing candidate-preparation future, verify the mesh greeting, then run the existing serialized publisher on the existing blocking pool while the original dispatcher continues to poll. Only finished publication permits promotion/retirement of the old link. No extra SSH operation, trust, dependency or routing engine. The additive lifecycle contract will explicitly require this ordering. Capturing the failing regression before the correction.

Paused only my local cohort launcher PID 1470756; its current allocated-TCP cohort continues to completion/cleanup and will not launch another cohort. Existing final gate and cohorts identify the 1371201e1 milestone; corrected-head artifacts, gates and final cohorts will supersede it.

Review finding on 1371201e1: my initial durable-retirement hook runs after the original SSH-backed channel is dropped and before the mesh dispatcher activates (remote_server_path.rs:730). A delayed publisher regression fails with UnexpectedEof when the original peer attempts a Ping during publication. This violates the D291 continuity bar even though ordinary joins pass. Correction: reuse the existing candidate-preparation future, verify the mesh greeting, then run the existing serialized publisher on the existing blocking pool while the original dispatcher continues to poll. Only finished publication permits promotion/retirement of the old link. No extra SSH operation, trust, dependency or routing engine. The additive lifecycle contract will explicitly require this ordering. Capturing the failing regression before the correction. Paused only my local cohort launcher PID 1470756; its current allocated-TCP cohort continues to completion/cleanup and will not launch another cohort. Existing final gate and cohorts identify the 1371201e1 milestone; corrected-head artifacts, gates and final cohorts will supersede it.
kayg commented 2026-10-17 01:27:47 +00:00
Author
Owner
Copy link

Pushed d9f010bd7: finish durable hint retirement in candidate preparation on the existing blocking pool, keeping the original dispatcher live until publication completes. A shared adapter also keeps cold reconnect storage off the async executor. Pushed 5540ac6e3 (contract ordering) and 5e6f3f32a (exact static ceilings), separately. No extra SSH request, dependency, engine or protocol frame.

Before correction the delayed publisher closes the original channel: failing real TLS/QUIC regression. Now that channel completes Ping/Pong while publication is blocked, then native activation succeeds: passing regression. Fmt, strict crate Clippy and 15 functional path tests pass; one perf-VM-only timing test is excluded from this host run. Complete gates.

Measured shipping sizes: x86 21,311,760 bytes (+8,128 versus 1371201e1, still −110,336 versus the original merge-base artifact), arm64 19,527,272 (+8,192). Cost is the existing blocking-pool adapter required by the failing continuity regression. No allowance; both exact size/linkage/trust checks. The superseded full-gate run spanned the draft correction and failed only its outdated size ceilings (+8,128/+8,192); raw output. It is not final-head evidence.

Source is frozen at 5e6f3f32a for controller, normalized comparison and final full fast, serially in one build slot. Fresh VM cohorts verify the rebuilt artifact hashes. Old cohort launcher PID 1470756 was stopped after its current 20/20 TCP cohort completed all cleanup; it and its SSH child 1771444 are both gone. Latest dev c47a438be has no mesh/remote runtime changes from 3b2183eaa, so the rootless/443 prerequisites remain missing.

Pushed d9f010bd7: finish durable hint retirement in candidate preparation on the existing blocking pool, keeping the original dispatcher live until publication completes. A shared adapter also keeps cold reconnect storage off the async executor. Pushed 5540ac6e3 (contract ordering) and 5e6f3f32a (exact static ceilings), separately. No extra SSH request, dependency, engine or protocol frame. Before correction the delayed publisher closes the original channel: [failing real TLS/QUIC regression](https://git.kayg.org/attachments/03c4e304-85c4-49d2-9736-fe0fded95f08). Now that channel completes Ping/Pong while publication is blocked, then native activation succeeds: [passing regression](https://git.kayg.org/attachments/126e85e9-90d2-4132-9282-2b5ce68ef10a). Fmt, strict crate Clippy and 15 functional path tests pass; one perf-VM-only timing test is excluded from this host run. [Complete gates](https://git.kayg.org/attachments/5ede9fae-9fbe-4574-8fb1-a7ba46f6b1cc). Measured shipping sizes: x86 21,311,760 bytes (+8,128 versus 1371201e1, still −110,336 versus the original merge-base artifact), arm64 19,527,272 (+8,192). Cost is the existing blocking-pool adapter required by the failing continuity regression. No allowance; [both exact size/linkage/trust checks](https://git.kayg.org/attachments/6ca3875b-8c46-45ea-a050-4915be2821ab). The superseded full-gate run spanned the draft correction and failed only its outdated size ceilings (+8,128/+8,192); [raw output](https://git.kayg.org/attachments/932796e1-1bb7-4f66-9592-27cee02e2965). It is not final-head evidence. Source is frozen at 5e6f3f32a for controller, normalized comparison and final full fast, serially in one build slot. Fresh VM cohorts verify the rebuilt artifact hashes. Old cohort launcher PID 1470756 was stopped after its current 20/20 TCP cohort completed all cleanup; it and its SSH child 1771444 are both gone. Latest dev c47a438be has no mesh/remote runtime changes from 3b2183eaa, so the rootless/443 prerequisites remain missing.
kayg commented 2026-10-17 02:05:14 +00:00
Author
Owner
Copy link

Final scripts/verify fast on frozen committed head 5e6f3f32a4d4c1656f5d97a89b955accce2e8fee: exit 0, no load exemptions. Verbatim Rust summary:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| test                      | PASS         |
| cargo-deny                | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

Verbatim final test output:

 37 pass
 0 fail
 34463 expect() calls
Ran 37 tests across 10 files. [3.40s]

Complete final gate, verbatim. TypeScript/compiler/bundle/oracle checks pass. Post-gate shipping hashes exactly match final VM inputs: x86 a9264293f5d20751607f309d3afe492c8b3c7dec8dfd0d9e7dfdc2d03e250a0b, arm64 cecf41302c0b29a698a314b47baf084d8136b966b91d3c5b5e57f851fe62c1bc. x86 receipt, arm64 receipt, controller/comparison rebuild transcript.

Final same-recipe normalized x86 comparison: base 21,356,048 bytes; head 21,249,808; delta −106,240. Exact refs/recipe/hashes. Shipping ordinary-recipe ceilings remain exactly 21,311,760 / 19,527,272 bytes.

Final artifact cohorts so far: LAN, 150 ms, agent port blocked, allocated private TCP, forwarding refusal, locked CA, NetBird SSH engine, and 150 ms / 1% packet loss each pass 20/20 with exact cleanup. Actual server-process restart and four five-case SSH-drop segments remain running. Rootless/443 runtime prerequisites still block full task acceptance; green gates do not change that. Final matrix receipts and completion/cleanup comment follow.

Final `scripts/verify fast` on frozen committed head `5e6f3f32a4d4c1656f5d97a89b955accce2e8fee`: exit 0, no load exemptions. Verbatim Rust summary: ```text +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | test | PASS | | cargo-deny | PASS | | static builds | PASS | +---------------------------+--------------+ ``` Verbatim final test output: ```text 37 pass 0 fail 34463 expect() calls Ran 37 tests across 10 files. [3.40s] ``` [Complete final gate, verbatim](https://git.kayg.org/attachments/a6e4cc73-7817-4bd5-a5ac-4101e07ee452). TypeScript/compiler/bundle/oracle checks pass. Post-gate shipping hashes exactly match final VM inputs: x86 a9264293f5d20751607f309d3afe492c8b3c7dec8dfd0d9e7dfdc2d03e250a0b, arm64 cecf41302c0b29a698a314b47baf084d8136b966b91d3c5b5e57f851fe62c1bc. [x86 receipt](https://git.kayg.org/attachments/7df064e3-d81d-40e3-babb-ddc29dd69e8d), [arm64 receipt](https://git.kayg.org/attachments/e78e9652-adf0-40ff-835a-c73db8236cac), [controller/comparison rebuild transcript](https://git.kayg.org/attachments/2886c38a-65d5-4cfb-bb7d-5c91f5fcd36a). Final same-recipe normalized x86 comparison: base 21,356,048 bytes; head 21,249,808; delta −106,240. [Exact refs/recipe/hashes](https://git.kayg.org/attachments/ac31dba4-b4f5-4a77-b85e-eb4fe4501207). Shipping ordinary-recipe ceilings remain exactly 21,311,760 / 19,527,272 bytes. Final artifact cohorts so far: LAN, 150 ms, agent port blocked, allocated private TCP, forwarding refusal, locked CA, NetBird SSH engine, and 150 ms / 1% packet loss each pass 20/20 with exact cleanup. Actual server-process restart and four five-case SSH-drop segments remain running. Rootless/443 runtime prerequisites still block full task acceptance; green gates do not change that. Final matrix receipts and completion/cleanup comment follow.
kayg commented 2026-10-17 02:19:40 +00:00
Author
Owner
Copy link

Acceptance infrastructure finding: the final original-harness restart cohort completed 19 joins, then sample 20 failed inside the process-stop helper before resumption; exact container/PID/network cleanup still completed. Unmodified raw receipt including the failure. That incomplete cohort is excluded as a whole, never counted as 20/20. The original helper omitted stderr, so that particular failure had no diagnostic beyond its assertion.

A separate deterministic native-process regression confirms a real observer race: opening /proc//stat, reaping the child, then reading the open descriptor returns ProcessLookupError/ESRCH, which the helper did not handle. Before: one error; after: both tests pass. The test executes the actual embedded observer, including a live-process refusal.

Pushed e03501af6: accept only ENOENT/ESRCH as disappeared processes, retain public stop-request metadata and bounded stop stderr; other errors still fail. No Rust, protocol, runtime binary or normal join-timing changes. Fmt, Python syntax, whitespace and both native observation tests pass. New harness SHA256 51710e6ad1b88741b932cf158fdc97cbcb373e5a49f484568bedac2efff0ea58, verified after copy. The fresh 20-case restart cohort is running, followed by four five-case SSH-drop segments; prior successful network/timing cohorts remain valid with their original recorded harness SHA. New final full fast is running on frozen committed e03501af6.

Acceptance infrastructure finding: the final original-harness restart cohort completed 19 joins, then sample 20 failed inside the process-stop helper before resumption; exact container/PID/network cleanup still completed. [Unmodified raw receipt including the failure](https://git.kayg.org/attachments/a6466281-b84e-4a45-8c0c-140cde7f00d7). That incomplete cohort is excluded as a whole, never counted as 20/20. The original helper omitted stderr, so that particular failure had no diagnostic beyond its assertion. A separate deterministic native-process regression confirms a real observer race: opening /proc/<owned child>/stat, reaping the child, then reading the open descriptor returns ProcessLookupError/ESRCH, which the helper did not handle. [Before: one error](https://git.kayg.org/attachments/6b06e5ea-0edb-4c74-9249-0719bb50e1d9); [after: both tests pass](https://git.kayg.org/attachments/d82aae42-0904-4b25-9e51-edabb6025bcd). The test executes the actual embedded observer, including a live-process refusal. Pushed e03501af6: accept only ENOENT/ESRCH as disappeared processes, retain public stop-request metadata and bounded stop stderr; other errors still fail. No Rust, protocol, runtime binary or normal join-timing changes. Fmt, Python syntax, whitespace and both native observation tests pass. New harness SHA256 51710e6ad1b88741b932cf158fdc97cbcb373e5a49f484568bedac2efff0ea58, verified after copy. The fresh 20-case restart cohort is running, followed by four five-case SSH-drop segments; prior successful network/timing cohorts remain valid with their original recorded harness SHA. New final full fast is running on frozen committed e03501af6.
kayg commented 2026-10-17 02:46:33 +00:00
Author
Owner
Copy link

Final VM acceptance receipts complete for the implemented paths: 200 accepted cases, every cohort 20/20, including 160 completed joins and 40 expected forwarding/CA refusals. The incomplete original restart cohort (19 joins plus one stop-observer error) is excluded as a whole; the corrected restart cohort is a fresh 20/20. Original timing/network receipt and excluded tail, corrected restart and all 20 SSH-drop retries, recomputed summary with artifact hashes, VM image/kernel and per-cohort load ranges, summary source. Quantiles: median p50, nearest-rank p95; 4 vCPU perf VM, same fixture image/kernel, per-sample load in raw logs.

Check-to-Done, ms Before p50/p95 After p50/p95
LAN, 20 each 1754 / 1924 1692 / 1877
150 ms RTT, 20 each 9953 / 10088 10640 / 10755

The conservative first-link checkpoint is authenticated facts complete, before mesh readiness/canary. Before→after p50/p95: LAN 1664/1814 → 1376.5/1548 ms; 150 ms 9256/9358 → 8860/8948 ms. First-link facts improve; total 150 ms completion adds 687/667 ms because it now waits for proved mesh activation and the final-path terminal proof. Do not claim the entire join adds only one RTT. Protocol operation counts are net zero extra SSH command round trips for Unix forwarding, at most one for allocated TCP, covered by the existing real command-count regression.

Other final cohorts: blocked agent port 20/20 (Done 1758.5/1921 ms), allocated private TCP 20/20 (1783.5/1969), NetBird SSH execution engine with synthetic network authorization 20/20 (2347/2821), 150 ms plus 1% per-packet loss each direction 20/20 (13503.5/15653). Forwarding denied and locked CA each give 20/20 clear refusals. Actual server-process restart after authenticated facts: 20/20, whole-controller 5582.7955/5939.467 ms. SSH master dropped at authenticated facts: 20/20, whole-controller 33534.473/34104.284 ms, including the existing 30 s readiness timeout; final successful attempt 1982.5/2251 ms. All accepted samples preserve the prior host, retire attempts/invitations and prove exact process/container/network cleanup.

Host gate e03501af6 hit one auth-socket test failure: responsive_socket_survives_a_held_terminal_open_with_an_existing_attachment, socket_clock.rs:81, “socket test stopped before its observed event”; all other gate rows pass. Raw gate. Saved head replay passes alone. Per #118, detached current dev c47a438be is built under this job’s scratch root with the same workspace feature graph, using the same single target folder. Head and baseline both pass the exact case in the paired load window. No baseline-failure exemption claimed and no assertions changed; rerunning the complete canonical fast gate on unchanged e03501af6. Full task acceptance remains blocked on the previously documented missing rootless Linux/outer 443 runtime paths.

Final VM acceptance receipts complete for the implemented paths: 200 accepted cases, every cohort 20/20, including 160 completed joins and 40 expected forwarding/CA refusals. The incomplete original restart cohort (19 joins plus one stop-observer error) is excluded as a whole; the corrected restart cohort is a fresh 20/20. [Original timing/network receipt and excluded tail](https://git.kayg.org/attachments/a6466281-b84e-4a45-8c0c-140cde7f00d7), [corrected restart and all 20 SSH-drop retries](https://git.kayg.org/attachments/db59c485-215c-48f3-a0ed-6915bb8a8a2e), [recomputed summary with artifact hashes, VM image/kernel and per-cohort load ranges](https://git.kayg.org/attachments/bc346bfd-265c-460a-9b6b-c15778da7d18), [summary source](https://git.kayg.org/attachments/7ae7bc69-c6fc-4ca7-9732-7fae01c25c6c). Quantiles: median p50, nearest-rank p95; 4 vCPU perf VM, same fixture image/kernel, per-sample load in raw logs. | Check-to-Done, ms | Before p50/p95 | After p50/p95 | | --- | --- | --- | | LAN, 20 each | 1754 / 1924 | 1692 / 1877 | | 150 ms RTT, 20 each | 9953 / 10088 | 10640 / 10755 | The conservative first-link checkpoint is authenticated facts complete, before mesh readiness/canary. Before→after p50/p95: LAN 1664/1814 → 1376.5/1548 ms; 150 ms 9256/9358 → 8860/8948 ms. First-link facts improve; total 150 ms completion adds 687/667 ms because it now waits for proved mesh activation and the final-path terminal proof. Do not claim the entire join adds only one RTT. Protocol operation counts are net zero extra SSH command round trips for Unix forwarding, at most one for allocated TCP, covered by the existing real command-count regression. Other final cohorts: blocked agent port 20/20 (Done 1758.5/1921 ms), allocated private TCP 20/20 (1783.5/1969), NetBird SSH execution engine with synthetic network authorization 20/20 (2347/2821), 150 ms plus 1% per-packet loss each direction 20/20 (13503.5/15653). Forwarding denied and locked CA each give 20/20 clear refusals. Actual server-process restart after authenticated facts: 20/20, whole-controller 5582.7955/5939.467 ms. SSH master dropped at authenticated facts: 20/20, whole-controller 33534.473/34104.284 ms, including the existing 30 s readiness timeout; final successful attempt 1982.5/2251 ms. All accepted samples preserve the prior host, retire attempts/invitations and prove exact process/container/network cleanup. Host gate e03501af6 hit one auth-socket test failure: responsive_socket_survives_a_held_terminal_open_with_an_existing_attachment, socket_clock.rs:81, “socket test stopped before its observed event”; all other gate rows pass. [Raw gate](https://git.kayg.org/attachments/bcfe4cb0-a4d5-4bfc-89df-3b9230714d00). Saved head replay passes alone. Per #118, detached current dev c47a438be is built under this job’s scratch root with the same workspace feature graph, using the same single target folder. Head and baseline both pass the exact case in the paired load window. No baseline-failure exemption claimed and no assertions changed; rerunning the complete canonical fast gate on unchanged e03501af6. Full task acceptance remains blocked on the previously documented missing rootless Linux/outer 443 runtime paths.
kayg commented 2026-10-17 02:52:58 +00:00
Author
Owner
Copy link

#118 comparison complete: saved head, same-window exact case and detached dev c47a438be, same-window exact case both exit 0. Baseline workspace build with matching feature graph. No failure exemption claimed, no assertion changed. Canonical full fast retry on unchanged e03501af6 is queued through the shared build slot.

All VM acceptance processes are finished; every sample has exact container-init-PID/container/network cleanup proof. Removed the owned VM job directory without removing any shared image, host interface or global lock/owner file. Cleanup receipt. Final-head controlled size comparison is unchanged at −106,240 bytes; e03501af6 refs/recipe/hashes. Runtime binaries remain identical to the accepted VM cohorts; this last commit changes only the harness observer and its regression.

#118 comparison complete: [saved head, same-window exact case](https://git.kayg.org/attachments/1b0d11bb-d8f7-4a8a-924e-a9a519ac5548) and [detached dev c47a438be, same-window exact case](https://git.kayg.org/attachments/481e6cb6-5281-45fc-8ad2-a14945249168) both exit 0. [Baseline workspace build with matching feature graph](https://git.kayg.org/attachments/86e80f8a-c751-4472-a198-88e9bcbbbf9b). No failure exemption claimed, no assertion changed. Canonical full fast retry on unchanged e03501af6 is queued through the shared build slot. All VM acceptance processes are finished; every sample has exact container-init-PID/container/network cleanup proof. Removed the owned VM job directory without removing any shared image, host interface or global lock/owner file. [Cleanup receipt](https://git.kayg.org/attachments/6aa7b122-162d-413a-baed-0e246bb2a255). Final-head controlled size comparison is unchanged at −106,240 bytes; [e03501af6 refs/recipe/hashes](https://git.kayg.org/attachments/cee055aa-3fd6-422e-8870-49954df00494). Runtime binaries remain identical to the accepted VM cohorts; this last commit changes only the harness observer and its regression.
kayg commented 2026-10-17 03:18:39 +00:00
Author
Owner
Copy link

Distinct gate-fixture finding during the unchanged-head retry: server_path_capability_is_advertised_on_native_and_fallback_transports failed at remote_client_tests.rs:603 with AddrInUse. UDP port-zero allocation had been treated as a TCP reservation, but the allocation pools are independent. Raw retry failure. This is not covered by a load exemption.

A controlled real TCP listener reproduces the same failure with the original allocation order: before. Pushed 47619c27d: client-only fixture reserves TCP first, retains TCP/UDP ownership together, retries only bind collisions for at most 16 candidates, and preserves the original native/fallback capability assertions and deadlines. The existing production server allocator already uses this order; no new server dependency or production API is introduced into client tests. Regression now passes; fmt, strict crate Clippy, 12 functional client tests pass, two perf-only diagnostics excluded from host execution.

This commit changes only cfg(test) code. Shipping runtime and protocol remain unchanged; all 200 accepted VM cases and their binary hashes remain the acceptance inputs. Final canonical fast is queued on frozen 47619c27d. No assertion weakened, timeout extended or test skipped. Rootless Linux and outer 443 runtime prerequisites remain missing, as recorded above.

Distinct gate-fixture finding during the unchanged-head retry: server_path_capability_is_advertised_on_native_and_fallback_transports failed at remote_client_tests.rs:603 with AddrInUse. UDP port-zero allocation had been treated as a TCP reservation, but the allocation pools are independent. [Raw retry failure](https://git.kayg.org/attachments/26ab6918-4c01-475c-8aec-763e85398d61). This is not covered by a load exemption. A controlled real TCP listener reproduces the same failure with the original allocation order: [before](https://git.kayg.org/attachments/2d3ee765-e425-4ba0-a004-d4e198352be2). Pushed 47619c27d: client-only fixture reserves TCP first, retains TCP/UDP ownership together, retries only bind collisions for at most 16 candidates, and preserves the original native/fallback capability assertions and deadlines. The existing production server allocator already uses this order; no new server dependency or production API is introduced into client tests. [Regression now passes](https://git.kayg.org/attachments/159c9f59-21b8-4678-b60e-1dee19c82cdd); [fmt, strict crate Clippy, 12 functional client tests pass](https://git.kayg.org/attachments/c4b000a3-ee21-4502-aed5-a76127160da9), two perf-only diagnostics excluded from host execution. This commit changes only cfg(test) code. Shipping runtime and protocol remain unchanged; all 200 accepted VM cases and their binary hashes remain the acceptance inputs. Final canonical fast is queued on frozen 47619c27d. No assertion weakened, timeout extended or test skipped. Rootless Linux and outer 443 runtime prerequisites remain missing, as recorded above.
kayg commented 2026-10-17 03:57:46 +00:00
Author
Owner
Copy link

Final implementation handoff: branch join-over-ssh, pushed head 47619c27d06f052faa646ad14bf05a611eb51720. No merge or deployment. Full task acceptance remains BLOCKED by missing rootless mesh and outer TCP/TLS 443 runtimes; these must be supplied through the owner/orchestrator process, not a foundational engine choice inside this job. Latest inspected dev 06be8b0000f7d476eeeeed3472a4605757151243 does not supply them. Issue remains open.

D291 implementation: enrollment rides the existing authenticated SSH session through a private Unix reverse forward (allocated private TCP fallback); forwarding refusal is explicit. Existing enrollment authentication and mutual TLS remain authoritative. The original link stays usable while the authenticated mesh candidate and durable bootstrap retirement complete; only then is the mesh connection promoted. Fresh interrupted installation recovery follows #230 ownership/journal checks.

Final scripts/verify fast exited 0 at this exact head, with no test exemption. Full transcript. Verbatim gate summaries:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| fmt                       | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| test                      | PASS         |
| cargo-deny                | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

 37 pass
 0 fail
 34463 expect() calls
Ran 37 tests across 10 files. [3.79s]

Perf VM (4 vCPU, kernel 6.12.111+deb13-amd64): final summary with per-cohort load and artifact hashes, main raw cases, corrected recovery cases. After: 10 cohorts × 20 cases = 200 accepted cases, 160 successful joins and 40 expected forwarding-disabled/CA-locked refusals; all cleanup checks passed. Cases include blocked agent port, private TCP fallback, NetBird SSH execution engine with synthetic network authorization, server-process restart at authenticated-facts boundary, SSH loss, and 150 ms RTT with 1% per-packet loss in each direction. The original incomplete server-restart cohort is excluded entirely; corrected observer regression and fresh 20-case replacement are recorded above. NetBird management/real peer deployment is not claimed.

Successful-attempt Check→Done p50/p95, milliseconds (20 before and 20 after per row):

Link Before After
LAN 1754 / 1924 1692 / 1877
150 ms RTT 9953 / 10088 10640 / 10755

Whole completion at 150 ms is +687/+667 ms because completion now proves the mesh link with a terminal canary. The ≤one-RTT claim is therefore NOT met for whole completion. The authenticated-facts checkpoint improves from 9256/9358 to 8860/8948 ms; command-count regression proves zero additional SSH command round trips for Unix forwarding and at most one for TCP fallback. Real TLS/QUIC tests prove the old link answers while mesh greeting and durable retirement are held. Slow/loss case 20/20, whole-controller p50/p95 13602.2875/15768.967 ms. Server restart 20/20, whole-controller 5582.7955/5939.467 ms. SSH loss 20/20, whole-controller 33534.473/34104.284 ms, including existing 30-second readiness wait; successful retry alone is not the end-to-end time.

Rootless and UDP-blocked diagnostics (20 each) completed tunneled enrollment but could not establish an ongoing path: these are honest no-path refusals with cleanup, NOT successful rootless/443 acceptance. Raw receipt.

Final shipping static stripped sizes: x86_64 21,311,760 bytes (receipt); aarch64 19,527,272 bytes (receipt). Both have no PT_INTERP. Shipping hashes match the accepted VM runtime inputs. x86 key-memory check passed; native arm execution is not claimed on this x86 host.

Author review completed: authenticated SSH/host binding/UID/CA checks preserved; private endpoint ownership checked; pending TLS/HTTP cancellation closes; per-source and global connection budgets retained; fresh recovery refuses unknown data; compile-out checked; no dependencies or foundational engines added; native transport continuity and retry checks passed. No UI changes.

VM cleanup proof. Local cleanup and final same-recipe size receipt follow below.

Same-recipe merge-base comparison: 21,356,048 → 21,249,808 bytes, delta −106,240 bytes, stripped static x86_64. Final comparison receipt preserves measured head e03501af6; the only later change is the cfg(test) port fixture in remote_client_tests.rs (included only under cfg(test) at remote_client.rs:391). Final shipping hashes are unchanged. A redundant final normalized rebuild was cancelled while waiting for a build slot, before Cargo started; it is not represented as a completed measurement. Identical approved recipe overlays and both measured artifact hashes are recorded in the receipt. The delta includes the imported #230 prerequisites.

Local cleanup proof: removed own temporary Git baseline, build targets, archives, binaries and fixtures; only evidence remains in the job scratch directory. Queued wrapper stopped by its own PID. Worktree clean; local and origin branch both 47619c27d06f052faa646ad14bf05a611eb51720. No owner fixtures, shared images, other jobs, merge or deployment touched.

Final implementation handoff: branch `join-over-ssh`, pushed head `47619c27d06f052faa646ad14bf05a611eb51720`. No merge or deployment. Full task acceptance remains BLOCKED by missing rootless mesh and outer TCP/TLS 443 runtimes; these must be supplied through the owner/orchestrator process, not a foundational engine choice inside this job. Latest inspected dev `06be8b0000f7d476eeeeed3472a4605757151243` does not supply them. Issue remains open. D291 implementation: enrollment rides the existing authenticated SSH session through a private Unix reverse forward (allocated private TCP fallback); forwarding refusal is explicit. Existing enrollment authentication and mutual TLS remain authoritative. The original link stays usable while the authenticated mesh candidate and durable bootstrap retirement complete; only then is the mesh connection promoted. Fresh interrupted installation recovery follows #230 ownership/journal checks. Final `scripts/verify fast` exited 0 at this exact head, with no test exemption. [Full transcript](https://git.kayg.org/attachments/d9672ddb-ee46-4d6c-849c-7db5eb1e876e). Verbatim gate summaries: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | fmt | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | test | PASS | | cargo-deny | PASS | | static builds | PASS | +---------------------------+--------------+ 37 pass 0 fail 34463 expect() calls Ran 37 tests across 10 files. [3.79s] ``` Perf VM (4 vCPU, kernel 6.12.111+deb13-amd64): [final summary with per-cohort load and artifact hashes](https://git.kayg.org/attachments/bc346bfd-265c-460a-9b6b-c15778da7d18), [main raw cases](https://git.kayg.org/attachments/a6466281-b84e-4a45-8c0c-140cde7f00d7), [corrected recovery cases](https://git.kayg.org/attachments/db59c485-215c-48f3-a0ed-6915bb8a8a2e). After: 10 cohorts × 20 cases = 200 accepted cases, 160 successful joins and 40 expected forwarding-disabled/CA-locked refusals; all cleanup checks passed. Cases include blocked agent port, private TCP fallback, NetBird SSH execution engine with synthetic network authorization, server-process restart at authenticated-facts boundary, SSH loss, and 150 ms RTT with 1% per-packet loss in each direction. The original incomplete server-restart cohort is excluded entirely; corrected observer regression and fresh 20-case replacement are recorded above. NetBird management/real peer deployment is not claimed. Successful-attempt Check→Done p50/p95, milliseconds (20 before and 20 after per row): | Link | Before | After | |---|---:|---:| | LAN | 1754 / 1924 | 1692 / 1877 | | 150 ms RTT | 9953 / 10088 | 10640 / 10755 | Whole completion at 150 ms is +687/+667 ms because completion now proves the mesh link with a terminal canary. The ≤one-RTT claim is therefore NOT met for whole completion. The authenticated-facts checkpoint improves from 9256/9358 to 8860/8948 ms; command-count regression proves zero additional SSH command round trips for Unix forwarding and at most one for TCP fallback. Real TLS/QUIC tests prove the old link answers while mesh greeting and durable retirement are held. Slow/loss case 20/20, whole-controller p50/p95 13602.2875/15768.967 ms. Server restart 20/20, whole-controller 5582.7955/5939.467 ms. SSH loss 20/20, whole-controller 33534.473/34104.284 ms, including existing 30-second readiness wait; successful retry alone is not the end-to-end time. Rootless and UDP-blocked diagnostics (20 each) completed tunneled enrollment but could not establish an ongoing path: these are honest no-path refusals with cleanup, NOT successful rootless/443 acceptance. [Raw receipt](https://git.kayg.org/attachments/8e0dc0aa-7140-40f6-ad8b-f028b17a5408). Final shipping static stripped sizes: x86_64 21,311,760 bytes ([receipt](https://git.kayg.org/attachments/18ac1d7e-000d-4538-b49a-a42f39dc1711)); aarch64 19,527,272 bytes ([receipt](https://git.kayg.org/attachments/c14e1c7d-593e-444b-9b59-6c7c9c30a709)). Both have no PT_INTERP. Shipping hashes match the accepted VM runtime inputs. x86 key-memory check passed; native arm execution is not claimed on this x86 host. Author review completed: authenticated SSH/host binding/UID/CA checks preserved; private endpoint ownership checked; pending TLS/HTTP cancellation closes; per-source and global connection budgets retained; fresh recovery refuses unknown data; compile-out checked; no dependencies or foundational engines added; native transport continuity and retry checks passed. No UI changes. [VM cleanup proof](https://git.kayg.org/attachments/6aa7b122-162d-413a-baed-0e246bb2a255). Local cleanup and final same-recipe size receipt follow below. Same-recipe merge-base comparison: 21,356,048 → 21,249,808 bytes, delta −106,240 bytes, stripped static x86_64. [Final comparison receipt](https://git.kayg.org/attachments/67be861c-cb8b-4e84-a2b7-437625c11347) preserves measured head `e03501af6`; the only later change is the cfg(test) port fixture in `remote_client_tests.rs` (included only under cfg(test) at `remote_client.rs:391`). Final shipping hashes are unchanged. A redundant final normalized rebuild was cancelled while waiting for a build slot, before Cargo started; it is not represented as a completed measurement. Identical approved recipe overlays and both measured artifact hashes are recorded in the receipt. The delta includes the imported #230 prerequisites. [Local cleanup proof](https://git.kayg.org/attachments/54667e52-b1eb-4e16-aec2-6b8334ee4107): removed own temporary Git baseline, build targets, archives, binaries and fixtures; only evidence remains in the job scratch directory. Queued wrapper stopped by its own PID. Worktree clean; local and origin branch both `47619c27d06f052faa646ad14bf05a611eb51720`. No owner fixtures, shared images, other jobs, merge or deployment touched.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
Labels
Clear labels   
launch-qa
Findings from launch QA.

No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/infraternal#233
No description provided.