kayg/infraternal

Watch

Fork 0

J318: rank source-backed GotaTun performance techniques (#260) #403

New issue

Open

opened 2026-11-06 16:45:28 +00:00 by kayg · 8 comments

kayg commented

2026-11-06 16:45:28 +00:00

Owner

Research-only job for #260. Compare GotaTun, upstream wireguard-go, Tailscale fork, BoringTun and kernel WireGuard techniques; record mechanisms, expected gains, risks, decision constraints and measurement plans. Deliver research/gotatun-tricks-2026-11-06.md on pushed research-gotatun-tricks. No product code or decision changes.

kayg commented

2026-11-06 16:45:39 +00:00

Author

Owner

J318 start: branch research-gotatun-tricks, base 37c93637e4 (dev). Research and documentation only. scripts/fj is absent from both main and dev; using the installed fj CLI with remote fjweb. Reading #260 J201/J88i/J225/J316 evidence; no perf VM work or product changes.

J318 start: branch research-gotatun-tricks, base 37c93637e49a415578416be037f972b9a716701c (dev). Research and documentation only. scripts/fj is absent from both main and dev; using the installed fj CLI with remote fjweb. Reading #260 J201/J88i/J225/J316 evidence; no perf VM work or product changes.

kayg referenced this issue

2026-11-06 16:45:39 +00:00

Head-to-head: infraternal mesh vs NetBird/Tailscale, infraternal sessions vs tmux/Zellij #260

kayg commented

2026-11-06 16:48:18 +00:00

Author

Owner

Source finding: fork ebeb8f0071c4103168b570407f9dc555562f4b79 uses MAX_PACKET_COUNT=100, UDP_SEGMENT_MAX_DATAGRAMS=64 and RecvManyBuf reserves 100 × (64 × 4096) = 25 MiB per instance, with idle madvise release. append_received_packets copies each GRO segment into a pool packet; TUN send also allocates/copies a vnet-header frame per packet. These are concrete memory/copy measurement targets, not claims of 25 MiB resident or per peer. Latest Tailscale source 79ead109 implements shared receive slabs and a distinct 2048-byte small pool. The report will give snapshot links and measurement limits. Verification: plain-shell quick/fast stop because xz is absent; nix develop quick passed, full fast retry is running.

kayg commented

2026-11-06 16:57:05 +00:00

Author

Owner

Research document is committed at 223d5d768b. It ranks 14 techniques, distinguishes shipping pin 84b7bb28 from diagnostic head ebeb8f0, records the failed spread calibration and rejected 64-packet cut, and includes historical 1,000-peer memory attribution limits. Validation: 35 external primary-source URLs returned HTTP 200; 5 local references resolve; git diff --check passed; final nix develop quick passed. Full fast gate is compiling the unchanged workspace and remains pending. No performance measurements or code changes by J318.

Research document is committed at 223d5d768b01b7fde1f1b010f1f3607edc1d37a5. It ranks 14 techniques, distinguishes shipping pin 84b7bb28 from diagnostic head ebeb8f0, records the failed spread calibration and rejected 64-packet cut, and includes historical 1,000-peer memory attribution limits. Validation: 35 external primary-source URLs returned HTTP 200; 5 local references resolve; git diff --check passed; final nix develop quick passed. Full fast gate is compiling the unchanged workspace and remains pending. No performance measurements or code changes by J318.

kayg referenced this issue

2026-11-06 17:00:51 +00:00

Head-to-head: infraternal mesh vs NetBird/Tailscale, infraternal sessions vs tmux/Zellij #260

kayg commented

2026-11-06 17:00:51 +00:00

Author

Owner

Published research: branch research-gotatun-tricks is pushed through origin SSH at 223d5d768b; full report is now a comment on #260. HTTPS push through fjweb had no configured Git credential, so the existing origin SSH remote was used successfully. Full fast gate remains running; this is publication status, not a green-gate claim.

Published research: branch research-gotatun-tricks is pushed through origin SSH at 223d5d768b01b7fde1f1b010f1f3607edc1d37a5; full report is now a comment on #260. HTTPS push through fjweb had no configured Git credential, so the existing origin SSH remote was used successfully. Full fast gate remains running; this is publication status, not a green-gate claim.

kayg commented

2026-11-06 18:00:56 +00:00

Author

Owner

Publication verified: full report is #260 comment 82600, remote head is 223d5d768b, and the worktree is clean. Full nix develop scripts/verify fast has advanced through CLI compile-out, clippy, SSH helper, fixture paths and release checks; it is still compiling/linking workspace tests. Process samples confirm increasing rustc CPU time and changing linker PIDs despite the quiet log. No final gate result yet; no performance run or code change was made.

Publication verified: full report is #260 comment 82600, remote head is 223d5d768b01b7fde1f1b010f1f3607edc1d37a5, and the worktree is clean. Full nix develop scripts/verify fast has advanced through CLI compile-out, clippy, SSH helper, fixture paths and release checks; it is still compiling/linking workspace tests. Process samples confirm increasing rustc CPU time and changing linker PIDs despite the quiet log. No final gate result yet; no performance run or code change was made.

kayg commented

2026-11-06 18:11:22 +00:00

Author

Owner

J318 verification finding at head 223d5d768b01b7fde1f1b010f1f3607edc1d37a5: the first full scripts/verify fast run reached the shim integration tests and reported shim startup/session failed: No such file or directory (os error 2). Tracing an isolated test showed its initial state-directory mkdir failing with ENOENT. The fixture default parent $HOME/.cache/codex-scratch/session-bounds did not exist; it assumes that parent exists. Created this scratch directory, with no repository change, and reran the exact failing test:

running 1 test
test detached_shell_exit_reaps_shim ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 30 filtered out; finished in 0.04s

Stopped the first already-failing gate attempt and restarted the full gate with the prerequisite present. Full gate result still pending. Evidence under $HOME/.cache/codex-scratch/j318/evidence/: verify-fast-missing-scratch.log, shim-test.strace, and current verify-fast-nix.log. The research report is already pushed and posted on #260; this finding is verification infrastructure, not an engine-performance result.

J318 verification finding at head `223d5d768b01b7fde1f1b010f1f3607edc1d37a5`: the first full `scripts/verify fast` run reached the shim integration tests and reported `shim startup/session failed: No such file or directory (os error 2)`. Tracing an isolated test showed its initial state-directory mkdir failing with ENOENT. The fixture default parent `$HOME/.cache/codex-scratch/session-bounds` did not exist; it assumes that parent exists. Created this scratch directory, with no repository change, and reran the exact failing test: ```text running 1 test test detached_shell_exit_reaps_shim ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 30 filtered out; finished in 0.04s ``` Stopped the first already-failing gate attempt and restarted the full gate with the prerequisite present. Full gate result still pending. Evidence under `$HOME/.cache/codex-scratch/j318/evidence/`: `verify-fast-missing-scratch.log`, `shim-test.strace`, and current `verify-fast-nix.log`. The research report is already pushed and posted on #260; this finding is verification infrastructure, not an engine-performance result.

kayg commented

2026-11-06 18:17:56 +00:00

Author

Owner

---- the_shell_and_its_children_never_hold_the_lifeline stdout ----

thread 'the_shell_and_its_children_never_hold_the_lifeline' (629487) panicked at crates/infraternal/tests/shim.rs:329:34:
called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    the_shell_and_its_children_never_hold_the_lifeline

test result: FAILED. 30 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 125.37s

error: test failed, to rerun pass `-p infraternal --test shim`
(test: 267s)

J318 full-gate finding after fixing the scratch-directory prerequisite: 30 shim tests pass; the remaining lifeline descriptor test times out reading terminal output. No product-code changes in this branch, and no claim that the timeout root cause is established. The gate is continuing through remaining rows. Verbatim output: ```text ---- the_shell_and_its_children_never_hold_the_lifeline stdout ---- thread 'the_shell_and_its_children_never_hold_the_lifeline' (629487) panicked at crates/infraternal/tests/shim.rs:329:34: called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" } note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace failures: the_shell_and_its_children_never_hold_the_lifeline test result: FAILED. 30 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 125.37s error: test failed, to rerun pass `-p infraternal --test shim` (test: 267s) ```

kayg commented

2026-11-06 18:27:31 +00:00

Author

Owner

J318 finish: research published at head 223d5d768b01b7fde1f1b010f1f3607edc1d37a5, branch research-gotatun-tricks (remote SHA verified; clean worktree). Base 37c93637e49a415578416be037f972b9a716701c.

Report on the branch.
Full report posted on #260.
14 ranked techniques cover mechanisms, expected effects, costs/risks, D289/D352 decision boundaries, source snapshots, and memory at 1,000 peers. No code, dependency or decision changed; no new performance run. 35 external primary-source URLs checked HTTP 200; five local references resolve; git show --check HEAD passes.

Verification is not green. Ran nix develop --command bash -c 'scripts/verify fast' on the final head. The first attempt exposed a missing scratch-directory prerequisite. After creating $HOME/.cache/codex-scratch/session-bounds, its isolated failure passed and a fresh full run passed 30/31 shim integration tests but timed out on the remaining lifeline test. Its root cause is not established; no unrelated code fix was made.

Verbatim full-gate failure output:

---- the_shell_and_its_children_never_hold_the_lifeline stdout ----

thread 'the_shell_and_its_children_never_hold_the_lifeline' (629487) panicked at crates/infraternal/tests/shim.rs:329:34:
called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    the_shell_and_its_children_never_hold_the_lifeline

test result: FAILED. 30 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 125.37s

error: test failed, to rerun pass `-p infraternal --test shim`
(test: 267s)

The gate continued afterward: process-per-user tests and policy checks completed; privileged integration reported the following verbatim:

privileged integration: SKIP (no change to install, proxy, mesh or shim code, or a crate they depend on)

I then stopped the already-failed gate during probe release exclusion compilation (process exit 143). Probe release exclusion, nightly fuzz reporter, static builds and the subsequent TypeScript gate are incomplete, not passing. There is no final all-row gate summary. Full captured output: $HOME/.cache/codex-scratch/j318/evidence/verify-fast-nix.log; first attempt: verify-fast-missing-scratch.log; diagnostic: shim-test.strace.

The documentation pre-check passed, separately from the failed/incomplete final gate. Verbatim:

QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done.

No merge, deployment, or issue closure. Top 5 by expected gain per effort: TUN coalescing; bounded fair queue service; CPU/IRQ locality; shared slabs and right-sized pools; bounded ordered parallel ring crypto.

J318 finish: research published at head `223d5d768b01b7fde1f1b010f1f3607edc1d37a5`, branch `research-gotatun-tricks` (remote SHA verified; clean worktree). Base `37c93637e49a415578416be037f972b9a716701c`. - [Report on the branch](https://git.kayg.org/kayg/infraternal/src/branch/research-gotatun-tricks/research/gotatun-tricks-2026-11-06.md). - [Full report posted on #260](https://git.kayg.org/kayg/infraternal/issues/260#issuecomment-82600). - 14 ranked techniques cover mechanisms, expected effects, costs/risks, D289/D352 decision boundaries, source snapshots, and memory at 1,000 peers. No code, dependency or decision changed; no new performance run. 35 external primary-source URLs checked HTTP 200; five local references resolve; `git show --check HEAD` passes. Verification is **not green**. Ran `nix develop --command bash -c 'scripts/verify fast'` on the final head. The first attempt exposed a missing scratch-directory prerequisite. After creating `$HOME/.cache/codex-scratch/session-bounds`, its isolated failure passed and a fresh full run passed 30/31 shim integration tests but timed out on the remaining lifeline test. Its root cause is not established; no unrelated code fix was made. Verbatim full-gate failure output: ```text ---- the_shell_and_its_children_never_hold_the_lifeline stdout ---- thread 'the_shell_and_its_children_never_hold_the_lifeline' (629487) panicked at crates/infraternal/tests/shim.rs:329:34: called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" } note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace failures: the_shell_and_its_children_never_hold_the_lifeline test result: FAILED. 30 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 125.37s error: test failed, to rerun pass `-p infraternal --test shim` (test: 267s) ``` The gate continued afterward: process-per-user tests and policy checks completed; privileged integration reported the following verbatim: ```text privileged integration: SKIP (no change to install, proxy, mesh or shim code, or a crate they depend on) ``` I then stopped the already-failed gate during `probe release exclusion` compilation (process exit 143). Probe release exclusion, nightly fuzz reporter, static builds and the subsequent TypeScript gate are **incomplete**, not passing. There is no final all-row gate summary. Full captured output: `$HOME/.cache/codex-scratch/j318/evidence/verify-fast-nix.log`; first attempt: `verify-fast-missing-scratch.log`; diagnostic: `shim-test.strace`. The documentation pre-check passed, separately from the failed/incomplete final gate. Verbatim: ```text QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done. ``` No merge, deployment, or issue closure. Top 5 by expected gain per effort: TUN coalescing; bounded fair queue service; CPU/IRQ locality; shared slabs and right-sized pools; bounded ordered parallel ring crypto.

No labels

launch-qa

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/infraternal#403

No description provided.

Rows
Columns