Mesh: measure WireGuard userspace engines and Linux kernel control (D163, D167) #81
Closed
opened 2026-10-07 11:12:18 +00:00 by kayg
·
20 comments
No Branch/Tag specified
main
j470b-server-mesh-rejoin
mesh-compat-382
j444b-d381-key-wins-flood
j461b-macos-utun-reader-thread
j475b-remove-host-motion
fix-reconnect-new-link
j481-d380
dev
j479-mesh-soak
fix/j476-host-facts
j475-motion-pill-split
j470-mac-node-join
root-upgrade-d376
j474-review-merge
latency-targets-120hz
nightly-suite-shards
gotatun-pin-j469
macos-utun-reader-thread
fix-flood-echo-stall
wip/j416
motion-fast-ease-out
mesh-addhost-j259
takeover-close-terminals-d368
kbd-instant-followups
fix-trust-key-double-click
nightly-suite-partial-report
appwin-icons-j207
rusqlite-slice1
swipe-content-first-d374
mesh-mtu-381
addhost-instant-open
glass-contrast-d365-mock
glass-contrast-d365
signout-followups-442
fix-startup-toast-red
research-gotatun-matrix-j317
themes-sync-j304
fix-load-flakes
fix-nix-zig-patch
fix-compute01-browser-baseline
ratchet-size-baseline-419
predict-auto-d130
revoke-coverage-d367
nightly-suite-queue-timeout
nightly-browser-suite
fix-rustsec-2026-0459
fonts-ioskeley-obel
log-retention-d375
fix-kbd-no-anim-427
fix-dev-red-specs-2
takeover-proof-305-j205
wip/j410-server-pump
ui-consistency-d372
fix-osc52-clipboard
libghostty-vt-1.0
forget-retired-host-d369
j404-merge-426
fix-setup-lock-race
fix-touch-select-426
revoke-access-report-ui
fix-predictive-echo-specs
fix-selinux-agent-exec
wip/j395-checkpoint2
wip/j395
j391-review-scratch
ui-consistency-j195-deferred-rebased
fix-rustc-paths-421
decisions-wording-341
fix-install-suite-red
fix-dev-rtt-specs
fix-pending-removal-align
fix-privileged-skip
fix-install-suite-404
review-j368
mesh-dns-j254
revoke-access-report-d356b
revoke-privilege-round5
fix-removal-confirm
review-j360
gotatun-rotation-367
deps-refresh-340
review-j354
review-J351
review-j350
addhost-takeover-d359
revoke-report-items-347
revoke-end-sessions-d362
review-j348
size-mesh-ssh-346
size-trim-337
revoke-access-diff-d356
j259-pre-rebase-b06c5a6
j305-ended-at-start-spec
addhost-sudo-consent
fix-layout-real-specs-411
fix-shell-open-411
fix-console-hosts-373
fix-shell-exit
fuzz-host-name-353
fix-host-details-410
macos-filter-j197
research-gotatun-tricks
revoke-p3-j216l
stale-state-pill
revoke-followups-j216k
revoke-progress-j216e
backup/macos01-20261106/lume/j197
backup/macos01-20261106/lume/appwin-icons-j207-stash0
backup/macos01-20261106/claude/stash-0
backup/macos01-20261106/claude/stash-1
backup/macos01-20261106/claude/J80-macos-s1
backup/macos01-20261106/claude/J80
backup/macos01-20261106/claude/J80-slice1-20261102
backup/macos01-20261106/claude/J16b-appwin-on-dev
backup/macos01-20261106/claude/appwin-launch-idle-146
backup/macos01-20261106/claude/J80-s1-run
wip/j259
trust-285-p3c
runtime-lock-j216f
mesh-ssh-p3-j175
tun-proof-p3b
trust-285-p3
terminal-polish-j271
tun-proof-p3
ssh-install-flakes-j216i
fuzz-fixes-j264
trust-285-a
trust-285-b
restart-oversized-fixture-j198
backup/macos01-20261106/lume/appwin-icons-j207
restart-bound-j216h
up-preflight-p3-j198
backup/macos01-20261106/lume/j251
up-preflight-j198
wip/up-preflight-j198-j245-p2
wip/up-preflight-j198-j245-p1
agent-fsync-offloop-j196
wip/up-preflight-j198-j245
settings-offscreen-j231
wip/up-preflight-j198-j226-inspection
wip/j103f
themes-j206
wip/up-preflight-j198-j226
flakes-j216c
backup/macos01-20261106/claude/j197
flakes-j216b
backup/macos01-20261106/claude/j197-red
wip/up-preflight-j198-review
flakes-j216
backup/macos01-20261106/claude/appwin-icons-j207
engineering-j208b
appwin-mock-j209
toast-stack-j165
engineering-j208
decisions-d351
decisions-d350
decisions-d349
appearance-presets-mock-j129
decisions-d348
docs-audit-spec
doc-coverage-baseline-raise
decisions-d347
decisions-d346b
decisions-d346
j88h-profile
ui-consistency-j195-deferred
ui-consistency-j195
engineering-practices-j194
preferred-login-ui-j166
signin-pools-263-j174
macos-node-265-s1
decisions-d345
decisions-d344
proxy-cgroup-j164
verify-quick-fuzz-329
proxy-start-io-j168
timing-flakes-j163
up-263-s1
mcp-events-j147
verify-quick-fuzz-j158
proxy-p3-j159
agent-resource-j157-detect
wip/j103d-final
remote-cancel-hang-j155
proxy-traefik-281-j62d
j149-verify-quick
proxy-traefik-281-j62b
j150-drop-unnamed
wip/j79d-red-r4
wip/j103d-d332
rootless-toast-j142
wip/j103d-head
wip/j103d
j125c-mcp
macos-build-j136
backup/macos01-20261106/claude/j136
wip/j79d-red-cli
wip/j79d-red-server
j121-link-liveness
j118c-tracer-failures
wip/j79c-head
release-desktop-j131
j124c-signed-macos
wip/j79c-red-p3
wip/j79c-red-p23
wip/j79c-red-p22
merge-j128-check
perf-deferred-j128
sidebar-toggle-260
j127-shortcut-changed-dot
j125-mcp
j118-tracer-failures
j126-account-summary
settings-sections-mock-j92
j120-pinch-lock
sidebar-292
backup/macos01-20261106/claude/j120
taste-2026-11-03
row-actions-242-215
backup/macos01-20261106/claude/stash-0-untracked
backup/macos01-20261106/claude/J80-265-s1-run
backup/macos01-20261106/claude/J80-macos-s1-20261102
j112-canvas-commands
ui-no-overshoot-scroll
fix-recovery-restart
backup/macos01-20261106/claude/stash-1-untracked
taste-2026-11-02
size-baseline-ratchet
machine-cards-structure-j108
j109-boot-copy-mockups
fix-dev-red
wip/up-263-s1
j95b-settings-rows
proxy-traefik-281
j95-option-rows
docs-human-j91
docs-keys-palette
fix-flood-1013-270-j38d
mesh-ssh-s1
mesh-ssh-s0
addhost-d291-tunnel
test-stability-load
mesh-relay-220
repro-296
shim-thp-off-24
mesh-pins-258-v2
mesh-dns
signin-d311-merged
docs-pages
ui-size-283
fix-ui-batch-specs
wip/j12b-edge-update
gate-on-arm
fix-rename-sync-293
trust-285-j60b
fix-focus-ring-clip
integ-ui-1101
edge-update-v2
signin-d311
taste-2026-11-01
layout-per-tab-250
ui-approved-splitopt-224
ui-approved-splitopt-224-on-dev
icon-text-centre-282
ui-approved-189-212-216
research-reverse-proxy
transfer-budget-320
review-mesh-pins-258-r2
sidebar-pill-259
review-mesh-relay-220
wip/j12-edge-update-v2
nix-package
status-crash-loop-237
fix-dev-test-failures
fix-fixture-socket-path
wip/mesh-relay-220-pre-dev-advance-rebase
fix-canvas-side-clip
research-head-to-head
frosted-real-256
review-mesh-pins-258
wip/mesh-relay-220-pre-final-rebase
review-layout-per-tab-250-r2
broad-suite-triage
nix-devshell
review-status-crash-loop-237-r3
wip/nix-devshell-pre-p3
taste-frosted-real
wip/code-serve-demo-2026-10-30
wip/code-serve-glass-2026-10-30
wip/code-serve-clusters-2026-10-30
wip/code-merge-release94-dev-2026-10-30
wip/rproxy-delta-scratch-2026-10-30
wip/stale-note-257-staged-2026-10-30
wip/d241-size-staged-2026-10-30
review-broad-suite-207
wip/mesh-pins-258-pre-split
wip/mesh-relay-220-pre-rebase
mesh-measure-220
plan-211-update
review-edge-autoupdate-r3
perf24-reconnect
review-stale-note-r2
release-prod-live
d241-shell-marker
review-mesh-pins-server-258
review-revocation-r3
review-setup-token-r2
review-sync-journal-r3
wip/review-mesh-pins-258-prune-review-1995fee46
frosted-scroll-256
sync-journal-r3
revocation-per-account-wip
review-tokio-asyncfd-0386
review-sync-p3s
wip/stash-8by64-stash0-2026-10-30
mesh-drag-r2
setup-token-data-wip
release-9-4
auth-busy-1013
mesh-view-overlap
sync-p3s
review-auth-busy-wake
review-readd-r8-wip
mesh-drag-bounds
audit-auth-93
app-windows-stream-start
review-authority-cap-r3
app-windows-r7-measure
readd-minimal-r6
readd-minimal-r5
app-windows-r7-soak
readd-minimal-r4
app-windows-r7
app-windows-r6
readd-minimal-r3
readd-minimal-r2
readd-minimal
merge-release8-dev
review-busy-page-liveness-r2
review-busy-page-liveness
mesh-view-trim
review-authority-read-retry-r2
review-remove-uninstall-r2
release-retry-reattach
release-retry
review-remove-uninstall
review-authority-read-retry
app-windows-ship
mesh-autoswitch-p2
mesh-ssh
mesh-relay
mesh-macos-node
screen-and-control
session-labels
review-real-logins-r5
review-real-logins-r4
review-real-logins-r3
shim-memory-r2
review-real-logins-r2
shim-memory
shadow-addhost-instant
shadow-real-logins
shadow-same-session-flood
shadow-prod-perf-fix
hosts-from-server
app-windows-real
app-windows-transport
release-prod-6w
mesh-policy-verify
quiet-close-toast
dev-merge-r4
review-mesh-agent-transport
review-app-windows-transport
settings-roundtrip
review-release-pipeline-r4
mesh-agent-transport
ended-strip-fix
review-audit-ca-update-d244
settings-rows
phone-width
signout-clear
fix-restart-tests
dev-merge-rp
signer-hardening
review-audit-ca-update
terminal-fidelity
review-audit-ca-update-wip
mesh-integrate
review-mesh-agent-transport-515d3712
review-mesh-dataplane-feec1988
mesh-dataplane
signout
research-canvas-node
wip/review-renderer-cost
wip/review-terminal-fidelity
padlock-glyph
review-release-pipeline-2e95b689
phone-polish
sidebar-polish
inbox-generic
review-mesh-dataplane-378d4055
prod-batch
ui-inbox
ui-ca-unlock
ui-machine-sections
review-mesh-dataplane-a5f744fb
tls-session-prod
session-bounds-prod
fix-dead-session
session-bounds
tls-session
palette-polish
shortcuts-d222
research-appmeta
phone-gutter
ui-centre-button
wip/ui-centre-button
wip/tls-session
review-addhost-breakit
ui-platform
renderer-cost
remote-agents
mesh-api
ssh-bootstrap
prod-hotfix2
prod-fixes
review-remote-agents
review-mesh-dataplane
session-life
research-browser
review-mesh-api
research-capture
review-mesh-api-round3
review-mesh-api-round1
gate-stability
review-gate-stability
review-relay-direct
review-mesh-api-findings
proto-sessions
font-picker
proto-addhost
proto-windows
mesh-core
latency-gate
research-bsd
research-discovery
keytray-mods
mock-core
ui-tweaks
fix-motion-sel
fix-motion
integrate2
fix-robust
t-web
t-server
vt-ghostty
fix-snap-echo
t-web-auth
review-vt2
t-agent
lane-protocol
lane-shim
lane-auth
s1-base
s1-contracts
s1-workspace
No results found.
No labels
launch-qa
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/infraternal#81
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Measure WireGuard engine candidates for the mesh (D163, D167). The job measures and reports; the owner decides (foundational-choice rule).
Rootless harness: user + network namespaces (
unshare -rn), where creating a wireguard link works on infraternal-dev (verified). Results in research/mesh-engines/REPORT.md with the raw numbers.Started on branch mesh-engines from base
159f437. Reading maintained userspace and Rust netlink candidates; next I will run reproducible host benchmarks and record platform limits.Host check: rootless successfully created/deleted a kernel wg link and a TUN device; is available. succeeds. Machine: 8-core i9-9900K, Debian 13, kernel 6.12.111; load average at check 8.03/5.54/7.10.
Correction: the previous comment was shell-expanded. Actual probes: unshare -rn created/deleted a kernel WireGuard link successfully; /dev/net/tun exists and ip tuntap add/delete succeeded; systemd-run --user --scope -p CPUWeight=10000 true exited successfully. Host: 8-core i9-9900K, Debian 13, kernel 6.12.111; load average 8.03/5.54/7.10 at the probe.
Candidate build step complete: BoringTun 0.7.1 release binary 1.6 MiB and GotaTun 0.9.2 default AWS-LC release build succeeded; Go wireguard-go stripped static binary is 3.2 MiB. GotaTun audit file audits/2026-02-17-Assured.md reports 12 findings (2 low, 9 notes, 1 good), with no major vulnerabilities; it flags unsafe code and maintenance concerns in the tun dependency. Next: five-run paired tunnel and netlink control measurements.
Finding: nlink 0.29.0 full-set apply with 1,000 peers returned EINVAL: bytes leftover after parsing attributes on kernel 6.12.111. Its set path emits one peer-list request and does not chunk. I will load the 1,000-peer baseline through the chunking client to measure nlink single-peer update and removal separately.
Candidate survey addendum: NepTUN is a maintained BSD-3-Clause Rust engine. GitHub Release v1.0.8 is dated 2025-07-21; upstream main was updated 2026-10-06 (source commit ce185155e1639403cc40e01260d0830065a49555). It has a Linux/macOS CLI and Windows library target, but the library leaves tunnel I/O to the integrator. Source: https://github.com/NordSecurity/NepTUN and https://github.com/NordSecurity/NepTUN/releases. I am building and adding it to the Linux dataplane comparison before writing the final recommendation.
Packaging finding: a stripped Go c-archive host with the measured engine path linked into a stripped Rust executable is 3,870,864 B versus a 332,688 B Rust baseline (+3,538,176 B); the archive itself is 4,332,236 B. The reproducible bridge and Rust host are being added under research/mesh-engines/go-embed/; standalone Go CLI was 3,313,824 B.
License finding: GotaTun CLI enables its optional tun feature and therefore includes tun 0.8.6, whose Cargo.toml declares WTFPL (source https://github.com/meh/rust-tun/blob/master/Cargo.toml). WTFPL is outside the approved license set. The GotaTun core can be built without that feature; its own MPL-2.0 license is allowed, but Windows/macOS TUN integration then needs a different adapter. Its Windows adapter loads wintun.dll at runtime. Recording this as a hard integration constraint in REPORT.md.
Candidate sweep: tailscale/wireguard-go main resolves to 469159ecf7d108dc1d44f8789dedbb463b7bd3a6 (last commit 2023-07-04; local checkout under /home/kayg/.cache/codex-scratch/mesh-engines/src/tailscale-wireguard-go), so it is not a maintained standalone candidate. Its security page reports no published advisories, but that does not change the stale-source finding. I am recording recent exploratory Go/Rust projects separately where target OS support or license evidence is incomplete.
Windows packaging finding: Wintun source labels the driver GPL-2.0-only, and the prebuilt wintun.dll has a separate custom license allowing redistribution only alongside software that uses its API (WireGuard/wintun prebuilt-binaries-license.txt). Neither is on the AGPL-compatible allowlist. Go/GotaTun can call Wintun, but this driver/package route is not license-cleared by the current rule; no Windows host was available to measure it. Evidence: https://github.com/WireGuard/wintun/blob/master/wintun.proj and https://github.com/WireGuard/wintun/blob/master/prebuilt-binaries-license.txt
Security review finding: current NepTUN source still contains in at commit ce185155e1639403cc40e01260d0830065a49555. This leaves the same inner-packet-length traffic-analysis exposure documented in upstream BoringTun #494; GotaTun's 2026 audit says its padding fix shipped in 0.3.0, and current source has padding logic. NepTUN has no audit located; its open macOS clock issue is #13. https://github.com/cloudflare/boringtun/issues/494 https://github.com/NordSecurity/NepTUN/issues/13
Correction to prior comment: the shell expanded my quoted source text. The complete finding is that current NepTUN source at neptun/src/noise/session.rs (commit ce185155e1639403cc40e01260d0830065a49555) retains a TODO for 16-byte data padding, leaving the inner packet-length traffic-analysis exposure noted in BoringTun issue 494. GotaTun fixed padding in its 0.3.0 line; the current source contains padding logic. NepTUN has no audit located, and issue 13 is open about macOS clock behavior. https://github.com/cloudflare/boringtun/issues/494 https://github.com/NordSecurity/NepTUN/issues/13
Harness commit
71d0b76882adds the rootless data-plane, netlink, artifact-size, and Go c-archive runners. Shell syntax passed; both locked netlink release builds completed offline, including the nlink feature build.Raw-results commit
f25c68d5ddretains five samples per data-plane and control candidate, environment capture, artifact bytes, and generated medians. Data-plane four-stream medians: kernel 2,658 Mbit/s; NepTUN 989; wireguard-go 1,541; BoringTun 586; GotaTun 514. Netlink applied 1,000 peers in 61,747 us (nl-wireguard) and 66,333 us (typed client); nlink full apply returned EINVAL. Files are under research/mesh-engines/results/.Report commit
e6bb136160records source revisions, platform TUN coverage, audits/security reports, dependency licenses, build commands, and measured trade-offs. Recommendation recorded for owner review: kernel WireGuard on Linux; typed netlink bindings plus bounded chunks or nl-wireguard for control; no userspace candidate currently meets all platform, license, and single-Rust-binary constraints.Fast gate blocked: scripts/verify is absent from this worktree; scripts/ contains only perf-run. The required command returned: zsh:1: no such file or directory: scripts/verify (exit 127). Shell syntax, generated summary comparison, and both locked offline Rust release builds passed. Research scope does not include adding product gate code; the gate entrypoint must be restored to the worktree to run the prescribed gate.
Finished on branch mesh-engines. Head:
e6bb136160.Recommendation: kernel WireGuard for Linux speed; typed minimal netlink or nl-wireguard for control. No measured userspace option clears the macOS, Windows, old Linux, AGPL-compatible dependency, and single Rust binary constraints. Wintun source is GPL-2.0-only and its prebuilt DLL has a separate custom license. Native macOS/Windows measurements remain unavailable. Kernel CPU counts wg-crypt workers only; userspace CPU counts both endpoints. Shared-host p99 is scheduler-sensitive.
Report: research/mesh-engines/REPORT.md. Raw samples and host evidence: research/mesh-engines/results/.
Required fast gate attempt: scripts/verify fast
Output (verbatim): zsh:1: no such file or directory: scripts/verify
Exit code: 127. The worktree contains scripts/perf-run but no scripts/verify. Bash syntax, summary regeneration, both locked offline netlink builds, and git diff --check passed.
Owner answers (2026-10-07):
Owner decisions recorded as D169 (main): our own typed netlink client for Linux; GotaTun for macOS nodes with our own utun adapter; Windows driver deferred.
Closed by the orchestrator during the 2026-10-28 handover (#254), on the owner's request to close finished issues. Evidence: Engine measurement report research/mesh-engines/REPORT.md delivered and owner decisions recorded as D169 (
a24ee0bd6in dev). Reopen if anything is still missing.kayg referenced this issue2026-10-28 17:53:54 +00:00