Mesh: our own WireGuard mesh as a binary role (grill)

kayg commented

2026-10-07 10:42:39 +00:00

Owner

Owner request (2026-10-07): build our own WireGuard mesh — open source, coordinated by infraternal, clients for web/macOS/iOS later. Mesh is one more role of the single binary: any server or infraternal agent node can be a mesh peer, a relay (D131) and/or a backup host (D147). Operator subcommand plus a UI checkbox.

Grill round 1 (asked 2026-10-07), awaiting answers:

Q1 purpose: node-to-node, user devices, or both staged (rec: both, staged; mesh stays optional per N2)
Q2 infraternal mesh up|down|status; deployment switch + per-host opt-in (rec)
Q3 coordination inside the infraternal server, ACLs from grants D154, private keys never leave nodes (rec)
Q4 NAT fallback through the existing relay role D128/D131 (rec)
Q5 IPv6-first ULA + IPv4 /16 chosen to avoid ranges discovery sees in use (rec)

Next round (blocked on round 1): privileges/CAP_NET_ADMIN, kernel vs userspace WireGuard (foundational: measured candidates, owner decides), DNS names, exit nodes and subnet routes, client order.

Owner request (2026-10-07): build our own WireGuard mesh — open source, coordinated by infraternal, clients for web/macOS/iOS later. Mesh is one more role of the single binary: any server or infraternal agent node can be a mesh peer, a relay (D131) and/or a backup host (D147). Operator subcommand plus a UI checkbox. Grill round 1 (asked 2026-10-07), awaiting answers: - Q1 purpose: node-to-node, user devices, or both staged (rec: both, staged; mesh stays optional per N2) - Q2 `infraternal mesh up|down|status`; deployment switch + per-host opt-in (rec) - Q3 coordination inside the infraternal server, ACLs from grants D154, private keys never leave nodes (rec) - Q4 NAT fallback through the existing relay role D128/D131 (rec) - Q5 IPv6-first ULA + IPv4 /16 chosen to avoid ranges discovery sees in use (rec) Next round (blocked on round 1): privileges/CAP_NET_ADMIN, kernel vs userspace WireGuard (foundational: measured candidates, owner decides), DNS names, exit nodes and subnet routes, client order.

kayg commented

2026-10-07 10:43:31 +00:00

Author

Owner

Owner answer Q1 (2026-10-07): agreed, staged. Stage 1 = node to node (server and infraternal agents). Stage 2 = clients (user devices: web, macOS, iOS).

kayg commented

2026-10-07 10:44:52 +00:00

Author

Owner

Owner answers (2026-10-07):

Q2: joining is AUTOMATIC: every infraternal agent joins the mesh. Access is granted visually: later, sticky arrows on the canvas from window to window attach two hosts and allow access. The point of the mesh is that it is extremely easy and fully UI-driven; the API, MCP and CLI must expose everything too (API-first parity).
Q5 (partial): it's WireGuard, so it coexists with other meshes on Linux (several WireGuard interfaces live side by side). Orchestrator note: interfaces coexist, but overlapping address ranges/routes would not, so the range must still be chosen automatically to avoid ranges in use (no user step). Follow-up asked.

Owner answers (2026-10-07): - Q2: joining is AUTOMATIC: every infraternal agent joins the mesh. Access is granted visually: later, sticky arrows on the canvas from window to window attach two hosts and allow access. The point of the mesh is that it is extremely easy and fully UI-driven; the API, MCP and CLI must expose everything too (API-first parity). - Q5 (partial): it's WireGuard, so it coexists with other meshes on Linux (several WireGuard interfaces live side by side). Orchestrator note: interfaces coexist, but overlapping address ranges/routes would not, so the range must still be chosen automatically to avoid ranges in use (no user step). Follow-up asked.

kayg commented

2026-10-07 10:48:51 +00:00

Author

Owner

Owner answers (2026-10-07):

Q3: the infraternal server is the coordination server. Nodes exchange keys through it and greet each other.
Q4: traffic falls back through the server by default, or through a relay if one exists. The server automatically runs a relay (consistent with D128/D131).
Q5: a FIXED IPv4 range, suggested 10.69.69.0/24 or 10.69.0.0/16. Orchestrator flagged: the owner's own LAN is 10.69.69.0/24 (monitor01 10.69.69.50, infraternal-dev 10.69.69.125), so both suggestions overlap it; /24 also caps the mesh at 254 nodes. Follow-up asked (Q5b).

Owner answers (2026-10-07): - Q3: the infraternal server is the coordination server. Nodes exchange keys through it and greet each other. - Q4: traffic falls back through the server by default, or through a relay if one exists. The server automatically runs a relay (consistent with D128/D131). - Q5: a FIXED IPv4 range, suggested 10.69.69.0/24 or 10.69.0.0/16. Orchestrator flagged: the owner's own LAN is 10.69.69.0/24 (monitor01 10.69.69.50, infraternal-dev 10.69.69.125), so both suggestions overlap it; /24 also caps the mesh at 254 nodes. Follow-up asked (Q5b).

kayg commented

2026-10-07 10:54:02 +00:00

Author

Owner

Owner answers (2026-10-07), round 2 closed:

Q5b: fixed IPv4 range 10.254.0.0/16 per deployment (orchestrator checked: no 10.254.x route or address on infraternal-dev; its interfaces are 10.69.69.0/24, 10.70.3.0/24, 172.17.0.0/16). The setup-time overlap check stays: warn and offer another range, never change silently.
Q5c: yes. Each node also gets a fixed IPv6 ULA address; the UI shows IPv4.
Q6: (a) deny by default after the automatic join; arrows grant access.
Q7: agreed, and very visual. Arrows can switch easily between individual nodes, tags and users as their endpoints. Direction is A→B initiates with replies flowing back; double-headed means both ways; all ports by default, narrowable to ports or discovered services. Each arrow is a D154 grant, with API/MCP/CLI parity.
Q8: yes. infraternal's own traffic (terminal, VNC, backups) moves automatically to the mesh path when it measures faster, falls back without breaking the session, and needs no arrow.

Owner answers (2026-10-07), round 2 closed: - Q5b: fixed IPv4 range **10.254.0.0/16** per deployment (orchestrator checked: no 10.254.x route or address on infraternal-dev; its interfaces are 10.69.69.0/24, 10.70.3.0/24, 172.17.0.0/16). The setup-time overlap check stays: warn and offer another range, never change silently. - Q5c: yes. Each node also gets a fixed IPv6 ULA address; the UI shows IPv4. - Q6: (a) deny by default after the automatic join; arrows grant access. - Q7: agreed, and very visual. Arrows can switch easily between individual nodes, tags and users as their endpoints. Direction is A→B initiates with replies flowing back; double-headed means both ways; all ports by default, narrowable to ports or discovered services. Each arrow is a D154 grant, with API/MCP/CLI parity. - Q8: yes. infraternal's own traffic (terminal, VNC, backups) moves automatically to the mesh path when it measures faster, falls back without breaking the session, and needs no arrow.

kayg commented

2026-10-07 10:58:29 +00:00

Author

Owner

Owner answers (2026-10-07), round 3:

Q9: the mesh runs with root privileges. Keep it simple: no split helper, no userspace-only fallback mode. (This fits the earlier rule that a root infraternal agent replaces the user agent when root becomes available.)
Q10: kernel WireGuard on Linux, userspace everywhere else. Per the foundational-choice rule, a job measures the userspace candidates (AGPL-compatible) and the owner picks among them.
Q11: yes to mesh DNS names. The owner asks whether the default TLD could be .infraternal; follow-up Q11b asked.
Q12: subnet routes and exit nodes both come later. Stage 1 stays simple and easy to understand.
Q13, Q14: still open.

Owner answers (2026-10-07), round 3: - Q9: the mesh runs with root privileges. Keep it simple: no split helper, no userspace-only fallback mode. (This fits the earlier rule that a root infraternal agent replaces the user agent when root becomes available.) - Q10: kernel WireGuard on Linux, userspace everywhere else. Per the foundational-choice rule, a job measures the userspace candidates (AGPL-compatible) and the owner picks among them. - Q11: yes to mesh DNS names. The owner asks whether the default TLD could be `.infraternal`; follow-up Q11b asked. - Q12: subnet routes and exit nodes both come later. Stage 1 stays simple and easy to understand. - Q13, Q14: still open.

kayg commented

2026-10-07 11:02:52 +00:00

Author

Owner

Owner answers (2026-10-07):

Q11b: the default suffix is .internal (no other TLD). It is changeable to any domain, e.g. the owner's NetBird domain is vpn.kayg.org, so names would become pve01.vpn.kayg.org.
Q13: needs more discussion. The owner notes that automatic expiry isn't user friendly. Orchestrator is splitting it into rotation / expiry / server-unreachable behaviour (Q13a–c).
Q14: owner asked for an explanation; re-asked with an example.

Owner answers (2026-10-07): - Q11b: the default suffix is `.internal` (no other TLD). It is changeable to any domain, e.g. the owner's NetBird domain is `vpn.kayg.org`, so names would become `pve01.vpn.kayg.org`. - Q13: needs more discussion. The owner notes that automatic expiry isn't user friendly. Orchestrator is splitting it into rotation / expiry / server-unreachable behaviour (Q13a–c). - Q14: owner asked for an explanation; re-asked with an example.

kayg commented

2026-10-07 11:05:14 +00:00

Author

Owner

Owner answers (2026-10-07):

Q13a: silent automatic key rotation, every 7 days by default (configurable).
Q13b: agreed. No membership expiry for nodes; removal pushes key drops to every peer instantly. User devices (stage 2) are decided separately.
Q13c: agreed. When the server is down, the mesh keeps working on the last known rules with no time limit; a removal reaches each peer when it reconnects; the UI shows peers that are behind on rules.
Q14: still open (re-explained with the kayg -> db01:5432 example).

Owner answers (2026-10-07): - Q13a: silent automatic key rotation, **every 7 days by default** (configurable). - Q13b: agreed. No membership expiry for nodes; removal pushes key drops to every peer instantly. User devices (stage 2) are decided separately. - Q13c: agreed. When the server is down, the mesh keeps working on the last known rules with no time limit; a removal reaches each peer when it reconnects; the UI shows peers that are behind on rules. - Q14: still open (re-explained with the kayg -> db01:5432 example).

kayg commented

2026-10-07 11:07:37 +00:00

Author

Owner

Owner answer (2026-10-07):

Q14: (a). Stage 1 arrows connect nodes and tags only. "User" is visible in the endpoint switch but greyed out ("Arrives with devices") until stage 2, when a user arrow means "this person's devices can reach this".

Open: Q15 (Linux-only first), Q16 (arrow path/RTT display), Q17 (stage 2 native client order; browsers cannot be mesh peers), Q18 (build order: core + engine measurement now, arrows after the canvas mockups).

Owner answer (2026-10-07): - Q14: (a). Stage 1 arrows connect nodes and tags only. "User" is visible in the endpoint switch but greyed out ("Arrives with devices") until stage 2, when a user arrow means "this person's devices can reach this". Open: Q15 (Linux-only first), Q16 (arrow path/RTT display), Q17 (stage 2 native client order; browsers cannot be mesh peers), Q18 (build order: core + engine measurement now, arrows after the canvas mockups).

kayg commented

2026-10-07 11:09:00 +00:00

Author

Owner

Owner answers (2026-10-07):

Q15: correct. Stage 1 mesh nodes are Linux only (kernel WireGuard); macOS and Windows nodes follow once the userspace engine is chosen.
Q16: agreed. Arrows show solid = direct, dashed = via relay (named), RTT on hover/tap only, and a quiet "rule not delivered yet" state.
Q17: stage 2 clients are macOS, then iOS (native; browsers cannot be mesh peers). Linux desktop and Windows clients are not scheduled.

Open: Q18 (build order).

Owner answers (2026-10-07): - Q15: correct. Stage 1 mesh nodes are Linux only (kernel WireGuard); macOS and Windows nodes follow once the userspace engine is chosen. - Q16: agreed. Arrows show solid = direct, dashed = via relay (named), RTT on hover/tap only, and a quiet "rule not delivered yet" state. - Q17: stage 2 clients are macOS, then iOS (native; browsers cannot be mesh peers). Linux desktop and Windows clients are not scheduled. Open: Q18 (build order).

kayg referenced this issue

2026-10-07 11:12:17 +00:00

Mesh core slice 1: coordination, join, rules, rotation (D163–D167) #80

kayg referenced this issue

2026-10-07 11:14:12 +00:00

Mesh core slice 1: coordination, join, rules, rotation (D163–D167) #80

kayg referenced this issue

2026-10-08 13:31:17 +00:00

Mesh API for the Mesh view: hosts, tags and grants (D163-D167, D180, D187) #108

kayg referenced this issue

2026-10-08 19:51:16 +00:00

Mesh API for the Mesh view: hosts, tags and grants (D163-D167, D180, D187) #108