Mesh: a leftover enabled snapshot without its key silently keeps the server's own node from starting #388

New issue

Open

opened 2026-11-06 09:36:24 +00:00 by kayg · 0 comments

kayg commented

2026-11-06 09:36:24 +00:00

Owner

Found by J175 during the #221 VM proof (compute01, build 60d5f09a8).

What happens. A server started on a host whose /run/infraternal holds a leftover enabled mesh snapshot (mesh-public.json with enabled: true and a full config, written 2026-11-04 by an earlier run) and a mesh-interface.json (2026-11-02), but no mesh-private-key, never brings up its own mesh node. Nothing is logged. The only signs are repeated Mesh storage unavailable … reason="Mesh server node is not connected." (store.rs:1875) on add-host, and mesh.list showing only the added hosts, all live: false, with no infraternal0 in the server's namespace. The server's embedded agent (agent --fd 3, state /run/infraternal) is the only reader of that directory; the installed infraternal agent uses /var/lib/infraternal/state.

Proof. After stopping the server and moving those two files aside (kept, then restored byte for byte), the next start brought the node up at once (infraternal0 10.254.0.3, fresh mesh-public.json, mesh-private-key, mesh-engine.json, mesh-routes.json), and all nodes went live within 40 s.

Expected. Either restore treats "enabled snapshot, no key" as a fresh node (or regenerates the key), or it fails loudly with a log line naming the file. Today a host where an earlier server ran can silently never join its own mesh. node_state.rs Store::open and the identity path in node_coordination::run (read_identity returns None and the loop retries silently) are the places to look.

Found by J175 during the #221 VM proof (compute01, build `60d5f09a8`). **What happens.** A server started on a host whose `/run/infraternal` holds a leftover enabled mesh snapshot (`mesh-public.json` with `enabled: true` and a full `config`, written 2026-11-04 by an earlier run) and a `mesh-interface.json` (2026-11-02), but no `mesh-private-key`, never brings up its own mesh node. Nothing is logged. The only signs are repeated `Mesh storage unavailable … reason="Mesh server node is not connected." (store.rs:1875)` on add-host, and `mesh.list` showing only the added hosts, all `live: false`, with no `infraternal0` in the server's namespace. The server's embedded agent (`agent --fd 3`, state `/run/infraternal`) is the only reader of that directory; the installed infraternal agent uses `/var/lib/infraternal/state`. **Proof.** After stopping the server and moving those two files aside (kept, then restored byte for byte), the next start brought the node up at once (`infraternal0` 10.254.0.3, fresh `mesh-public.json`, `mesh-private-key`, `mesh-engine.json`, `mesh-routes.json`), and all nodes went live within 40 s. **Expected.** Either restore treats "enabled snapshot, no key" as a fresh node (or regenerates the key), or it fails loudly with a log line naming the file. Today a host where an earlier server ran can silently never join its own mesh. `node_state.rs` `Store::open` and the identity path in `node_coordination::run` (`read_identity` returns `None` and the loop retries silently) are the places to look.

kayg referenced this issue

2026-11-06 09:36:56 +00:00

Mesh: SSH over the mesh (D197 M1, D235) #221

No labels

launch-qa

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/infraternal#388

No description provided.

Rows
Columns