Uninstall through the product leaves the login-binary cache and bootstrap cache on the host #360

New issue

Open

opened 2026-11-05 02:01:58 +00:00 by kayg · 1 comment

kayg commented

2026-11-05 02:01:58 +00:00

Owner

Found during the J175 mesh SSH two-VM proof (#221, comment 79163, finding 4). This isn't mesh SSH work, so it gets its own issue here.

What happened. compute01 (infraternal-dev-01-compute01) was added through add-host as meshtest (rootful), then removed through the product (/remote/hosts/remove/{start,finish} with uninstall: true, fresh owner passkey). The server answered {"host_cleanup":"requested","state":"removed","message":"10.69.69.218 was removed. Its infraternal agent is uninstalling itself."}. Checked on the host about 20 s later:

Left behind	Evidence
`/var/cache/infraternal-login-bin/infraternal-801-2c9380-187b1f0-6aebcfbc-2d56fa4f-6aebcfbf-1dd7a304`	25,670,128 B, written 2026-11-05 01:37:23 +0100 at install. Its sha256 starts `977d8ab03b650660`, the same as the installed build (`mesh-ssh-s1-j175` at `eb746547c`)
`/var/lib/infraternal.bootstrap-cache/payload`	0700 dir, `payload` 0700 with the same hash `977d8ab03b650660`, written 01:37:16 +0100
`/var/lib/infraternal.bootstrap-cache/owner`	29 B, written 2026-11-04 20:14 +0100, so from an earlier install on this host. Contents not printed
`infraternal-login-166905b256a843ada3469cda6605d117-0.service`	Listed `loaded active running` right after the uninstall (the `__shim --login-worker` for the proof owner, uid 0). It was gone (`MainPID=0`) about a minute later, so this may only be ordering, not a leak

Removed correctly: the infraternal agent unit, /etc/ssh/ssh_config.d/50-infraternal-mesh.conf, /etc/ssh/infraternal-mesh-known_hosts, /run/infraternal-mesh-ssh, and the infraternal0 interface.

/run/infraternal, /run/infraternal-login-locks and /run/infraternal-login-records also still exist on compute01. They hold entries from many jobs (person locks dated back to 2026-11-03, mesh-interface.json from 2026-11-02), so I can't tell from this run alone whether uninstall should have removed them. A clean add-host → uninstall on a fresh VM would settle it.

Cleanup done by J175. Only the files provably from this run were removed by hand: the login-bin copy (hash matched), one person lock, and an old-path socket from an earlier J175 build. The bootstrap cache was left, because its owner file predates the run.

Expected. Either uninstall removes the login-binary cache and the bootstrap cache (or keeps them for a documented reason, such as a fast re-add), or contracts/bootstrap.md says what survives an uninstall and why.

Code: crates/plugins/remote/src/uninstall.rs (what it removes), the bootstrap cache writer, and the login-binary cache in crates/infraternal-agent/src/login_binary.rs.

Found during the J175 mesh SSH two-VM proof (#221, comment 79163, finding 4). This isn't mesh SSH work, so it gets its own issue here. **What happened.** compute01 (`infraternal-dev-01-compute01`) was added through add-host as `meshtest` (rootful), then removed through the product (`/remote/hosts/remove/{start,finish}` with `uninstall: true`, fresh owner passkey). The server answered `{"host_cleanup":"requested","state":"removed","message":"10.69.69.218 was removed. Its infraternal agent is uninstalling itself."}`. Checked on the host about 20 s later: | Left behind | Evidence | |---|---| | `/var/cache/infraternal-login-bin/infraternal-801-2c9380-187b1f0-6aebcfbc-2d56fa4f-6aebcfbf-1dd7a304` | 25,670,128 B, written 2026-11-05 01:37:23 +0100 at install. Its sha256 starts `977d8ab03b650660`, the same as the installed build (`mesh-ssh-s1-j175` at `eb746547c`) | | `/var/lib/infraternal.bootstrap-cache/payload` | 0700 dir, `payload` 0700 with the same hash `977d8ab03b650660`, written 01:37:16 +0100 | | `/var/lib/infraternal.bootstrap-cache/owner` | 29 B, written 2026-11-04 20:14 +0100, so from an earlier install on this host. Contents not printed | | `infraternal-login-166905b256a843ada3469cda6605d117-0.service` | Listed `loaded active running` right after the uninstall (the `__shim --login-worker` for the proof owner, uid 0). It was gone (`MainPID=0`) about a minute later, so this may only be ordering, not a leak | Removed correctly: the infraternal agent unit, `/etc/ssh/ssh_config.d/50-infraternal-mesh.conf`, `/etc/ssh/infraternal-mesh-known_hosts`, `/run/infraternal-mesh-ssh`, and the `infraternal0` interface. `/run/infraternal`, `/run/infraternal-login-locks` and `/run/infraternal-login-records` also still exist on compute01. They hold entries from many jobs (person locks dated back to 2026-11-03, `mesh-interface.json` from 2026-11-02), so I can't tell from this run alone whether uninstall should have removed them. A clean add-host → uninstall on a fresh VM would settle it. **Cleanup done by J175.** Only the files provably from this run were removed by hand: the login-bin copy (hash matched), one person lock, and an old-path socket from an earlier J175 build. The bootstrap cache was left, because its `owner` file predates the run. **Expected.** Either uninstall removes the login-binary cache and the bootstrap cache (or keeps them for a documented reason, such as a fast re-add), or `contracts/bootstrap.md` says what survives an uninstall and why. Code: `crates/plugins/remote/src/uninstall.rs` (what it removes), the bootstrap cache writer, and the login-binary cache in `crates/infraternal-agent/src/login_binary.rs`.

kayg commented

2026-11-05 04:39:01 +00:00

Author

Owner

The same thing happened again on a fresh add-host → uninstall with the J175 build at d83a3242e, 2026-11-05 about 04:31–04:35 UTC on compute01. After the product uninstall:

/var/cache/infraternal-login-bin/infraternal-801-2a001e-… was left. Its sha256 starts 414920349f0b7601, the same as the installed build.
/var/lib/infraternal.bootstrap-cache/payload was left, rewritten at this install (05:30:32 +0100). owner is unchanged from 2026-11-04 20:14.
New: login records survive uninstall too. These were written by add-host's "Checking a terminal through the infraternal agent" step, one set per proof owner, and match both add-host runs to the minute (03:35 and 04:30 UTC):
- /run/infraternal-login-locks/<person>.person.lock
- /run/infraternal-login-locks/infraternal-login-<person>-0.lock
- /run/infraternal-login-locks/<person>.revoked (written at uninstall)
- /run/infraternal-login-records/infraternal-login-<person>-0.record, plus its hard link ZPhC.reserved

/run clears on reboot, but a host that is removed and added again keeps collecting these until then.

All these files were from J175's own runs, and J175 removed them by hand. The bootstrap cache stays, as before. Mesh SSH's own state (redirect tables, client files, key socket, infraternal0) was fully removed by this uninstall.

The same thing happened again on a fresh add-host → uninstall with the J175 build at `d83a3242e`, 2026-11-05 about 04:31–04:35 UTC on compute01. After the product uninstall: - `/var/cache/infraternal-login-bin/infraternal-801-2a001e-…` was left. Its sha256 starts `414920349f0b7601`, the same as the installed build. - `/var/lib/infraternal.bootstrap-cache/payload` was left, rewritten at this install (05:30:32 +0100). `owner` is unchanged from 2026-11-04 20:14. - **New: login records survive uninstall too.** These were written by add-host's "Checking a terminal through the infraternal agent" step, one set per proof owner, and match both add-host runs to the minute (03:35 and 04:30 UTC): - `/run/infraternal-login-locks/<person>.person.lock` - `/run/infraternal-login-locks/infraternal-login-<person>-0.lock` - `/run/infraternal-login-locks/<person>.revoked` (written at uninstall) - `/run/infraternal-login-records/infraternal-login-<person>-0.record`, plus its hard link `ZPhC.reserved` `/run` clears on reboot, but a host that is removed and added again keeps collecting these until then. All these files were from J175's own runs, and J175 removed them by hand. The bootstrap cache stays, as before. Mesh SSH's own state (redirect tables, client files, key socket, `infraternal0`) was fully removed by this uninstall.

kayg referenced this issue

2026-11-05 04:47:11 +00:00

Mesh: SSH over the mesh (D197 M1, D235) #221

kayg referenced this issue

2026-11-05 09:36:29 +00:00

Mesh: SSH over the mesh (D197 M1, D235) #221

kayg referenced this issue

2026-11-05 11:07:57 +00:00

Mesh: SSH over the mesh (D197 M1, D235) #221