SSH sessions: a user's shell rc that starts a multiplexer takes over the PTY #32
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found by #27. The SSH test runner's
.bashrcstarts Zellij, which took over the test's PTY (owner_loss_terminates_shelltimeouts). The tests now use a clean shell. Product sessions request a PTY and the user's login shell, so a real user whose.bashrc/.zshrcauto-starts zellij, tmux or screen gets the multiplexer inside infraternal's terminal.That may be what the user wants, or it may not. Decide for step 14b (SSH in the product): a per-host option ("run login shell" or "run a plain shell" / a custom command), how persistent sessions and replay behave with a multiplexer inside, and a test with a real rc file that starts a multiplexer. Cite the OpenSSH and shell startup semantics. Not a test defect.
Starting step 14b on branch ssh-product at base
ee62153839. Scope: SSH host settings/trust, stream sessions, session close, per-host shell option, real-sshd tests and benchmark. No UI and no push.Code orientation: crates/ssh already has strict host-key decisions and a bounded persistent session manager, but EmbeddedAgent::open_frame_at still returns HostOffline for ssh and DELETE /sessions dispatches only persistent exec termination. The host inventory currently has only the local host and no SSH settings. I am adding the per-host shell contract first, then wiring these missing product paths. OpenSSH runs exec requests through the account shell (-c); GNU Bash may read startup files for remote commands, so plain mode can suppress startup files in the requested Bash process but cannot bypass a remote account's own shell wrapper. Sources: https://man.openbsd.org/sshd.8 and https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html
Checkpoint head
373dab8. Real isolated sshd suite: 15 passed, 0 failed; the new rc/tmux shell-option test: 1 passed, 0 failed.cargo check -p infraternal-serverpassed after building the production web bundle. SSH OPEN now reaches the existingcrates/sshsession manager; the host API stores only address, port, user, adefaultcredential reference and shell mode. An append-only migration stores SSH settings and changed-key alerts. I found that the core replay path treated an absent resume offset as the output tail and rejected offsets ahead of output; both contradicted §3.11 and are corrected. Remaining work is full product-path tests, detached exit and revocation verification, generated API artifacts, and required gates.SSH product checkpoint on
ssh-productat78d3d58(baseee62153). The real server plus isolatedsshdtest covers host settings, first-key trust and changed-key refusal, AI trust plan, SSH OPEN/type/resize/replay/detach/close, detached exit, revocation, and a 40-keystroke round-trip smoke. Retry cleanup is fenced by the creating request ID; a realsshdtest confirms a failed retry does not terminate the prior shell. The integration gate now runs the product-path test.scripts/verify fastoutput (verbatim):PASS fast: migrations are append-only
PASS fast: JavaScript dependencies
PASS fast: build web app
PASS fast: cargo fmt --check
PASS fast: cargo clippy
PASS fast: cargo test
PASS fast: svelte-check
PASS fast: UI package svelte-check
PASS fast: stream package typecheck
PASS fast: Vitest
PASS fast: UI package Vitest
PASS fast: stream package Vitest
PASS fast: OpenAPI and TypeScript client drift
PASS fast: terminal TypeScript
PASS fast: terminal Vitest
Integration is pending while another VM job owns the release build.
SSH integration finding at
1cda2b4: the first integration run passed release, real HTTP, Docker/Podman, stream authorization, resource API and CLI/MCP, then timed out in the real-sshd large-output replay test. The timeout was a test-helper regression after the documented future-offset clamp: the helper opened an attachment early and took the writer lease before the queued command ran. It now waits on the live session output offset without attaching. The targeted replay test and full isolated SSH suite pass (18/18);scripts/verify fastalso passes in full after this fix. The full integration rerun is pending while other VM jobs run release/browser suites.Step 14b SSH product finished on branch
ssh-productat head990996540a5491e7bcfa5d2fd67875b63b596737(baseee62153839982d068586e3890bfe8749db8f1dab). No push or issue close.API/contract excerpt (
docs/contracts.md§1.9, §3.11):POST /api/v1/hosts(hosts:create) andPUT /api/v1/hosts/{host_id}/ssh(hosts:update) store address, port, user,credential_ref, and shell mode; the only current credential reference isdefault, whose public key is available atGET /api/v1/ssh/credentials/default. Private key bytes remain on the infraternal agent.GET /api/v1/hosts/{host_id}/ssh/host-keyprobes without trusting.POSTto that path requireshosts:trust_key, a matching observed fingerprint, and a plan forai_agenttokens. Changed keys refuse SSH and create an inbox item. StreamOPENkindsshrequiresssh:connect;SshParamsprovides PTY dimensions, persistence, open key, session ID and resume offset.GET/DELETE /api/v1/sessionslist and terminate real SSH sessions. Per-hostshell.modeislogin(default),plain(exec bash --noprofile --norc -i), orcustomcommand; an rc-started multiplexer runs inside the persistent PTY, whose bytes are replayed.Verification: full isolated SSH contract suite passed 18/18. The real server/isolated sshd product test covers first trust, changed-key refusal, shell configuration, stream type/resize/replay/detach, API close, detached exit, revocation, AI trust plan, and 40 keystroke round trips. OpenAPI and TypeScript client were regenerated and passed drift checking. The branch is clean.
scripts/verify fastoutput (verbatim):PASS fast: migrations are append-only
PASS fast: JavaScript dependencies
PASS fast: build web app
PASS fast: cargo fmt --check
PASS fast: cargo clippy
PASS fast: cargo test
PASS fast: svelte-check
PASS fast: UI package svelte-check
PASS fast: stream package typecheck
PASS fast: Vitest
PASS fast: UI package Vitest
PASS fast: stream package Vitest
PASS fast: OpenAPI and TypeScript client drift
PASS fast: terminal TypeScript
PASS fast: terminal Vitest
scripts/verify integrationoutput (verbatim):PASS integration: JavaScript dependencies
PASS integration: build web app
PASS integration: build release binary
PASS integration: real HTTP behavior
PASS integration: Docker socket _ping
PASS integration: rootless Podman socket _ping
PASS integration: engine adapters against Docker and Podman
PASS integration: TypeScript stream client against Docker and Podman
PASS integration: stream authorization and CLOSE regressions against Docker
PASS integration: persistent PTY termination preempts blocked send
PASS integration: resource HTTP API against Docker and rootless Podman
PASS integration: compiled CLI and MCP against real server and engines
PASS integration: SSH contract against isolated real sshd
PASS integration: SSH product API and stream against isolated real sshd
Independent SSH product review started. Branch
ssh-product, baseee62153, head9909965. Auditing the changed paths againstdocs/contracts.mdand running the prescribed gates.Review findings with code evidence and proposed real-sshd test cases are recorded in
/mnt/hdd/infraternal/artifacts/review-ssh-product/findings.md: F1 nonpersistent SSH OPEN bypasses capacity, F2 SSH mutations can survive completion-audit failure, F3 product tests omit untrusted/changed-key OPEN refusal, F4 host-key GET probes have no connection admission limit, F5 an OPEN mismatch can lose its inbox alert if the follow-up probe fails.scripts/verify fastpassed; integration is running.Independent review finished. Branch
ssh-product, baseee62153, head990996540a5491e7bcfa5d2fd67875b63b596737. No source edits or commits.Independent review: ssh-product
9909965againstee62153F1 — blocks-merge — Nonpersistent SSH OPEN bypasses every SSH session admission limit
Evidence:
crates/ssh/src/session.rs:622-645only callscheck_capacityand increments pending counts whenpersistentis true.validate_open_paramsat :473-476 acceptspersist:false;create_sessionat :245-259 still opens a TCP connection and PTY, thenstart_sessionretains it. A caller can open arbitrarily many ephemeral SSH sessions across WebSocket connections, including concurrent 10–20 s handshakes to a slow host. The 64 global/16 per-user persistent limits do not cover them, and no host-level admission/backoff is present.Test case: with the isolated real sshd, issue >64
sshOPENs withpersist:falsefor one user while keeping each channel open; assert that the first 16 or a specified separate ephemeral cap succeeds and later OPENs returnresource_exhausted, and count sshd connections. Repeat against a deliberately slow sshd handshake to check pending admission. Current code admits all (subject only to external fd/sshd limits).F2 — blocks-merge — Successful SSH mutations can survive failure of their completion audit
Evidence:
crates/server/src/resources.rs:876-900inserts the host before writing the accepted audit; :999-1021 replaces SSH settings before that audit; :1345-1380 trusts/replaces a host key and resolves its alert before the accepted audit. Each returnsinternalif the final audit fails but leaves the mutation in effect.docs/contracts.md§4 says an action is not performed if its audit write fails. For key trust, this leaves an active trusted key without the required accepted trust record.Test case: using the real sshd fixture and a SQLite fault-injection trigger that rejects only the
acceptedssh.host_key.trustaudit insert, POST trust for a previously untrusted fingerprint; assert the response fails and a subsequent SSH OPEN still reportshost_key_needs_trustand the store has no trusted fingerprint. Analogous failure-injection cases should cover host create and settings update. Current code mutates before the rejected audit.F3 — blocks-merge — Product-path tests never assert that SSH OPEN refuses unknown or changed host keys
Evidence:
crates/server/tests/ssh_product.rs:223-245probes then trusts before its first stream OPEN. At :534-545 it changes the real sshd key but tests only the GET probe and alert, then trusts the replacement. The directcrates/ssh/tests/real_sshd.rs:956-979does assertSshClient::connectrefuses unknown and changed keys, but cannot detect a server/embedded infraternal agent regression that auto-trusts or ignores an OPEN error. The explicit job requirement asks whether the product path would catch auto-trust; it would not.Test case: in the existing isolated real sshd product fixture, send a stream SSH OPEN before the trust POST and assert
OPEN_ERR ssh_errorwithdetails.ssh.stage=host_key, noOPEN_OK, nostream_sessionsrow, and no remote shell. After changing the sshd host key, send another new OPEN before replacement and assertOPEN_ERR host_key_mismatchplus one unresolved inbox item. Repeat the refused OPEN with the sameopen_keyand attemptsession_idreattach after rotation.F4 — blocks-merge — Read-only host-key probes have no connection admission limit
Evidence:
crates/server/src/resources.rs:1104-1118allows any caller withhosts:readto run a GET probe;crates/agent/src/lib.rs:621-645callsSshClient::connectfor each request;crates/ssh/src/client.rs:85-124can wait up to 10 seconds for TCP, another 10 for handshake and another 10 for authentication. No semaphore, per-host concurrency limit or retry backoff appears on this path. A slow or hostile sshd can hold arbitrarily many infraternal-agent sockets and tasks from a low-privilege read token.Test case: the per-run isolated real sshd harness should accept connections and delay its SSH banner; issue many simultaneous host-key GETs with only
hosts:read. Assert an explicit bound on concurrent accepted sockets, promptresource_exhausted/rate-limit replies for excess requests, and recovery after the stalled sockets time out. The current path spawns one connection per request.F5 — blocks-merge — A stream host-key mismatch can lose its inbox alert
Evidence:
crates/server/src/stream.rs:1607-1650noticesOPEN_ERR host_key_mismatchonly after the failed OPEN, then spawns an independent task to fetch settings and re-probe. That task silently returns when settings lookup or the second probe fails, when the stored expected fingerprint is absent, or when the second probe sees a different state. The first mismatch is therefore not durably recorded.docs/contracts.md§1.9 requires an actionable inbox item on mismatch.Test case: using the isolated real sshd, trust key A, restart the listener on the same port with key B, send a stream OPEN and assert
host_key_mismatch; make the listener unavailable before the follow-up probe (a deterministic barrier/failure injection around that probe can control timing), then assert an unresolved inbox item with A and B. Current code returns from the spawned task without storing one.F6 — blocks-merge — Detach drops accepted SSH input before the remote shell receives it (integration gate fails)
Evidence:
crates/ssh/src/session.rs:708-738returns success after placing input in the boundedcommandsqueue. An immediately following stream CLOSE makesrun_sshdrop the attachment (crates/agent/src/lib.rs:1117-1120), andInner::detachatcrates/ssh/src/session.rs:756-765clearswriter_lease.drive_ptyat :969-973 discards any queued input whose lease is no longer current. Thus a DATA then CLOSE sequence can silently lose the DATA, leaving the remote shell running while its database row stays Detached. The required integration run reproduced this incrates/server/tests/ssh_product.rs:432-463: afterexit 7\nfollowed by CLOSE, the five-second exit-status assertion failed withstate: Detached, exit_code: None.Test case: the existing isolated real sshd product test is already a regression test: send
exit 7\nas DATA, immediately send CLOSE on that channel, then require the session to become Exited with code 7. Add a remote side-effect command before CLOSE to assert accepted bytes were delivered even when no output is produced.Gate output (verbatim)
scripts/verify fast(exit 0):scripts/verify integration(exit 1; started with load5 5.92):