kayg/infraternal-v0
Archived
Watch
1
0
Fork
You've already forked infraternal-v0
 0
Code Issues 92 Pull requests Projects Releases Packages Wiki Activity Actions

Step 8b: Engine follow-ups #15

Open
opened 2026-09-21 23:11:05 +00:00 by kayg · 22 comments
kayg commented 2026-09-21 23:11:05 +00:00
Owner
Copy link

Deferred from the step 8 verify (#4):

  • Podman container list drops health status and pod identity (inspect keeps them).
  • Dropping an attached PtySession leaves exec helpers running on all engines; needs a cancellation path. Required before step 13 (terminal/exec) merges.
  • Benchmark allocation reporting.

Step 8b in docs/PLAN.md.

Deferred from the step 8 verify (#4): - Podman container list drops health status and pod identity (inspect keeps them). - Dropping an attached PtySession leaves exec helpers running on all engines; needs a cancellation path. Required before step 13 (terminal/exec) merges. - Benchmark allocation reporting. Step 8b in docs/PLAN.md.
kayg commented 2026-09-21 23:12:07 +00:00
Author
Owner
Copy link

Starting step 8b on branch engine-followups from base 7f83404c3e. Scope is crates/engine only: real-engine regressions for list metadata and PTY cleanup, implementation, and allocation-reporting benchmark coverage.

Starting step 8b on branch engine-followups from base 7f83404c3edad552286fbd6db60fb270a8bc1436. Scope is crates/engine only: real-engine regressions for list metadata and PTY cleanup, implementation, and allocation-reporting benchmark coverage.
kayg commented 2026-09-21 23:19:50 +00:00
Author
Owner
Copy link

Base-head regression evidence (base 7f83404c3e): real metadata test list_preserves_health_and_pod_identity_on_all_real_engines failed at integration.rs:386 with list health None vs inspect Some(Healthy). Real PTY test dropping_pty_session_terminates_exec_process_and_helpers_on_all_real_engines failed after drop on Docker first: container shell PID 7 and helper PID 13 remained in ps output; the base Drop only aborted the output producer. Test guards cleaned the test containers.

Base-head regression evidence (base 7f83404c3edad552286fbd6db60fb270a8bc1436): real metadata test list_preserves_health_and_pod_identity_on_all_real_engines failed at integration.rs:386 with list health None vs inspect Some(Healthy). Real PTY test dropping_pty_session_terminates_exec_process_and_helpers_on_all_real_engines failed after drop on Docker first: container shell PID 7 and helper PID 13 remained in ps output; the base Drop only aborted the output producer. Test guards cleaned the test containers.
kayg commented 2026-09-21 23:31:19 +00:00
Author
Owner
Copy link

Found while gating the dev merge head (ca6f3e7): docker_and_podman_share_the_real_engine_contract (crates/engine/tests/integration.rs:353-358) takes the first event after subscribing and asserts it belongs to its own container. With another job creating containers on the same engines concurrently, a foreign event arrives first and the test fails. Filter events by the test container id (or the run label) until the expected one arrives, within the timeout. Add to this step.

Found while gating the dev merge head (ca6f3e7): `docker_and_podman_share_the_real_engine_contract` (crates/engine/tests/integration.rs:353-358) takes the first event after subscribing and asserts it belongs to its own container. With another job creating containers on the same engines concurrently, a foreign event arrives first and the test fails. Filter events by the test container id (or the run label) until the expected one arrives, within the timeout. Add to this step.
kayg commented 2026-09-21 23:41:23 +00:00
Author
Owner
Copy link

Regression tests are now isolated in commit e15b1ce. Base-head results remain recorded above: metadata list failed with None vs Some(Healthy; PTY drop left the exec shell/helper PIDs in the container process table. The tests cover Docker, rootless Podman, and rootful Podman through sudo.

Regression tests are now isolated in commit e15b1ce. Base-head results remain recorded above: metadata list failed with None vs Some(Healthy; PTY drop left the exec shell/helper PIDs in the container process table. The tests cover Docker, rootless Podman, and rootful Podman through sudo.
kayg commented 2026-09-21 23:44:12 +00:00
Author
Owner
Copy link

Integration gate finding: with the gate's default parallel test execution, the existing global event-stream assertion in docker_and_podman_share_the_real_engine_contract consumed another test container's event (actor 3698de2a… vs guarded 78ca2d67…). Six other engine integration tests, including both follow-up regressions, passed. The test must filter events to its guarded container rather than assuming the first daemon event is local.

Integration gate finding: with the gate's default parallel test execution, the existing global event-stream assertion in docker_and_podman_share_the_real_engine_contract consumed another test container's event (actor 3698de2a… vs guarded 78ca2d67…). Six other engine integration tests, including both follow-up regressions, passed. The test must filter events to its guarded container rather than assuming the first daemon event is local.
kayg commented 2026-09-21 23:52:39 +00:00
Author
Owner
Copy link

ISSUE UPDATE kayg/infraternal#15
branch: engine-followups
base: 7f83404c3e
head: a9f9a9e900
fixed: Podman native list/inspect metadata now preserves health and pod identity in normalized containers.
fixed: PtySession drop and explicit close interrupt and await exec termination, including helper cleanup.
fixed: the engine benchmark now reports log-framing allocation counts via a counting allocator.
regression tests (fail before → pass after): list_preserves_health_and_pod_identity_on_all_real_engines — base listed health as None while inspect reported Some(Healthy) → passes on Docker, rootless Podman, and rootful Podman.
regression tests (fail before → pass after): pty_session_drop_and_close_terminate_exec_processes_on_all_real_engines — base left the Docker shell/helper processes alive after drop → passes for drop and close with container/host process-table checks.
gates:
PASS fast: JavaScript dependencies
PASS fast: build web app
PASS fast: cargo fmt --check
PASS fast: cargo clippy
PASS fast: cargo test
PASS fast: svelte-check
PASS fast: UI package svelte-check
PASS fast: Vitest
PASS fast: UI package Vitest
PASS fast: OpenAPI and TypeScript client drift
PASS integration: JavaScript dependencies
PASS integration: build web app
PASS integration: build release binary
PASS integration: real HTTP behavior
PASS integration: Docker socket _ping
PASS integration: rootless Podman socket _ping
PASS integration: engine adapters against Docker and Podman
PASS bench: JavaScript dependencies
PASS bench: build web app
PASS bench: criterion smoke (numbers from this VM are not reportable)
PASS bench: proto codec criterion smoke (numbers from this VM are not reportable)
PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable)
leftover test containers: none

ISSUE UPDATE kayg/infraternal#15 branch: engine-followups base: 7f83404c3edad552286fbd6db60fb270a8bc1436 head: a9f9a9e900836ca2a1477bb22988572797e04a11 fixed: Podman native list/inspect metadata now preserves health and pod identity in normalized containers. fixed: PtySession drop and explicit close interrupt and await exec termination, including helper cleanup. fixed: the engine benchmark now reports log-framing allocation counts via a counting allocator. regression tests (fail before → pass after): list_preserves_health_and_pod_identity_on_all_real_engines — base listed health as None while inspect reported Some(Healthy) → passes on Docker, rootless Podman, and rootful Podman. regression tests (fail before → pass after): pty_session_drop_and_close_terminate_exec_processes_on_all_real_engines — base left the Docker shell/helper processes alive after drop → passes for drop and close with container/host process-table checks. gates: PASS fast: JavaScript dependencies PASS fast: build web app PASS fast: cargo fmt --check PASS fast: cargo clippy PASS fast: cargo test PASS fast: svelte-check PASS fast: UI package svelte-check PASS fast: Vitest PASS fast: UI package Vitest PASS fast: OpenAPI and TypeScript client drift PASS integration: JavaScript dependencies PASS integration: build web app PASS integration: build release binary PASS integration: real HTTP behavior PASS integration: Docker socket _ping PASS integration: rootless Podman socket _ping PASS integration: engine adapters against Docker and Podman PASS bench: JavaScript dependencies PASS bench: build web app PASS bench: criterion smoke (numbers from this VM are not reportable) PASS bench: proto codec criterion smoke (numbers from this VM are not reportable) PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable) leftover test containers: none
kayg commented 2026-09-21 23:54:24 +00:00
Author
Owner
Copy link

Build job engine-followups (gpt-5.6-luna/max) finished:

ISSUE UPDATE kayg/infraternal#15
branch: engine-followups
base: 7f83404c3edad552286fbd6db60fb270a8bc1436
head: a9f9a9e900836ca2a1477bb22988572797e04a11
fixed: Podman native list/inspect metadata now preserves health and pod identity in normalized containers.
fixed: PtySession drop and explicit close interrupt and await exec termination, including helper cleanup.
fixed: the engine benchmark now reports log-framing allocation counts via a counting allocator.
regression tests (fail before → pass after): list_preserves_health_and_pod_identity_on_all_real_engines — base listed health as None while inspect reported Some(Healthy) → passes on Docker, rootless Podman, and rootful Podman.
regression tests (fail before → pass after): pty_session_drop_and_close_terminate_exec_processes_on_all_real_engines — base left the Docker shell/helper processes alive after drop → passes for drop and close with container/host process-table checks.
gates:
PASS fast: JavaScript dependencies
PASS fast: build web app
PASS fast: cargo fmt --check
PASS fast: cargo clippy
PASS fast: cargo test
PASS fast: svelte-check
PASS fast: UI package svelte-check
PASS fast: Vitest
PASS fast: UI package Vitest
PASS fast: OpenAPI and TypeScript client drift
PASS integration: JavaScript dependencies
PASS integration: build web app
PASS integration: build release binary
PASS integration: real HTTP behavior
PASS integration: Docker socket _ping
PASS integration: rootless Podman socket _ping
PASS integration: engine adapters against Docker and Podman
PASS bench: JavaScript dependencies
PASS bench: build web app
PASS bench: criterion smoke (numbers from this VM are not reportable)
PASS bench: proto codec criterion smoke (numbers from this VM are not reportable)
PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable)
leftover test containers: none

It started before the flaky event-test note was filed, so a small job engine-flaky fixes that on the same branch; then one review.

Build job `engine-followups` (gpt-5.6-luna/max) finished: ``` ISSUE UPDATE kayg/infraternal#15 branch: engine-followups base: 7f83404c3edad552286fbd6db60fb270a8bc1436 head: a9f9a9e900836ca2a1477bb22988572797e04a11 fixed: Podman native list/inspect metadata now preserves health and pod identity in normalized containers. fixed: PtySession drop and explicit close interrupt and await exec termination, including helper cleanup. fixed: the engine benchmark now reports log-framing allocation counts via a counting allocator. regression tests (fail before → pass after): list_preserves_health_and_pod_identity_on_all_real_engines — base listed health as None while inspect reported Some(Healthy) → passes on Docker, rootless Podman, and rootful Podman. regression tests (fail before → pass after): pty_session_drop_and_close_terminate_exec_processes_on_all_real_engines — base left the Docker shell/helper processes alive after drop → passes for drop and close with container/host process-table checks. gates: PASS fast: JavaScript dependencies PASS fast: build web app PASS fast: cargo fmt --check PASS fast: cargo clippy PASS fast: cargo test PASS fast: svelte-check PASS fast: UI package svelte-check PASS fast: Vitest PASS fast: UI package Vitest PASS fast: OpenAPI and TypeScript client drift PASS integration: JavaScript dependencies PASS integration: build web app PASS integration: build release binary PASS integration: real HTTP behavior PASS integration: Docker socket _ping PASS integration: rootless Podman socket _ping PASS integration: engine adapters against Docker and Podman PASS bench: JavaScript dependencies PASS bench: build web app PASS bench: criterion smoke (numbers from this VM are not reportable) PASS bench: proto codec criterion smoke (numbers from this VM are not reportable) PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable) leftover test containers: none ``` It started before the flaky event-test note was filed, so a small job `engine-flaky` fixes that on the same branch; then one review.
kayg commented 2026-09-21 23:55:06 +00:00
Author
Owner
Copy link

Started engine-followups at base a9f9a9e900. Reading the contracts and inspecting the engine event integration tests; I will reproduce the race, apply the smallest test-only fix, and run the integration gate.

Started engine-followups at base a9f9a9e900836ca2a1477bb22988572797e04a11. Reading the contracts and inspecting the engine event integration tests; I will reproduce the race, apply the smallest test-only fix, and run the integration gate.
kayg commented 2026-09-22 00:01:29 +00:00
Author
Owner
Copy link

Finding: the engine integration suite has one event consumer, in exercise_engine; the other event-related matches are adapter/protocol code, not test assertions.

Reproduction with background create/start/remove churn and the old first-event assertion: targeted Docker+Podman test failed at integration.rs:688; foreign actor_id 3d98f75770ce3d032a38ad71c808e69a224d558dc7d4bea94860657e96e846fa arrived before test actor_id 7558242b4117d59a364b3296e499235994a05b55dab6dcc7ab4cdf549bb12fe5.

After restoring the ID-filtered receive loop: cargo test: 1 passed, 6 filtered out (1 suite, 90.70s). The churn task uses unique labeled throwaways and RAII cleanup.

Finding: the engine integration suite has one event consumer, in exercise_engine; the other event-related matches are adapter/protocol code, not test assertions. Reproduction with background create/start/remove churn and the old first-event assertion: targeted Docker+Podman test failed at integration.rs:688; foreign actor_id 3d98f75770ce3d032a38ad71c808e69a224d558dc7d4bea94860657e96e846fa arrived before test actor_id 7558242b4117d59a364b3296e499235994a05b55dab6dcc7ab4cdf549bb12fe5. After restoring the ID-filtered receive loop: cargo test: 1 passed, 6 filtered out (1 suite, 90.70s). The churn task uses unique labeled throwaways and RAII cleanup.
kayg commented 2026-09-22 00:07:33 +00:00
Author
Owner
Copy link

ISSUE UPDATE kayg/infraternal#15
branch: engine-followups
head: a6d97acace
fixed: exercise_engine now creates/removes three foreign labeled throwaways in a background task and consumes events until the test container ID arrives.
fail before → pass after: old first-event assertion failed on a foreign actor ID; the ID-filtered loop passed (targeted test: 1 passed, 6 filtered out).
gates:
PASS integration: JavaScript dependencies
PASS integration: build web app
PASS integration: build release binary
PASS integration: real HTTP behavior
PASS integration: Docker socket _ping
PASS integration: rootless Podman socket _ping
PASS integration: engine adapters against Docker and Podman

ISSUE UPDATE kayg/infraternal#15 branch: engine-followups head: a6d97acace46bafe3b1ea63aab84f6c9bd1c68fa fixed: exercise_engine now creates/removes three foreign labeled throwaways in a background task and consumes events until the test container ID arrives. fail before → pass after: old first-event assertion failed on a foreign actor ID; the ID-filtered loop passed (targeted test: 1 passed, 6 filtered out). gates: PASS integration: JavaScript dependencies PASS integration: build web app PASS integration: build release binary PASS integration: real HTTP behavior PASS integration: Docker socket _ping PASS integration: rootless Podman socket _ping PASS integration: engine adapters against Docker and Podman
kayg commented 2026-09-22 00:10:07 +00:00
Author
Owner
Copy link

Review started: detached at a6d97acace; base 7f83404c3e. Scope is independent review only; no tracked-file edits or commits.

Review started: detached at a6d97acace46bafe3b1ea63aab84f6c9bd1c68fa; base 7f83404c3edad552286fbd6db60fb270a8bc1436. Scope is independent review only; no tracked-file edits or commits.
kayg commented 2026-09-22 00:36:13 +00:00
Author
Owner
Copy link

Finding (should fix): the no-runtime PtySession drop path does not terminate the exec. crates/engine/src/lib.rs:482-488 calls producer.abort() only; the moved stdin is then dropped, with no PTY interrupt and no termination wait. A temporary ignored Docker probe dropped the session from a plain thread and, after 500 ms, docker exec <container> ps still showed /bin/sh -c echo drop-fallback-marker; while :; do sleep 1; done running (PID 7). The current-runtime drop path and explicit close passed their probes, so this is specifically the fallback/durability gap.

Finding (should fix): the no-runtime `PtySession` drop path does not terminate the exec. `crates/engine/src/lib.rs:482-488` calls `producer.abort()` only; the moved stdin is then dropped, with no PTY interrupt and no termination wait. A temporary ignored Docker probe dropped the session from a plain thread and, after 500 ms, `docker exec <container> ps` still showed `/bin/sh -c echo drop-fallback-marker; while :; do sleep 1; done` running (PID 7). The current-runtime drop path and explicit close passed their probes, so this is specifically the fallback/durability gap.
kayg commented 2026-09-22 00:39:31 +00:00
Author
Owner
Copy link

Finding (blocking): canceling PtySession::close() can permanently lose cleanup. close() sets closed and takes producer before awaiting (crates/engine/src/lib.rs:503-511); if its task is canceled, Drop sees producer == None and returns at :470-472. A temporary ignored Docker probe used trap 'sleep 10' INT; ... while ... so termination stayed pending, aborted the close task after 100 ms, waited 500 ms, and docker exec <container> ps still showed the attached /bin/sh -c trap 'sleep 10' INT; ... running (PID 7). The session must retain or hand off cancellation ownership so an aborted close still performs the cleanup handshake.

Finding (blocking): canceling `PtySession::close()` can permanently lose cleanup. `close()` sets `closed` and takes `producer` before awaiting (`crates/engine/src/lib.rs:503-511`); if its task is canceled, `Drop` sees `producer == None` and returns at `:470-472`. A temporary ignored Docker probe used `trap 'sleep 10' INT; ... while ...` so termination stayed pending, aborted the close task after 100 ms, waited 500 ms, and `docker exec <container> ps` still showed the attached `/bin/sh -c trap 'sleep 10' INT; ...` running (PID 7). The session must retain or hand off cancellation ownership so an aborted close still performs the cleanup handshake.
kayg commented 2026-09-22 00:40:56 +00:00
Author
Owner
Copy link

Finding (should fix): current-runtime Drop launches unbounded detached cleanup work. Every drop at crates/engine/src/lib.rs:482-485 calls Handle::spawn; each task can poll inspect_exec for up to five seconds, and there is no bounded queue, semaphore, supervisor, or join ownership. close() is async and does not block the runtime, but a rapid drop of many attached sessions creates one concurrent cleanup task per session and can amplify engine requests. The real-engine drop probe passed for one session; this is the missing bound/durability guard.

Finding (should fix): current-runtime `Drop` launches unbounded detached cleanup work. Every drop at `crates/engine/src/lib.rs:482-485` calls `Handle::spawn`; each task can poll `inspect_exec` for up to five seconds, and there is no bounded queue, semaphore, supervisor, or join ownership. `close()` is async and does not block the runtime, but a rapid drop of many attached sessions creates one concurrent cleanup task per session and can amplify engine requests. The real-engine drop probe passed for one session; this is the missing bound/durability guard.
kayg commented 2026-09-22 00:42:02 +00:00
Author
Owner
Copy link

Review finished: engine-followups @ a6d97acace
verdict: merge after fixes

Findings:

  1. [blocking] PtySession::close() sets closed and takes producer before its first await; canceling close then makes Drop return with no cleanup. A Docker probe with trap 'sleep 10' INT aborted close after 100 ms and still showed the attached shell running in docker exec ps.
  2. [blocking] The no-runtime Drop fallback only aborts the output producer and drops stdin. A Docker probe dropping from a plain thread still showed /bin/sh -c ... while ... running after 500 ms.
  3. [should fix] Current-runtime Drop spawns one detached cleanup task per session with no bound, supervisor, or join ownership; each can poll for five seconds and amplify engine requests during rapid drops.

Verified claims:

  • Podman and Docker list metadata: passed real Docker, rootless Podman, and rootful Podman health/pod identity coverage.
  • Normal PTY drop/close, close twice, flowing output, separate-task drop, delayed exit, and attached-container removal: passed the real-engine probes; the two blocking lifecycle cases above remain.
  • Allocation reporting: direct engine benchmark printed engine=docker log framing allocations: records=10000 bytes=468890 allocations=31822 deallocations=31822 reallocations=12 bytes_allocated=1391797 bytes_deallocated=1391797 bytes_reallocated=562 and engine=podman log framing allocations: records=10000 bytes=468890 allocations=27642 deallocations=27644 reallocations=16 bytes_allocated=1150197 bytes_deallocated=1150485 bytes_reallocated=914.
  • Event integration: foreign churn ran concurrently and the assertion filtered by the target container id.
  • Leftover audit: no labeled Docker, rootless Podman, or rootful Podman containers; no review/benchmark processes remained.

Gates (verbatim):
PASS fast: JavaScript dependencies
PASS fast: build web app
PASS fast: cargo fmt --check
PASS fast: cargo clippy
PASS fast: cargo test
PASS fast: svelte-check
PASS fast: UI package svelte-check
PASS fast: Vitest
PASS fast: UI package Vitest
PASS fast: OpenAPI and TypeScript client drift
PASS integration: JavaScript dependencies
PASS integration: build web app
PASS integration: build release binary
PASS integration: real HTTP behavior
PASS integration: Docker socket _ping
PASS integration: rootless Podman socket _ping
PASS integration: engine adapters against Docker and Podman
PASS bench: JavaScript dependencies
PASS bench: build web app
PASS bench: criterion smoke (numbers from this VM are not reportable)
PASS bench: proto codec criterion smoke (numbers from this VM are not reportable)
PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable)

Review finished: engine-followups @ a6d97acace46bafe3b1ea63aab84f6c9bd1c68fa verdict: merge after fixes Findings: 1. [blocking] `PtySession::close()` sets `closed` and takes `producer` before its first await; canceling close then makes `Drop` return with no cleanup. A Docker probe with `trap 'sleep 10' INT` aborted close after 100 ms and still showed the attached shell running in `docker exec ps`. 2. [blocking] The no-runtime `Drop` fallback only aborts the output producer and drops stdin. A Docker probe dropping from a plain thread still showed `/bin/sh -c ... while ...` running after 500 ms. 3. [should fix] Current-runtime `Drop` spawns one detached cleanup task per session with no bound, supervisor, or join ownership; each can poll for five seconds and amplify engine requests during rapid drops. Verified claims: - Podman and Docker list metadata: passed real Docker, rootless Podman, and rootful Podman health/pod identity coverage. - Normal PTY drop/close, close twice, flowing output, separate-task drop, delayed exit, and attached-container removal: passed the real-engine probes; the two blocking lifecycle cases above remain. - Allocation reporting: direct engine benchmark printed `engine=docker log framing allocations: records=10000 bytes=468890 allocations=31822 deallocations=31822 reallocations=12 bytes_allocated=1391797 bytes_deallocated=1391797 bytes_reallocated=562` and `engine=podman log framing allocations: records=10000 bytes=468890 allocations=27642 deallocations=27644 reallocations=16 bytes_allocated=1150197 bytes_deallocated=1150485 bytes_reallocated=914`. - Event integration: foreign churn ran concurrently and the assertion filtered by the target container id. - Leftover audit: no labeled Docker, rootless Podman, or rootful Podman containers; no review/benchmark processes remained. Gates (verbatim): PASS fast: JavaScript dependencies PASS fast: build web app PASS fast: cargo fmt --check PASS fast: cargo clippy PASS fast: cargo test PASS fast: svelte-check PASS fast: UI package svelte-check PASS fast: Vitest PASS fast: UI package Vitest PASS fast: OpenAPI and TypeScript client drift PASS integration: JavaScript dependencies PASS integration: build web app PASS integration: build release binary PASS integration: real HTTP behavior PASS integration: Docker socket _ping PASS integration: rootless Podman socket _ping PASS integration: engine adapters against Docker and Podman PASS bench: JavaScript dependencies PASS bench: build web app PASS bench: criterion smoke (numbers from this VM are not reportable) PASS bench: proto codec criterion smoke (numbers from this VM are not reportable) PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable)
kayg commented 2026-09-22 00:43:42 +00:00
Author
Owner
Copy link

Review ef-review: merge after fixes (2 blocking: canceled close and no-runtime drop can leave exec running; 1 should-fix: unbounded detached cleanup tasks). One fix round ef-fix launched; no further review, the orchestrator checks fail-first evidence and merges.

Review `ef-review`: merge after fixes (2 blocking: canceled close and no-runtime drop can leave exec running; 1 should-fix: unbounded detached cleanup tasks). One fix round `ef-fix` launched; no further review, the orchestrator checks fail-first evidence and merges.
kayg commented 2026-09-22 00:44:35 +00:00
Author
Owner
Copy link

Starting the single fix round on branch engine-followups at base a6d97ac. Scope is exec cleanup lifecycle: reproduce canceled close(), drop outside Tokio, and detached cleanup growth; then add failing regressions, fix, verify fast/integration/bench across Docker, rootless Podman, and test sshd. No push or issue close.

Starting the single fix round on branch engine-followups at base a6d97ac. Scope is exec cleanup lifecycle: reproduce canceled close(), drop outside Tokio, and detached cleanup growth; then add failing regressions, fix, verify fast/integration/bench across Docker, rootless Podman, and test sshd. No push or issue close.
kayg commented 2026-09-22 00:58:33 +00:00
Author
Owner
Copy link

Failing regression on base a6d97ac: the modified real-engine PTY test moved drop(session) to a thread without Tokio; Docker run 1309339-1790038684738 left container exec shell PID 7 and helper PID 13 after the 5s cleanup wait. The test failed at crates/engine/tests/integration.rs:655 before reaching the cancellation case. This reproduces finding 2.

Failing regression on base a6d97ac: the modified real-engine PTY test moved drop(session) to a thread without Tokio; Docker run 1309339-1790038684738 left container exec shell PID 7 and helper PID 13 after the 5s cleanup wait. The test failed at crates/engine/tests/integration.rs:655 before reaching the cancellation case. This reproduces finding 2.
kayg commented 2026-09-22 01:39:16 +00:00
Author
Owner
Copy link

Failing regression on base a6d97ac for finding 1: a temporary unit probe polled close() once while its PTY write was pending, then canceled the future. cargo test -p infraternal-engine --lib cancelled_close_does_not_leave_producer_detached failed after 1.00s at crates/engine/src/lib.rs:817 with cancelled close left its producer running; the pre-fix close had moved the producer into the canceled future and Drop had no cleanup left to run.

Failing regression on base a6d97ac for finding 1: a temporary unit probe polled close() once while its PTY write was pending, then canceled the future. cargo test -p infraternal-engine --lib cancelled_close_does_not_leave_producer_detached failed after 1.00s at crates/engine/src/lib.rs:817 with `cancelled close left its producer running`; the pre-fix close had moved the producer into the canceled future and Drop had no cleanup left to run.
kayg commented 2026-09-22 01:39:16 +00:00
Author
Owner
Copy link

Fix regression evidence: the final internal cancellation probe passed (test tests::cancelled_close_aborts_producer_before_cleanup_can_be_cancelled ... ok). The real ignored PTY matrix passed all three engines after the worker fix: test pty_session_drop_and_close_terminate_exec_processes_on_all_real_engines ... ok; result 1 passed, 0 failed, finished in 96.65s. This matrix covered canceled close, no-Tokio-thread drop, and ordinary close.

Fix regression evidence: the final internal cancellation probe passed (`test tests::cancelled_close_aborts_producer_before_cleanup_can_be_cancelled ... ok`). The real ignored PTY matrix passed all three engines after the worker fix: `test pty_session_drop_and_close_terminate_exec_processes_on_all_real_engines ... ok`; result 1 passed, 0 failed, finished in 96.65s. This matrix covered canceled close, no-Tokio-thread drop, and ordinary close.
kayg commented 2026-09-22 02:13:52 +00:00
Author
Owner
Copy link

ISSUE UPDATE kayg/infraternal#15
branch: engine-followups
base: a6d97ac
head: da212f7
fixed: canceled close loses cleanup -> producer aborts synchronously, then owned PTY cleanup is queued before close awaits its acknowledgement; cancellation leaves the worker request alive.
fixed: drop outside Tokio -> every engine owns one dedicated Tokio cleanup worker; PtySession Drop uses bounded nonblocking try_send.
fixed: detached cleanup tasks -> one bounded worker queue with one reserved slot per live exec; Drop never spawns a per-session task, and the full/disconnected fallback only aborts the local producer and drops stdin.
regression tests (fail before -> pass after):
canceled close: temporary probe failed on a6d97ac with "cancelled close left its producer running" -> explicit cleanup_tests::cancelled_close_aborts_producer_before_cleanup_can_be_cancelled passed; real three-engine PTY matrix passed.
drop outside Tokio: a6d97ac Docker repro left exec shell PID 7 and helper PID 13 after the 5s wait -> real Docker, rootless Podman, and rootful Podman PTY matrix passed.
detached cleanup growth: a6d97ac Drop path spawned one detached task per session -> final code has one bounded worker and bounded queue; fast, integration, and bench gates passed.
gates:
PASS fast: JavaScript dependencies
PASS fast: build web app
PASS fast: cargo fmt --check
PASS fast: cargo clippy
PASS fast: cargo test
PASS fast: svelte-check
PASS fast: UI package svelte-check
PASS fast: Vitest
PASS fast: UI package Vitest
PASS fast: OpenAPI and TypeScript client drift
PASS integration: JavaScript dependencies
PASS integration: build web app
PASS integration: build release binary
PASS integration: real HTTP behavior
PASS integration: Docker socket _ping
PASS integration: rootless Podman socket _ping
PASS integration: engine adapters against Docker and Podman
PASS bench: JavaScript dependencies
PASS bench: build web app
PASS bench: criterion smoke (numbers from this VM are not reportable)
PASS bench: proto codec criterion smoke (numbers from this VM are not reportable)
PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable)
leftover containers/processes: none from this job

ISSUE UPDATE kayg/infraternal#15 branch: engine-followups base: a6d97ac head: da212f7 fixed: canceled close loses cleanup -> producer aborts synchronously, then owned PTY cleanup is queued before close awaits its acknowledgement; cancellation leaves the worker request alive. fixed: drop outside Tokio -> every engine owns one dedicated Tokio cleanup worker; PtySession Drop uses bounded nonblocking try_send. fixed: detached cleanup tasks -> one bounded worker queue with one reserved slot per live exec; Drop never spawns a per-session task, and the full/disconnected fallback only aborts the local producer and drops stdin. regression tests (fail before -> pass after): canceled close: temporary probe failed on a6d97ac with "cancelled close left its producer running" -> explicit cleanup_tests::cancelled_close_aborts_producer_before_cleanup_can_be_cancelled passed; real three-engine PTY matrix passed. drop outside Tokio: a6d97ac Docker repro left exec shell PID 7 and helper PID 13 after the 5s wait -> real Docker, rootless Podman, and rootful Podman PTY matrix passed. detached cleanup growth: a6d97ac Drop path spawned one detached task per session -> final code has one bounded worker and bounded queue; fast, integration, and bench gates passed. gates: PASS fast: JavaScript dependencies PASS fast: build web app PASS fast: cargo fmt --check PASS fast: cargo clippy PASS fast: cargo test PASS fast: svelte-check PASS fast: UI package svelte-check PASS fast: Vitest PASS fast: UI package Vitest PASS fast: OpenAPI and TypeScript client drift PASS integration: JavaScript dependencies PASS integration: build web app PASS integration: build release binary PASS integration: real HTTP behavior PASS integration: Docker socket _ping PASS integration: rootless Podman socket _ping PASS integration: engine adapters against Docker and Podman PASS bench: JavaScript dependencies PASS bench: build web app PASS bench: criterion smoke (numbers from this VM are not reportable) PASS bench: proto codec criterion smoke (numbers from this VM are not reportable) PASS bench: engine adapter criterion smoke (numbers from this VM are not reportable) leftover containers/processes: none from this job
kayg commented 2026-09-22 02:21:37 +00:00
Author
Owner
Copy link

Fix ef-fix (da212f7): bounded engine-owned cleanup worker; fail-first tests reported. The report lacked verbatim gates, so the orchestrator merged to dev on the VM (a655978) and ran scripts/verify fast and integration on the merge head: all PASS, including engine adapters against Docker and Podman; 0 leftover test containers on Docker and Podman. Step 8b done.

Fix `ef-fix` (da212f7): bounded engine-owned cleanup worker; fail-first tests reported. The report lacked verbatim gates, so the orchestrator merged to dev on the VM (a655978) and ran `scripts/verify fast` and `integration` on the merge head: all PASS, including engine adapters against Docker and Podman; 0 leftover test containers on Docker and Podman. Step 8b done.
Commenting is not possible because the repository is archived.
No Branch/Tag specified
Branches Tags
main
dev
perf-gates
provider-docker-podman
fix-networkidle
fix-apierror-panics
fix-revoke-panic
wal-race
fix-private-storage
provider-runtime
plugin-design
server-channels
scope-v01
deploy-git-fix
sec-hardening
gates-tiering
sidebar-sync
multi-tab
cli-eof
podman-flake
ssh-reattach
agent-tunnel
process-boundary
shadcn
stream-params
cli-ssh
ssh-product
container-live
flaky-tests
s1-e2e
ssh-test-isolation
cli-stream
research-lxc
container-live-agent
ui-apis
grant-shrink
deploy-o2
ui-redesign
migration-fix
cli
stream
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/infraternal-v0#15
No description provided.