kayg/calternal

Watch

Fork 0

[tests] Investigate calternal-cli loopback request failures #389

New issue

Closed

opened 2026-09-29 03:40:02 +00:00 by kayg · 4 comments

kayg commented

2026-09-29 03:40:02 +00:00

Owner

Evidence

The one full workspace cargo test run on 2026-09-29 failed in crates/calternal-cli/tests/output_contract.rs after compilation completed.

The three failing tests were:

ls_follows_all_pages_and_returns_a_complete_json_listing
limited_ls_preserves_the_cursor_in_json_and_reports_the_count_in_human_output
connection_resets_retry_safe_read_requests

Each child CLI process reported error sending request for url (http://127.0.0.1:<ephemeral-port>/...). The test helpers bind a loopback TcpListener and serve the response on a thread. The other 12 tests in output_contract passed. The test output ended with:

test result: FAILED. 12 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 26.85s
error: test failed, to rerun pass `-p calternal-cli --test output_contract`

This run did not change any test expectations. The gate was run once, as required. Please investigate whether the failure comes from the mock listener, the CLI HTTP client, or shared-host conditions. No root cause is confirmed yet.

## Evidence The one full workspace `cargo test` run on 2026-09-29 failed in `crates/calternal-cli/tests/output_contract.rs` after compilation completed. The three failing tests were: - `ls_follows_all_pages_and_returns_a_complete_json_listing` - `limited_ls_preserves_the_cursor_in_json_and_reports_the_count_in_human_output` - `connection_resets_retry_safe_read_requests` Each child CLI process reported `error sending request for url (http://127.0.0.1:<ephemeral-port>/...)`. The test helpers bind a loopback `TcpListener` and serve the response on a thread. The other 12 tests in `output_contract` passed. The test output ended with: ```text test result: FAILED. 12 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 26.85s error: test failed, to rerun pass `-p calternal-cli --test output_contract` ``` This run did not change any test expectations. The gate was run once, as required. Please investigate whether the failure comes from the mock listener, the CLI HTTP client, or shared-host conditions. No root cause is confirmed yet.

kayg referenced this issue

2026-09-29 04:01:09 +00:00

App Passwords UI: shared select and date picker, input surface token, one sheet title #377

kayg commented

2026-09-29 04:36:28 +00:00

Author

Owner

Starting #389 and #387 investigation on branch job/test-infra, based at 00015e652a153485d9e7adda1c5ee61bc17300db (dev). I read the repo contract, glossary, relevant design sections, and both issue threads; both issues currently have no comments. I am tracing the loopback listener lifecycle and the adversarial probe's availability/timeout handling before changing the harness.

Starting #389 and #387 investigation on branch `job/test-infra`, based at `00015e652a153485d9e7adda1c5ee61bc17300db` (`dev`). I read the repo contract, glossary, relevant design sections, and both issue threads; both issues currently have no comments. I am tracing the loopback listener lifecycle and the adversarial probe's availability/timeout handling before changing the harness.

kayg commented

2026-09-29 04:38:36 +00:00

Author

Owner

Evidence: crates/calternal-cli/tests/output_contract.rs has nonblocking loopback fixtures with fixed 2-second and 4-second accept deadlines (run_retry_whoami and run_ls). They poll every 10 ms and return without completing the expected request sequence. If the child CLI or fixture thread is delayed on the shared host, the listener can stop accepting before the request arrives; the CLI then reports a loopback connection failure. I will keep the test listener alive until the child finishes, with a separate long safety deadline, so host scheduling does not decide whether the mock server exists.

Evidence: `crates/calternal-cli/tests/output_contract.rs` has nonblocking loopback fixtures with fixed 2-second and 4-second accept deadlines (`run_retry_whoami` and `run_ls`). They poll every 10 ms and return without completing the expected request sequence. If the child CLI or fixture thread is delayed on the shared host, the listener can stop accepting before the request arrives; the CLI then reports a loopback connection failure. I will keep the test listener alive until the child finishes, with a separate long safety deadline, so host scheduling does not decide whether the mock server exists.

kayg referenced this issue

2026-09-29 09:10:20 +00:00

[adversarial] Review API probe timeouts under mixed load #387

kayg commented

2026-09-29 09:10:38 +00:00

Author

Owner

Finished #389 and the #387 harness investigation on branch job/test-infra.

Head: 0eaa6ab4e4d18f2e0dcc60e7ed0245a9cdcd2dd5 (includes one clean merge of dev; not merged into dev). Commits: d19dca90 stabilizes CLI loopback fixtures; 3e853b39 sends API-only adversarial probes to the backend.

Files:

crates/calternal-cli/tests/output_contract.rs
tests/adversarial/attack.py
tests/adversarial/run.sh

Gate output:

cargo fmt --check
(exit 0; no output)

cargo test -p calternal-cli --test output_contract
Finished `test` profile [unoptimized + debuginfo] target(s) in 22m 31s
15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.12s

cargo clippy -p calternal-cli --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 26m 19s

cargo clean
Removed 14061 files, 6.9GiB total

git push origin job/test-infra
Everything up-to-date

The focused test output ended with the exact result line: test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.12s.

The full workspace clippy gate was interrupted after the job exceeded the owner’s ~4-hour limit. The full workspace cargo test gate was not run. The API-only adversarial round exited 1 with 163 findings, mostly SLOW results while other builds and adversarial probes were active; the server reported alive at the end. Direct backend Task-storm and Calendar Event-from-Log timeouts are recorded on #367. The #387 comment records which original Calendar Log this and Journal timeouts did not reproduce, plus existing DAV/Reminders probe findings. I removed the generated web build and local dependency output after the server probe.

Decisions not covered by DESIGN.md: the loopback fixture listener stays available until its CLI child exits, with a 60-second cleanup bound; these mock HTTP commands use a 30-second client timeout. The API-only adversarial mode targets the real backend directly so the local frontend proxy cannot turn upstream errors into synthetic API 502s.

Known gaps: workspace clippy and workspace tests remain incomplete due to the timebox. Existing adversarial expectations for DAV discovery, DAV VALARM, and the Reminders unsupported field/delete sequence remain unchanged for orchestrator review.

Finished #389 and the #387 harness investigation on branch `job/test-infra`. Head: `0eaa6ab4e4d18f2e0dcc60e7ed0245a9cdcd2dd5` (includes one clean merge of `dev`; not merged into `dev`). Commits: `d19dca90` stabilizes CLI loopback fixtures; `3e853b39` sends API-only adversarial probes to the backend. Files: - `crates/calternal-cli/tests/output_contract.rs` - `tests/adversarial/attack.py` - `tests/adversarial/run.sh` Gate output: ```text cargo fmt --check (exit 0; no output) cargo test -p calternal-cli --test output_contract Finished `test` profile [unoptimized + debuginfo] target(s) in 22m 31s 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.12s cargo clippy -p calternal-cli --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 26m 19s cargo clean Removed 14061 files, 6.9GiB total git push origin job/test-infra Everything up-to-date ``` The focused test output ended with the exact result line: `test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.12s`. The full workspace clippy gate was interrupted after the job exceeded the owner’s ~4-hour limit. The full workspace `cargo test` gate was not run. The API-only adversarial round exited 1 with 163 findings, mostly `SLOW` results while other builds and adversarial probes were active; the server reported alive at the end. Direct backend Task-storm and Calendar Event-from-Log timeouts are recorded on #367. The #387 comment records which original Calendar Log this and Journal timeouts did not reproduce, plus existing DAV/Reminders probe findings. I removed the generated web build and local dependency output after the server probe. Decisions not covered by DESIGN.md: the loopback fixture listener stays available until its CLI child exits, with a 60-second cleanup bound; these mock HTTP commands use a 30-second client timeout. The API-only adversarial mode targets the real backend directly so the local frontend proxy cannot turn upstream errors into synthetic API 502s. Known gaps: workspace clippy and workspace tests remain incomplete due to the timebox. Existing adversarial expectations for DAV discovery, DAV `VALARM`, and the Reminders unsupported field/delete sequence remain unchanged for orchestrator review.

kayg referenced this issue from a commit

2026-09-29 20:08:01 +00:00

Merge job/test-infra (#389 #387: cancellable CLI loopback listener, backend-direct API probes)

kayg commented

2026-10-03 11:55:34 +00:00

Author

Owner

Fixed in d06030b5ca (origin/dev); the CLI loopback listener and backend-direct request tests cover the reported transport failures.

Fixed in d06030b5ca (origin/dev); the CLI loopback listener and backend-direct request tests cover the reported transport failures.

kayg closed this issue

2026-10-03 11:55:35 +00:00

kayg referenced this issue