Server startup fixture exceeds its deadline while an upgrade Home is locked #1192

Open
opened 2026-10-06 08:35:56 +00:00 by kayg · 0 comments
Owner

Observed during #1140 verification on branch job/wizard-1140, head abd4812a0a. The startup test and its 15-second expectation were not changed. The cause is not established. Do not change the deadline to hide this finding.

The test holds the Notes Home change lock, then calls build_live_app. HTTP should bind while the upgrade backfill waits for that lock. The 15-second timeout instead returns Elapsed at crates/calternal-server/src/wire.rs:8469.

Full server command:

CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=4 TMPDIR=$PWD/target/tmp cargo test -p calternal-server -- --test-threads=4

Output:

wire::tests::startup_serves_http_while_upgrade_backfills_wait failed:
HTTP startup waited for an upgrade backfill: Elapsed(())
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 266 filtered out; finished in 19.57s
test result: FAILED. 255 passed; 1 failed; 11 ignored; 0 measured; 0 filtered out; finished in 102.92s

An isolated retry used the same deadline and a fresh process:

CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=4 TMPDIR=$PWD/target/tmp RUST_MIN_STACK=8388608 cargo test -p calternal-server wire::tests::startup_serves_http_while_upgrade_backfills_wait -- --ignored --exact --test-threads=1

Output:

HTTP startup waited for an upgrade backfill: Elapsed(())
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 266 filtered out; finished in 97.22s

An earlier full server unit run on this job passed 256 tests in 240.20s. Thus the two failures do not yet prove a source regression or a load-only cause. Money's 119 unit tests and 27 import review tests pass; all production upgrade fixtures pass in the failing full server run. Web check has zero errors and all 1883 web tests pass. The final server report must retain this failed gate until the cause is checked.

Observed during #1140 verification on branch job/wizard-1140, head abd4812a0a6a59a48e68f1aabeae2aafed31cbe0. The startup test and its 15-second expectation were not changed. The cause is not established. Do not change the deadline to hide this finding. The test holds the Notes Home change lock, then calls build_live_app. HTTP should bind while the upgrade backfill waits for that lock. The 15-second timeout instead returns Elapsed at crates/calternal-server/src/wire.rs:8469. Full server command: ```sh CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=4 TMPDIR=$PWD/target/tmp cargo test -p calternal-server -- --test-threads=4 ``` Output: ``` wire::tests::startup_serves_http_while_upgrade_backfills_wait failed: HTTP startup waited for an upgrade backfill: Elapsed(()) test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 266 filtered out; finished in 19.57s test result: FAILED. 255 passed; 1 failed; 11 ignored; 0 measured; 0 filtered out; finished in 102.92s ``` An isolated retry used the same deadline and a fresh process: ```sh CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=4 TMPDIR=$PWD/target/tmp RUST_MIN_STACK=8388608 cargo test -p calternal-server wire::tests::startup_serves_http_while_upgrade_backfills_wait -- --ignored --exact --test-threads=1 ``` Output: ``` HTTP startup waited for an upgrade backfill: Elapsed(()) test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 266 filtered out; finished in 97.22s ``` An earlier full server unit run on this job passed 256 tests in 240.20s. Thus the two failures do not yet prove a source regression or a load-only cause. Money's 119 unit tests and 27 import review tests pass; all production upgrade fixtures pass in the failing full server run. Web check has zero errors and all 1883 web tests pass. The final server report must retain this failed gate until the cause is checked.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#1192
No description provided.