kayg/infraternal

Watch

Fork 0

dev: core-screen #rtt and platform-consistency specs fail #430

New issue

Open

opened 2026-11-08 15:41:38 +00:00 by kayg · 3 comments

kayg commented

2026-11-08 15:41:38 +00:00

Owner

J370 found these tracer browser specs failing on unmodified origin/dev (7c8bd2221):

two core-screen.spec.ts tests: #rtt never shows a number;
platform-consistency.spec.ts:184.

Plan: bisect first-parent dev merges, decide product regression vs stale test, fix with the spec as the red test, and check whether production (f6f62a875) users see the RTT readout broken (read-only, via the test server). Branch fix-dev-rtt-specs, base 77b1255cb. Estimate: 2h.

J370 found these tracer browser specs failing on unmodified origin/dev (7c8bd2221): - two `core-screen.spec.ts` tests: `#rtt` never shows a number; - `platform-consistency.spec.ts:184`. Plan: bisect first-parent dev merges, decide product regression vs stale test, fix with the spec as the red test, and check whether production (f6f62a875) users see the RTT readout broken (read-only, via the test server). Branch `fix-dev-rtt-specs`, base 77b1255cb. Estimate: 2h.

kayg commented

2026-11-08 16:59:54 +00:00

Author

Owner

J374 start. Branch fix-dev-rtt-specs, base 77b1255cb (origin/dev). Estimate 2h. Ran the three failing specs on compute01 through build-slot (worktree ~/worktrees/j374, logs in ~/j374-evidence/ on compute01).

Bisect result: none of today's merges (#409, #391, #367, #416, mesh DNS, #404) broke these. core-screen.spec.ts + platform-consistency.spec.ts fail identically at 6ffe37e06 (11-07 evening), f6f62a875 (prod), 37c93637e (11-06) and older. Logs: ~/j374-evidence/pw-<sha>.log, summary results.txt.

Cause of the #rtt failure (product bug, not a stale test). Probe spec at dabeccdd9 (parent of #317) shows 319 ms at once; at fe5e4261f (#317, 11-03) it shows ·· for 10 s until a key is pressed (~/j374-evidence/probe-results.txt). Instrumented run (probe-instr.log):

a probe sent at t=5024 is answered and reported;
the terminal attaches, startPings() blanks the readout (onRoundTrip(undefined)) and sends another probe;
its answer arrives 604 ms after the last report and the 2 s report throttle in acceptPong swallows it.
The readout stays ·· until an input at least 2 s later. The throttle was added with #317; startPings (blank then re-probe) predates it, so the two were never reconciled.

Fix: sampleRoundTrip(clearedReadout) resets the throttle, as resetPings already does on reconnect; startPings passes true. Red unit test first (connection-idle.test.ts): 1 fail, then 60 pass.

Still open: with the RTT fix, core-screen.spec.ts:151 fails later on the paper palette fill (the palette is deliberately 58% overlay over page colour since D219, 28612f19f, so the spec's "palette == root material" is stale) and platform-consistency.spec.ts:184 (393x852 paper) shows a scrollbar at rest. Investigating both.

**J374 start.** Branch `fix-dev-rtt-specs`, base 77b1255cb (origin/dev). Estimate 2h. Ran the three failing specs on compute01 through build-slot (worktree `~/worktrees/j374`, logs in `~/j374-evidence/` on compute01). **Bisect result: none of today's merges (#409, #391, #367, #416, mesh DNS, #404) broke these.** `core-screen.spec.ts` + `platform-consistency.spec.ts` fail identically at 6ffe37e06 (11-07 evening), f6f62a875 (prod), 37c93637e (11-06) and older. Logs: `~/j374-evidence/pw-<sha>.log`, summary `results.txt`. **Cause of the `#rtt` failure (product bug, not a stale test).** Probe spec at dabeccdd9 (parent of #317) shows `319 ms` at once; at fe5e4261f (#317, 11-03) it shows `··` for 10 s until a key is pressed (`~/j374-evidence/probe-results.txt`). Instrumented run (`probe-instr.log`): 1. a probe sent at t=5024 is answered and reported; 2. the terminal attaches, `startPings()` blanks the readout (`onRoundTrip(undefined)`) and sends another probe; 3. its answer arrives 604 ms after the last report and the 2 s report throttle in `acceptPong` swallows it. The readout stays `··` until an input at least 2 s later. The throttle was added with #317; `startPings` (blank then re-probe) predates it, so the two were never reconciled. Fix: `sampleRoundTrip(clearedReadout)` resets the throttle, as `resetPings` already does on reconnect; `startPings` passes `true`. Red unit test first (`connection-idle.test.ts`): 1 fail, then 60 pass. **Still open:** with the RTT fix, `core-screen.spec.ts:151` fails later on the paper palette fill (the palette is deliberately 58% overlay over page colour since D219, 28612f19f, so the spec's "palette == root material" is stale) and `platform-consistency.spec.ts:184` (393x852 paper) shows a scrollbar at rest. Investigating both.

kayg commented

2026-11-08 18:39:03 +00:00

Author

Owner

J374 progress (branch fix-dev-rtt-specs, pushed, head 2521dc7f9 = base 77b1255cb + origin/dev 098f2c445 merged + 4 commits).

Three separate causes, none from today's merges (#409, #391, #367, #416, mesh DNS, #404 are all cleared by the bisect above):

#rtt never shows a number: product bug, fixed. Details in the first comment. Commit b6bc4cbf1; unit test red (1 fail) then green (60 pass).
core-screen.spec.ts palette assertion: stale test, since D219 (28612f19f, 10-10). The RTT wait was masking it. The palette's fill is deliberately color-mix(in oklab, tint 58%, page) so glass never lets terminal text read through: alpha 0.58 x 0.78 + 0.42 = 0.8724, which is exactly what the failure printed (oklab(0.98062 ... / 0.8724) against rgb(251 251 251 / 0.78)). The old expectation (palette == pill material) stopped being true on 10-10. Spec now states the D219 contract. Commit 840b839b4. Reproduced at 378b97da4 (10-11), the day after D219.
platform-consistency.spec.ts:184: the test measured the page, not a bar. No bar is drawn at rest (thumb is transparent; the gutter is empty). The strip spread came from the palette's rounded bottom-right corner entering the clip (last 4 of 542 rows: lo 609, hi 728), from the page's own vertical gradient showing through the glass (ink: 14 over 408 rows, so the old darkest-to-brightest spread grows with list height), and from measuring while the palette was still fading in. Now: largest step between neighbouring rows, clip stops one corner radius short, wait for the palette's animations. Also fixed two neighbours the slower measurement exposed (hide-clock started at the wheel; capture tests retry the mock reset once like openTerminal). Commit 763f19415. Platform file x3: 24 of 24 pass.

Evidence on compute01 ~/j374-evidence/: results.txt/pw-<sha>.log (bisect), probe-results.txt (RTT at dabeccdd9 / fe5e4261f / dev), probe-instr.log (ping trace), strip.log/strip2.log (strip rows), green1.log (core-screen + platform after fix: 11 passed before the platform edits; 1 flake fixed after), plat1.log (24 passed), quick.log (verify quick PASS at 2521dc7f9).

Production (f6f62a875, same JS as the test server): users are mostly fine; the bug is a race. Read-only Playwright against https://test.ssh.kayg.org with the test-admin sign-in (rtt-live-test-server.txt, rtt-live-test-server-reload.txt): after sign-in #rtt read 146 ms at once and stayed; after a cold reload with no input it read 148 ms within 2 s. So the readout works on the real path. It fails only when a probe is answered under 2 s before a terminal attaches (the mock's attach is later than the real server's). A person on a slow attach (large replay, high latency) or a reconnect that probes on return can hit it and see ·· until their next key press. I did not reproduce that on the live test server (no safe way to force the ordering there).

verify fast queued on compute01; will post the verbatim output at the finish.

**J374 progress (branch `fix-dev-rtt-specs`, pushed, head 2521dc7f9 = base 77b1255cb + origin/dev 098f2c445 merged + 4 commits).** Three separate causes, none from today's merges (#409, #391, #367, #416, mesh DNS, #404 are all cleared by the bisect above): 1. **`#rtt` never shows a number: product bug, fixed.** Details in the first comment. Commit b6bc4cbf1; unit test red (1 fail) then green (60 pass). 2. **`core-screen.spec.ts` palette assertion: stale test, since D219 (28612f19f, 10-10).** The RTT wait was masking it. The palette's fill is deliberately `color-mix(in oklab, tint 58%, page)` so glass never lets terminal text read through: alpha 0.58 x 0.78 + 0.42 = 0.8724, which is exactly what the failure printed (`oklab(0.98062 ... / 0.8724)` against `rgb(251 251 251 / 0.78)`). The old expectation (palette == pill material) stopped being true on 10-10. Spec now states the D219 contract. Commit 840b839b4. Reproduced at 378b97da4 (10-11), the day after D219. 3. **`platform-consistency.spec.ts:184`: the test measured the page, not a bar.** No bar is drawn at rest (thumb is transparent; the gutter is empty). The strip spread came from the palette's rounded bottom-right corner entering the clip (last 4 of 542 rows: lo 609, hi 728), from the page's own vertical gradient showing through the glass (ink: 14 over 408 rows, so the old darkest-to-brightest spread grows with list height), and from measuring while the palette was still fading in. Now: largest step between neighbouring rows, clip stops one corner radius short, wait for the palette's animations. Also fixed two neighbours the slower measurement exposed (hide-clock started at the wheel; capture tests retry the mock reset once like `openTerminal`). Commit 763f19415. Platform file x3: 24 of 24 pass. Evidence on compute01 `~/j374-evidence/`: `results.txt`/`pw-<sha>.log` (bisect), `probe-results.txt` (RTT at dabeccdd9 / fe5e4261f / dev), `probe-instr.log` (ping trace), `strip.log`/`strip2.log` (strip rows), `green1.log` (core-screen + platform after fix: 11 passed before the platform edits; 1 flake fixed after), `plat1.log` (24 passed), `quick.log` (`verify quick` PASS at 2521dc7f9). **Production (f6f62a875, same JS as the test server): users are mostly fine; the bug is a race.** Read-only Playwright against https://test.ssh.kayg.org with the test-admin sign-in (`rtt-live-test-server.txt`, `rtt-live-test-server-reload.txt`): after sign-in `#rtt` read `146 ms` at once and stayed; after a cold reload with no input it read `148 ms` within 2 s. So the readout works on the real path. It fails only when a probe is answered under 2 s before a terminal attaches (the mock's attach is later than the real server's). A person on a slow attach (large replay, high latency) or a reconnect that probes on return can hit it and see `··` until their next key press. I did not reproduce that on the live test server (no safe way to force the ordering there). `verify fast` queued on compute01; will post the verbatim output at the finish.

kayg commented

2026-11-08 20:22:49 +00:00

Author

Owner

J374 finish. Branch fix-dev-rtt-specs, head 2521dc7f9 (pushed). Base 77b1255cb, origin/dev 098f2c445 merged (no conflicts).

Commits: b6bc4cbf1 (product fix + unit test), 840b839b4 (core-screen palette spec), 763f19415 (platform-consistency spec), 0d41cdff6 (merge dev), 2521dc7f9 (D345 doc comments the gate asked for).

Red/green.

New unit test in connection-idle.test.ts: red before the fix (1 fail, 59 pass), 60 pass after.
core-screen.spec.ts + platform-consistency.spec.ts on unmodified dev: 3 failed, 8 passed (pw-77b1255cb.log). After the fix: core-screen passes (green1.log), and the platform file passes 24 of 24 over three repeats (plat1.log).
Targeted browser run of 14 related spec files on 2521dc7f9 (sub.log): 105 passed, 13 skipped, 2 failed. Both failures also fail on unmodified dev (below).

verify quick (compute01, 2521dc7f9, quick.log), VERIFY_EXIT=0:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| scope guard               | PASS         |
| migration names           | PASS         |
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| ui literals               | PASS         |
| tracer typecheck          | PASS         |
+---------------------------+--------------+

verify fast (compute01, 2521dc7f9, fast.log), VERIFY_EXIT=0; the bun suite at the end reports 50 pass, 0 fail:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| migration names           | PASS         |
| migration tooling tests   | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| Traefik component manifest | PASS         |
| release component publication | PASS         |
| test                      | PASS         |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| privileged integration    | SKIP         |
| probe release exclusion   | PASS         |
| nightly fuzz reporter     | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

privileged integration reports SKIP; I did not look into why.

Still red after this branch, not caused by it:

terminal.spec.ts:5 (first run, passkey login, resume, latency probe). On unmodified dev it fails at terminal-scenario.ts:111 (round trip never above 60 ms: the same RTT bug). With my fix it passes that step and fails at terminal-scenario.ts:155: after inject("p") in no-echo mode, predictiveEcho().pendingText never becomes "p" within 30 s. Not investigated: past 2x my estimate and outside the task. Logs: cmp-base.log (dev) and sub.log (branch).
predictive-echo.spec.ts:530 fails on unmodified dev too (cmp-base.log). On the branch run it died with socket hang up (ECONNRESET) on GET /__mock/state through the latency proxy, the same proxy flake openTerminal retries once. I did not look for a different cause on dev.

Production (f6f62a875, same JS as the test server): people see the readout in the normal cold-load case (146 to 148 ms at once, see the earlier comment) and see ·· only in the race described there. Production keeps that race until this fix ships.

Gate gap worth a look. fast.log has no Playwright output (0 matches), and the tracer suite lives in verify_latency (scripts/verify:31). That is how these specs stayed red on dev while verify fast was green.

Process notes. I ran one pkill -f over ssh by mistake (the brief forbids it). It stopped my own jobs and my ssh shell, and I removed the leftover test server by exact PID afterwards. My 48 GB target dir and both VM worktrees are deleted. Logs and screenshots stay in ~/j374-evidence/ on compute01. The red unit-test output is /tmp/j374-red-unit.txt on the code host.

**J374 finish.** Branch `fix-dev-rtt-specs`, head **2521dc7f9** (pushed). Base 77b1255cb, origin/dev 098f2c445 merged (no conflicts). Commits: b6bc4cbf1 (product fix + unit test), 840b839b4 (core-screen palette spec), 763f19415 (platform-consistency spec), 0d41cdff6 (merge dev), 2521dc7f9 (D345 doc comments the gate asked for). **Red/green.** - New unit test in `connection-idle.test.ts`: red before the fix (1 fail, 59 pass), 60 pass after. - `core-screen.spec.ts` + `platform-consistency.spec.ts` on unmodified dev: 3 failed, 8 passed (`pw-77b1255cb.log`). After the fix: core-screen passes (`green1.log`), and the platform file passes 24 of 24 over three repeats (`plat1.log`). - Targeted browser run of 14 related spec files on 2521dc7f9 (`sub.log`): 105 passed, 13 skipped, 2 failed. Both failures also fail on unmodified dev (below). **`verify quick`** (compute01, 2521dc7f9, `quick.log`), `VERIFY_EXIT=0`: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | scope guard | PASS | | migration names | PASS | | Nix Cargo vendoring | PASS | | fmt | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | ui literals | PASS | | tracer typecheck | PASS | +---------------------------+--------------+ ``` **`verify fast`** (compute01, 2521dc7f9, `fast.log`), `VERIFY_EXIT=0`; the bun suite at the end reports `50 pass, 0 fail`: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | Nix Cargo vendoring | PASS | | fmt | PASS | | migration names | PASS | | migration tooling tests | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | Traefik component manifest | PASS | | release component publication | PASS | | test | PASS | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | privileged integration | SKIP | | probe release exclusion | PASS | | nightly fuzz reporter | PASS | | static builds | PASS | +---------------------------+--------------+ ``` `privileged integration` reports SKIP; I did not look into why. **Still red after this branch, not caused by it:** 1. `terminal.spec.ts:5` (first run, passkey login, resume, latency probe). On unmodified dev it fails at `terminal-scenario.ts:111` (round trip never above 60 ms: the same RTT bug). With my fix it passes that step and fails at `terminal-scenario.ts:155`: after `inject("p")` in no-echo mode, `predictiveEcho().pendingText` never becomes `"p"` within 30 s. Not investigated: past 2x my estimate and outside the task. Logs: `cmp-base.log` (dev) and `sub.log` (branch). 2. `predictive-echo.spec.ts:530` fails on unmodified dev too (`cmp-base.log`). On the branch run it died with `socket hang up (ECONNRESET)` on `GET /__mock/state` through the latency proxy, the same proxy flake `openTerminal` retries once. I did not look for a different cause on dev. **Production** (f6f62a875, same JS as the test server): people see the readout in the normal cold-load case (146 to 148 ms at once, see the earlier comment) and see `··` only in the race described there. Production keeps that race until this fix ships. **Gate gap worth a look.** `fast.log` has no Playwright output (0 matches), and the tracer suite lives in `verify_latency` (`scripts/verify:31`). That is how these specs stayed red on dev while `verify fast` was green. **Process notes.** I ran one `pkill -f` over ssh by mistake (the brief forbids it). It stopped my own jobs and my ssh shell, and I removed the leftover test server by exact PID afterwards. My 48 GB target dir and both VM worktrees are deleted. Logs and screenshots stay in `~/j374-evidence/` on compute01. The red unit-test output is `/tmp/j374-red-unit.txt` on the `code` host.

kayg referenced this issue

2026-11-08 20:27:49 +00:00

J394: two tracer specs fail on dev (terminal.spec.ts:5 prediction, predictive-echo.spec.ts:530) #438

kayg referenced this issue

2026-11-08 21:43:06 +00:00

J386: tracer browser baseline is red on compute01 #434

kayg referenced this issue

2026-11-08 21:44:24 +00:00

J382: forget retired-unreachable hosts (D369) #435

kayg referenced this issue

2026-11-08 21:51:37 +00:00

J394: two tracer specs fail on dev (terminal.spec.ts:5 prediction, predictive-echo.spec.ts:530) #438