e2e: suite stability from the BM14b repeat-each study (vim :noh, lazy-ui reload mark, links Try again) #895

Closed

kayg wants to merge 0 commits from fix/suite-stability-bm14b into dev

kayg commented

2026-11-03 11:57:13 +00:00

Owner

Stability fixes from the BM14b repeat-each study (#891 comment has the full table).

Three test fixes, each against a failure that passed one comparison run but is not deterministic:

Test	Cause	Fix
chromium-1440 `vim.spec.ts` "/ searches in normal mode…"	Esc from a half-typed `?cher` removes the marks and puts the earlier search's 3 back ~60 ms later (probe: `0,3,3,…`). `:noh` sent inside that gap clears nothing and the marks return.	Wait for the 3 marks before `:`; wait for the bar's prefix and focused field before typing `noh`.
webkit-1440 `lazy-ui.spec.ts` "failed … idle warming recovers"	Reload check used `performance.timeOrigin`; WebKit derives it from the wall clock, BM14b saw ...041 vs ...042 with no reload.	Mark the window (`__sameWindow`); a reload loses it.
webkit-1440 `links.spec.ts` "Try again"	5 s URL check came before the 15 s wait for the server job that moves the page.	Wait for the job first, then the URL.

No timeout, threshold or retry was changed.

Numbers (repeat-each 20, `--workers=1 --retries=0 CI=1`)

"plain" = head and baseline running side by side; "load" = plus one ordinary slot Playwright job. vim on dev-01, WebKit on dev-02.

Case	origin/dev before (plain / load)	train head before (plain / load)	after: origin/dev + fix	after: train head + fix
vim "/ searches…"	16/20, 18/20	19/20, 19/20	20/20, 20/20	20/20, 20/20
lazy-ui ×3 chunks	60/60, 60/60	60/60, 60/60	60/60, 60/60	60/60, 60/60
links "Try again"	20/20, 20/20	20/20, 20/20	20/20, 20/20	20/20, 20/20

The two WebKit cases never failed in these runs, so the two WebKit fixes are by analysis, not by a before/after (stated in the #891 comment).

🤖 Generated with Claude Code

Stability fixes from the BM14b repeat-each study (#891 comment has the full table). Three test fixes, each against a failure that passed one comparison run but is not deterministic: | Test | Cause | Fix | |---|---|---| | chromium-1440 `vim.spec.ts` "/ searches in normal mode…" | Esc from a half-typed `?cher` removes the marks and puts the earlier search's 3 back ~60 ms later (probe: `0,3,3,…`). `:noh` sent inside that gap clears nothing and the marks return. | Wait for the 3 marks before `:`; wait for the bar's prefix and focused field before typing `noh`. | | webkit-1440 `lazy-ui.spec.ts` "failed … idle warming recovers" | Reload check used `performance.timeOrigin`; WebKit derives it from the wall clock, BM14b saw ...041 vs ...042 with no reload. | Mark the window (`__sameWindow`); a reload loses it. | | webkit-1440 `links.spec.ts` "Try again" | 5 s URL check came before the 15 s wait for the server job that moves the page. | Wait for the job first, then the URL. | No timeout, threshold or retry was changed. ## Numbers (repeat-each 20, `--workers=1 --retries=0 CI=1`) "plain" = head and baseline running side by side; "load" = plus one ordinary slot Playwright job. vim on dev-01, WebKit on dev-02. | Case | origin/dev before (plain / load) | train head before (plain / load) | after: origin/dev + fix | after: train head + fix | |---|---|---|---|---| | vim "/ searches…" | 16/20, 18/20 | 19/20, 19/20 | 20/20, 20/20 | 20/20, 20/20 | | lazy-ui ×3 chunks | 60/60, 60/60 | 60/60, 60/60 | 60/60, 60/60 | 60/60, 60/60 | | links "Try again" | 20/20, 20/20 | 20/20, 20/20 | 20/20, 20/20 | 20/20, 20/20 | The two WebKit cases never failed in these runs, so the two WebKit fixes are by analysis, not by a before/after (stated in the #891 comment). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

kayg added 1 commit

2026-11-03 11:57:13 +00:00

e2e: vim :noh waits for the marks to come back; lazy-ui marks the window; links waits for the job first

CI / server (pull_request) Has been cancelled

Details

CI / rust (pull_request) Has been cancelled

Details

CI / web (pull_request) Has been cancelled

Details

Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled

Details

Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled

Details

3c1d8deb19

- vim.spec '/ searches...': Esc from a half-typed '?cher' removes the marks and puts the
  earlier search's three back about 60 ms later (probe: 0,3,3,...). :noh sent inside that gap
  clears nothing and the marks then return. Wait for the 3 before ':', and for the bar's
  field to hold the focus before typing. Flaked 4/20 on origin/dev, 1/20 on the BM14b head.
- lazy-ui.spec 'failed ... idle warming recovers': detect a reload with a mark on the window,
  not performance.timeOrigin (WebKit derives it from the wall clock; BM14b saw it move by 1 ms).
- links.spec 'Try again': wait for the server's job (15 s poll) before the 5 s URL check.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

kayg referenced this pull request

2026-11-03 11:57:45 +00:00

Merge train BM14b #891

kayg referenced this pull request

2026-11-03 13:35:21 +00:00

Merge train BM14c #897

kayg referenced this pull request

2026-11-03 16:19:14 +00:00

Merge train BM14c #897

kayg referenced this pull request

2026-11-03 19:59:29 +00:00

Merge train BM14c #897