kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 190 Pull requests 61 Projects Releases Packages Wiki Activity Actions 26

Merge train BM14b #891

Closed
kayg wants to merge 125 commits from train/2026-11-03-bm14b into dev
pull from: train/2026-11-03-bm14b
merge into: kayg:dev
Conversation 6 Commits 125 Files changed 88 +8091 -969
kayg commented 2026-11-03 06:21:03 +00:00
Owner
Copy link

BM14b, based on origin/dev 41f2efa387b74175971a838f602535089d0d5fb4. Head 6a1749a4251539f877515d4265c9b732b4965b76. Do not merge.

PR Current fetched head Outcome
#850 1812cc5e10 included --no-ff
#851 80934507d2 included --no-ff
#852 08b6d9adc8 included --no-ff
#829 97d790f692 included --no-ff
#835 3755691ddc included --no-ff

Excluded #842/#876 as requested. Mechanical #852 conflict resolution retains dev pull hint and expects 25 lazy imports, matching BM14. First-run gates and full split browser matrix pending, CI=1, workers=1, retries=0. No fixes will be made. Known candidates: vim.spec.ts:341/:380; sidebar-edge.spec.ts:8; webkit-1440 editor.spec.ts:173; webkit-390 missing-links.spec.ts:245. A candidate is exempt only if reproduced in its single baseline comparison. Results will be posted in comments. Artifacts stay outside the repo.

BM14b, based on origin/dev `41f2efa387b74175971a838f602535089d0d5fb4`. Head `6a1749a4251539f877515d4265c9b732b4965b76`. Do not merge. | PR | Current fetched head | Outcome | |---|---|---| | #850 | 1812cc5e10b8008e28acaa2f19234ff8123dfd73 | included --no-ff | | #851 | 80934507d2dd288505f968f1252e06b1dae8d071 | included --no-ff | | #852 | 08b6d9adc892248243bea934df2823fb280f5d0e | included --no-ff | | #829 | 97d790f69259b0308598ca0aafd2407ef1a1226e | included --no-ff | | #835 | 3755691ddc63a740d36a4782f2163a3876f58cfc | included --no-ff | Excluded #842/#876 as requested. Mechanical #852 conflict resolution retains dev pull hint and expects 25 lazy imports, matching BM14. First-run gates and full split browser matrix pending, CI=1, workers=1, retries=0. No fixes will be made. Known candidates: vim.spec.ts:341/:380; sidebar-edge.spec.ts:8; webkit-1440 editor.spec.ts:173; webkit-390 missing-links.spec.ts:245. A candidate is exempt only if reproduced in its single baseline comparison. Results will be posted in comments. Artifacts stay outside the repo.
kayg added 169 commits 2026-11-03 06:21:03 +00:00 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The room under a phone's Style sheet loads with the idle screens
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
7a6d5adf4c
Presets: the thumbnail's page is not .page; backdrop, opacity and width specs follow the new Style sheet
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
c1ef1b78ed
Merge port/style-2
Some checks failed
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
a0f425ac08 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts:
#	apps/web/scripts/lazy-ui-plugin.ts
Merge #835 into BM14b
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
6a1749a425
kayg commented 2026-11-03 06:29:19 +00:00
Author
Owner
Copy link

Assembly complete at 6a1749a4251539f877515d4265c9b732b4965b76, base 41f2efa387b74175971a838f602535089d0d5fb4. All five current heads in the description were fetched before assembly and merged with --no-ff in the requested order. #842/#876 excluded. Only mechanical resolution: #852 lazy import count 25, preserving dev PullHint and the style stack, as in BM14.

Train locks are held by live owner shells on both VMs with EXIT cleanup, SLOT_TRAIN=1, CI=1, disk TMPDIR $HOME/bm14b-tmp. Existing slot jobs are draining before gates. Initial free space: dev-01 252 GiB, dev-02 319 GiB; each gate checks the 50 GiB floor. Playwright starts directly with the corrected external .mts wrapper and explicit testDir/webServer.cwd. Chromium half on dev-01, WebKit half on dev-02, including admin-setup on each VM, preserving dependencies within each half. Workers 1, retries 0. No train case will be retried, and failures receive one origin/dev comparison. No product fixes or merge.

Assembly complete at `6a1749a4251539f877515d4265c9b732b4965b76`, base `41f2efa387b74175971a838f602535089d0d5fb4`. All five current heads in the description were fetched before assembly and merged with `--no-ff` in the requested order. #842/#876 excluded. Only mechanical resolution: #852 lazy import count 25, preserving dev PullHint and the style stack, as in BM14. Train locks are held by live owner shells on both VMs with EXIT cleanup, `SLOT_TRAIN=1`, `CI=1`, disk TMPDIR `$HOME/bm14b-tmp`. Existing slot jobs are draining before gates. Initial free space: dev-01 252 GiB, dev-02 319 GiB; each gate checks the 50 GiB floor. Playwright starts directly with the corrected external `.mts` wrapper and explicit `testDir`/`webServer.cwd`. Chromium half on dev-01, WebKit half on dev-02, including admin-setup on each VM, preserving dependencies within each half. Workers 1, retries 0. No train case will be retried, and failures receive one origin/dev comparison. No product fixes or merge.
kayg commented 2026-11-03 06:53:08 +00:00
Author
Owner
Copy link

First-run WebKit startup gate FAILED before test discovery/attempts. Exact command, in Nix from apps/web on dev-02:

SLOT_TRAIN=1 CI=1 TMPDIR=$HOME/bm14b-tmp CALTERNAL_BIN=$HOME/Developer/Worktrees/bm14b/target/debug/calternal E2E_DIST=$HOME/Developer/Worktrees/bm14b/apps/web/dist BM14B_HALF=webkit bunx playwright test --config=$HOME/bm14b-train/playwright.config.mts --output=$HOME/bm14b-train/matrix-results --workers=1 --retries=0 --reporter=line,json

First error:

Error: Timed out waiting 60000ms from config.webServer.

Reporter has empty suites and zero test attempts. Prior output contains Vite's extensionless-import warnings, no earlier error. This is not an exempt known flake and remains a failed gate. A single origin/dev startup comparison is pending. To fulfill the full matrix, the still-unattempted cases are receiving their first attempts in a separate invocation, with DEBUG=pw:webserver, the unchanged .mts wrapper and unchanged product sources. No failed case is retried and the startup failure is not erased. This follows BM14's separate accounting for startup failure versus unattempted cases.

Completed first-run gates so far: frozen install PASS on both VMs; fmt PASS; clippy PASS; dev-02 Rust build PASS and web build PASS. Workspace tests still running on dev-01.

First-run WebKit startup gate FAILED before test discovery/attempts. Exact command, in Nix from `apps/web` on dev-02: ```sh SLOT_TRAIN=1 CI=1 TMPDIR=$HOME/bm14b-tmp CALTERNAL_BIN=$HOME/Developer/Worktrees/bm14b/target/debug/calternal E2E_DIST=$HOME/Developer/Worktrees/bm14b/apps/web/dist BM14B_HALF=webkit bunx playwright test --config=$HOME/bm14b-train/playwright.config.mts --output=$HOME/bm14b-train/matrix-results --workers=1 --retries=0 --reporter=line,json ``` First error: ```text Error: Timed out waiting 60000ms from config.webServer. ``` Reporter has empty suites and zero test attempts. Prior output contains Vite's extensionless-import warnings, no earlier error. This is not an exempt known flake and remains a failed gate. A single origin/dev startup comparison is pending. To fulfill the full matrix, the still-unattempted cases are receiving their first attempts in a separate invocation, with `DEBUG=pw:webserver`, the unchanged `.mts` wrapper and unchanged product sources. No failed case is retried and the startup failure is not erased. This follows BM14's separate accounting for startup failure versus unattempted cases. Completed first-run gates so far: frozen install PASS on both VMs; fmt PASS; clippy PASS; dev-02 Rust build PASS and web build PASS. Workspace tests still running on dev-01.
kayg commented 2026-11-03 08:54:28 +00:00
Author
Owner
Copy link

BM14b full first run: RED. Do not merge. Head 6a1749a4251539f877515d4265c9b732b4965b76, base origin/dev 41f2efa387b74175971a838f602535089d0d5fb4. Rerun from scratch by the test sub-agent after the earlier Codex run stopped on a usage limit (its two bm14b-owner units were stopped and stale locks removed; the Codex run's WebKit start-up timeout is not counted). Merged --no-ff in order: #850 1812cc5e, #851 80934507, #852 08b6d9ad, #829 97d790f6, #835 3755691d (all current heads). #842 and #876 excluded. CI=1, SLOT_TRAIN=1, on-disk TMPDIR, --workers=1 --retries=0, no case retried. Disk checked before every step (dev-01 at least 252 GiB free, dev-02 at least 317 GiB).

Gates

Gate Command Result
install (both VMs) bun install --frozen-lockfile pass
fmt cargo fmt --all --check pass
clippy cargo clippy --workspace --all-targets --locked -j4 -- -D warnings pass
Rust cargo test --workspace -j4 pass (the two #829 auth tests that failed in BM14 pass)
cargo build (both) cargo build -j4 pass
web build (both) cd apps/web && bun run build pass
bundle PERF_DIST=$PWD/apps/web/dist perf/web.sh --bundle-only --gate pass
case paths node ci/check-case-paths.mjs pass
evidence node ci/check-no-evidence.mjs pass

Playwright matrix (every project, one attempt each, retry index 0, no reporter errors)

Command, from apps/web in nix develop: SLOT_TRAIN=1 CI=1 CALTERNAL_BIN=… E2E_DIST=… bunx playwright test --config=<.mts wrapper> --workers=1 --retries=0 --reporter=line,json. Chromium half on dev-01, WebKit half on dev-02, admin-setup on each.

VM / project Passed Failed Skipped
chromium / chromium-1440 440 2 625
chromium / chromium-390 896 2 169
chromium / throttled-1440 * 1 1 2
chromium / throttled-390 * 1 3 0
chromium / admin-chromium-1440 12 0 9
chromium / admin-chromium-390 12 0 9
chromium / flow, passkeys, admin-setup 1, 1, 1 0 0
chromium / prefs-chromium 7 0 0
chromium / look-chromium-1440, -390 5, 5 0 0
chromium / contrast 9 0 0
chromium / sidebar-band-chromium 3 0 0
webkit / webkit-1440 438 2 627
webkit / webkit-390 895 0 172
webkit / admin-webkit-1440, -390 9, 9 0 9, 9
webkit / admin-setup 1 0 0
webkit / prefs-webkit 3 0 4
webkit / look-webkit-1440, -390 4, 4 0 1, 1
webkit / sidebar-band-webkit 3 0 0
Total 2760 10 see rows

* Both throttled projects depend on the chromium projects, which had failures, so Playwright did not run them (8 "did not run"). Their first attempts were made right after, in one separate invocation on the unchanged train binary and dist with dependencies removed. Those results are the ones in the table.

Failures (first error), each with one origin/dev comparison

Baseline 41f2efa3, built and run the same way, exact cases only, one attempt. Every case below passed on the baseline, so none reproduces there.

Case First error origin/dev
chromium-1440 glass.spec.ts:76 flashes.frames expected > 20, received 13 (line 259) pass
chromium-1440 vim.spec.ts:341 (on the known-flake list) .cm-searchMatch count expected 0, received 3 pass, so it did not reproduce on the baseline
chromium-390 note-bars.spec.ts:100 (Attach, Divider) bar key expected attach, received tools after the tap (line 109) pass
chromium-390 touch-select.spec.ts:312 test timeout 30000 ms pass
throttled-1440 editor.spec.ts:57 (cpu x6) test timeout 30000 ms pass
throttled-390 editor.spec.ts:57 (cpu x6) test timeout 30000 ms, page closed in helpers.ts:123 pass
throttled-390 note-bars-motion.spec.ts:71 (Style sheet) longest main-thread task expected < 250, received 376.9 pass
throttled-390 note-bars-motion.spec.ts:71 (Attach menu) longest main-thread task expected < 250, received 258.4 pass
webkit-1440 lazy-ui.spec.ts:70 performance.timeOrigin expected 1793689770041, received ...042 (line 91) pass
webkit-1440 links.spec.ts:101 not.toHaveURL(/chg-n7hesu/) still on the note URL after Try again (line 121) pass

Known flakes that did not occur: vim.spec.ts:380, sidebar-edge.spec.ts:8, webkit-1440 editor.spec.ts:173, webkit-390 missing-links.spec.ts:245. webkit-390 had no failures this time. No product fixes were made.

**BM14b full first run: RED. Do not merge.** Head `6a1749a4251539f877515d4265c9b732b4965b76`, base origin/dev `41f2efa387b74175971a838f602535089d0d5fb4`. Rerun from scratch by the test sub-agent after the earlier Codex run stopped on a usage limit (its two `bm14b-owner` units were stopped and stale locks removed; the Codex run's WebKit start-up timeout is not counted). Merged `--no-ff` in order: #850 `1812cc5e`, #851 `80934507`, #852 `08b6d9ad`, #829 `97d790f6`, #835 `3755691d` (all current heads). #842 and #876 excluded. `CI=1`, `SLOT_TRAIN=1`, on-disk TMPDIR, `--workers=1 --retries=0`, no case retried. Disk checked before every step (dev-01 at least 252 GiB free, dev-02 at least 317 GiB). ## Gates | Gate | Command | Result | |---|---|---| | install (both VMs) | `bun install --frozen-lockfile` | pass | | fmt | `cargo fmt --all --check` | pass | | clippy | `cargo clippy --workspace --all-targets --locked -j4 -- -D warnings` | pass | | Rust | `cargo test --workspace -j4` | pass (the two #829 auth tests that failed in BM14 pass) | | cargo build (both) | `cargo build -j4` | pass | | web build (both) | `cd apps/web && bun run build` | pass | | bundle | `PERF_DIST=$PWD/apps/web/dist perf/web.sh --bundle-only --gate` | pass | | case paths | `node ci/check-case-paths.mjs` | pass | | evidence | `node ci/check-no-evidence.mjs` | pass | ## Playwright matrix (every project, one attempt each, retry index 0, no reporter errors) Command, from `apps/web` in `nix develop`: `SLOT_TRAIN=1 CI=1 CALTERNAL_BIN=… E2E_DIST=… bunx playwright test --config=<.mts wrapper> --workers=1 --retries=0 --reporter=line,json`. Chromium half on dev-01, WebKit half on dev-02, admin-setup on each. | VM / project | Passed | Failed | Skipped | |---|---:|---:|---:| | chromium / chromium-1440 | 440 | 2 | 625 | | chromium / chromium-390 | 896 | 2 | 169 | | chromium / throttled-1440 * | 1 | 1 | 2 | | chromium / throttled-390 * | 1 | 3 | 0 | | chromium / admin-chromium-1440 | 12 | 0 | 9 | | chromium / admin-chromium-390 | 12 | 0 | 9 | | chromium / flow, passkeys, admin-setup | 1, 1, 1 | 0 | 0 | | chromium / prefs-chromium | 7 | 0 | 0 | | chromium / look-chromium-1440, -390 | 5, 5 | 0 | 0 | | chromium / contrast | 9 | 0 | 0 | | chromium / sidebar-band-chromium | 3 | 0 | 0 | | webkit / webkit-1440 | 438 | 2 | 627 | | webkit / webkit-390 | 895 | 0 | 172 | | webkit / admin-webkit-1440, -390 | 9, 9 | 0 | 9, 9 | | webkit / admin-setup | 1 | 0 | 0 | | webkit / prefs-webkit | 3 | 0 | 4 | | webkit / look-webkit-1440, -390 | 4, 4 | 0 | 1, 1 | | webkit / sidebar-band-webkit | 3 | 0 | 0 | | **Total** | **2760** | **10** | see rows | \* Both throttled projects depend on the chromium projects, which had failures, so Playwright did not run them (8 "did not run"). Their first attempts were made right after, in one separate invocation on the unchanged train binary and dist with dependencies removed. Those results are the ones in the table. ## Failures (first error), each with one origin/dev comparison Baseline `41f2efa3`, built and run the same way, exact cases only, one attempt. Every case below passed on the baseline, so none reproduces there. | Case | First error | origin/dev | |---|---|---| | chromium-1440 `glass.spec.ts:76` | `flashes.frames` expected > 20, received 13 (line 259) | pass | | chromium-1440 `vim.spec.ts:341` (on the known-flake list) | `.cm-searchMatch` count expected 0, received 3 | pass, so it did not reproduce on the baseline | | chromium-390 `note-bars.spec.ts:100` (Attach, Divider) | bar key expected `attach`, received `tools` after the tap (line 109) | pass | | chromium-390 `touch-select.spec.ts:312` | test timeout 30000 ms | pass | | throttled-1440 `editor.spec.ts:57` (cpu x6) | test timeout 30000 ms | pass | | throttled-390 `editor.spec.ts:57` (cpu x6) | test timeout 30000 ms, page closed in `helpers.ts:123` | pass | | throttled-390 `note-bars-motion.spec.ts:71` (Style sheet) | longest main-thread task expected < 250, received 376.9 | pass | | throttled-390 `note-bars-motion.spec.ts:71` (Attach menu) | longest main-thread task expected < 250, received 258.4 | pass | | webkit-1440 `lazy-ui.spec.ts:70` | `performance.timeOrigin` expected 1793689770041, received ...042 (line 91) | pass | | webkit-1440 `links.spec.ts:101` | `not.toHaveURL(/chg-n7hesu/)` still on the note URL after Try again (line 121) | pass | Known flakes that did not occur: `vim.spec.ts:380`, `sidebar-edge.spec.ts:8`, webkit-1440 `editor.spec.ts:173`, webkit-390 `missing-links.spec.ts:245`. webkit-390 had no failures this time. No product fixes were made.
kayg commented 2026-11-03 11:57:45 +00:00
Author
Owner
Copy link

BM14b: repeat-each study of the 10 red cases

Every case run --repeat-each=20 --workers=1 --retries=0 CI=1, --no-deps, train head 6a1749a4 and origin/dev 41f2efa3 side by side on the same VM (Chromium on dev-01, WebKit on dev-02), each run with its own --output. "plain" = the two side by side; "load" = plus one ordinary ~/bin/slot Playwright job (editor+links specs) for the whole phase. Host load average from other agents was 4–24 during these runs (the editor cases hit the 30 s limit only when it was high). My own first plain pass on dev-01 was contaminated by my probe runs sharing test-results; those cases were rerun clean and only the clean numbers are below.

Pass counts, passed/20 (plain, load):

Case (train failure) origin/dev train head Class Cause
chromium-1440 glass.spec.ts:76 20, 20 0, 0 (first pass 4, 0) (a) train-only dee3e961 (#850) swapped the fixed 450×4 ms waits for still(). A settled page draws no frames (the caret blink gave most of the old ones): the screencast holds 15–19 frames, the old test 40+, and frames > 20 fails every time.
chromium-1440 vim.spec.ts:341 16, 18 19, 19 (b) flaky on both Esc from ?cher removes the marks and puts the earlier 3 back ~60 ms later (probe 0,3,3,…); :noh sent in that gap clears nothing. Rate: 4/20 and 2/20 on dev, 1/20 and 1/20 on head.
chromium-390 note-bars.spec.ts:100 spec does not exist on dev 20, 20 not reproduced New in #850. Also whole file ×8 on dev-02: 104/104 pass.
chromium-390 touch-select.spec.ts:312 20, 20 20, 20 not reproduced Whole file ×8: 128/128 on head and on dev.
throttled-1440 editor.spec.ts:57 20, 20 20, 20 (c) load-only, worse on head One test did all the work at 6x CPU: open 5.5 s, type 55 keys 6 s, palette toggles 3.5 s + 2.4 s = ~18 s of the 30 s on a quiet host. Head is ~4 s slower than dev (it opens the palette twice, dev clicked a button).
throttled-390 editor.spec.ts:57 19, 20 (1 timeout) 16, 20 (4 timeouts) (c) load-only, worse on head same; dev also timed out once (32.6 s)
throttled-390 note-bars-motion.spec.ts:71 Style sheet spec does not exist on dev 20, 19 intermittent, not fixed longest task at 6x is 106–208 ms plain, up to 299 ms under load (budget 250).
same, Attach menu – 20, 20 not reproduced 85–172 ms
webkit-1440 lazy-ui.spec.ts:70 (3 chunks) 60, 60 60, 60 not reproduced performance.timeOrigin moved 1 ms with no reload; fix by analysis (below).
webkit-1440 links.spec.ts:101 20, 20 20, 20 not reproduced

The pkill -u cua -f ms-playwright on dev-02 (08:00 UTC / 09:00 CET) does not explain the two WebKit failures. They happened at 07:09:29Z and 07:10:43Z, about 50 minutes earlier, and are assertion failures with the browser alive. None of my dev-02 runs overlap it (first start 09:04Z) and none of my logs have a closed-browser signature.

Why dee3e961 did not hold for editor.spec.ts:57

It made toggleSource wait for the dialog and the mode (correct, not faster). The test still spends ~18 s of its 30 s at 6x, so any host load above ~1.7× runs out of time, and the "page closed in helpers.ts:123" is just the step that was running at the 30 s mark. The same commit is what broke glass.spec.ts:76.

Fixes (nothing loosened: no timeout, threshold or retry changed)

On #850 (port/style, pushed d252a902)

  • 3744b595 glass: wait for the screencast to hold >24 frames (state) after the caret step, threshold > 20 untouched.
  • d252a902 editor: the throttled test is split into "loading and typing" and "switching modes" (×1 and ×6), each a fresh page; shift check unchanged.

New PR #895 fix/suite-stability-bm14b → dev (3c1d8deb): vim waits for the marks to return and the field to hold focus; lazy-ui marks the window instead of timeOrigin; links waits for the server job before the URL check.

After the fixes, passed/20 (plain, load):

Case origin/dev + fix train head + fix
glass:76 – (no change on dev: 20, 20) 20, 20
vim:341 20, 20 20, 20
editor loading+typing, 1440 / 390 – 20, 20 / 20, 20 (max 14.7 s of 30)
editor switching modes, 1440 / 390 – 20, 20 / 20, 20 (max 18.4 s of 30)
lazy-ui ×3, links 60, 60 / 20, 20 60, 60 / 20, 20

Open

  • note-bars-motion.spec.ts:71 Style sheet is not fixed. A first attempt (wait for idle warming before tracing) made it worse (6/40 failed plain) and is reverted, not pushed. The traces say the Style sheet mounts in one click/frame task of ~150–260 ms at 6x against the 250 ms budget, so it is passable only by luck on a shared host. It needs the sheet to mount in smaller pieces (an app change in #850's area), or a decision about the budget; I did not touch either.
  • note-bars.spec.ts:100 and touch-select.spec.ts:312 could not be reproduced (0 failures in 80 and 160 runs); no change made. A single full-suite run is the only place they have shown up.
  • The two WebKit fixes are by analysis (no failure was reproduced to show before/after).
  • Artifacts (logs, JSON per run) stay on the VMs in ~/sf-res; nothing is committed.
## BM14b: repeat-each study of the 10 red cases Every case run `--repeat-each=20 --workers=1 --retries=0 CI=1`, `--no-deps`, train head `6a1749a4` and origin/dev `41f2efa3` **side by side on the same VM** (Chromium on dev-01, WebKit on dev-02), each run with its own `--output`. "plain" = the two side by side; "load" = plus one ordinary `~/bin/slot` Playwright job (editor+links specs) for the whole phase. Host load average from other agents was 4–24 during these runs (the editor cases hit the 30 s limit only when it was high). My own first plain pass on dev-01 was contaminated by my probe runs sharing `test-results`; those cases were rerun clean and only the clean numbers are below. Pass counts, passed/20 (plain, load): | Case (train failure) | origin/dev | train head | Class | Cause | |---|---|---|---|---| | chromium-1440 `glass.spec.ts:76` | 20, 20 | **0, 0** (first pass 4, 0) | **(a) train-only** | `dee3e961` (#850) swapped the fixed `450×4 ms` waits for `still()`. A settled page draws no frames (the caret blink gave most of the old ones): the screencast holds 15–19 frames, the old test 40+, and `frames > 20` fails every time. | | chromium-1440 `vim.spec.ts:341` | 16, 18 | 19, 19 | **(b) flaky on both** | Esc from `?cher` removes the marks and puts the earlier 3 back ~60 ms later (probe `0,3,3,…`); `:noh` sent in that gap clears nothing. Rate: 4/20 and 2/20 on dev, 1/20 and 1/20 on head. | | chromium-390 `note-bars.spec.ts:100` | spec does not exist on dev | 20, 20 | not reproduced | New in #850. Also whole file ×8 on dev-02: 104/104 pass. | | chromium-390 `touch-select.spec.ts:312` | 20, 20 | 20, 20 | not reproduced | Whole file ×8: 128/128 on head and on dev. | | throttled-1440 `editor.spec.ts:57` | 20, 20 | 20, 20 | **(c) load-only, worse on head** | One test did all the work at 6x CPU: open 5.5 s, type 55 keys 6 s, palette toggles 3.5 s + 2.4 s = ~18 s of the 30 s on a quiet host. Head is ~4 s slower than dev (it opens the palette twice, dev clicked a button). | | throttled-390 `editor.spec.ts:57` | 19, 20 (1 timeout) | 16, 20 (4 timeouts) | **(c) load-only, worse on head** | same; dev also timed out once (32.6 s) | | throttled-390 `note-bars-motion.spec.ts:71` Style sheet | spec does not exist on dev | 20, 19 | intermittent, **not fixed** | longest task at 6x is 106–208 ms plain, up to 299 ms under load (budget 250). | | same, Attach menu | – | 20, 20 | not reproduced | 85–172 ms | | webkit-1440 `lazy-ui.spec.ts:70` (3 chunks) | 60, 60 | 60, 60 | not reproduced | `performance.timeOrigin` moved 1 ms with no reload; fix by analysis (below). | | webkit-1440 `links.spec.ts:101` | 20, 20 | 20, 20 | not reproduced | | **The `pkill -u cua -f ms-playwright` on dev-02 (08:00 UTC / 09:00 CET) does not explain the two WebKit failures.** They happened at 07:09:29Z and 07:10:43Z, about 50 minutes earlier, and are assertion failures with the browser alive. None of my dev-02 runs overlap it (first start 09:04Z) and none of my logs have a closed-browser signature. ### Why `dee3e961` did not hold for `editor.spec.ts:57` It made `toggleSource` wait for the dialog and the mode (correct, not faster). The test still spends ~18 s of its 30 s at 6x, so any host load above ~1.7× runs out of time, and the "page closed in `helpers.ts:123`" is just the step that was running at the 30 s mark. The same commit is what broke `glass.spec.ts:76`. ## Fixes (nothing loosened: no timeout, threshold or retry changed) **On #850 (`port/style`, pushed `d252a902`)** - `3744b595` glass: wait for the screencast to hold >24 frames (state) after the caret step, threshold `> 20` untouched. - `d252a902` editor: the throttled test is split into "loading and typing" and "switching modes" (×1 and ×6), each a fresh page; shift check unchanged. **New PR #895 `fix/suite-stability-bm14b` → dev (`3c1d8deb`)**: vim waits for the marks to return and the field to hold focus; lazy-ui marks the window instead of `timeOrigin`; links waits for the server job before the URL check. After the fixes, passed/20 (plain, load): | Case | origin/dev + fix | train head + fix | |---|---|---| | glass:76 | – (no change on dev: 20, 20) | 20, 20 | | vim:341 | 20, 20 | 20, 20 | | editor loading+typing, 1440 / 390 | – | 20, 20 / 20, 20 (max 14.7 s of 30) | | editor switching modes, 1440 / 390 | – | 20, 20 / 20, 20 (max 18.4 s of 30) | | lazy-ui ×3, links | 60, 60 / 20, 20 | 60, 60 / 20, 20 | ### Open - **`note-bars-motion.spec.ts:71` Style sheet is not fixed.** A first attempt (wait for idle warming before tracing) made it worse (6/40 failed plain) and is reverted, not pushed. The traces say the Style sheet mounts in one click/frame task of ~150–260 ms at 6x against the 250 ms budget, so it is passable only by luck on a shared host. It needs the sheet to mount in smaller pieces (an app change in #850's area), or a decision about the budget; I did not touch either. - `note-bars.spec.ts:100` and `touch-select.spec.ts:312` could not be reproduced (0 failures in 80 and 160 runs); no change made. A single full-suite run is the only place they have shown up. - The two WebKit fixes are by analysis (no failure was reproduced to show before/after). - Artifacts (logs, JSON per run) stay on the VMs in `~/sf-res`; nothing is committed.
kayg commented 2026-11-03 13:32:26 +00:00
Author
Owner
Copy link

Style sheet open: task timings before / after (6x CPU, throttled-390, dev-01)

Cause (trace of note-bars-motion.spec.ts:71): the tap opened the sheet in one task of 150–300 ms: Svelte mounted the whole sheet (strip + Page tab) in the tap's frame, Selector asked document.fonts.ready (a getter that forces a style recalc, ~39 ms at 6x), scrollLeft = index * clientWidth forced a layout, and the first pointer press of a page flipped data-last-input, which a * selector depends on (whole-page restyle, ~70 ms at 6x, in the pointerdown listener).

Fix (#850, commits fdddc714, e2146619, 3764d0b9, bcd5ef36, af6fb861):

  1. Selector asks fonts.ready only while a font is still loading.
  2. The sheet opens in steps, each its own frame: (0) a shell held at the Page tab's height (last measured, else a measured guess), so the capsule springs to its size at once; (1) the tab strip and three empty pages, two frames on (a rAF asked for inside the tap still runs in the tap's frame); (2) the shown page. Other tabs unchanged (intent / 2.5 s / on choose). The strip and the page fade in over 140 ms inside the morph's own 170 ms crossfade.
  3. data-last-input is written only when entering or leaving "keyboard" (the only value CSS reads), so the first tap costs no restyle.

Longest main-thread task, rest to Style sheet (there), spec's own trace:

runs min median max pass
before (d252a902) 6 246 ~270 306 1 / 6
after, plain --repeat-each=20 20 72 ~95 163 20 / 20 (Attach 20 / 20)
after, plain, earlier build (e2146619 only) 20 81 ~113 298 19 / 20
after, + one slot job (editor+links), run A 20 77 ~104 263 19 / 20 (one 263 ms outlier)
after, + one slot job, run B 20 77 ~84 126 20 / 20

Probe of one open at 6x (task, ms): click task 239–263 → 75–82; the Playwright pointerdown listener's forced recalc 71 → gone from the page's side; next tasks are the strip frame (~55) and the page frame (~60).
Host load average of other agents was 2–10 during these runs; the 263 outlier was one run with main-thread totals 2x its neighbours (host contention), but I cannot rule out a slow tail of ~1 in 40.

Look: frames at 1x (CDP screencast, 390 phone) before / after: the morph ends in the identical sheet; in the first 2–3 frames of the growth the capsule is empty for ~35–50 ms longer, then the strip and page fade in (no layout jump: the room is held).

Gates (dev-01, ~/bin/slot): bundle budgets perf/web.sh --bundle-only --gate ok (first JS 233.5 / 240 KB brotli); packages/ui test:primitives 55 pass, apps/web test 96 pass; 16 Style/bar/glass/sheet specs on chromium-390 (336 pass, 24 skipped), chromium-1440 (67 pass), throttled-390 (2 pass). apps/web check fails on src/lib/actions.gen.ts is out of date with contracts/actions.json (not touched by this work; present on the base).

Stack: #851 now has 994278bb (merge of the new #850 head; this stack merges port/style rather than rebasing) + d94771b6 (held room measured for its taller Page tab: 362 / 451 px) and #852 has 656b61bf (merge of #851; lazy-ui import count 25 = pull hint + covers). note-bars-motion.spec.ts Style sheet on both: 6 / 6 pass (longest 75–100 and 83–184 ms).

## Style sheet open: task timings before / after (6x CPU, throttled-390, dev-01) **Cause** (trace of `note-bars-motion.spec.ts:71`): the tap opened the sheet in one task of 150–300 ms: Svelte mounted the whole sheet (strip + Page tab) in the tap's frame, `Selector` asked `document.fonts.ready` (a getter that forces a style recalc, ~39 ms at 6x), `scrollLeft = index * clientWidth` forced a layout, and the first pointer press of a page flipped `data-last-input`, which a `*` selector depends on (whole-page restyle, ~70 ms at 6x, in the pointerdown listener). **Fix** (#850, commits fdddc714, e2146619, 3764d0b9, bcd5ef36, af6fb861): 1. `Selector` asks `fonts.ready` only while a font is still loading. 2. The sheet opens in steps, each its own frame: (0) a shell held at the Page tab's height (last measured, else a measured guess), so the capsule springs to its size at once; (1) the tab strip and three empty pages, two frames on (a rAF asked for inside the tap still runs in the tap's frame); (2) the shown page. Other tabs unchanged (intent / 2.5 s / on choose). The strip and the page fade in over 140 ms inside the morph's own 170 ms crossfade. 3. `data-last-input` is written only when entering or leaving "keyboard" (the only value CSS reads), so the first tap costs no restyle. **Longest main-thread task, `rest to Style sheet` (`there`), spec's own trace:** | | runs | min | median | max | pass | |---|---|---|---|---|---| | before (d252a902) | 6 | 246 | ~270 | 306 | **1 / 6** | | after, plain `--repeat-each=20` | 20 | 72 | ~95 | 163 | 20 / 20 (Attach 20 / 20) | | after, plain, earlier build (e2146619 only) | 20 | 81 | ~113 | 298 | 19 / 20 | | after, + one slot job (editor+links), run A | 20 | 77 | ~104 | 263 | **19 / 20** (one 263 ms outlier) | | after, + one slot job, run B | 20 | 77 | ~84 | 126 | 20 / 20 | Probe of one open at 6x (task, ms): click task 239–263 → 75–82; the Playwright pointerdown listener's forced recalc 71 → gone from the page's side; next tasks are the strip frame (~55) and the page frame (~60). Host load average of other agents was 2–10 during these runs; the 263 outlier was one run with main-thread totals 2x its neighbours (host contention), but I cannot rule out a slow tail of ~1 in 40. **Look**: frames at 1x (CDP screencast, 390 phone) before / after: the morph ends in the identical sheet; in the first 2–3 frames of the growth the capsule is empty for ~35–50 ms longer, then the strip and page fade in (no layout jump: the room is held). **Gates** (dev-01, `~/bin/slot`): bundle budgets `perf/web.sh --bundle-only --gate` ok (first JS 233.5 / 240 KB brotli); `packages/ui test:primitives` 55 pass, `apps/web test` 96 pass; 16 Style/bar/glass/sheet specs on chromium-390 (336 pass, 24 skipped), chromium-1440 (67 pass), throttled-390 (2 pass). `apps/web check` fails on `src/lib/actions.gen.ts is out of date with contracts/actions.json` (not touched by this work; present on the base). **Stack**: #851 now has `994278bb` (merge of the new #850 head; this stack merges `port/style` rather than rebasing) + `d94771b6` (held room measured for its taller Page tab: 362 / 451 px) and #852 has `656b61bf` (merge of #851; lazy-ui import count 25 = pull hint + covers). `note-bars-motion.spec.ts` Style sheet on both: 6 / 6 pass (longest 75–100 and 83–184 ms).
kayg commented 2026-11-04 13:05:03 +00:00
Author
Owner
Copy link

Closing: BM14b was a study train; BM14c (#897), BM14d (#907) and BM14e (#914) replaced it. Branch kept.

Closing: BM14b was a study train; BM14c (#897), BM14d (#907) and BM14e (#914) replaced it. Branch kept.
kayg closed this pull request 2026-11-04 13:05:03 +00:00
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!891
No description provided.