kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 190 Pull requests 61 Projects Releases Packages Wiki Activity Actions 26

Merge train BM14e #914

Closed
kayg wants to merge 0 commits from train/2026-11-04-bm14e into dev
pull from: train/2026-11-04-bm14e
merge into: kayg:dev
Conversation 13 Commits - Files changed - +50766 -1318
kayg commented 2026-11-04 08:19:16 +00:00
Owner
Copy link

BM14e. Head 6825355eb929b7ef439e45bd909d8814f6a6a515. Do not merge.

Base note: origin/dev is still 41f2efa3 (BM13). BM14d (#907) was built on it and is not merged, so the brief's "41f2efa3 = c163abae" was wrong. This train is built on BM14d's head c163abae (all nine PRs below) plus #908. Gate baseline ("dev") = c163abae.

PR Head merged Outcome
#850 6eb81f62 via BM14d, clean
#851 d94771b6 via BM14d, clean
#852 07b6770a via BM14d, clean
#829 72eb0eb1 via BM14d, clean
#835 46be23e3 via BM14d, clean
#895 3c1d8deb via BM14d, clean
#905 8973b7e6 via BM14d, clean
#899 8484849c via BM14d, clean
#906 809a33c2 via BM14d, clean
#908 2bb672ff --no-ff, clean

All heads re-verified against Forgejo before merging. Gate results follow in comments.

BM14e. Head `6825355eb929b7ef439e45bd909d8814f6a6a515`. **Do not merge.** Base note: origin/dev is still `41f2efa3` (BM13). BM14d (#907) was built on it and is not merged, so the brief's "41f2efa3 = c163abae" was wrong. This train is built on BM14d's head `c163abae` (all nine PRs below) plus #908. Gate baseline ("dev") = c163abae. | PR | Head merged | Outcome | |---|---|---| | #850 | 6eb81f62 | via BM14d, clean | | #851 | d94771b6 | via BM14d, clean | | #852 | 07b6770a | via BM14d, clean | | #829 | 72eb0eb1 | via BM14d, clean | | #835 | 46be23e3 | via BM14d, clean | | #895 | 3c1d8deb | via BM14d, clean | | #905 | 8973b7e6 | via BM14d, clean | | #899 | 8484849c | via BM14d, clean | | #906 | 809a33c2 | via BM14d, clean | | #908 | 2bb672ff | --no-ff, clean | All heads re-verified against Forgejo before merging. Gate results follow in comments.
kayg added 220 commits 2026-11-04 08:19:16 +00:00 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The room under a phone's Style sheet loads with the idle screens
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
7a6d5adf4c
Presets: the thumbnail's page is not .page; backdrop, opacity and width specs follow the new Style sheet
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
c1ef1b78ed
Merge port/style-2
Some checks failed
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
a0f425ac08 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dee3e961 replaced the fixed 450 ms x4 waits with still(); a settled page
draws no frames, so the screencast held 15-19 frames instead of 40+ and
'frames > 20' failed 16/20 unloaded and 20/20 under load. Wait for the
frame count (state), threshold unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Opening the note (5 s), typing 55 keys (6 s) and two palette round trips
(6 s) at 6x CPU took 18 s of a 30 s test on a quiet machine and hit the
limit whenever the machine was shared. Loading+typing and switching modes
are now separate tests; nothing about the timeout or the shift check changed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
e2e: vim :noh waits for the marks to come back; lazy-ui marks the window; links waits for the job first
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
3c1d8deb19 
- vim.spec '/ searches...': Esc from a half-typed '?cher' removes the marks and puts the
  earlier search's three back about 60 ms later (probe: 0,3,3,...). :noh sent inside that gap
  clears nothing and the marks then return. Wait for the 3 before ':', and for the bar's
  field to hold the focus before typing. Flaked 4/20 on origin/dev, 1/20 on the BM14b head.
- lazy-ui.spec 'failed ... idle warming recovers': detect a reload with a mark on the window,
  not performance.timeOrigin (WebKit derives it from the wall clock; BM14b saw it move by 1 ms).
- links.spec 'Try again': wait for the server's job (15 s poll) before the 5 s URL check.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
yrs 0.28.0 counts the whole merged string again after every squash of two
string items, so a run of appends costs the block so far each time (O(n^2)).
Vendored copy with the one-line fix: add the two lengths.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
test(notes): appends cost the same early and late, UTF-16 length stays exact
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
76759d9da5 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Patch to the merged upstream commit until a release after 0.28.0 ships.
Cargo.lock follows in the next commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Halves of a run are noisy on a loaded machine (3.2x once in a full suite).
A 5 s bound on 6,000 appends separates the quadratic case (over a minute)
from the fixed one (well under a second) with a wide margin.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An owner reading their own note never depends on a grant row. For
every credential except an app password the resource check reduces to
the note-ID size and read-scope checks, so run those directly instead
of a database round trip for each queued frame and each delivered
frame. App passwords keep the full check (revocation is in the db).
Same allow/deny outcome for every caller; non-owner paths untouched.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The owner's own live frames paid a clone, a boxed future and a home
lookup per queued frame and again per written frame. Answer them in
Home and Notes with the same size and scope checks, for non-app-password
owner credentials only; everyone else takes the full path unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A build older than #898 could not deserialize Requirement::ExpectedOwners, so
one such record took a whole Home offline on rollback. Accept the variant in
the reader and validator; admission still refuses it (this build has no
probe). Writes are unchanged. Test reads a real record written by #898.
Deploy doc: roll back past #898 only to a build with this reader.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
style: cargo fmt
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
5f40792840 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
perf(web): leave unused remote-cursor code out of first paint (-983 B brotli)
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
809a33c2b4 
yCollab only uses y-codemirror.next remote selections when it is given an
awareness, and the editor always passes null. The module still shipped.
A build-time empty stand-in drops it; yCollab's own import keeps working.

first-paint JS: brotli 238797 -> 237814 B (-983), raw 845550 -> 841512 B (-4038)
first-paint CSS: unchanged (19250 B brotli)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The upstream pin (26cc6ae3) is 18 commits past 0.28.0 and some change how
unusual input is read (e.g. rejecting malformed updates), which could make a
stored document unopenable. Vendor the release with only the squash length fix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An owner reading their own note never depends on a grant row. For every
credential except an app password the resource check reduces to the
note-ID size and read-scope checks, so run those directly instead of a
database round trip for each delivered frame. App passwords keep the
full check (revocation is in the db). Same allow/deny outcome for every
caller; non-owner paths untouched.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Answer the owner's own connection in Home with the same size and scope
checks, without cloning the caller or awaiting the sharing layer.
Everyone else takes the full path unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The probe stood in for a recipient using the owner. An owner never waits
on a grant lookup, so use a real recipient caller: same recheck, same
assertions, no dependency on the owner path.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The probe stood in for a recipient using the owner. An owner never waits
on a grant lookup, so use a real recipient caller: same recheck, same
assertions, no dependency on the owner path.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
# Conflicts:
#	crates/auth/src/sharing.rs
#	crates/notes/src/home.rs
#	crates/notes/src/room/sync.rs
Opening a note for its owner made two sharing lookups (read, then edit)
plus the delivery check, for a person who needs none: the owner's access
is fixed by the credential. Answer owner Read and Edit with the same size
and scope checks and an Owner result, for session and development
credentials; app passwords (revocable), recipients and every other
operation take the full path unchanged. The sequence is passed through as
given; owner delivery never reads it.

Access logic touched: owner Read/Edit resolution only; outcomes are the
same allow/deny set.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Opening a note for its owner made two sharing lookups (read, then edit)
plus the delivery check, for a person who needs none: the owner's access
is fixed by the credential. Answer owner Read and Edit with the same size
and scope checks and an Owner result, for session and development
credentials; app passwords (revocable), recipients and every other
operation take the full path unchanged. The sequence is passed through as
given; owner delivery never reads it.

Access logic touched: owner Read/Edit resolution only; outcomes are the
same allow/deny set.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
note-bars.css held every bar state's rules and was imported by main.ts, so the
first stylesheet carried the block bar, the typing strip, the kind chip and the
strip fades (5.4 KB of 8.5 KB) before anyone selects a block or types. Move
those to note-bars-lazy.css, imported by BlockBar and Strip (lazy, in the
idle-screens chunk; Vite resolves the import after its CSS, so there is no
unstyled frame). The rest bar's rules and the pill radius stay eager.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Merge #906 (perf/first-paint-diet @ 809a33c2) into train BM14d
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
c163abaea6 
The rest bar uses none of them; the block bar and the strip (which load note-bars-lazy.css) do.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
No component, test or script names these classes any more (the two-line phone bar they drew was replaced by the note bars).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
perf(web): load the @ / [[ / # picker list's look after the first paint
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
2bb672ff3b 
The list only shows once you type a trigger. Its stylesheet is warmed when the browser is idle and the picker's first lookup waits for it, so the list never shows unstyled.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Merge PR #908 (perf/first-css-diet) into train BM14e
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
6825355eb9
kayg commented 2026-11-04 11:58:16 +00:00
Author
Owner
Copy link

BM14e status (12:55 CEST, in progress)

Head 6825355eb929b7ef439e45bd909d8814f6a6a515. Base note: origin/dev is still 41f2efa3 (BM13) and does not contain BM14d (#907, unmerged). The brief's "41f2efa3 = c163abae" is wrong, so the train is c163abae (BM14d head, all nine PRs) + #908, and the gate baseline ("dev") is c163abae.

Gates (first run, CI=1, --retries=0): install, fmt, clippy -D warnings, cargo test --workspace -j4 (1497 passed, 0 failed), cargo build, web build, perf/web.sh --bundle-only --gate, check-case-paths, check-no-evidence: all pass.

Bundle (brotli q11, bundle-size.mjs, exact bytes), c163abae vs train: first CSS 19637 → 18998 (-639 B; vs dev 41f2efa3 19250: -252). First JS 238416 → 238420 (+4 B, raw +317 B: the picker lazy-load hook). By the brief's "any growth = RED" this is growth. It is 4 bytes, and against real dev 41f2efa3 first JS is smaller (233.2 → 232.83 KB). Total JS 363.406 → 363.333 KB.

Playwright matrix, first run

Half Project Passed Failed
dev-01 chromium chromium-1440 443 0
chromium-390 899 0
admin-chromium-1440 / -390, admin-setup, flow, passkeys 12 / 12 / 1 / 1 / 1 0
contrast, look-chromium-1440 / -390, prefs-chromium, sidebar-band-chromium 9 / 5 / 5 / 7 / 3 0
throttled-1440 2 1 (editor.spec.ts:109 no layout shift while switching modes ×6)
throttled-390 did not run (dependency); first attempt --no-deps: 5 passed 0
dev-02 webkit webkit-1440 435 5
webkit-390 853 6
look-webkit-1440 3 1
admin-setup, admin-webkit-1440 / -390, prefs-webkit, sidebar-band-webkit 1 / 9 / 9 / 3 / 3 0
look-webkit-390 did not run (dependency on look-webkit-1440) rerun pending

WebKit first-run failures (12): webkit-1440 cover.spec.ts:112, style-sheet.spec.ts:26, swipe-rows.spec.ts:365, switching.spec.ts:102, toast-bar.spec.ts:40; webkit-390 home-screen.spec.ts:70 (beforeAll timeout), list-search.spec.ts:37, note-bars-fit.spec.ts:67, safe-areas.spec.ts:117, swipe-rows.spec.ts:365, trash-search.spec.ts:14; look-webkit-1440 backdrop-page.spec.ts:34. Most are 5 s or 30 s timeouts.

x10 results so far (--no-deps --repeat-each=10)

  • editor.spec.ts:109 throttled-1440: train 10/10 pass, dev (c163abae) 10/10 pass; whole file editor.spec.ts x10 on throttled-1440: train and dev both exit 0. Does not reproduce, so it does not block (stability issue to open).
  • The 12 WebKit cases: not run yet. My first attempt on dev-02 ran without BM14E_HALF=webkit and selected Chromium projects, so those runs errored (no result). Relaunched at 12:55 with the right config on train and on dev (c163abae baseline built on dev-02). Results will follow in the next comment.

Infra

dev-01 train lock released at 12:51 (nothing more needs dev-01 for now). dev-02 lock kept while the WebKit x10 runs. Perf: both perf locks held since 11:52 (a ~50 min control-host network stall cost me part of the window); server arms (dev / train / dev2 control, rotating) running on perf-01, browser arms + picker first-open next.

## BM14e status (12:55 CEST, in progress) Head `6825355eb929b7ef439e45bd909d8814f6a6a515`. **Base note:** origin/dev is still `41f2efa3` (BM13) and does not contain BM14d (#907, unmerged). The brief's "41f2efa3 = c163abae" is wrong, so the train is c163abae (BM14d head, all nine PRs) + #908, and the gate baseline ("dev") is c163abae. **Gates (first run, `CI=1`, `--retries=0`):** install, fmt, clippy `-D warnings`, `cargo test --workspace -j4` (1497 passed, 0 failed), cargo build, web build, `perf/web.sh --bundle-only --gate`, check-case-paths, check-no-evidence: all pass. **Bundle (brotli q11, `bundle-size.mjs`, exact bytes), c163abae vs train:** first CSS 19637 → **18998** (-639 B; vs dev 41f2efa3 19250: -252). First JS 238416 → **238420 (+4 B**, raw +317 B: the picker lazy-load hook). By the brief's "any growth = RED" this is growth. It is 4 bytes, and against real dev 41f2efa3 first JS is smaller (233.2 → 232.83 KB). Total JS 363.406 → 363.333 KB. ### Playwright matrix, first run | Half | Project | Passed | Failed | |---|---|---:|---:| | dev-01 chromium | chromium-1440 | 443 | 0 | | | chromium-390 | 899 | 0 | | | admin-chromium-1440 / -390, admin-setup, flow, passkeys | 12 / 12 / 1 / 1 / 1 | 0 | | | contrast, look-chromium-1440 / -390, prefs-chromium, sidebar-band-chromium | 9 / 5 / 5 / 7 / 3 | 0 | | | throttled-1440 | 2 | **1** (`editor.spec.ts:109` no layout shift while switching modes ×6) | | | throttled-390 | did not run (dependency); first attempt `--no-deps`: **5 passed** | 0 | | dev-02 webkit | webkit-1440 | 435 | **5** | | | webkit-390 | 853 | **6** | | | look-webkit-1440 | 3 | **1** | | | admin-setup, admin-webkit-1440 / -390, prefs-webkit, sidebar-band-webkit | 1 / 9 / 9 / 3 / 3 | 0 | | | look-webkit-390 | did not run (dependency on look-webkit-1440) | rerun pending | WebKit first-run failures (12): webkit-1440 `cover.spec.ts:112`, `style-sheet.spec.ts:26`, `swipe-rows.spec.ts:365`, `switching.spec.ts:102`, `toast-bar.spec.ts:40`; webkit-390 `home-screen.spec.ts:70` (beforeAll timeout), `list-search.spec.ts:37`, `note-bars-fit.spec.ts:67`, `safe-areas.spec.ts:117`, `swipe-rows.spec.ts:365`, `trash-search.spec.ts:14`; look-webkit-1440 `backdrop-page.spec.ts:34`. Most are 5 s or 30 s timeouts. ### x10 results so far (`--no-deps --repeat-each=10`) - `editor.spec.ts:109` throttled-1440: **train 10/10 pass, dev (c163abae) 10/10 pass**; whole file `editor.spec.ts` x10 on throttled-1440: train and dev both exit 0. Does not reproduce, so it does not block (stability issue to open). - The 12 WebKit cases: **not run yet**. My first attempt on dev-02 ran without `BM14E_HALF=webkit` and selected Chromium projects, so those runs errored (no result). Relaunched at 12:55 with the right config on train and on dev (c163abae baseline built on dev-02). Results will follow in the next comment. ### Infra dev-01 train lock released at 12:51 (nothing more needs dev-01 for now). dev-02 lock kept while the WebKit x10 runs. Perf: both perf locks held since 11:52 (a ~50 min control-host network stall cost me part of the window); server arms (dev / train / dev2 control, rotating) running on perf-01, browser arms + picker first-open next.
kayg commented 2026-11-04 13:41:11 +00:00
Author
Owner
Copy link

BM14e: WebKit and Chromium x10 reruns, bundle vs real dev

Head 6825355eb929b7ef439e45bd909d8814f6a6a515. "dev" in the x10 columns = c163abae (BM14d head) built the same way on the same VM (origin/dev 41f2efa3 does not contain BM14d). All runs --no-deps --repeat-each=10, CI=1, retries 0, --workers=1, dev-01 (Chromium) and dev-02 (WebKit). Failure counts are failed runs / runs.

First-run failure project train x10 c163abae x10 ABAB x3 (interleaved train / dev, 30 runs each) train total / dev total class
editor.spec.ts:109 switching modes x6 throttled-1440 (dev-01) 0/10 0/10 (whole editor.spec.ts x10: 0 fails both) 0 / 0 not reproduced
cover.spec.ts:112 webkit-1440 1/10 0/10 0 / 1 1/40 / 1/40 flake on both
style-sheet.spec.ts:26 webkit-1440 0/10 0/10 - 0 / 0 not reproduced
swipe-rows.spec.ts:365 webkit-1440 0/10 0/10 - 0 / 0 not reproduced
switching.spec.ts:102 webkit-1440 1/10 0/10 1 / 0 2/40 / 0/40 train-only, 2 timeouts (30 s)
toast-bar.spec.ts:40 webkit-1440 3/10 0/10 2 / 1 5/40 / 1/40 flake on both, higher on train (Fisher p about 0.2)
backdrop-page.spec.ts:34 look-webkit-1440 1/10 2/10 - 1/10 / 2/10 flake on both
home-screen.spec.ts:70 (24 params, shared beforeAll) webkit-390 1 beforeAll timeout (30 s) in 10 0 - 1 / 0 train-only, single
list-search.spec.ts:37 webkit-390 0/10 0/10 - 0 / 0 not reproduced
note-bars-fit.spec.ts:67 Style sheet fits every tab webkit-390 22/60 20/60 - 22/60 / 20/60 fails about 1 in 3 on both (pre-existing on c163abae)
safe-areas.spec.ts:117 webkit-390 1/40 (portrait light, 30 s) 0/40 - 1 / 0 train-only, single
swipe-rows.spec.ts:365 webkit-390 0/10 0/10 - 0 / 0 not reproduced
trash-search.spec.ts:14 (row) webkit-390 2/10 1/10 - 2 / 1 flake on both
look-webkit-390 (first attempt, --no-deps) look-webkit-390 4 passed, 1 skipped, 0 failed
throttled-390 (first attempt, --no-deps) throttled-390 5 passed

Reading it plainly:

  • Not reproduced on either side: editor:109, style-sheet:26, swipe-rows:365 (both widths), list-search:37.
  • Reproduces on both train and c163abae at similar rates, so it is not from #908: note-bars-fit:67 (about 35%), trash-search:14 row, backdrop-page:34, cover:112, toast-bar:40 (train 5/40 vs dev 1/40).
  • Train only, not seen on c163abae: switching:102 2/40, home-screen:70 1 beforeAll, safe-areas:117 1/40. All are 30 s timeouts on dev-02 with load average about 9 during the x10 runs, and each recurred at most twice.
  • By the flaky rule as written ("a failure that reproduces always blocks"), these reproducing cases mean the WebKit half is not clean. None is attributable to #908 alone: #908 is CSS loading for the @ [[ # picker, and the same specs fail on the BM14d head. Stability issues to open: note-bars-fit:67 (WebKit 390 timeouts), toast-bar:40, switching:102.

Bundle vs real origin/dev 41f2efa3 (what prod runs) and vs c163abae

Exact brotli q11 bytes from scripts/bundle-size.mjs --json:

build first JS first CSS
dev 41f2efa3 (BM14d report, 233.200 / 18.799 KiB) about 238 797 19 250
c163abae (BM14d head) 238 416 19 637
train 6825355e 238 420 18 998
Train vs dev 41f2efa3: first JS about -377 B, first CSS -252 B. Train vs c163abae: first JS +4 B (raw +317 B), first CSS -639 B. Total JS 363.333 KB (c163abae 363.406). All inside budgets (--gate pass). No growth against real dev.
## BM14e: WebKit and Chromium x10 reruns, bundle vs real dev Head `6825355eb929b7ef439e45bd909d8814f6a6a515`. "dev" in the x10 columns = c163abae (BM14d head) built the same way on the same VM (origin/dev `41f2efa3` does not contain BM14d). All runs `--no-deps --repeat-each=10`, `CI=1`, retries 0, `--workers=1`, dev-01 (Chromium) and dev-02 (WebKit). Failure counts are failed runs / runs. | First-run failure | project | train x10 | c163abae x10 | ABAB x3 (interleaved train / dev, 30 runs each) | train total / dev total | class | |---|---|---:|---:|---|---|---| | `editor.spec.ts:109` switching modes x6 | throttled-1440 (dev-01) | 0/10 | 0/10 | (whole `editor.spec.ts` x10: 0 fails both) | 0 / 0 | not reproduced | | `cover.spec.ts:112` | webkit-1440 | 1/10 | 0/10 | 0 / 1 | 1/40 / 1/40 | flake on both | | `style-sheet.spec.ts:26` | webkit-1440 | 0/10 | 0/10 | - | 0 / 0 | not reproduced | | `swipe-rows.spec.ts:365` | webkit-1440 | 0/10 | 0/10 | - | 0 / 0 | not reproduced | | `switching.spec.ts:102` | webkit-1440 | 1/10 | 0/10 | 1 / 0 | **2/40 / 0/40** | train-only, 2 timeouts (30 s) | | `toast-bar.spec.ts:40` | webkit-1440 | 3/10 | 0/10 | 2 / 1 | **5/40 / 1/40** | flake on both, higher on train (Fisher p about 0.2) | | `backdrop-page.spec.ts:34` | look-webkit-1440 | 1/10 | 2/10 | - | 1/10 / 2/10 | flake on both | | `home-screen.spec.ts:70` (24 params, shared beforeAll) | webkit-390 | 1 beforeAll timeout (30 s) in 10 | 0 | - | 1 / 0 | train-only, single | | `list-search.spec.ts:37` | webkit-390 | 0/10 | 0/10 | - | 0 / 0 | not reproduced | | `note-bars-fit.spec.ts:67` Style sheet fits every tab | webkit-390 | 22/60 | 20/60 | - | 22/60 / 20/60 | **fails about 1 in 3 on both** (pre-existing on c163abae) | | `safe-areas.spec.ts:117` | webkit-390 | 1/40 (portrait light, 30 s) | 0/40 | - | 1 / 0 | train-only, single | | `swipe-rows.spec.ts:365` | webkit-390 | 0/10 | 0/10 | - | 0 / 0 | not reproduced | | `trash-search.spec.ts:14` (row) | webkit-390 | 2/10 | 1/10 | - | 2 / 1 | flake on both | | look-webkit-390 (first attempt, `--no-deps`) | look-webkit-390 | 4 passed, 1 skipped, 0 failed | | | | | | throttled-390 (first attempt, `--no-deps`) | throttled-390 | 5 passed | | | | | Reading it plainly: - Not reproduced on either side: `editor:109`, `style-sheet:26`, `swipe-rows:365` (both widths), `list-search:37`. - Reproduces on **both** train and c163abae at similar rates, so it is not from #908: `note-bars-fit:67` (about 35%), `trash-search:14` row, `backdrop-page:34`, `cover:112`, `toast-bar:40` (train 5/40 vs dev 1/40). - Train only, not seen on c163abae: `switching:102` 2/40, `home-screen:70` 1 beforeAll, `safe-areas:117` 1/40. All are 30 s timeouts on dev-02 with load average about 9 during the x10 runs, and each recurred at most twice. - By the flaky rule as written ("a failure that reproduces always blocks"), these reproducing cases mean the WebKit half is not clean. None is attributable to #908 alone: #908 is CSS loading for the `@ [[ #` picker, and the same specs fail on the BM14d head. Stability issues to open: `note-bars-fit:67` (WebKit 390 timeouts), `toast-bar:40`, `switching:102`. ## Bundle vs real origin/dev `41f2efa3` (what prod runs) and vs c163abae Exact brotli q11 bytes from `scripts/bundle-size.mjs --json`: | build | first JS | first CSS | |---|---:|---:| | dev `41f2efa3` (BM14d report, 233.200 / 18.799 KiB) | about 238 797 | 19 250 | | c163abae (BM14d head) | 238 416 | 19 637 | | **train 6825355e** | **238 420** | **18 998** | Train vs dev 41f2efa3: first JS about **-377 B**, first CSS **-252 B**. Train vs c163abae: first JS **+4 B** (raw +317 B), first CSS **-639 B**. Total JS 363.333 KB (c163abae 363.406). All inside budgets (`--gate` pass). No growth against real dev.
kayg commented 2026-11-04 14:24:40 +00:00
Author
Owner
Copy link

BM14e perf: train 6825355e vs c163abae (BM14d head), ABAB + dev-vs-dev control

perf-01, ~/bin/perf-lock held on perf-01 and perf-02 (perf-01 first, then perf-02) in two consecutive holder units (11:52-13:49 and 13:49-stopped), both units now stopped on both VMs. Release binaries built on dev-01 and relayed, sha256 checked: dev calternal 16b2785c…, train 30f2ac9d…, control = byte copy of dev (16b2785c…); one calternal-perf (55322a58…, from dev) drives every arm. Every round runs all three arms (dev, train, ctl) in rotating order. Seed 1. "dev" here is c163abae because real origin/dev 41f2efa3 does not contain BM14d; BM14d's train-vs-41f2efa3 results stand for the rest. #908 only changes web CSS/JS loading, so the server binary logic is unchanged.

Server (calternal-perf --sizes 10k,100k --runs 5 --cold-runs 0 --idle-secs 0 --storm-edits 1; 6 rounds, round 1 builds the fixtures and is not counted, so 5 interleaved rounds)

Values = median over the 5 rounds of the per-round p50 / p95 (ms; mem in MB). "<" columns = in how many of the 5 rounds the arm's p50 / p95 was below dev's.

metric notes dev p50/p95 train p50/p95 ctl p50/p95 train vs dev p50/p95 ctl vs dev p50/p95 train<dev p50 (of 5) ctl<dev p50 train<dev p95 ctl<dev p95
home.first_page 10000 2.256 / 4.296 2.308 / 6.747 2.405 / 5.322 +2% / +57% +7% / +24% 3 4 1 2
home.first_page 100000 3.76 / 7.889 3.796 / 6.472 4.078 / 4.925 +1% / -18% +8% / -38% 2 2 3 4
home.page 10000 0.7928 / 1.452 0.75 / 1.73 0.754 / 1.205 -5% / +19% -5% / -17% 4 4 2 3
home.page 100000 0.9924 / 1.505 1.023 / 1.633 0.9579 / 1.451 +3% / +9% -3% / -4% 2 3 2 3
home.page_big 10000 16.27 / 19.72 16.6 / 19.52 16.64 / 20.2 +2% / -1% +2% / +2% 2 2 2 1
home.page_big 100000 24.01 / 35.4 19.23 / 29.98 22.57 / 32.19 -20% / -15% -6% / -9% 4 4 4 3
home.page_next 10000 0.7825 / 1.107 0.7739 / 1.083 0.7149 / 0.9996 -1% / -2% -9% / -10% 3 4 3 3
home.page_next 100000 0.7872 / 1.142 0.7635 / 1.139 0.7935 / 1.23 -3% / -0% +1% / +8% 5 4 3 1
home.page_tag 10000 5.081 / 6.943 5.1 / 6.607 5.093 / 6.377 +0% / -5% +0% / -8% 1 2 3 3
home.page_tag 100000 56.71 / 62.05 55.08 / 60.52 56.3 / 60.32 -3% / -2% -1% / -3% 4 3 3 4
home.page_title 10000 0.6363 / 0.7947 0.6656 / 0.8713 0.6186 / 0.8316 +5% / +10% -3% / +5% 2 4 2 3
home.page_title 100000 0.884 / 1.309 0.8895 / 1.313 0.8804 / 1.151 +1% / +0% -0% / -12% 3 2 2 3
journal.catch_up 10000 28.83 / 33.65 28.37 / 30.94 28.51 / 33.2 -2% / -8% -1% / -1% 3 3 3 3
journal.catch_up 100000 320.3 / 352.3 295.4 / 313.4 312.8 / 340.3 -8% / -11% -2% / -3% 4 3 4 3
journal.entries 10000 1.035e+04 / 1.07e+04 1.035e+04 / 1.07e+04 1.035e+04 / 1.07e+04 +0% / +0% +0% / +0% 1 1 1 1
journal.entries 100000 1.004e+05 / 1.007e+05 1.004e+05 / 1.007e+05 1.004e+05 / 1.007e+05 -0% / -0% -0% / -0% 4 5 4 5
journal.page 10000 2.417 / 2.642 2.381 / 2.723 2.392 / 2.608 -1% / +3% -1% / -1% 4 3 2 3
journal.page 100000 2.548 / 4.223 2.483 / 3.159 2.616 / 3.677 -3% / -25% +3% / -13% 3 3 4 3
mem.after_first_page 10000 21.41 / 21.5 21.44 / 22.07 21.46 / 22.35 +0% / +3% +0% / +4% 2 2 0 2
mem.after_first_page 100000 22.67 / 23.61 22.82 / 24.11 22.77 / 23.22 +1% / +2% +0% / -2% 3 2 2 4
mem.after_opens 10000 40.1 / 42.48 40.8 / 42.54 41.05 / 42.9 +2% / +0% +2% / +1% 2 2 2 3
mem.after_opens 100000 96.69 / 100.4 95.71 / 101.6 98.66 / 102.8 -1% / +1% +2% / +2% 4 2 1 2
mem.after_search 10000 45.57 / 75.66 46.96 / 75.38 47.59 / 74.87 +3% / -0% +4% / -1% 2 1 2 4
mem.after_search 100000 109.1 / 119.4 106.1 / 113.9 108.7 / 119 -3% / -5% -0% / -0% 4 3 3 2
mem.idle 10000 18.71 / 19.33 18.88 / 19.41 19.24 / 20.1 +1% / +0% +3% / +4% 2 0 1 0
mem.idle 100000 18.4 / 20.45 18.75 / 20.51 19.34 / 20.73 +2% / +0% +5% / +1% 0 1 2 2
mem.peak 10000 45.21 / 75.46 46.39 / 74.92 47.24 / 74.41 +3% / -1% +4% / -1% 2 1 2 2
mem.peak 100000 127.1 / 129.9 124.9 / 126.8 124.6 / 133.6 -2% / -2% -2% / +3% 3 3 4 2
open.api_get 10000 3.361 / 12.28 3.326 / 10.78 3.478 / 12.69 -1% / -12% +3% / +3% 2 2 3 2
open.api_get 100000 5.099 / 15.37 4.241 / 12.29 4.447 / 10.03 -17% / -20% -13% / -35% 4 3 4 5
open.long_note 10000 45.01 / 53.8 47.21 / 54.9 49.13 / 55.52 +5% / +2% +9% / +3% 3 1 3 2
open.long_note 100000 80.59 / 101.1 66.45 / 83.6 80.82 / 95.96 -18% / -17% +0% / -5% 4 2 3 3
open.note 10000 0.754 / 1.445 0.7766 / 1.435 0.7398 / 1.415 +3% / -1% -2% / -2% 2 1 3 4
open.note 100000 4.704 / 8.07 2.36 / 7.493 4.231 / 7.404 -50% / -7% -10% / -8% 4 3 4 3
search.catch_up 10000 2.005 / 4.011 2.006 / 4.01 2.005 / 4.016 +0% / -0% -0% / +0% 2 3 2 2
search.catch_up 100000 2.012 / 58.12 2.007 / 58.14 2.02 / 56.15 -0% / +0% +0% / -3% 4 2 2 2
search.first_query 10000 4.659 / 23.47 5.276 / 35.34 5.022 / 38.78 +13% / +51% +8% / +65% 1 3 0 1
search.first_query 100000 6.555 / 29.12 6.309 / 24.01 7.082 / 23.7 -4% / -18% +8% / -19% 3 3 1 2
search.query 10000 1.66 / 4.144 1.554 / 4.166 1.588 / 3.927 -6% / +1% -4% / -5% 5 4 3 4
search.query 100000 1.918 / 4.892 1.731 / 4.218 1.873 / 4.516 -10% / -14% -2% / -8% 5 4 4 3
shutdown 10000 64.01 / 64.37 64.02 / 64.78 64.1 / 85.18 +0% / +1% +0% / +32% 1 1 2 1
shutdown 100000 85.12 / 85.29 84.25 / 88.37 85.19 / 85.64 -1% / +4% +0% / +0% 5 3 3 2
startup.healthz 10000 5.115 / 7.226 4.926 / 8.07 4.985 / 8.21 -4% / +12% -3% / +14% 4 3 2 2
startup.healthz 100000 6.617 / 7.549 5.808 / 7.213 6.215 / 8.016 -12% / -4% -6% / +6% 2 2 3 1
storm.converge 10000 171 / 172.3 171.2 / 172.3 170.8 / 172.2 +0% / +0% -0% / -0% 0 2 2 2
storm.converge 100000 171.9 / 175.2 170.7 / 176 167.2 / 171.8 -1% / +0% -3% / -2% 4 3 3 4
storm.disk_written 10000 0.5366 / 0.7127 0.5407 / 2.535 0.5366 / 0.8438 +1% / +256% +0% / +18% 1 2 1 2
storm.disk_written 100000 0.6021 / 3.953 0.5939 / 0.9871 0.5816 / 0.9544 -1% / -75% -3% / -76% 2 3 5 4
storm.dropped_clients 10000 0 / 0 0 / 0 0 / 0 +0% / +0% +0% / +0% 0 0 0 0
storm.dropped_clients 100000 0 / 0 0 / 0 0 / 0 +0% / +0% +0% / +0% 0 0 0 0
storm.edits_per_s 10000 0 / 0 0 / 0 0 / 0 +0% / +0% +0% / +0% 0 0 0 0
storm.edits_per_s 100000 0 / 0 0 / 0 0 / 0 +0% / +0% +0% / +0% 0 0 0 0
storm.outside_edits 10000 1 / 1 1 / 1 1 / 1 +0% / +0% +0% / +0% 0 0 0 0
storm.outside_edits 100000 1 / 1 1 / 1 1 / 1 +0% / +0% +0% / +0% 0 0 0 0
storm.typing 10000 2.634e-05 / 3.756e-05 2.709e-05 / 3.098e-05 2.507e-05 / 3.245e-05 +3% / -18% -5% / -14% 3 3 3 4
storm.typing 100000 2.589e-05 / 3.269e-05 2.829e-05 / 3.813e-05 2.864e-05 / 3.352e-05 +9% / +17% +11% / +3% 1 1 1 3
sync.durable_write 10000 19.42 / 23.6 20.56 / 27.98 20.73 / 26.2 +6% / +19% +7% / +11% 2 1 1 1
sync.durable_write 100000 23.19 / 35.38 20.35 / 30.18 23.53 / 35.26 -12% / -15% +1% / -0% 4 4 4 3
sync.round_trip 10000 0.2146 / 0.3339 0.2165 / 0.3451 0.2097 / 0.3387 +1% / +3% -2% / +1% 2 3 2 1
sync.round_trip 100000 0.2354 / 0.35 0.2335 / 0.336 0.2153 / 0.3428 -1% / -4% -9% / -2% 4 5 4 3
sync.save_ack 10000 2.981 / 4.225 3.163 / 3.844 3.124 / 4.355 +6% / -9% +5% / +3% 1 0 3 2
sync.save_ack 100000 3.295 / 4.755 3.231 / 4.14 3.273 / 4.575 -2% / -13% -1% / -4% 5 4 3 3
trash.delete 10000 51.42 / 80.82 52.89 / 92.54 54.92 / 80.27 +3% / +14% +7% / -1% 3 2 1 2
trash.delete 100000 99.53 / 217 75.49 / 115.4 99.81 / 226.4 -24% / -47% +0% / +4% 4 4 4 2
trash.forget 10000 7.642 / 13.99 9.544 / 13.78 9.568 / 12.69 +25% / -1% +25% / -9% 1 2 3 3
trash.forget 100000 10.81 / 18.4 10.11 / 14.81 10.76 / 16.34 -7% / -20% -1% / -11% 4 3 4 3
trash.restore 10000 61.71 / 111.5 62.84 / 114.1 63.43 / 113.6 +2% / +2% +3% / +2% 1 2 1 1
trash.restore 100000 67.45 / 127.4 63.56 / 105.9 66.35 / 115.5 -6% / -17% -2% / -9% 5 2 3 2
uploads.response 10000 109.5 / 146.5 142.1 / 159.2 143.3 / 158.3 +30% / +9% +31% / +8% 2 1 1 1
uploads.response 100000 152 / 174.1 142.8 / 165.3 150.9 / 178 -6% / -5% -1% / +2% 4 4 5 1
uploads.round_trip 10000 0.2618 / 0.4115 0.2592 / 0.4304 0.2593 / 0.4111 -1% / +5% -1% / -0% 4 3 2 2
uploads.round_trip 100000 0.2804 / 0.4412 0.2796 / 0.4335 0.2792 / 0.4398 -0% / -2% -0% / -0% 3 4 2 3
uploads.save_ack 10000 2.877 / 7.144 3.325 / 9.694 3.386 / 8.847 +16% / +36% +18% / +24% 2 2 1 1
uploads.save_ack 100000 3.538 / 10.08 3.461 / 8.875 3.601 / 10.69 -2% / -12% +2% / +6% 4 3 5 2

Reading: sync.round_trip (10k +1%/+3%, 100k -1%/-4%), open.note (10k +3%/-1%; 100k p50 is bimodal in dev itself: 4.7 vs control 4.2 vs train 2.4), search.query, sync.save_ack, durable_write and memory are flat or better, and everything that moves for train also moves for the control. search.first_query: 100k -4% p50 / -18% p95 (train below dev in 3/5 p50 rounds); 10k +13% p50 / +51% p95, but the same-binary control is +8% / +65% in those rounds (dev's own spread is wide), so it is noise, not the picker CSS.

Browser (Chromium 4x CPU slowdown, 10k Home, boot / list-search / blaze / typing perf specs + picker first-open probe; 5 rounds, each arm served by its own binary and dist)

metric dev p50/p95 train p50/p95 ctl p50/p95 train vs dev p50/p95 ctl vs dev p50/p95 train<dev p50 ctl<dev p50 rounds
blaze.back_missed 1 / 1 1 / 1 1 / 1 +0.0% / +0.0% +0.0% / +0.0% 0/5 0/5 5
blaze.back_shown 31.6 / 47.3 31.8 / 50.8 31.3 / 47.5 +0.6% / +7.4% -0.9% / +0.4% 1/5 3/5 5
blaze.cls 0 / 0 0 / 0 0 / 0 +0.0% / +0.0% +0.0% / +0.0% 0/5 0/5 5
blaze.cpu 69.48 / 69.48 71.33 / 71.33 70.26 / 70.26 +2.7% / +2.7% +1.1% / +1.1% 0/5 2/5 5
blaze.dom_growth 1 / 1 1 / 1 1 / 1 +0.0% / +0.0% +0.0% / +0.0% 0/5 1/5 5
blaze.heap_growth 1.87 / 1.87 1.86 / 1.86 1.88 / 1.88 -0.5% / -0.5% +0.5% / +0.5% 4/5 3/5 5
blaze.live 105 / 105 107 / 107 90 / 90 +1.9% / +1.9% -14.3% / -14.3% 2/5 3/5 5
blaze.long_task_max 138 / 138 150 / 150 148 / 148 +8.7% / +8.7% +7.2% / +7.2% 0/5 2/5 5
blaze.long_tasks 7 / 7 9 / 9 8 / 8 +28.6% / +28.6% +14.3% / +14.3% 1/5 2/5 5
blaze.missed 0 / 0 0 / 0 0 / 0 +0.0% / +0.0% +0.0% / +0.0% 1/5 1/5 5
blaze.requests 52 / 52 52 / 52 52 / 52 +0.0% / +0.0% +0.0% / +0.0% 0/5 0/5 5
blaze.shown 37.8 / 60.5 39.8 / 66 36.6 / 59.6 +5.3% / +9.1% -3.2% / -1.5% 2/5 4/5 5
boot.cls 0 / 0.01 0 / 0.01 0 / 0.01 +0.0% / +0.0% +0.0% / +0.0% 0/5 0/5 5
boot.cold_blocked 434 / 530 434 / 495 452 / 513 +0.0% / -6.6% +4.1% / -3.2% 2/5 3/5 5
boot.cold_cpu 1285 / 1418 1310 / 1311 1311 / 1400 +2.0% / -7.5% +2.0% / -1.3% 1/5 2/5 5
boot.cold_fcp 720 / 780 716 / 768 728 / 752 -0.6% / -1.5% +1.1% / -3.6% 1/5 1/5 5
boot.cold_list 885.6 / 980.4 887.2 / 924 851.9 / 953.5 +0.2% / -5.8% -3.8% / -2.7% 2/5 3/5 5
boot.cold_note 885.5 / 980.4 907.5 / 943.9 910.6 / 968 +2.5% / -3.7% +2.8% / -1.3% 2/5 1/5 5
boot.cold_tbt 110.2 / 160.8 110.9 / 143 98 / 142 +0.6% / -11.1% -11.1% / -11.7% 1/5 3/5 5
boot.cold_tti 1755 / 1830 1765 / 1809 1788 / 1815 +0.5% / -1.1% +1.9% / -0.8% 2/5 1/5 5
boot.heap 11.91 / 11.91 11.92 / 11.92 11.91 / 11.91 +0.1% / +0.1% +0.0% / +0.0% 0/5 0/5 5
boot.warm_list 297.7 / 592.5 284.3 / 575.3 297.6 / 577.7 -4.5% / -2.9% -0.0% / -2.5% 4/5 3/5 5
boot.warm_note 457.6 / 810.3 432.7 / 782.9 439.3 / 787.5 -5.4% / -3.4% -4.0% / -2.8% 4/5 2/5 5
boot.warm_tbt 5 / 22 0 / 22 4 / 28 -100.0% / +0.0% -20.0% / +27.3% 3/5 3/5 5
boot.warm_tti 1155 / 1698 464.7 / 1720 1141 / 1709 -59.8% / +1.3% -1.2% / +0.6% 3/5 2/5 5
list_search.blank_frames@10000 0 / 0 0 / 0 0 / 0 +0.0% / +0.0% +0.0% / +0.0% 0/5 0/5 5
list_search.key@10000 5.9 / 7.8 5.8 / 7.5 5.9 / 8.2 -1.7% / -3.8% +0.0% / +5.1% 3/5 3/5 5
list_search.warmup_key@10000 6.1 / 30.8 5.8 / 22.9 6.9 / 25.5 -4.9% / -25.6% +13.1% / -17.2% 3/5 1/5 5
picker.at.first_open_dom 315.3 / 332.4 301.1 / 320.8 291.6 / 311 -4.5% / -3.5% -7.5% / -6.4% 3/5 4/5 5
picker.at.first_open_painted 361.9 / 426.5 365.6 / 428 332.1 / 416.6 +1.0% / +0.4% -8.2% / -2.3% 2/5 5/5 5
picker.tag.first_open_dom 259.6 / 275.6 257.3 / 279.4 255 / 267.3 -0.9% / +1.4% -1.8% / -3.0% 3/5 4/5 5
picker.tag.first_open_painted 371.2 / 407.8 371.3 / 403.2 343.5 / 409.4 +0.0% / -1.1% -7.5% / +0.4% 3/5 4/5 5
picker.wiki.first_open_dom 352.7 / 367.5 344.1 / 362.2 342.3 / 365 -2.4% / -1.4% -2.9% / -0.7% 2/5 3/5 5
picker.wiki.first_open_painted 417.7 / 449.1 409.9 / 435.6 423.5 / 438 -1.9% / -3.0% +1.4% / -2.5% 3/5 1/5 5
type.long_note 57.8 / 146 56.9 / 145.4 55.6 / 146.5 -1.6% / -0.4% -3.8% / +0.3% 3/5 3/5 5
type.long_note_cpu 25.84 / 25.84 25.11 / 25.11 25.53 / 25.53 -2.8% / -2.8% -1.2% / -1.2% 4/5 3/5 5
type.nested_list 235.8 / 1298 350.3 / 1143 194.9 / 829.5 +48.6% / -11.9% -17.3% / -36.1% 2/5 2/5 5
type.nested_list_cpu 48.84 / 48.84 50.33 / 50.33 51.09 / 51.09 +3.1% / +3.1% +4.6% / +4.6% 1/5 2/5 5
  • FCP: boot.cold_fcp train 716 / 768 vs dev 720 / 780 (-0.6% / -1.5%), control +1.1% / -3.6%. First note painted (cold_note) +2.5% / -3.7%, control +2.8% / -1.3%. Flat.
  • Typing: type.long_note -1.6% / -0.4%, cpu -2.8%. type.nested_list p50 train 350 vs dev 236 (+49%), but the same binary swings from 168 to 456 ms between dev rounds and 180 to 848 ms in the control (bimodal); train is below dev in 2/5 rounds, p95 -12%, cpu per key +3.1% (control +4.6%). Not a regression, but the noisiest number here.
  • Picker first open (@, [[, #; fresh page per sample, 8 samples per round, keydown of the trigger to the first option painted; its CSS now loads after first paint):
    trigger dev p50 / p95 train p50 / p95 control p50 / p95 train vs dev train<dev rounds
    @ 361.9 / 426.5 365.6 / 428 332.1 / 416.6 +1.0% / +0.4% 2/5
    [[ 417.7 / 449.1 409.9 / 435.6 423.5 / 438 -1.9% / -3.0% 3/5
    # 371.2 / 407.8 371.3 / 403.2 343.5 / 409.4 +0.0% / -1.1% 3/5
    The picker's first lookup waiting for its CSS costs nothing measurable (differences are smaller than dev-vs-control, which is up to -8%). The DOM-ready time (first option in the DOM) is -4.5% / -2.4% / -0.9% for @ / [[ / #.
  • blaze.long_tasks 7 -> 9 (control 8; per-round dev 5-12, train 6-15): within the same-binary spread. blaze.shown +5.3% / +9.1% (control -3.2% / -1.5%), rounds 2/5 below dev; I would watch it in the next train but it is inside dev-to-dev movement seen in BM14d (dev range 34.7-37 there vs 36.6-37.8 here).

Perf verdict

No regression found: server paths and cold FCP / first note / list search / typing / picker first-open are flat or better against the control; first CSS -639 B and first JS +4 B vs c163abae (-252 B / about -377 B vs real dev).

BM14e verdict: RED (strict flaky rule)

  • Gates: all pass (fmt, clippy, 1497 Rust tests, builds, bundle gate, case paths, evidence).
  • Bundle: no growth vs real dev 41f2efa3 (first JS about -377 B, first CSS -252 B); vs c163abae first JS +4 B, CSS -639 B.
  • Perf: no regression (above).
  • Chromium: 1 first-run failure (editor.spec.ts:109 throttled-1440), 0/10 on train and 0/10 on c163abae, whole file 0 fails: flake, does not block.
  • WebKit: 12 first-run failures. style-sheet:26, swipe-rows:365 (both widths), list-search:37 did not reproduce. The others do reproduce in x10, but mostly on both sides: note-bars-fit:67 fails about 1 in 3 on train (22/60) and on c163abae (20/60); toast-bar:40 5/40 train vs 1/40 dev; trash-search:14, backdrop-page:34, cover:112 on both. Train-only singles: switching:102 2/40 vs 0/40, home-screen:70 one beforeAll timeout, safe-areas:117 1/40 (30 s timeouts on a dev-02 at load average about 9). Because reproducing failures exist, the WebKit half is not clean under the rule as written; none looks caused by #908 (same specs fail on the BM14d head).
  • Follow-ups to open as stability issues: note-bars-fit.spec.ts:67 (WebKit 390), toast-bar.spec.ts:40, switching.spec.ts:102.
## BM14e perf: train 6825355e vs c163abae (BM14d head), ABAB + dev-vs-dev control perf-01, `~/bin/perf-lock` held on perf-01 **and** perf-02 (perf-01 first, then perf-02) in two consecutive holder units (11:52-13:49 and 13:49-stopped), both units now stopped on both VMs. Release binaries built on dev-01 and relayed, sha256 checked: dev `calternal` `16b2785c…`, train `30f2ac9d…`, control = byte copy of dev (`16b2785c…`); one `calternal-perf` (`55322a58…`, from dev) drives every arm. Every round runs all three arms (dev, train, ctl) in rotating order. Seed 1. "dev" here is c163abae because real origin/dev `41f2efa3` does not contain BM14d; BM14d's train-vs-41f2efa3 results stand for the rest. #908 only changes web CSS/JS loading, so the server binary logic is unchanged. ### Server (`calternal-perf --sizes 10k,100k --runs 5 --cold-runs 0 --idle-secs 0 --storm-edits 1`; 6 rounds, round 1 builds the fixtures and is not counted, so 5 interleaved rounds) Values = median over the 5 rounds of the per-round p50 / p95 (ms; mem in MB). "<" columns = in how many of the 5 rounds the arm's p50 / p95 was below dev's. | metric | notes | dev p50/p95 | train p50/p95 | ctl p50/p95 | train vs dev p50/p95 | ctl vs dev p50/p95 | train<dev p50 (of 5) | ctl<dev p50 | train<dev p95 | ctl<dev p95 | |---|---|---|---|---|---|---|---|---|---|---| | home.first_page | 10000 | 2.256 / 4.296 | 2.308 / 6.747 | 2.405 / 5.322 | +2% / +57% | +7% / +24% | 3 | 4 | 1 | 2 | | home.first_page | 100000 | 3.76 / 7.889 | 3.796 / 6.472 | 4.078 / 4.925 | +1% / -18% | +8% / -38% | 2 | 2 | 3 | 4 | | home.page | 10000 | 0.7928 / 1.452 | 0.75 / 1.73 | 0.754 / 1.205 | -5% / +19% | -5% / -17% | 4 | 4 | 2 | 3 | | home.page | 100000 | 0.9924 / 1.505 | 1.023 / 1.633 | 0.9579 / 1.451 | +3% / +9% | -3% / -4% | 2 | 3 | 2 | 3 | | home.page_big | 10000 | 16.27 / 19.72 | 16.6 / 19.52 | 16.64 / 20.2 | +2% / -1% | +2% / +2% | 2 | 2 | 2 | 1 | | home.page_big | 100000 | 24.01 / 35.4 | 19.23 / 29.98 | 22.57 / 32.19 | -20% / -15% | -6% / -9% | 4 | 4 | 4 | 3 | | home.page_next | 10000 | 0.7825 / 1.107 | 0.7739 / 1.083 | 0.7149 / 0.9996 | -1% / -2% | -9% / -10% | 3 | 4 | 3 | 3 | | home.page_next | 100000 | 0.7872 / 1.142 | 0.7635 / 1.139 | 0.7935 / 1.23 | -3% / -0% | +1% / +8% | 5 | 4 | 3 | 1 | | home.page_tag | 10000 | 5.081 / 6.943 | 5.1 / 6.607 | 5.093 / 6.377 | +0% / -5% | +0% / -8% | 1 | 2 | 3 | 3 | | home.page_tag | 100000 | 56.71 / 62.05 | 55.08 / 60.52 | 56.3 / 60.32 | -3% / -2% | -1% / -3% | 4 | 3 | 3 | 4 | | home.page_title | 10000 | 0.6363 / 0.7947 | 0.6656 / 0.8713 | 0.6186 / 0.8316 | +5% / +10% | -3% / +5% | 2 | 4 | 2 | 3 | | home.page_title | 100000 | 0.884 / 1.309 | 0.8895 / 1.313 | 0.8804 / 1.151 | +1% / +0% | -0% / -12% | 3 | 2 | 2 | 3 | | journal.catch_up | 10000 | 28.83 / 33.65 | 28.37 / 30.94 | 28.51 / 33.2 | -2% / -8% | -1% / -1% | 3 | 3 | 3 | 3 | | journal.catch_up | 100000 | 320.3 / 352.3 | 295.4 / 313.4 | 312.8 / 340.3 | -8% / -11% | -2% / -3% | 4 | 3 | 4 | 3 | | journal.entries | 10000 | 1.035e+04 / 1.07e+04 | 1.035e+04 / 1.07e+04 | 1.035e+04 / 1.07e+04 | +0% / +0% | +0% / +0% | 1 | 1 | 1 | 1 | | journal.entries | 100000 | 1.004e+05 / 1.007e+05 | 1.004e+05 / 1.007e+05 | 1.004e+05 / 1.007e+05 | -0% / -0% | -0% / -0% | 4 | 5 | 4 | 5 | | journal.page | 10000 | 2.417 / 2.642 | 2.381 / 2.723 | 2.392 / 2.608 | -1% / +3% | -1% / -1% | 4 | 3 | 2 | 3 | | journal.page | 100000 | 2.548 / 4.223 | 2.483 / 3.159 | 2.616 / 3.677 | -3% / -25% | +3% / -13% | 3 | 3 | 4 | 3 | | mem.after_first_page | 10000 | 21.41 / 21.5 | 21.44 / 22.07 | 21.46 / 22.35 | +0% / +3% | +0% / +4% | 2 | 2 | 0 | 2 | | mem.after_first_page | 100000 | 22.67 / 23.61 | 22.82 / 24.11 | 22.77 / 23.22 | +1% / +2% | +0% / -2% | 3 | 2 | 2 | 4 | | mem.after_opens | 10000 | 40.1 / 42.48 | 40.8 / 42.54 | 41.05 / 42.9 | +2% / +0% | +2% / +1% | 2 | 2 | 2 | 3 | | mem.after_opens | 100000 | 96.69 / 100.4 | 95.71 / 101.6 | 98.66 / 102.8 | -1% / +1% | +2% / +2% | 4 | 2 | 1 | 2 | | mem.after_search | 10000 | 45.57 / 75.66 | 46.96 / 75.38 | 47.59 / 74.87 | +3% / -0% | +4% / -1% | 2 | 1 | 2 | 4 | | mem.after_search | 100000 | 109.1 / 119.4 | 106.1 / 113.9 | 108.7 / 119 | -3% / -5% | -0% / -0% | 4 | 3 | 3 | 2 | | mem.idle | 10000 | 18.71 / 19.33 | 18.88 / 19.41 | 19.24 / 20.1 | +1% / +0% | +3% / +4% | 2 | 0 | 1 | 0 | | mem.idle | 100000 | 18.4 / 20.45 | 18.75 / 20.51 | 19.34 / 20.73 | +2% / +0% | +5% / +1% | 0 | 1 | 2 | 2 | | mem.peak | 10000 | 45.21 / 75.46 | 46.39 / 74.92 | 47.24 / 74.41 | +3% / -1% | +4% / -1% | 2 | 1 | 2 | 2 | | mem.peak | 100000 | 127.1 / 129.9 | 124.9 / 126.8 | 124.6 / 133.6 | -2% / -2% | -2% / +3% | 3 | 3 | 4 | 2 | | open.api_get | 10000 | 3.361 / 12.28 | 3.326 / 10.78 | 3.478 / 12.69 | -1% / -12% | +3% / +3% | 2 | 2 | 3 | 2 | | open.api_get | 100000 | 5.099 / 15.37 | 4.241 / 12.29 | 4.447 / 10.03 | -17% / -20% | -13% / -35% | 4 | 3 | 4 | 5 | | open.long_note | 10000 | 45.01 / 53.8 | 47.21 / 54.9 | 49.13 / 55.52 | +5% / +2% | +9% / +3% | 3 | 1 | 3 | 2 | | open.long_note | 100000 | 80.59 / 101.1 | 66.45 / 83.6 | 80.82 / 95.96 | -18% / -17% | +0% / -5% | 4 | 2 | 3 | 3 | | open.note | 10000 | 0.754 / 1.445 | 0.7766 / 1.435 | 0.7398 / 1.415 | +3% / -1% | -2% / -2% | 2 | 1 | 3 | 4 | | open.note | 100000 | 4.704 / 8.07 | 2.36 / 7.493 | 4.231 / 7.404 | -50% / -7% | -10% / -8% | 4 | 3 | 4 | 3 | | search.catch_up | 10000 | 2.005 / 4.011 | 2.006 / 4.01 | 2.005 / 4.016 | +0% / -0% | -0% / +0% | 2 | 3 | 2 | 2 | | search.catch_up | 100000 | 2.012 / 58.12 | 2.007 / 58.14 | 2.02 / 56.15 | -0% / +0% | +0% / -3% | 4 | 2 | 2 | 2 | | search.first_query | 10000 | 4.659 / 23.47 | 5.276 / 35.34 | 5.022 / 38.78 | +13% / +51% | +8% / +65% | 1 | 3 | 0 | 1 | | search.first_query | 100000 | 6.555 / 29.12 | 6.309 / 24.01 | 7.082 / 23.7 | -4% / -18% | +8% / -19% | 3 | 3 | 1 | 2 | | search.query | 10000 | 1.66 / 4.144 | 1.554 / 4.166 | 1.588 / 3.927 | -6% / +1% | -4% / -5% | 5 | 4 | 3 | 4 | | search.query | 100000 | 1.918 / 4.892 | 1.731 / 4.218 | 1.873 / 4.516 | -10% / -14% | -2% / -8% | 5 | 4 | 4 | 3 | | shutdown | 10000 | 64.01 / 64.37 | 64.02 / 64.78 | 64.1 / 85.18 | +0% / +1% | +0% / +32% | 1 | 1 | 2 | 1 | | shutdown | 100000 | 85.12 / 85.29 | 84.25 / 88.37 | 85.19 / 85.64 | -1% / +4% | +0% / +0% | 5 | 3 | 3 | 2 | | startup.healthz | 10000 | 5.115 / 7.226 | 4.926 / 8.07 | 4.985 / 8.21 | -4% / +12% | -3% / +14% | 4 | 3 | 2 | 2 | | startup.healthz | 100000 | 6.617 / 7.549 | 5.808 / 7.213 | 6.215 / 8.016 | -12% / -4% | -6% / +6% | 2 | 2 | 3 | 1 | | storm.converge | 10000 | 171 / 172.3 | 171.2 / 172.3 | 170.8 / 172.2 | +0% / +0% | -0% / -0% | 0 | 2 | 2 | 2 | | storm.converge | 100000 | 171.9 / 175.2 | 170.7 / 176 | 167.2 / 171.8 | -1% / +0% | -3% / -2% | 4 | 3 | 3 | 4 | | storm.disk_written | 10000 | 0.5366 / 0.7127 | 0.5407 / 2.535 | 0.5366 / 0.8438 | +1% / +256% | +0% / +18% | 1 | 2 | 1 | 2 | | storm.disk_written | 100000 | 0.6021 / 3.953 | 0.5939 / 0.9871 | 0.5816 / 0.9544 | -1% / -75% | -3% / -76% | 2 | 3 | 5 | 4 | | storm.dropped_clients | 10000 | 0 / 0 | 0 / 0 | 0 / 0 | +0% / +0% | +0% / +0% | 0 | 0 | 0 | 0 | | storm.dropped_clients | 100000 | 0 / 0 | 0 / 0 | 0 / 0 | +0% / +0% | +0% / +0% | 0 | 0 | 0 | 0 | | storm.edits_per_s | 10000 | 0 / 0 | 0 / 0 | 0 / 0 | +0% / +0% | +0% / +0% | 0 | 0 | 0 | 0 | | storm.edits_per_s | 100000 | 0 / 0 | 0 / 0 | 0 / 0 | +0% / +0% | +0% / +0% | 0 | 0 | 0 | 0 | | storm.outside_edits | 10000 | 1 / 1 | 1 / 1 | 1 / 1 | +0% / +0% | +0% / +0% | 0 | 0 | 0 | 0 | | storm.outside_edits | 100000 | 1 / 1 | 1 / 1 | 1 / 1 | +0% / +0% | +0% / +0% | 0 | 0 | 0 | 0 | | storm.typing | 10000 | 2.634e-05 / 3.756e-05 | 2.709e-05 / 3.098e-05 | 2.507e-05 / 3.245e-05 | +3% / -18% | -5% / -14% | 3 | 3 | 3 | 4 | | storm.typing | 100000 | 2.589e-05 / 3.269e-05 | 2.829e-05 / 3.813e-05 | 2.864e-05 / 3.352e-05 | +9% / +17% | +11% / +3% | 1 | 1 | 1 | 3 | | sync.durable_write | 10000 | 19.42 / 23.6 | 20.56 / 27.98 | 20.73 / 26.2 | +6% / +19% | +7% / +11% | 2 | 1 | 1 | 1 | | sync.durable_write | 100000 | 23.19 / 35.38 | 20.35 / 30.18 | 23.53 / 35.26 | -12% / -15% | +1% / -0% | 4 | 4 | 4 | 3 | | sync.round_trip | 10000 | 0.2146 / 0.3339 | 0.2165 / 0.3451 | 0.2097 / 0.3387 | +1% / +3% | -2% / +1% | 2 | 3 | 2 | 1 | | sync.round_trip | 100000 | 0.2354 / 0.35 | 0.2335 / 0.336 | 0.2153 / 0.3428 | -1% / -4% | -9% / -2% | 4 | 5 | 4 | 3 | | sync.save_ack | 10000 | 2.981 / 4.225 | 3.163 / 3.844 | 3.124 / 4.355 | +6% / -9% | +5% / +3% | 1 | 0 | 3 | 2 | | sync.save_ack | 100000 | 3.295 / 4.755 | 3.231 / 4.14 | 3.273 / 4.575 | -2% / -13% | -1% / -4% | 5 | 4 | 3 | 3 | | trash.delete | 10000 | 51.42 / 80.82 | 52.89 / 92.54 | 54.92 / 80.27 | +3% / +14% | +7% / -1% | 3 | 2 | 1 | 2 | | trash.delete | 100000 | 99.53 / 217 | 75.49 / 115.4 | 99.81 / 226.4 | -24% / -47% | +0% / +4% | 4 | 4 | 4 | 2 | | trash.forget | 10000 | 7.642 / 13.99 | 9.544 / 13.78 | 9.568 / 12.69 | +25% / -1% | +25% / -9% | 1 | 2 | 3 | 3 | | trash.forget | 100000 | 10.81 / 18.4 | 10.11 / 14.81 | 10.76 / 16.34 | -7% / -20% | -1% / -11% | 4 | 3 | 4 | 3 | | trash.restore | 10000 | 61.71 / 111.5 | 62.84 / 114.1 | 63.43 / 113.6 | +2% / +2% | +3% / +2% | 1 | 2 | 1 | 1 | | trash.restore | 100000 | 67.45 / 127.4 | 63.56 / 105.9 | 66.35 / 115.5 | -6% / -17% | -2% / -9% | 5 | 2 | 3 | 2 | | uploads.response | 10000 | 109.5 / 146.5 | 142.1 / 159.2 | 143.3 / 158.3 | +30% / +9% | +31% / +8% | 2 | 1 | 1 | 1 | | uploads.response | 100000 | 152 / 174.1 | 142.8 / 165.3 | 150.9 / 178 | -6% / -5% | -1% / +2% | 4 | 4 | 5 | 1 | | uploads.round_trip | 10000 | 0.2618 / 0.4115 | 0.2592 / 0.4304 | 0.2593 / 0.4111 | -1% / +5% | -1% / -0% | 4 | 3 | 2 | 2 | | uploads.round_trip | 100000 | 0.2804 / 0.4412 | 0.2796 / 0.4335 | 0.2792 / 0.4398 | -0% / -2% | -0% / -0% | 3 | 4 | 2 | 3 | | uploads.save_ack | 10000 | 2.877 / 7.144 | 3.325 / 9.694 | 3.386 / 8.847 | +16% / +36% | +18% / +24% | 2 | 2 | 1 | 1 | | uploads.save_ack | 100000 | 3.538 / 10.08 | 3.461 / 8.875 | 3.601 / 10.69 | -2% / -12% | +2% / +6% | 4 | 3 | 5 | 2 | Reading: sync.round_trip (10k +1%/+3%, 100k -1%/-4%), open.note (10k +3%/-1%; 100k p50 is bimodal in dev itself: 4.7 vs control 4.2 vs train 2.4), search.query, sync.save_ack, durable_write and memory are flat or better, and everything that moves for train also moves for the control. `search.first_query`: 100k **-4% p50 / -18% p95** (train below dev in 3/5 p50 rounds); 10k +13% p50 / +51% p95, but the same-binary control is +8% / +65% in those rounds (dev's own spread is wide), so it is noise, not the picker CSS. ### Browser (Chromium 4x CPU slowdown, 10k Home, `boot` / `list-search` / `blaze` / `typing` perf specs + picker first-open probe; 5 rounds, each arm served by its own binary and dist) | metric | dev p50/p95 | train p50/p95 | ctl p50/p95 | train vs dev p50/p95 | ctl vs dev p50/p95 | train<dev p50 | ctl<dev p50 | rounds | |---|---|---|---|---|---|---|---|---| | blaze.back_missed | 1 / 1 | 1 / 1 | 1 / 1 | +0.0% / +0.0% | +0.0% / +0.0% | 0/5 | 0/5 | 5 | | blaze.back_shown | 31.6 / 47.3 | 31.8 / 50.8 | 31.3 / 47.5 | +0.6% / +7.4% | -0.9% / +0.4% | 1/5 | 3/5 | 5 | | blaze.cls | 0 / 0 | 0 / 0 | 0 / 0 | +0.0% / +0.0% | +0.0% / +0.0% | 0/5 | 0/5 | 5 | | blaze.cpu | 69.48 / 69.48 | 71.33 / 71.33 | 70.26 / 70.26 | +2.7% / +2.7% | +1.1% / +1.1% | 0/5 | 2/5 | 5 | | blaze.dom_growth | 1 / 1 | 1 / 1 | 1 / 1 | +0.0% / +0.0% | +0.0% / +0.0% | 0/5 | 1/5 | 5 | | blaze.heap_growth | 1.87 / 1.87 | 1.86 / 1.86 | 1.88 / 1.88 | -0.5% / -0.5% | +0.5% / +0.5% | 4/5 | 3/5 | 5 | | blaze.live | 105 / 105 | 107 / 107 | 90 / 90 | +1.9% / +1.9% | -14.3% / -14.3% | 2/5 | 3/5 | 5 | | blaze.long_task_max | 138 / 138 | 150 / 150 | 148 / 148 | +8.7% / +8.7% | +7.2% / +7.2% | 0/5 | 2/5 | 5 | | blaze.long_tasks | 7 / 7 | 9 / 9 | 8 / 8 | +28.6% / +28.6% | +14.3% / +14.3% | 1/5 | 2/5 | 5 | | blaze.missed | 0 / 0 | 0 / 0 | 0 / 0 | +0.0% / +0.0% | +0.0% / +0.0% | 1/5 | 1/5 | 5 | | blaze.requests | 52 / 52 | 52 / 52 | 52 / 52 | +0.0% / +0.0% | +0.0% / +0.0% | 0/5 | 0/5 | 5 | | blaze.shown | 37.8 / 60.5 | 39.8 / 66 | 36.6 / 59.6 | +5.3% / +9.1% | -3.2% / -1.5% | 2/5 | 4/5 | 5 | | boot.cls | 0 / 0.01 | 0 / 0.01 | 0 / 0.01 | +0.0% / +0.0% | +0.0% / +0.0% | 0/5 | 0/5 | 5 | | boot.cold_blocked | 434 / 530 | 434 / 495 | 452 / 513 | +0.0% / -6.6% | +4.1% / -3.2% | 2/5 | 3/5 | 5 | | boot.cold_cpu | 1285 / 1418 | 1310 / 1311 | 1311 / 1400 | +2.0% / -7.5% | +2.0% / -1.3% | 1/5 | 2/5 | 5 | | boot.cold_fcp | 720 / 780 | 716 / 768 | 728 / 752 | -0.6% / -1.5% | +1.1% / -3.6% | 1/5 | 1/5 | 5 | | boot.cold_list | 885.6 / 980.4 | 887.2 / 924 | 851.9 / 953.5 | +0.2% / -5.8% | -3.8% / -2.7% | 2/5 | 3/5 | 5 | | boot.cold_note | 885.5 / 980.4 | 907.5 / 943.9 | 910.6 / 968 | +2.5% / -3.7% | +2.8% / -1.3% | 2/5 | 1/5 | 5 | | boot.cold_tbt | 110.2 / 160.8 | 110.9 / 143 | 98 / 142 | +0.6% / -11.1% | -11.1% / -11.7% | 1/5 | 3/5 | 5 | | boot.cold_tti | 1755 / 1830 | 1765 / 1809 | 1788 / 1815 | +0.5% / -1.1% | +1.9% / -0.8% | 2/5 | 1/5 | 5 | | boot.heap | 11.91 / 11.91 | 11.92 / 11.92 | 11.91 / 11.91 | +0.1% / +0.1% | +0.0% / +0.0% | 0/5 | 0/5 | 5 | | boot.warm_list | 297.7 / 592.5 | 284.3 / 575.3 | 297.6 / 577.7 | -4.5% / -2.9% | -0.0% / -2.5% | 4/5 | 3/5 | 5 | | boot.warm_note | 457.6 / 810.3 | 432.7 / 782.9 | 439.3 / 787.5 | -5.4% / -3.4% | -4.0% / -2.8% | 4/5 | 2/5 | 5 | | boot.warm_tbt | 5 / 22 | 0 / 22 | 4 / 28 | -100.0% / +0.0% | -20.0% / +27.3% | 3/5 | 3/5 | 5 | | boot.warm_tti | 1155 / 1698 | 464.7 / 1720 | 1141 / 1709 | -59.8% / +1.3% | -1.2% / +0.6% | 3/5 | 2/5 | 5 | | list_search.blank_frames@10000 | 0 / 0 | 0 / 0 | 0 / 0 | +0.0% / +0.0% | +0.0% / +0.0% | 0/5 | 0/5 | 5 | | list_search.key@10000 | 5.9 / 7.8 | 5.8 / 7.5 | 5.9 / 8.2 | -1.7% / -3.8% | +0.0% / +5.1% | 3/5 | 3/5 | 5 | | list_search.warmup_key@10000 | 6.1 / 30.8 | 5.8 / 22.9 | 6.9 / 25.5 | -4.9% / -25.6% | +13.1% / -17.2% | 3/5 | 1/5 | 5 | | picker.at.first_open_dom | 315.3 / 332.4 | 301.1 / 320.8 | 291.6 / 311 | -4.5% / -3.5% | -7.5% / -6.4% | 3/5 | 4/5 | 5 | | picker.at.first_open_painted | 361.9 / 426.5 | 365.6 / 428 | 332.1 / 416.6 | +1.0% / +0.4% | -8.2% / -2.3% | 2/5 | 5/5 | 5 | | picker.tag.first_open_dom | 259.6 / 275.6 | 257.3 / 279.4 | 255 / 267.3 | -0.9% / +1.4% | -1.8% / -3.0% | 3/5 | 4/5 | 5 | | picker.tag.first_open_painted | 371.2 / 407.8 | 371.3 / 403.2 | 343.5 / 409.4 | +0.0% / -1.1% | -7.5% / +0.4% | 3/5 | 4/5 | 5 | | picker.wiki.first_open_dom | 352.7 / 367.5 | 344.1 / 362.2 | 342.3 / 365 | -2.4% / -1.4% | -2.9% / -0.7% | 2/5 | 3/5 | 5 | | picker.wiki.first_open_painted | 417.7 / 449.1 | 409.9 / 435.6 | 423.5 / 438 | -1.9% / -3.0% | +1.4% / -2.5% | 3/5 | 1/5 | 5 | | type.long_note | 57.8 / 146 | 56.9 / 145.4 | 55.6 / 146.5 | -1.6% / -0.4% | -3.8% / +0.3% | 3/5 | 3/5 | 5 | | type.long_note_cpu | 25.84 / 25.84 | 25.11 / 25.11 | 25.53 / 25.53 | -2.8% / -2.8% | -1.2% / -1.2% | 4/5 | 3/5 | 5 | | type.nested_list | 235.8 / 1298 | 350.3 / 1143 | 194.9 / 829.5 | +48.6% / -11.9% | -17.3% / -36.1% | 2/5 | 2/5 | 5 | | type.nested_list_cpu | 48.84 / 48.84 | 50.33 / 50.33 | 51.09 / 51.09 | +3.1% / +3.1% | +4.6% / +4.6% | 1/5 | 2/5 | 5 | - **FCP:** `boot.cold_fcp` train 716 / 768 vs dev 720 / 780 (-0.6% / -1.5%), control +1.1% / -3.6%. First note painted (`cold_note`) +2.5% / -3.7%, control +2.8% / -1.3%. Flat. - **Typing:** `type.long_note` -1.6% / -0.4%, cpu -2.8%. `type.nested_list` p50 train 350 vs dev 236 (+49%), but the same binary swings from 168 to 456 ms between dev rounds and 180 to 848 ms in the control (bimodal); train is below dev in 2/5 rounds, p95 -12%, cpu per key +3.1% (control +4.6%). Not a regression, but the noisiest number here. - **Picker first open (`@`, `[[`, `#`; fresh page per sample, 8 samples per round, keydown of the trigger to the first option painted; its CSS now loads after first paint):** | trigger | dev p50 / p95 | train p50 / p95 | control p50 / p95 | train vs dev | train<dev rounds | |---|---|---|---|---|---| | `@` | 361.9 / 426.5 | 365.6 / 428 | 332.1 / 416.6 | +1.0% / +0.4% | 2/5 | | `[[` | 417.7 / 449.1 | 409.9 / 435.6 | 423.5 / 438 | -1.9% / -3.0% | 3/5 | | `#` | 371.2 / 407.8 | 371.3 / 403.2 | 343.5 / 409.4 | +0.0% / -1.1% | 3/5 | The picker's first lookup waiting for its CSS costs nothing measurable (differences are smaller than dev-vs-control, which is up to -8%). The DOM-ready time (first option in the DOM) is -4.5% / -2.4% / -0.9% for `@` / `[[` / `#`. - `blaze.long_tasks` 7 -> 9 (control 8; per-round dev 5-12, train 6-15): within the same-binary spread. `blaze.shown` +5.3% / +9.1% (control -3.2% / -1.5%), rounds 2/5 below dev; I would watch it in the next train but it is inside dev-to-dev movement seen in BM14d (dev range 34.7-37 there vs 36.6-37.8 here). ### Perf verdict No regression found: server paths and cold FCP / first note / list search / typing / picker first-open are flat or better against the control; first CSS -639 B and first JS +4 B vs c163abae (-252 B / about -377 B vs real dev). ## BM14e verdict: RED (strict flaky rule) - Gates: all pass (fmt, clippy, 1497 Rust tests, builds, bundle gate, case paths, evidence). - Bundle: no growth vs real dev `41f2efa3` (first JS about -377 B, first CSS -252 B); vs c163abae first JS +4 B, CSS -639 B. - Perf: no regression (above). - Chromium: 1 first-run failure (`editor.spec.ts:109` throttled-1440), 0/10 on train and 0/10 on c163abae, whole file 0 fails: flake, does not block. - WebKit: 12 first-run failures. `style-sheet:26`, `swipe-rows:365` (both widths), `list-search:37` did not reproduce. The others **do reproduce in x10**, but mostly on both sides: `note-bars-fit:67` fails about 1 in 3 on train (22/60) and on c163abae (20/60); `toast-bar:40` 5/40 train vs 1/40 dev; `trash-search:14`, `backdrop-page:34`, `cover:112` on both. Train-only singles: `switching:102` 2/40 vs 0/40, `home-screen:70` one beforeAll timeout, `safe-areas:117` 1/40 (30 s timeouts on a dev-02 at load average about 9). Because reproducing failures exist, the WebKit half is not clean under the rule as written; none looks caused by #908 (same specs fail on the BM14d head). - Follow-ups to open as stability issues: `note-bars-fit.spec.ts:67` (WebKit 390), `toast-bar.spec.ts:40`, `switching.spec.ts:102`.
bm14e-perf-raw.tar.gz
819 KiB
kayg commented 2026-11-04 17:03:35 +00:00
Author
Owner
Copy link

BM14e flaky WebKit specs: classification, root cause, fix (new head 1aed57bc)

Root cause (one, shared by all): the dev-02 virtual disk stalls on fsync. Probe, 2000 x (4 KB write + fsync) on the idle host: p50 24.8 ms, p99 752 ms, max 4.4 s, 10 stalls over 1 s. The server makes every write durable, and the Playwright harness put every test server's data in /tmp on that disk. Traces of the failures show the request hanging with no response (notes.undo, notes.restore, notes.delete status -1) while the spec waits 5 s. Same fsync probe on /dev/shm: max 0.2 ms. Not a product bug, not a race in the specs, and not from any BM14d PR or #908, so no bisect was needed (every spec also fails on origin/dev).

Fix: #937 (fix/e2e-data-on-ram, commit 19021eb7, cherry-picked on the train as 1aed57bc): the harness keeps the servers' data dirs under /dev/shm (fallback tmpdir, PERF=1 keeps the real disk), removed on runner exit. No assertion, timeout, retry or skip touched.

Method: --no-deps --repeat-each=10 --retries=0 --workers=1 CI=1, one group at a time through ~/bm14f/x10.sh -> ~/bin/slot, dev-02 (load 4-10; other agents' jobs were also running). Builds: dev = origin/dev 41f2efa3 (own binary + dist), c163 = BM14d c163abae, train = 6825355e. Failed runs / runs. note-bars-fit has 60 runs per x10 of which 20 are skipped by design (the 1440 variants on the 390 project).

spec (project) dev 41f2efa3 c163abae train 6825355e class train + fix (1aed57bc)
note-bars-fit:67 (webkit-390) 0/40 0/40 0/40 (1st-run failures not reproduced on a quiet host) pre-existing/load only 40/40 pass
trash-search:14 (webkit-390, 4 variants) 3/40 (undo/restore hang) 2/40 5/40 (a) pre-existing 40/40 pass
toast-bar:40 (webkit-1440) 1/10 (delete hang) 0/10 0/10 (a) pre-existing 10/10
switching:102 (webkit-1440) 1/10 (30 s timeout) 0/10 0/10 (a) pre-existing 10/10
cover:112 (webkit-1440) 0/10 0/10 0/10 (a) pre-existing (fails on both in earlier runs) 10/10
backdrop-page:34 (look-webkit-1440) 0/10 1/10 2/10 (saved setting not seen in 5 s) (a) pre-existing 10/10
safe-areas:117 (webkit-390) 0/40 0/40 1/40 (30 s) (a) fsync stall, single 40/40 pass
home-screen:70 (webkit-390, beforeAll) 0/240 1/240 beforeAll timeout 1/240 beforeAll timeout (a) fsync stall in seed 240/240 pass

Control: the train build with only the data dir on /dev/shm (before the harness change was cherry-picked) also ran all groups clean: 360/360, 30/30, 10/10.

No spec classified (b) or (c): nothing is introduced by BM14d or #908. Notes: one first launch of the 390 group hit a harness port race (already used, freePort picked a port twice); it ran no tests and was simply started again, result above. Not fixed here.

PR #937 (train: #914), new train head 1aed57bc06a82df2734e1da147fa32d453ebe061
Results after (n/10): note-bars-fit:67 10/10 (40/40 runs), trash-search:14 10/10 (40/40), toast-bar:40 10/10, switching:102 10/10, cover:112 10/10, backdrop-page:34 10/10, safe-areas:117 10/10 (40/40), home-screen:70 10/10 (240/240).

## BM14e flaky WebKit specs: classification, root cause, fix (new head `1aed57bc`) **Root cause (one, shared by all):** the dev-02 virtual disk stalls on fsync. Probe, 2000 x (4 KB write + fsync) on the idle host: p50 24.8 ms, p99 752 ms, max 4.4 s, 10 stalls over 1 s. The server makes every write durable, and the Playwright harness put every test server's data in `/tmp` on that disk. Traces of the failures show the request hanging with no response (`notes.undo`, `notes.restore`, `notes.delete` status -1) while the spec waits 5 s. Same fsync probe on `/dev/shm`: max 0.2 ms. Not a product bug, not a race in the specs, and not from any BM14d PR or #908, so no bisect was needed (every spec also fails on origin/dev). **Fix:** #937 (`fix/e2e-data-on-ram`, commit `19021eb7`, cherry-picked on the train as `1aed57bc`): the harness keeps the servers' data dirs under `/dev/shm` (fallback `tmpdir`, `PERF=1` keeps the real disk), removed on runner exit. No assertion, timeout, retry or skip touched. Method: `--no-deps --repeat-each=10 --retries=0 --workers=1 CI=1`, one group at a time through `~/bm14f/x10.sh` -> `~/bin/slot`, dev-02 (load 4-10; other agents' jobs were also running). Builds: dev = origin/dev `41f2efa3` (own binary + dist), c163 = BM14d `c163abae`, train = `6825355e`. Failed runs / runs. note-bars-fit has 60 runs per x10 of which 20 are skipped by design (the 1440 variants on the 390 project). | spec (project) | dev 41f2efa3 | c163abae | train 6825355e | class | train + fix (`1aed57bc`) | |---|---|---|---|---|---| | note-bars-fit:67 (webkit-390) | 0/40 | 0/40 | 0/40 (1st-run failures not reproduced on a quiet host) | pre-existing/load only | **40/40 pass** | | trash-search:14 (webkit-390, 4 variants) | 3/40 (undo/restore hang) | 2/40 | 5/40 | (a) pre-existing | **40/40 pass** | | toast-bar:40 (webkit-1440) | 1/10 (delete hang) | 0/10 | 0/10 | (a) pre-existing | **10/10** | | switching:102 (webkit-1440) | 1/10 (30 s timeout) | 0/10 | 0/10 | (a) pre-existing | **10/10** | | cover:112 (webkit-1440) | 0/10 | 0/10 | 0/10 | (a) pre-existing (fails on both in earlier runs) | **10/10** | | backdrop-page:34 (look-webkit-1440) | 0/10 | 1/10 | 2/10 (saved setting not seen in 5 s) | (a) pre-existing | **10/10** | | safe-areas:117 (webkit-390) | 0/40 | 0/40 | 1/40 (30 s) | (a) fsync stall, single | **40/40 pass** | | home-screen:70 (webkit-390, beforeAll) | 0/240 | 1/240 beforeAll timeout | 1/240 beforeAll timeout | (a) fsync stall in seed | **240/240 pass** | Control: the train build with only the data dir on `/dev/shm` (before the harness change was cherry-picked) also ran all groups clean: 360/360, 30/30, 10/10. No spec classified (b) or (c): nothing is introduced by BM14d or #908. Notes: one first launch of the 390 group hit a harness port race (`already used`, `freePort` picked a port twice); it ran no tests and was simply started again, result above. Not fixed here. PR https://git.kayg.org/kayg/calternal/pulls/937 (train: https://git.kayg.org/kayg/calternal/pulls/914), new train head `1aed57bc06a82df2734e1da147fa32d453ebe061` Results after (n/10): note-bars-fit:67 10/10 (40/40 runs), trash-search:14 10/10 (40/40), toast-bar:40 10/10, switching:102 10/10, cover:112 10/10, backdrop-page:34 10/10, safe-areas:117 10/10 (40/40), home-screen:70 10/10 (240/240).
kayg commented 2026-11-04 17:03:43 +00:00
Author
Owner
Copy link

Correction to the table above: trash-search:14 on dev 41f2efa3 was 2/40 (not 3/40; one click, one row, both an unanswered notes.undo). All other cells are as posted. Also: note-bars-fit:67 did not fail in any of my runs (0/40 on dev, c163abae, train), so its earlier 22/60 first-run failures came from the loaded host, not from a build.

Correction to the table above: trash-search:14 on dev 41f2efa3 was **2/40** (not 3/40; one `click`, one `row`, both an unanswered `notes.undo`). All other cells are as posted. Also: note-bars-fit:67 did not fail in any of my runs (0/40 on dev, c163abae, train), so its earlier 22/60 first-run failures came from the loaded host, not from a build.
kayg commented 2026-11-04 20:09:36 +00:00
Author
Owner
Copy link

BM14e full Playwright rerun — head 1aed57bc06a82df2734e1da147fa32d453ebe061

Both halves ran with CI=1, --workers=1, --retries=0, and SLOT_TRAIN=1 through ~/bin/slot. The train worktree used its apps/web/dist and bm14e-base/target/debug/calternal; the baseline was 41f2efa3 with both #937 harness commits applied.

First run

  • dev-01 Chromium projects (chromium-1440, chromium-390, flow, passkeys, admin-setup, admin-chromium-1440, admin-chromium-390, prefs-chromium, look-chromium-1440, look-chromium-390, contrast, sidebar-band-chromium): 1,400 passed, 816 skipped, 1 failed; exit 1.
  • dev-01 throttled projects (throttled-1440, throttled-390, --no-deps): 8 passed, 2 skipped, 0 failed; exit 0.
  • dev-02 WebKit projects (webkit-1440, webkit-390, admin-webkit-1440, admin-webkit-390, prefs-webkit, look-webkit-1440, look-webkit-390, sidebar-band-webkit, plus the selected contrast project): 1,387 passed, 831 skipped, 4 failed; exit 1. The line reporter showed 825 skipped and 6 not run.

No already used or EADDRINUSE port collisions occurred.

The exact first-run invocations, from the relevant apps/web worktree, were:

SLOT_TRAIN=1 ~/bin/slot bunx playwright test --project=chromium-1440 --project=chromium-390 --project=flow --project=passkeys --project=admin-setup --project=admin-chromium-1440 --project=admin-chromium-390 --project=prefs-chromium --project=look-chromium-1440 --project=look-chromium-390 --project=contrast --project=sidebar-band-chromium --workers=1 --retries=0 --reporter=line,json
SLOT_TRAIN=1 ~/bin/slot bunx playwright test --project=webkit-1440 --project=webkit-390 --project=admin-webkit-1440 --project=admin-webkit-390 --project=prefs-webkit --project=look-webkit-1440 --project=look-webkit-390 --project=sidebar-band-webkit --workers=1 --retries=0 --reporter=line,json
SLOT_TRAIN=1 ~/bin/slot bunx playwright test --project=throttled-1440 --project=throttled-390 --no-deps --workers=1 --retries=0 --reporter=line,json

Failure repeats (--repeat-each=10, --no-deps, retries 0)

First-run failure Train dev 41f2efa3 with #937 harness Classification
note-bars.spec.ts:100, chromium-390 (30 s wait for Divider menu item) 10/10 passed The file is absent at 41f2efa3; after copying the train spec, 0/10 passed because the older app never entered the newer tools mode. The original timeout did not recur on train. Not a comparable baseline case; follow-up issue #943.
composer-tasks.spec.ts:51, webkit-1440 and webkit-390 20/20 passed 20/20 passed Did not reproduce on either side; non-blocking. Stability follow-up #942.
signout.spec.ts:63, webkit-1440 10/10 passed 8/10 passed, 2 failed Reproduces on baseline; blocks under the strict rule, but was not reproduced on train.
contrast.spec.ts:174, project contrast 6/10 passed at 1440 and 2/10 at 390 (14/20 failed) Same counts: 6/10 at 1440 and 2/10 at 390 (14/20 failed) Reproduces on train and baseline; blocks under the strict rule and predates this train. The 390 failure reports contrast 1.75 where 3 is required.

The test selectors were e2e/note-bars.spec.ts:100 --project=chromium-390, e2e/composer-tasks.spec.ts:51 --project=webkit-1440 --project=webkit-390, e2e/signout.spec.ts:63 --project=webkit-1440, and e2e/contrast.spec.ts:174 --project=contrast. Each used --repeat-each=10 --no-deps --workers=1 --retries=0 --reporter=line,json through the train slot. Baseline-only selectors used --grep where needed to match the test title.

#937 port guard

#937 head is 0c451d6b1b441ba6f6132c33cc4be9dad17adc3f, a separate port-guard commit. Full Vitest verification on that worktree passed: SLOT_TRAIN=1 ~/bin/slot bun install --frozen-lockfile, then SLOT_TRAIN=1 ~/bin/slot bun run test from apps/web — 9 files, 97 tests passed, including src/lib/harness.test.ts.

Verdict: RED under the strict flaky rule. The contrast failures reproduce at the same rate on train and 41f2efa3; sign-out also fails on the baseline. No port clash occurred. The dev-01 and dev-02 train locks have both been released.

## BM14e full Playwright rerun — head `1aed57bc06a82df2734e1da147fa32d453ebe061` Both halves ran with `CI=1`, `--workers=1`, `--retries=0`, and `SLOT_TRAIN=1` through `~/bin/slot`. The train worktree used its `apps/web/dist` and `bm14e-base/target/debug/calternal`; the baseline was `41f2efa3` with both #937 harness commits applied. ### First run - **dev-01 Chromium projects** (`chromium-1440`, `chromium-390`, `flow`, `passkeys`, `admin-setup`, `admin-chromium-1440`, `admin-chromium-390`, `prefs-chromium`, `look-chromium-1440`, `look-chromium-390`, `contrast`, `sidebar-band-chromium`): 1,400 passed, 816 skipped, 1 failed; exit 1. - **dev-01 throttled projects** (`throttled-1440`, `throttled-390`, `--no-deps`): 8 passed, 2 skipped, 0 failed; exit 0. - **dev-02 WebKit projects** (`webkit-1440`, `webkit-390`, `admin-webkit-1440`, `admin-webkit-390`, `prefs-webkit`, `look-webkit-1440`, `look-webkit-390`, `sidebar-band-webkit`, plus the selected `contrast` project): 1,387 passed, 831 skipped, 4 failed; exit 1. The line reporter showed 825 skipped and 6 not run. No `already used` or `EADDRINUSE` port collisions occurred. The exact first-run invocations, from the relevant `apps/web` worktree, were: ```sh SLOT_TRAIN=1 ~/bin/slot bunx playwright test --project=chromium-1440 --project=chromium-390 --project=flow --project=passkeys --project=admin-setup --project=admin-chromium-1440 --project=admin-chromium-390 --project=prefs-chromium --project=look-chromium-1440 --project=look-chromium-390 --project=contrast --project=sidebar-band-chromium --workers=1 --retries=0 --reporter=line,json SLOT_TRAIN=1 ~/bin/slot bunx playwright test --project=webkit-1440 --project=webkit-390 --project=admin-webkit-1440 --project=admin-webkit-390 --project=prefs-webkit --project=look-webkit-1440 --project=look-webkit-390 --project=sidebar-band-webkit --workers=1 --retries=0 --reporter=line,json SLOT_TRAIN=1 ~/bin/slot bunx playwright test --project=throttled-1440 --project=throttled-390 --no-deps --workers=1 --retries=0 --reporter=line,json ``` ### Failure repeats (`--repeat-each=10`, `--no-deps`, retries 0) | First-run failure | Train | dev `41f2efa3` with #937 harness | Classification | |---|---:|---:|---| | `note-bars.spec.ts:100`, `chromium-390` (30 s wait for Divider menu item) | 10/10 passed | The file is absent at `41f2efa3`; after copying the train spec, 0/10 passed because the older app never entered the newer `tools` mode. The original timeout did not recur on train. | Not a comparable baseline case; follow-up issue [#943](https://git.kayg.org/kayg/calternal/issues/943). | | `composer-tasks.spec.ts:51`, `webkit-1440` and `webkit-390` | 20/20 passed | 20/20 passed | Did not reproduce on either side; non-blocking. Stability follow-up [#942](https://git.kayg.org/kayg/calternal/issues/942). | | `signout.spec.ts:63`, `webkit-1440` | 10/10 passed | 8/10 passed, 2 failed | Reproduces on baseline; blocks under the strict rule, but was not reproduced on train. | | `contrast.spec.ts:174`, project `contrast` | 6/10 passed at 1440 and 2/10 at 390 (14/20 failed) | Same counts: 6/10 at 1440 and 2/10 at 390 (14/20 failed) | Reproduces on train and baseline; blocks under the strict rule and predates this train. The 390 failure reports contrast 1.75 where 3 is required. | The test selectors were `e2e/note-bars.spec.ts:100 --project=chromium-390`, `e2e/composer-tasks.spec.ts:51 --project=webkit-1440 --project=webkit-390`, `e2e/signout.spec.ts:63 --project=webkit-1440`, and `e2e/contrast.spec.ts:174 --project=contrast`. Each used `--repeat-each=10 --no-deps --workers=1 --retries=0 --reporter=line,json` through the train slot. Baseline-only selectors used `--grep` where needed to match the test title. ### #937 port guard #937 head is `0c451d6b1b441ba6f6132c33cc4be9dad17adc3f`, a separate port-guard commit. Full Vitest verification on that worktree passed: `SLOT_TRAIN=1 ~/bin/slot bun install --frozen-lockfile`, then `SLOT_TRAIN=1 ~/bin/slot bun run test` from `apps/web` — 9 files, 97 tests passed, including `src/lib/harness.test.ts`. **Verdict: RED under the strict flaky rule.** The contrast failures reproduce at the same rate on train and `41f2efa3`; sign-out also fails on the baseline. No port clash occurred. The dev-01 and dev-02 train locks have both been released.
kayg commented 2026-11-05 03:17:32 +00:00
Author
Owner
Copy link

BM14e contrast: cause, fix PR #944, interim result (head 96d1aa25)

Failing test. contrast.spec.ts "the palette keeps its contrast over a photo and over a page of huge words" (1440 and 390). Reproduced on dev-01 (--repeat-each on train, no deps). Three separate causes, all found by reading the failing glyph names and screenshots:

  1. Product, 1440 light: the palette's note snippet (.sn, --ink-3) read 4.23 to 4.40 (needs 4.5) over the page of huge black words. Fix in #944: --glass-ink-3 (light) #4a423b → #403a34 (as deep as ink 2), one token, so every quiet label on any glass follows. No local override.
  2. Test, 390: the phone palette's glass grows on a JS spring (lib/hug.ts) that starts when the Recent rows arrive, after still() (it only sees CSS animations). The words were measured over a glass that had not reached them yet: "Try" at 1.55 to 1.87 (the 1.75 in the first report). Fix: hugSettled() waits until the palette's box and glass clip stay unchanged for 30 frames. It is a transient animation state, not a persisting legibility bug.
  3. Test, the "sometimes" at 1440 (the non-determinism): the contrast project's server keeps its notes for the whole run and Playwright starts a new worker for every repeat, so seedLists() added 4 notes per run. After about 6 to 11 runs there were a dozen rows in the list pane, i.e. a dozen lines of 40 px / 900 black words behind the palette's results, and the rows read 2.6 (date) and 2.2 (icon). The product cannot promise AA over any amount of such text with see-through glass (and kayg's rule is no near-opaque fill), so the spec now seeds its rows ONCE by fixed id (contrast-keep, contrast-gone-1..3, only when the server doesn't have them) and measures the same rows every run.

Gate: contrast.spec.ts palette-over-photo test, --repeat-each=20 --no-deps --workers=1 --retries=0, train head 96d1aa25, dev-01 Chromium: 20/20 passed at 1440 and 20/20 passed at 390 (40 passed, 0 failed, 0 flaky).

Fix PR commits (all test-only except the token): 6fa8c53a (token + first spec edit), 8438d68f/9ca82ae7/13796a4c (spec iterations, final state = fixed ids + hugSettled), final contrast.spec.ts on PR #944 head == train head's. Earlier attempts that did not work and were dropped: fixed titles only (the Note info sheet then saw every older "Kept" note as an unlinked mention and measured its accent Link buttons), and a once-per-worker memo (workers restart per repeat).

Full matrix on 96d1aa25 is being evaluated now; results follow.

## BM14e contrast: cause, fix PR #944, interim result (head `96d1aa25`) **Failing test.** `contrast.spec.ts` "the palette keeps its contrast over a photo and over a page of huge words" (1440 and 390). Reproduced on dev-01 (`--repeat-each` on train, no deps). Three separate causes, all found by reading the failing glyph names and screenshots: 1. **Product, 1440 light:** the palette's note snippet (`.sn`, `--ink-3`) read 4.23 to 4.40 (needs 4.5) over the page of huge black words. Fix in [#944](https://git.kayg.org/kayg/calternal/pulls/944): `--glass-ink-3` (light) `#4a423b` → `#403a34` (as deep as ink 2), one token, so every quiet label on any glass follows. No local override. 2. **Test, 390:** the phone palette's glass grows on a JS spring (`lib/hug.ts`) that starts when the Recent rows arrive, after `still()` (it only sees CSS animations). The words were measured over a glass that had not reached them yet: "Try" at 1.55 to 1.87 (the 1.75 in the first report). Fix: `hugSettled()` waits until the palette's box and glass clip stay unchanged for 30 frames. It is a transient animation state, not a persisting legibility bug. 3. **Test, the "sometimes" at 1440 (the non-determinism):** the contrast project's server keeps its notes for the whole run and Playwright starts a new worker for every repeat, so `seedLists()` added 4 notes per run. After about 6 to 11 runs there were a dozen rows in the list pane, i.e. a dozen lines of 40 px / 900 black words behind the palette's results, and the rows read 2.6 (date) and 2.2 (icon). The product cannot promise AA over any amount of such text with see-through glass (and kayg's rule is no near-opaque fill), so the spec now seeds its rows ONCE by fixed id (`contrast-keep`, `contrast-gone-1..3`, only when the server doesn't have them) and measures the same rows every run. **Gate: `contrast.spec.ts` palette-over-photo test, `--repeat-each=20 --no-deps --workers=1 --retries=0`, train head `96d1aa25`, dev-01 Chromium:** 20/20 passed at 1440 and 20/20 passed at 390 (40 passed, 0 failed, 0 flaky). Fix PR commits (all test-only except the token): 6fa8c53a (token + first spec edit), 8438d68f/9ca82ae7/13796a4c (spec iterations, final state = fixed ids + hugSettled), final `contrast.spec.ts` on PR #944 head == train head's. Earlier attempts that did not work and were dropped: fixed titles only (the Note info sheet then saw every older "Kept" note as an unlinked mention and measured its accent Link buttons), and a once-per-worker memo (workers restart per repeat). Full matrix on `96d1aa25` is being evaluated now; results follow.
kayg commented 2026-11-05 03:36:16 +00:00
Author
Owner
Copy link

BM14e final: full matrix on train head 96d1aa25 (contrast fixed; RED on other failures)

Fix PR: #944. Train head 96d1aa25 = 1aed57bc + token fix + contrast.spec.ts fixes. Full matrix, first run, CI=1 --workers=1 --retries=0, SLOT_TRAIN=1 via ~/bm14e-final.sh, train locks released.

Contrast: all contrast project tests passed in the full run, and the repeat gate is 20/20 at 1440 and 20/20 at 390 (previous comment).

dev-01 Chromium (12 projects): 1400 passed, 816 skipped, 1 failed. Throttled (--no-deps): all passed.
dev-02 WebKit (8 projects): 1395 passed, 825 skipped, 2 failed.

First-run failure Train, --repeat-each=10 dev 9377d711 (+ fresh dist), --repeat-each=10 Verdict
bar-taps.spec.ts "one tap fires every control: style sheet" (chromium-390, Add a cover tap missed) 1/10 and 2/10 failed (two runs); also 3/20 failed on the PRE-FIX train head 1aed57bc 0/10 failed Reproduces on train, not on dev. Blocks. Not caused by #944 (fails on 1aed57bc too). Likely a regression from a PR in this train touching the Cover tab or the Style sheet taps; needs its own issue and a bisect.
signout.spec.ts:63 (webkit-1440) 0/10 failed 2/10 failed Reproduces on dev, not on train. Blocks under the strict rule, already in prod.
review6-signout.spec.ts:7 "cancelling one of two sign-outs…" (webkit-390, BroadcastChannel=false) 1/10 failed (BroadcastChannel=true: 1 timedOut in 10) 1/10 failed Reproduces on dev and train at the same rate. Blocks under the strict rule, already in prod.

Verdict: RED. The contrast failure is fixed (20/20 both widths, passed in the full run). The train is still blocked by one train-only regression (bar-taps style sheet "Add a cover", about 10 to 15 % of runs) and two sign-out flakes that already exist on dev.

Notes: webkit projects depend on the contrast project, which runs Chromium only; the contrast results above come from the dev-01 run.

## BM14e final: full matrix on train head `96d1aa25` (contrast fixed; RED on other failures) Fix PR: [#944](https://git.kayg.org/kayg/calternal/pulls/944). Train head `96d1aa25` = `1aed57bc` + token fix + `contrast.spec.ts` fixes. Full matrix, first run, `CI=1 --workers=1 --retries=0`, `SLOT_TRAIN=1` via `~/bm14e-final.sh`, train locks released. **Contrast: all `contrast` project tests passed in the full run, and the repeat gate is 20/20 at 1440 and 20/20 at 390** (previous comment). **dev-01 Chromium (12 projects):** 1400 passed, 816 skipped, 1 failed. Throttled (`--no-deps`): all passed. **dev-02 WebKit (8 projects):** 1395 passed, 825 skipped, 2 failed. | First-run failure | Train, `--repeat-each=10` | dev `9377d711` (+ fresh dist), `--repeat-each=10` | Verdict | |---|---|---|---| | `bar-taps.spec.ts` "one tap fires every control: style sheet" (`chromium-390`, `Add a cover` tap missed) | 1/10 and 2/10 failed (two runs); also 3/20 failed on the PRE-FIX train head `1aed57bc` | 0/10 failed | **Reproduces on train, not on dev. Blocks. Not caused by #944** (fails on `1aed57bc` too). Likely a regression from a PR in this train touching the Cover tab or the Style sheet taps; needs its own issue and a bisect. | | `signout.spec.ts:63` (`webkit-1440`) | 0/10 failed | 2/10 failed | Reproduces on dev, not on train. Blocks under the strict rule, already in prod. | | `review6-signout.spec.ts:7` "cancelling one of two sign-outs…" (`webkit-390`, BroadcastChannel=false) | 1/10 failed (BroadcastChannel=true: 1 timedOut in 10) | 1/10 failed | Reproduces on dev and train at the same rate. Blocks under the strict rule, already in prod. | **Verdict: RED.** The contrast failure is fixed (20/20 both widths, passed in the full run). The train is still blocked by one train-only regression (bar-taps style sheet "Add a cover", about 10 to 15 % of runs) and two sign-out flakes that already exist on dev. Notes: `webkit` projects depend on the `contrast` project, which runs Chromium only; the contrast results above come from the dev-01 run.
kayg commented 2026-11-05 05:14:10 +00:00
Author
Owner
Copy link

BM14e red: item 1 root cause (bar-taps "Add a cover") — a test race on the staged Page tab, fixed on #851

What fails. bar-taps.spec.ts "one tap fires every control: style sheet", chromium-390. On the unmodified spec it fails 4/20 on the train build (dev-01). Not only "Add a cover": the misses were Slate, Sage, Sand, Adjust See-through, Adjust Frost and Add a cover, always a Page-tab control.

Evidence. I logged, only after a miss, where the tap went. Every miss, whatever the label, tapped the same point (225, 912), which is below the 844 px viewport, and that box is the last button on the sheet (Slate): elementFromPoint there was null.

Cause. The Style sheet opens in steps (strip, then the Page tab's looks, rows, then colours, each in its own frame / 40 ms timer; dbbf83a9, #851) while the capsule springs. The spec waited only for the sheet to be visible, then read the button names. If the Page tab was not built yet, names.indexOf(label) was -1, and all.nth(-1) is the last button, so the spec tapped a button that was off screen (or the wrong one). A user cannot tap a control that is not drawn yet, and a tap on a drawn control lands where it is drawn (hit-testing follows the current layout), so this is not a product hit-target bug. It shows only on the train because dev has no staged Page tab (#850 to #852 are not in dev). The same lazy nth(i) could also shift when the other tabs are built 2.5 s after opening.

Fix (test only, on the culprit PR #851, commit df43f76f on port/style-2; merged forward into #852 port/style-3 as aab43742):

  • settled() waits until the sheet's buttons have stopped arriving and moving for 12 frames in a row;
  • the button to tap is found and marked by its label (not by its index), and a missing label fails by name.

Train: cherry-picked as d76f051b on train/2026-11-04-bm14e. Gate (--repeat-each=20) results follow below.

## BM14e red: item 1 root cause (bar-taps "Add a cover") — a test race on the staged Page tab, fixed on #851 **What fails.** `bar-taps.spec.ts` "one tap fires every control: style sheet", `chromium-390`. On the unmodified spec it fails 4/20 on the train build (dev-01). Not only "Add a cover": the misses were `Slate`, `Sage`, `Sand`, `Adjust See-through`, `Adjust Frost` and `Add a cover`, always a Page-tab control. **Evidence.** I logged, only after a miss, where the tap went. Every miss, whatever the label, tapped the same point **(225, 912)**, which is below the 844 px viewport, and that box is the *last* button on the sheet (`Slate`): `elementFromPoint` there was `null`. **Cause.** The Style sheet opens in steps (strip, then the Page tab's looks, rows, then colours, each in its own frame / 40 ms timer; `dbbf83a9`, #851) while the capsule springs. The spec waited only for the sheet to be visible, then read the button names. If the Page tab was not built yet, `names.indexOf(label)` was `-1`, and `all.nth(-1)` is the *last* button, so the spec tapped a button that was off screen (or the wrong one). A user cannot tap a control that is not drawn yet, and a tap on a drawn control lands where it is drawn (hit-testing follows the current layout), so this is not a product hit-target bug. It shows only on the train because dev has no staged Page tab (#850 to #852 are not in dev). The same lazy `nth(i)` could also shift when the other tabs are built 2.5 s after opening. **Fix** (test only, on the culprit PR #851, commit `df43f76f` on `port/style-2`; merged forward into #852 `port/style-3` as `aab43742`): - `settled()` waits until the sheet's buttons have stopped arriving and moving for 12 frames in a row; - the button to tap is found and marked by its label (not by its index), and a missing label fails by name. Train: cherry-picked as `d76f051b` on `train/2026-11-04-bm14e`. Gate (`--repeat-each=20`) results follow below.
kayg commented 2026-11-05 08:59:25 +00:00
Author
Owner
Copy link

BM14e red: items 2 and 3 (sign-out), a new vim flake, and the matrix reruns

All fixes are test-only. Train head now 68491858 (was 96d1aa25); the three new commits are cherry-picks (-x) of the PR commits below.

Item 2: signout.spec.ts:63 (webkit-1440) — test race, fixed in new PR #952

Failed on its last line: GET /auth/me for the old session expected 401, got 200 (4 of 15 on the train tree under load; 2/10 on dev). Signing out anyway while offline leaves the logout pending by design (wipe.ts finishLogout): it is sent when the device is back online or on the next page load. The sign-in screen (what empty() waits for) is up before that request lands, and the reload right after going online can cancel the first try. So the product is eventually consistent and the test asked too soon. It now polls (30 s) for the same 401.

Item 3: review6-signout.spec.ts:7 (webkit-390) — fixture stall on the disk, fixed in #952

Fails during the fixture's setup ("Test timeout of 30000ms exceeded while setting up device", /healthz never 200), before any sign-out step: 2 in 80 in my repeat runs. I put a stall watchdog in a scratch copy of the fixture; at 10 s it dumped the server:

PID STAT ELAPSED WCHAN                  CMD
... Dl   00:10   jbd2_log_wait_commit   .../calternal serve --create --listen 127.0.0.1:33265 ...

State D in jbd2_log_wait_commit: the server was blocked in an fsync on the busy virtual disk, because this fixture's data folder was in the disk's temp dir (the train's main harness already keeps its servers' data on /dev/shm, 1aed57bc, but this fixture spawns its own). Fix: the fixture's folder goes on /dev/shm when there is one (PERF=1 keeps the disk) and is removed after the test (after the server has exited). No timeout, retry or assertion was changed. The same slow disk is why the logout write in item 2 landed late.

Gates on the train (--no-deps --workers=1 --retries=0, train lock + SLOT_TRAIN=1, dev-02 WebKit)

Spec Command core Result
bar-taps.spec.ts "style sheet" (chromium-390, dev-01) --repeat-each=20 20/20 (was 4/20 failed)
signout.spec.ts:63 + review6-signout.spec.ts:7 (webkit-1440 and webkit-390, both BroadcastChannel variants) --repeat-each=20 120/120
vim.spec.ts ": commands in the bar" (chromium-1440) --repeat-each=20 20/20 (was 5/10 failed)
note-bars.spec.ts whole file (chromium-390) --repeat-each=30 390/390 (was 1 failure in 5 file runs)

New first-run failures in the full matrix, flaky rule

First reruns (head e541f89f, Chromium): note-bars.spec.ts:100, search-palette.spec.ts:47, vim.spec.ts:387.

  • vim.spec.ts ": commands in the bar" — blocks, fixed in new PR #954. --repeat-each=10: train 5/10 failed, dev 3/10 failed (reproduces on both, so it was already on dev). :w answers "Saved" only once the server's receipt covers the edits (packages/editor/test/vim.test.ts); the spec sent :w right after :s/:%s. It now waits for the page's own save state first (article.page[data-save=saved]).
  • note-bars.spec.ts "Attach › Divider" — blocks, fixed on #850 (port/style, 60b30bce; merged forward into #851 9741e860 and #852 63dd4ffa). It failed in two of the three full runs, 0/20 alone, 1/5 as a whole file (so it only shows with earlier tests in the same worker; the test is not on dev). With a probe in a scratch copy, two failing runs logged: pointerdown Attach at 207/240 ms, click, data-key → attach at +10 ms, back to tools 100 to 250 ms later with nothing focused. The test's poll for attach missed so short a visit. I could not catch what closes the menu (0 of 25 failed with the heavier probe), so I do not claim a product bug; the test tapped as soon as the bar showed, while its sibling Attach test first waits for still(page). It now does the same. Gate above: 390/390. Open risk: if the menu really does close itself 100 to 250 ms after a very early tap, that is a product issue worth its own look; the evidence is the two logs above.
  • search-palette.spec.ts:47 — does not block. --repeat-each=10 on the train: 10/10 passed; passed in the following full run. Needs a stability issue.

Second full runs (head a632c60f): Chromium 1400 passed / 1 failed (note-bars Divider, above); throttled all passed. WebKit: 1395 passed / 2 failed (touch-select.spec.ts:163, trash-header-geometry.spec.ts:5, both webkit-390, "Test timeout of 30000ms exceeded while setting up page", while another agent's cargo job was running on dev-02). Flaky rule, --repeat-each=10: train 20/20 passed and dev 20/20 passed (both tests), so neither blocks; each needs a stability issue.

The Divider fix changed the head to 68491858; I am rerunning the full Chromium matrix on it now (the Divider test is skipped on WebKit, so the WebKit run stands).

## BM14e red: items 2 and 3 (sign-out), a new vim flake, and the matrix reruns All fixes are test-only. Train head now `68491858` (was `96d1aa25`); the three new commits are cherry-picks (`-x`) of the PR commits below. ### Item 2: `signout.spec.ts:63` (webkit-1440) — test race, fixed in new PR [#952](https://git.kayg.org/kayg/calternal/pulls/952) Failed on its last line: `GET /auth/me` for the old session expected 401, got 200 (4 of 15 on the train tree under load; 2/10 on dev). Signing out *anyway while offline* leaves the logout **pending by design** (`wipe.ts` `finishLogout`): it is sent when the device is back online or on the next page load. The sign-in screen (what `empty()` waits for) is up before that request lands, and the reload right after going online can cancel the first try. So the product is eventually consistent and the test asked too soon. It now polls (30 s) for the same 401. ### Item 3: `review6-signout.spec.ts:7` (webkit-390) — fixture stall on the disk, fixed in #952 Fails during the *fixture's setup* ("Test timeout of 30000ms exceeded while setting up device", `/healthz` never 200), before any sign-out step: 2 in 80 in my repeat runs. I put a stall watchdog in a scratch copy of the fixture; at 10 s it dumped the server: ``` PID STAT ELAPSED WCHAN CMD ... Dl 00:10 jbd2_log_wait_commit .../calternal serve --create --listen 127.0.0.1:33265 ... ``` State `D` in `jbd2_log_wait_commit`: the server was blocked in an fsync on the busy virtual disk, because this fixture's data folder was in the disk's temp dir (the train's main harness already keeps its servers' data on `/dev/shm`, `1aed57bc`, but this fixture spawns its own). Fix: the fixture's folder goes on `/dev/shm` when there is one (`PERF=1` keeps the disk) and is removed after the test (after the server has exited). No timeout, retry or assertion was changed. The same slow disk is why the logout write in item 2 landed late. ### Gates on the train (`--no-deps --workers=1 --retries=0`, train lock + `SLOT_TRAIN=1`, dev-02 WebKit) | Spec | Command core | Result | |---|---|---| | `bar-taps.spec.ts` "style sheet" (chromium-390, dev-01) | `--repeat-each=20` | **20/20** (was 4/20 failed) | | `signout.spec.ts:63` + `review6-signout.spec.ts:7` (webkit-1440 and webkit-390, both BroadcastChannel variants) | `--repeat-each=20` | **120/120** | | `vim.spec.ts` ": commands in the bar" (chromium-1440) | `--repeat-each=20` | **20/20** (was 5/10 failed) | | `note-bars.spec.ts` whole file (chromium-390) | `--repeat-each=30` | **390/390** (was 1 failure in 5 file runs) | ### New first-run failures in the full matrix, flaky rule First reruns (head `e541f89f`, Chromium): `note-bars.spec.ts:100`, `search-palette.spec.ts:47`, `vim.spec.ts:387`. - **`vim.spec.ts` ": commands in the bar" — blocks, fixed in new PR [#954](https://git.kayg.org/kayg/calternal/pulls/954).** `--repeat-each=10`: train 5/10 failed, dev 3/10 failed (reproduces on both, so it was already on dev). `:w` answers "Saved" only once the server's receipt covers the edits (`packages/editor/test/vim.test.ts`); the spec sent `:w` right after `:s`/`:%s`. It now waits for the page's own save state first (`article.page[data-save=saved]`). - **`note-bars.spec.ts` "Attach › Divider" — blocks, fixed on #850 (`port/style`, `60b30bce`; merged forward into #851 `9741e860` and #852 `63dd4ffa`).** It failed in two of the three full runs, 0/20 alone, 1/5 as a whole file (so it only shows with earlier tests in the same worker; the test is not on dev). With a probe in a scratch copy, two failing runs logged: `pointerdown Attach` at 207/240 ms, `click`, `data-key` → `attach` at +10 ms, back to `tools` 100 to 250 ms later with nothing focused. The test's poll for `attach` missed so short a visit. I could not catch what closes the menu (0 of 25 failed with the heavier probe), so I do **not** claim a product bug; the test tapped as soon as the bar showed, while its sibling Attach test first waits for `still(page)`. It now does the same. Gate above: 390/390. Open risk: if the menu really does close itself 100 to 250 ms after a very early tap, that is a product issue worth its own look; the evidence is the two logs above. - **`search-palette.spec.ts:47` — does not block.** `--repeat-each=10` on the train: 10/10 passed; passed in the following full run. Needs a stability issue. Second full runs (head `a632c60f`): Chromium 1400 passed / 1 failed (`note-bars` Divider, above); throttled all passed. WebKit: 1395 passed / 2 failed (`touch-select.spec.ts:163`, `trash-header-geometry.spec.ts:5`, both webkit-390, "Test timeout of 30000ms exceeded while setting up page", while another agent's cargo job was running on dev-02). Flaky rule, `--repeat-each=10`: **train 20/20 passed and dev 20/20 passed** (both tests), so neither blocks; each needs a stability issue. The Divider fix changed the head to `68491858`; I am rerunning the full Chromium matrix on it now (the Divider test is skipped on WebKit, so the WebKit run stands).
kayg commented 2026-11-05 10:12:15 +00:00
Author
Owner
Copy link

BM14e final: GREEN on train head 68491858

Full matrix, first run, CI=1 --workers=1 --retries=0, SLOT_TRAIN=1 under the train lock (released; no ~/.train.lock left on either host).

Host Projects Head Result
dev-01 Chromium 12 projects (incl. contrast, flow, passkeys, admin-*) 68491858 1401 passed, 0 failed, 0 flaky, 816 skipped; throttled (--no-deps) passed
dev-02 WebKit 8 projects a632c60f 1395 passed, 2 failed (touch-select.spec.ts:163, trash-header-geometry.spec.ts:5, "setting up page" timeouts under another agent's load); --repeat-each=10: train 20/20, dev 20/20, so flaky, non-blocking.

68491858 differs from a632c60f only in note-bars.spec.ts (its Divider test, skipped on WebKit), so the WebKit run stands for the final head.

Fix PRs: #851 (df43f76f bar-taps; merged forward into #852, aab43742), #850 (60b30bce note-bars Divider; merged forward as #851 9741e860, #852 63dd4ffa), new #952 (sign-out specs + fixture), new #954 (vim :w). Train cherry-picks: d76f051b, a57e1351, e541f89f, a632c60f, 68491858.

Needs stability issues (flaky rule, not blocking): search-palette.spec.ts:47 (once in a full run, 10/10 on the train), touch-select.spec.ts:163 and trash-header-geometry.spec.ts:5 (WebKit, setup timeouts under host load).

Open risks: (1) If the Attach menu really closes itself 100 to 250 ms after a very early tap (two probe logs, not reproduced with heavier probes), that could be a product issue; the test now waits for still(page) like its sibling. (2) Merging port/style forward into #851 also pulled in note-bars-lazy.css, which the stack branch lacked; the train already had it. (3) #952's fixture uses its own small /dev/shm helper because the train's dataDir() isn't in dev yet; fold them together when 1aed57bc lands.

## BM14e final: GREEN on train head `68491858` Full matrix, first run, `CI=1 --workers=1 --retries=0`, `SLOT_TRAIN=1` under the train lock (released; no `~/.train.lock` left on either host). | Host | Projects | Head | Result | |---|---|---|---| | dev-01 Chromium | 12 projects (incl. `contrast`, `flow`, `passkeys`, `admin-*`) | `68491858` | **1401 passed, 0 failed, 0 flaky**, 816 skipped; throttled (`--no-deps`) passed | | dev-02 WebKit | 8 projects | `a632c60f` | 1395 passed, 2 failed (`touch-select.spec.ts:163`, `trash-header-geometry.spec.ts:5`, "setting up page" timeouts under another agent's load); `--repeat-each=10`: train 20/20, dev 20/20, so **flaky, non-blocking**. | `68491858` differs from `a632c60f` only in `note-bars.spec.ts` (its Divider test, skipped on WebKit), so the WebKit run stands for the final head. **Fix PRs:** #851 (`df43f76f` bar-taps; merged forward into #852, `aab43742`), #850 (`60b30bce` note-bars Divider; merged forward as #851 `9741e860`, #852 `63dd4ffa`), new #952 (sign-out specs + fixture), new #954 (vim `:w`). Train cherry-picks: `d76f051b`, `a57e1351`, `e541f89f`, `a632c60f`, `68491858`. **Needs stability issues (flaky rule, not blocking):** `search-palette.spec.ts:47` (once in a full run, 10/10 on the train), `touch-select.spec.ts:163` and `trash-header-geometry.spec.ts:5` (WebKit, setup timeouts under host load). **Open risks:** (1) If the Attach menu really closes itself 100 to 250 ms after a very early tap (two probe logs, not reproduced with heavier probes), that could be a product issue; the test now waits for `still(page)` like its sibling. (2) Merging `port/style` forward into #851 also pulled in `note-bars-lazy.css`, which the stack branch lacked; the train already had it. (3) #952's fixture uses its own small `/dev/shm` helper because the train's `dataDir()` isn't in dev yet; fold them together when `1aed57bc` lands.
kayg commented 2026-11-05 23:13:03 +00:00
Author
Owner
Copy link

Flaky lane (U4): the 'Attach menu closes itself' observation is a real product bug, not a test artefact. Cause and fix in PR #977.

Flaky lane (U4): the 'Attach menu closes itself' observation is a real product bug, not a test artefact. Cause and fix in PR #977.
kayg commented 2026-11-07 17:53:08 +00:00
Author
Owner
Copy link

Landed in dev via 12103f1c97 — closing.

Landed in dev via 12103f1c973f1e810dd4a9a3bff5a705514d70b0 — closing.
kayg closed this pull request 2026-11-07 17:53:09 +00:00
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!914
No description provided.