kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 190 Pull requests 61 Projects Releases Packages Wiki Activity Actions 20

test: stabilize Playwright readiness and scroll observation #856

Closed
kayg wants to merge 0 commits from fix/playwright-train-flakes into dev
pull from: fix/playwright-train-flakes
merge into: kayg:dev
Conversation 10 Commits - Files changed -
kayg commented 2026-11-02 12:01:04 +00:00
Owner
Copy link

Make browser fixtures wait for the state they exercise: search typo readiness, the phone account-section landing, save receipts, link-card geometry and preference replies. The refusal integration loads its module graph before measurement; list-fit records the first qualifying browser scroll under its existing deadline.

Current validation and investigation. The historical renamed-link failure remains unattributed; its test and product code are unchanged. Earlier qualification.

Make browser fixtures wait for the state they exercise: search typo readiness, the phone account-section landing, save receipts, link-card geometry and preference replies. The refusal integration loads its module graph before measurement; list-fit records the first qualifying browser scroll under its existing deadline. [Current validation and investigation](https://git.kayg.org/kayg/calternal/pulls/856#issuecomment-76224). The historical renamed-link failure remains unattributed; its test and product code are unchanged. [Earlier qualification](https://git.kayg.org/kayg/calternal/pulls/856#issuecomment-76227).
flake-wall-artifacts.tar.gz
87 KiB 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
e2e: drop the recovery receipt wait (the receipt tag does not fire on rejoin)
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
85845e3d11 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
kayg commented 2026-11-02 18:43:00 +00:00
Author
Owner
Copy link

BM9 search-corpus repair plan, before source push.

Every cumulative train merge from #779 through #856 passed the isolated spec (1492 queries / 0 misses each), so the requested cold-run sequence does not identify a first bad product merge. Full results: #869 (comment).

The setup contract in this PR's apps/web/e2e/search-corpus.spec.ts:74-76 is incomplete: first/last exact results do not establish typo-vocabulary readiness. crates/search/src/index.rs:229-275 deliberately caches expensive vocabularies, serves the prior vocabulary while stale, and delays another background build until its five-second refresh interval. QueryOutput.indexing only tracks the document indexer (crates/search/src/lib.rs:208), not this vocabulary. Those production paths are unchanged by BM9.

A temporary probe retained outside the repository seeds 6000 background notes / 300000 generated terms, warms a typo on that older vocabulary, then executes the unchanged corpus assertions. It failed with 48/1492 misses (limit 2%). The first mrmalade response was {"hits":[],"indexing":false,"matched":0,...} despite the exact endpoint readiness checks succeeding. This demonstrates the same missing-typo failure mechanism; it does not prove the historical train's exact cache state or reproduce its exact 1106 count.

Repair here: wait for each corpus note's exact hit, then for mrmalade to find this run's marmalade note before starting the measured/asserted query loop. Keep all 1492 rank/typo assertions and the 2% limit. Use signal polling, no sleeps or retry setting changes. Enable both desktop gate projects (WebKit currently skips these tests). Retain the existing corpus test as the regression gate and rerun the warmed-vocabulary probe red/green, then 3 independent invocations per requested project on the full BM9 scratch merge. No production search algorithm change, no train branch change. Historical first-bad source attribution remains unproven; #856 owns the setup repair.

BM9 search-corpus repair plan, before source push. Every cumulative train merge from #779 through #856 passed the isolated spec (1492 queries / 0 misses each), so the requested cold-run sequence does **not** identify a first bad product merge. Full results: https://git.kayg.org/kayg/calternal/pulls/869#issuecomment-75542. The setup contract in this PR's `apps/web/e2e/search-corpus.spec.ts:74-76` is incomplete: first/last exact results do not establish typo-vocabulary readiness. `crates/search/src/index.rs:229-275` deliberately caches expensive vocabularies, serves the prior vocabulary while stale, and delays another background build until its five-second refresh interval. `QueryOutput.indexing` only tracks the document indexer (`crates/search/src/lib.rs:208`), not this vocabulary. Those production paths are unchanged by BM9. A temporary probe retained outside the repository seeds 6000 background notes / 300000 generated terms, warms a typo on that older vocabulary, then executes the unchanged corpus assertions. It failed with **48/1492 misses** (limit 2%). The first `mrmalade` response was `{"hits":[],"indexing":false,"matched":0,...}` despite the exact endpoint readiness checks succeeding. This demonstrates the same missing-typo failure mechanism; it does not prove the historical train's exact cache state or reproduce its exact 1106 count. Repair here: wait for each corpus note's exact hit, then for `mrmalade` to find this run's marmalade note before starting the measured/asserted query loop. Keep all 1492 rank/typo assertions and the 2% limit. Use signal polling, no sleeps or retry setting changes. Enable both desktop gate projects (WebKit currently skips these tests). Retain the existing corpus test as the regression gate and rerun the warmed-vocabulary probe red/green, then 3 independent invocations per requested project on the full BM9 scratch merge. No production search algorithm change, no train branch change. Historical first-bad source attribution remains unproven; #856 owns the setup repair.
kayg commented 2026-11-02 18:49:43 +00:00
Author
Owner
Copy link

BM9 source repairs and final targeted gates.

PR Fix commit / new source head Change
#827 76f11dc4ba06468a2a0ed4d4592cea6038206696 collapse optional caller + notes.write scope check into a let-chain at crates/notes/src/room/edit.rs:43; authorization behavior/order unchanged
#856 b89e39bb313bb06b92da274124c6bbde30e2250a wait for all corpus exact hits and a fresh typo result before the 1492-query gate; actually execute both desktop projects

Confirmed setup fault and attribution limit. The readiness checks at apps/web/e2e/search-corpus.spec.ts:74-76 in parent 85845e3d checked only the first/last exact words. crates/search/src/index.rs:229-275 serves an older expensive typo vocabulary until its five-second background refresh is eligible/completed. Document indexing and vocabulary refresh are separate; crates/search/src/lib.rs reports only document indexing in QueryOutput.indexing. Therefore exact hits, even with indexing:false, do not establish that newly seeded words can be corrected. The repair is at current search-corpus.spec.ts:71-77: establish each exact corpus hit, then poll mrmalade until it finds this run's marmalade note. No fixed wait, longer timeout, weakened rank limit, removed query, or production search change.

All 14 cumulative merges passed cold (1492 queries / 0 misses each). Thus there is no demonstrated first-bad merge from the requested isolated sequence, and this investigation cannot honestly attribute the historical 1106 misses to a particular new product commit. #856 owns the incomplete corpus setup and its repair. The full-train observation remains real; it has not been dismissed as a flake. Complete cumulative sequence and commands.

Red/green reproduction. A throwaway probe outside the repository writes 6000 background notes, each with 50 unique generated alphabetic terms, after the native test server starts. It waits for warmvocabulary and typo warmvocabulry to find a separate warm-up note, then executes the corpus setup and unchanged 1492-query assertions. Before the repair: 48/1492 misses, one corpus test failed and the ranking test passed. Its first typo response was:

{"hits":[],"indexing":false,"matched":0}

After the repair, the same probe: 0/1492 misses, 2 tests passed. The first typo response corrected mrmalade to marmalade and returned the current run's note. This proves the readiness gap and fixes that reproduced failure mechanism; it does not recreate the historical train's exact cache state/count. The existing full corpus/ranking tests and all 1492 assertions remain. The new typo-readiness assertion also fails if the vocabulary never catches up.

Full BM9 scratch qualification. On dev-01, scratch 73abdbce9821f30e903668f8fe7781028410a1ec is original train head 383eb8f8 plus clean --no-ff merges of the two source repairs. Verified source merge parents are the full heads above. No train branch was changed or pushed.

Gate Result
requested workspace clippy on #827 + temporary #779 Nix tooling overlay PASS, exit 0
workspace clippy on full scratch PASS, exit 0
locked server build on full scratch PASS, exit 0
frozen web dependencies + web build PASS
chromium-1440, independent runs 1 / 2 / 3 2 passed each; 1492 queries / 0 misses each
webkit-1440, executed runs 2 / 3 / 4 2 passed each; 1492 queries / 0 misses each
control and VM worktree status / diff check clean

Totals for the six executed final spec invocations: 12 passed, 0 failed, 0 skipped; 8952 corpus queries, 0 misses. One additional WebKit invocation (run 1) failed before any test ran because native config selected an already-used port (http://127.0.0.1:13393/). That startup error is retained; an additional independent invocation supplied the third executed WebKit run. The original six-invocation wrapper retains exit 1 for that startup failure; the additional run exits 0. No attempted failing browser case was retried or hidden.

Exact commands, all builds/tests on dev-01 via Nix and ~/bin/slot, with CI=1, CARGO_PROFILE_DEV_DEBUG=0, CARGO_PROFILE_TEST_DEBUG=0, CARGO_INCREMENTAL=0:

cargo clippy --workspace --all-targets --locked -j4 -- -D warnings
cargo build --locked -j4
bun install --frozen-lockfile
# apps/web
bun run build
# CALTERNAL_BIN=<scratch>/target/debug/calternal E2E_DIST=<scratch>/apps/web/dist
bunx playwright test e2e/search-corpus.spec.ts --project=chromium-1440 --no-deps --retries=0 --workers=1 --reporter=line
bunx playwright test e2e/search-corpus.spec.ts --project=webkit-1440 --no-deps --retries=0 --workers=1 --reporter=line

Each final test invocation uses a fresh TMPDIR and native server/data setup. WebKit's prior unconditional skip in this spec was removed for the requested 1440px gate. No wall-clock performance assertion was added. Full workspace tests/full browser suite were not rerun; BM9's existing results cover those unrelated gates. All own jobs finished. Own target, dist, dependency directory, large probe Homes and traces were removed from the VM; only small logs/scripts remain outside the repository. No settings, merges, or deployment.

Open risk: historical first-bad product attribution is unresolved despite exhaustive cold cumulative runs. Treat this as a verified corpus readiness repair with green targeted qualification, not proof that the original full train is now green without a new train run.

BM9 source repairs and final targeted gates. | PR | Fix commit / new source head | Change | |---|---|---| | #827 | `76f11dc4ba06468a2a0ed4d4592cea6038206696` | collapse optional caller + notes.write scope check into a let-chain at `crates/notes/src/room/edit.rs:43`; authorization behavior/order unchanged | | #856 | `b89e39bb313bb06b92da274124c6bbde30e2250a` | wait for all corpus exact hits and a fresh typo result before the 1492-query gate; actually execute both desktop projects | **Confirmed setup fault and attribution limit.** The readiness checks at `apps/web/e2e/search-corpus.spec.ts:74-76` in parent `85845e3d` checked only the first/last exact words. `crates/search/src/index.rs:229-275` serves an older expensive typo vocabulary until its five-second background refresh is eligible/completed. Document indexing and vocabulary refresh are separate; `crates/search/src/lib.rs` reports only document indexing in `QueryOutput.indexing`. Therefore exact hits, even with `indexing:false`, do not establish that newly seeded words can be corrected. The repair is at current `search-corpus.spec.ts:71-77`: establish each exact corpus hit, then poll `mrmalade` until it finds this run's marmalade note. No fixed wait, longer timeout, weakened rank limit, removed query, or production search change. All **14 cumulative merges passed cold** (1492 queries / 0 misses each). Thus there is **no demonstrated first-bad merge from the requested isolated sequence**, and this investigation cannot honestly attribute the historical 1106 misses to a particular new product commit. #856 owns the incomplete corpus setup and its repair. The full-train observation remains real; it has not been dismissed as a flake. [Complete cumulative sequence and commands](https://git.kayg.org/kayg/calternal/pulls/869#issuecomment-75542). **Red/green reproduction.** A throwaway probe outside the repository writes 6000 background notes, each with 50 unique generated alphabetic terms, after the native test server starts. It waits for `warmvocabulary` and typo `warmvocabulry` to find a separate warm-up note, then executes the corpus setup and unchanged 1492-query assertions. Before the repair: 48/1492 misses, one corpus test failed and the ranking test passed. Its first typo response was: ```json {"hits":[],"indexing":false,"matched":0} ``` After the repair, the same probe: 0/1492 misses, 2 tests passed. The first typo response corrected `mrmalade` to `marmalade` and returned the current run's note. This proves the readiness gap and fixes that reproduced failure mechanism; it does not recreate the historical train's exact cache state/count. The existing full corpus/ranking tests and all 1492 assertions remain. The new typo-readiness assertion also fails if the vocabulary never catches up. **Full BM9 scratch qualification.** On dev-01, scratch `73abdbce9821f30e903668f8fe7781028410a1ec` is original train head `383eb8f8` plus clean --no-ff merges of the two source repairs. Verified source merge parents are the full heads above. No train branch was changed or pushed. | Gate | Result | |---|---| | requested workspace clippy on #827 + temporary #779 Nix tooling overlay | PASS, exit 0 | | workspace clippy on full scratch | PASS, exit 0 | | locked server build on full scratch | PASS, exit 0 | | frozen web dependencies + web build | PASS | | chromium-1440, independent runs 1 / 2 / 3 | 2 passed each; 1492 queries / 0 misses each | | webkit-1440, executed runs 2 / 3 / 4 | 2 passed each; 1492 queries / 0 misses each | | control and VM worktree status / diff check | clean | Totals for the six executed final spec invocations: **12 passed, 0 failed, 0 skipped; 8952 corpus queries, 0 misses.** One additional WebKit invocation (run 1) failed before any test ran because native config selected an already-used port (`http://127.0.0.1:13393/`). That startup error is retained; an additional independent invocation supplied the third executed WebKit run. The original six-invocation wrapper retains exit 1 for that startup failure; the additional run exits 0. No attempted failing browser case was retried or hidden. Exact commands, all builds/tests on dev-01 via Nix and `~/bin/slot`, with CI=1, CARGO_PROFILE_DEV_DEBUG=0, CARGO_PROFILE_TEST_DEBUG=0, CARGO_INCREMENTAL=0: ```sh cargo clippy --workspace --all-targets --locked -j4 -- -D warnings cargo build --locked -j4 bun install --frozen-lockfile # apps/web bun run build # CALTERNAL_BIN=<scratch>/target/debug/calternal E2E_DIST=<scratch>/apps/web/dist bunx playwright test e2e/search-corpus.spec.ts --project=chromium-1440 --no-deps --retries=0 --workers=1 --reporter=line bunx playwright test e2e/search-corpus.spec.ts --project=webkit-1440 --no-deps --retries=0 --workers=1 --reporter=line ``` Each final test invocation uses a fresh TMPDIR and native server/data setup. WebKit's prior unconditional skip in this spec was removed for the requested 1440px gate. No wall-clock performance assertion was added. Full workspace tests/full browser suite were not rerun; BM9's existing results cover those unrelated gates. All own jobs finished. Own target, dist, dependency directory, large probe Homes and traces were removed from the VM; only small logs/scripts remain outside the repository. No settings, merges, or deployment. Open risk: historical first-bad product attribution is unresolved despite exhaustive cold cumulative runs. Treat this as a verified corpus readiness repair with green targeted qualification, not proof that the original full train is now green without a new train run.
kayg commented 2026-11-02 23:10:03 +00:00
Author
Owner
Copy link

BM11 sign-out waiting repair plan. The matched matrix reproduced the original review4 typing timeout twice under one ordinary release build. A read-only pointer probe now reproduces the same mechanism on origin/dev: pointerdown targets Sign out; the existing account-section landing runs; pointerup and click target DIV instead. The button never activates. Settings.svelte schedules this scroll after the section is rendered; that code is identical on dev and train. The current accountReady helper only waits for heading/passkey content.

Repair in this PR: activate the settings tab and wait until the phone Account heading reaches its actual scroll-margin landing position before clicking Sign out. Retain the passkey readiness check. No fixed sleep, longer timeout, weakened logout/editing assertion, or production change. The existing original sign-out cases are the red/green regression gates. A separate pointer probe checks cancellation too. Qualification: 15 fresh zero-retry executions of each requested case on the original train product with the repaired tests, plus a green pointer probe; preserve every earlier failure and both probe startup errors in the BM11 report. Editor has not reproduced in its matched matrix; its historical cause is not established.

BM11 sign-out waiting repair plan. The matched matrix reproduced the original review4 typing timeout twice under one ordinary release build. A read-only pointer probe now reproduces the same mechanism on origin/dev: pointerdown targets Sign out; the existing account-section landing runs; pointerup and click target DIV instead. The button never activates. Settings.svelte schedules this scroll after the section is rendered; that code is identical on dev and train. The current accountReady helper only waits for heading/passkey content. Repair in this PR: activate the settings tab and wait until the phone Account heading reaches its actual scroll-margin landing position before clicking Sign out. Retain the passkey readiness check. No fixed sleep, longer timeout, weakened logout/editing assertion, or production change. The existing original sign-out cases are the red/green regression gates. A separate pointer probe checks cancellation too. Qualification: 15 fresh zero-retry executions of each requested case on the original train product with the repaired tests, plus a green pointer probe; preserve every earlier failure and both probe startup errors in the BM11 report. Editor has not reproduced in its matched matrix; its historical cause is not established.
kayg commented 2026-11-02 23:25:13 +00:00
Author
Owner
Copy link

BM11 sign-out repair pushed: commit/head 26548949efef7e6e4109ccc8e158f21a12f2b026 (fix/playwright-train-flakes).

The shared accountReady helper now foregrounds settings and waits for the phone Account heading to reach its actual section landing before Sign out receives a pointer click. It retains passkey readiness and all existing product assertions, without a fixed sleep or timeout increase.

Baseline pointer probes confirmed both pre-existing flakes: typing 13 passed / 2 lost clicks; cancellation 14 passed / 1 lost click. Both failures show pointer-down on Sign out, section scroll, pointer-up/click on DIV. The original train typing timeout reproduced twice before repair.

Final original-train product qualification with this exact fixture: typing 15/15, cancellation 15/15, editor 15/15; pointer probe 3/3, zero retries. git diff --check and the train no-evidence gate passed. Editor's historical failure remains unattributed after 37 train + 22 dev passes; no editor patch is included.

Full per-case verdicts, matched/replacement counts, exact commands, binary provenance, complete failed-attempt accounting and sanitized attachment: #883 (comment). All owned jobs finished and large VM outputs were cleaned. No current-head CI status or review was published when checked.

BM11 sign-out repair pushed: commit/head `26548949efef7e6e4109ccc8e158f21a12f2b026` (`fix/playwright-train-flakes`). The shared accountReady helper now foregrounds settings and waits for the phone Account heading to reach its actual section landing before Sign out receives a pointer click. It retains passkey readiness and all existing product assertions, without a fixed sleep or timeout increase. Baseline pointer probes confirmed both pre-existing flakes: typing 13 passed / 2 lost clicks; cancellation 14 passed / 1 lost click. Both failures show pointer-down on Sign out, section scroll, pointer-up/click on DIV. The original train typing timeout reproduced twice before repair. Final original-train product qualification with this exact fixture: typing **15/15**, cancellation **15/15**, editor **15/15**; pointer probe **3/3**, zero retries. git diff --check and the train no-evidence gate passed. Editor's historical failure remains unattributed after 37 train + 22 dev passes; no editor patch is included. Full per-case verdicts, matched/replacement counts, exact commands, binary provenance, complete failed-attempt accounting and sanitized attachment: https://git.kayg.org/kayg/calternal/pulls/883#issuecomment-76036. All owned jobs finished and large VM outputs were cleaned. No current-head CI status or review was published when checked.
kayg commented 2026-11-03 01:23:48 +00:00
Author
Owner
Copy link

Flake-wall investigation started from #856 head 26548949, owned control worktree flake-wall. The requested similarly named fix/parallel-test-flakes is a separate branch; repairs will target this PR's actual fix/playwright-train-flakes branch.

Plan: reserve idle dev-01 using the new train gate, build independent BM12 (1e776f9e) and origin/dev (55f35fcd) products, then run 15 attempts per product idle and 15 per product with one ordinary slot build as load, identical train test definitions. Record all failures and timing samples; bisect only if a product difference is established. Separately reproduce refusal navigation and renamed-link failures before changing their readiness contract, then 15 clean listed-project attempts per repaired case. No retries, larger limits, other-job termination, train edits, merges, or repo evidence. Slot behavior will be checked with live/stale locks, train bypass, and an already-running job on both VMs; the control common brief gets the short Trains rule.

Flake-wall investigation started from #856 head 26548949, owned control worktree `flake-wall`. The requested similarly named fix/parallel-test-flakes is a separate branch; repairs will target this PR's actual fix/playwright-train-flakes branch. Plan: reserve idle dev-01 using the new train gate, build independent BM12 (1e776f9e) and origin/dev (55f35fcd) products, then run 15 attempts per product idle and 15 per product with one ordinary slot build as load, identical train test definitions. Record all failures and timing samples; bisect only if a product difference is established. Separately reproduce refusal navigation and renamed-link failures before changing their readiness contract, then 15 clean listed-project attempts per repaired case. No retries, larger limits, other-job termination, train edits, merges, or repo evidence. Slot behavior will be checked with live/stale locks, train bypass, and an already-running job on both VMs; the control common brief gets the short Trains rule.
kayg commented 2026-11-03 01:28:39 +00:00
Author
Owner
Copy link

Train gate installed and verified on both build VMs.

Paths: claude@calternal-dev-01-compute02:~/bin/slot and cua@calternal-dev-02-std:~/bin/slot; original scripts retained at ~/bin/slot.pre-flake-wall. Control-host code common brief /home/kayg/Developer/Worktrees/BRIEF-COMMON.md now has the short Trains rule, ready for kayg to copy to minipc.

Lock contract: write <train-name> <owner-pid>\n to ~/.train.lock, keep that owner alive, remove its lock in an EXIT trap, and run train jobs with SLOT_TRAIN=1. A gone PID makes the lock stale. New ordinary slot calls check the gate before searching and again after acquiring a slot. Existing jobs are unaffected. No job was killed. For the matched experiment, the reservation owner additionally waited for every existing slot lock to drain before preparation.

Fresh behavior verification after the final script update: 6/6 checks passed on each VM, in a temporary HOME with the actual installed script: live lock blocks a new ordinary job; SLOT_TRAIN=1 bypasses it; an ordinary job already running finishes while the lock remains; removing the lock releases the waiting job; a nonexistent PID does not block; a live lock without a trailing newline still blocks. The unterminated-file check failed against the first candidate (ordinary job incorrectly started), then passed after the parser correction. bash -n passed on each installed candidate. Test scratch directories were removed.

Installed script (evidence kept here, not in the repo):

#!/usr/bin/env bash
# slot <cmd...>: run a heavy command holding one of ~/.slots/max build slots.
# Trains write '<train-name> <owner-pid>' to ~/.train.lock and use SLOT_TRAIN=1.
set -u
d="$HOME/.slots"; mkdir -p "$d"; n=$(cat "$d/max" 2>/dev/null || echo 2)
train_active() {
  [[ ${SLOT_TRAIN:-0} == 1 ]] && return 1
  local name= pid= extra=
  # read still returns the fields when a writer omits the final newline.
  read -r name pid extra 2>/dev/null < "$HOME/.train.lock" || :
  [[ $pid =~ ^[0-9]+$ && $pid -gt 0 ]] && kill -0 "$pid" 2>/dev/null
}
while :; do
  if train_active; then
    sleep 15
    continue
  fi
  for i in $(seq 1 "$n"); do
    exec {fd}>"$d/$i.lock"
    if flock -n "$fd"; then
      # A train may have arrived while we were finding a free slot.
      if train_active; then
        exec {fd}>&-
        break
      fi
      echo "$$ $(date -Is) $*" > "$d/$i.holder"
      "$@"; rc=$?
      : > "$d/$i.holder"; exit $rc
    fi
    exec {fd}>&-
  done
  sleep 15
done
Train gate installed and verified on both build VMs. Paths: `claude@calternal-dev-01-compute02:~/bin/slot` and `cua@calternal-dev-02-std:~/bin/slot`; original scripts retained at `~/bin/slot.pre-flake-wall`. Control-host `code` common brief `/home/kayg/Developer/Worktrees/BRIEF-COMMON.md` now has the short Trains rule, ready for kayg to copy to minipc. Lock contract: write `<train-name> <owner-pid>\n` to `~/.train.lock`, keep that owner alive, remove its lock in an EXIT trap, and run train jobs with `SLOT_TRAIN=1`. A gone PID makes the lock stale. New ordinary slot calls check the gate before searching and again after acquiring a slot. Existing jobs are unaffected. No job was killed. For the matched experiment, the reservation owner additionally waited for every existing slot lock to drain before preparation. Fresh behavior verification after the final script update: **6/6 checks passed on each VM**, in a temporary HOME with the actual installed script: live lock blocks a new ordinary job; `SLOT_TRAIN=1` bypasses it; an ordinary job already running finishes while the lock remains; removing the lock releases the waiting job; a nonexistent PID does not block; a live lock without a trailing newline still blocks. The unterminated-file check failed against the first candidate (ordinary job incorrectly started), then passed after the parser correction. `bash -n` passed on each installed candidate. Test scratch directories were removed. Installed script (evidence kept here, not in the repo): ```bash #!/usr/bin/env bash # slot <cmd...>: run a heavy command holding one of ~/.slots/max build slots. # Trains write '<train-name> <owner-pid>' to ~/.train.lock and use SLOT_TRAIN=1. set -u d="$HOME/.slots"; mkdir -p "$d"; n=$(cat "$d/max" 2>/dev/null || echo 2) train_active() { [[ ${SLOT_TRAIN:-0} == 1 ]] && return 1 local name= pid= extra= # read still returns the fields when a writer omits the final newline. read -r name pid extra 2>/dev/null < "$HOME/.train.lock" || : [[ $pid =~ ^[0-9]+$ && $pid -gt 0 ]] && kill -0 "$pid" 2>/dev/null } while :; do if train_active; then sleep 15 continue fi for i in $(seq 1 "$n"); do exec {fd}>"$d/$i.lock" if flock -n "$fd"; then # A train may have arrived while we were finding a free slot. if train_active; then exec {fd}>&- break fi echo "$$ $(date -Is) $*" > "$d/$i.holder" "$@"; rc=$? : > "$d/$i.holder"; exit $rc fi exec {fd}>&- done sleep 15 done ```
kayg commented 2026-11-03 01:43:33 +00:00
Author
Owner
Copy link

Matched list-fit matrix complete, original 1500 ms Chromium assertion retained, zero retries. Independent BM12 product 1e776f9e and origin/dev 55f35fcd, built separately on dev-01; same targeted test definition plus read-only scroll-event diagnostics. The measurement is captured before diagnostic output to avoid charging that output to the assertion. The train reservation drained all existing slot jobs before setup.

Condition/product Passed Failed Playwright observation median / max, ms Actual scroll event median / max after command start, ms
idle-base 15/15 0 587 / 860 486 / 597
idle-train 15/15 0 573 / 873 478 / 621
loaded-base 15/15 0 839 / 1126 709 / 877
loaded-train 15/15 0 817 / 1142 608 / 898

The one controlled load was SLOT_TRAIN=1 CARGO_TARGET_DIR=$HOME/flake-wall/load/target ~/bin/slot nix develop -c bash -c "cargo build --release --workspace --locked -j4", an ordinary workspace build in a separate target directory. SLOT_TRAIN only admitted that requested load through our own reservation; no other ordinary job entered. It ran from 07:01:39 to 07:07:20 IST and overlapped every loaded sample. Both loaded invocations were serial; no second browser test job ran during this matrix. The load build passed.

Exact test command (apps/web, per product and condition): CALTERNAL_BIN=<worktree>/target/debug/calternal E2E_DIST=<worktree>/apps/web/dist ~/bin/slot nix develop <worktree> -c bash -c 'bunx playwright test list-fit.spec.ts:67 --project=chromium-1440 --no-deps --workers=1 --retries=0 --repeat-each=15 --reporter=line,json --output=<outside-repo-results>', with CI=1, SLOT_TRAIN=1 and independent TMPDIRs. No skipped or retried cases.

Verdict so far: no demonstrated train-specific slowdown, and the historical 1654 ms failure did not reproduce under the requested single-build load. This does not establish that load alone caused BM12, and there is no justified first-bad PR to bisect from these results. Browser events consistently precede Playwright observation; a controlled below-budget-response probe is investigating that separate measurement weakness. Linux frame figures are functional diagnostics, not GPU qualification.

Matched list-fit matrix complete, original 1500 ms Chromium assertion retained, zero retries. Independent BM12 product `1e776f9e` and origin/dev `55f35fcd`, built separately on dev-01; same targeted test definition plus read-only scroll-event diagnostics. The measurement is captured before diagnostic output to avoid charging that output to the assertion. The train reservation drained all existing slot jobs before setup. | Condition/product | Passed | Failed | Playwright observation median / max, ms | Actual scroll event median / max after command start, ms | |---|---:|---:|---:|---:| | idle-base | 15/15 | 0 | 587 / 860 | 486 / 597 | | idle-train | 15/15 | 0 | 573 / 873 | 478 / 621 | | loaded-base | 15/15 | 0 | 839 / 1126 | 709 / 877 | | loaded-train | 15/15 | 0 | 817 / 1142 | 608 / 898 | The one controlled load was `SLOT_TRAIN=1 CARGO_TARGET_DIR=$HOME/flake-wall/load/target ~/bin/slot nix develop -c bash -c "cargo build --release --workspace --locked -j4"`, an ordinary workspace build in a separate target directory. SLOT_TRAIN only admitted that requested load through our own reservation; no other ordinary job entered. It ran from 07:01:39 to 07:07:20 IST and overlapped every loaded sample. Both loaded invocations were serial; no second browser test job ran during this matrix. The load build passed. Exact test command (apps/web, per product and condition): `CALTERNAL_BIN=<worktree>/target/debug/calternal E2E_DIST=<worktree>/apps/web/dist ~/bin/slot nix develop <worktree> -c bash -c 'bunx playwright test list-fit.spec.ts:67 --project=chromium-1440 --no-deps --workers=1 --retries=0 --repeat-each=15 --reporter=line,json --output=<outside-repo-results>'`, with CI=1, SLOT_TRAIN=1 and independent TMPDIRs. No skipped or retried cases. Verdict so far: **no demonstrated train-specific slowdown, and the historical 1654 ms failure did not reproduce under the requested single-build load.** This does not establish that load alone caused BM12, and there is no justified first-bad PR to bisect from these results. Browser events consistently precede Playwright observation; a controlled below-budget-response probe is investigating that separate measurement weakness. Linux frame figures are functional diagnostics, not GPU qualification.
kayg commented 2026-11-03 02:03:05 +00:00
Author
Owner
Copy link

Flake-wall report: two source repairs pushed; the historical renamed-link failure remains unresolved. This is not a green replacement train and no merge is authorized.

PR #856, branch fix/playwright-train-flakes, final source head cf14dc43c1b2cd7ffdd61cb188346e26b2a40f65. The owned control worktree is /home/kayg/Developer/Worktrees/flake-wall. The similarly named fix/parallel-test-flakes branch is a separate PR's work and was untouched.

Repair Commit
Refusal integration waits for its document's loaded module graph across Vite reloads 63faf32c81e637ea6f8d9e541e69c27cd59eefc6
List-fit records the actual qualifying browser scroll under the existing deadline cf14dc43c1b2cd7ffdd61cb188346e26b2a40f65

1. chromium-1440 list-fit: no demonstrated train-specific slowdown; a measurement fault was reproduced and repaired.

Independent products were built from BM12 1e776f9e810c46c91bc752a117ed4f43ebd3938f and origin/dev 55f35fcde3bc91ca3781055577b93e93819f6cd4, with separate binaries and production distributions. The original test definition was identical on both products. A read-only browser scroll listener captured the event timestamp; the original elapsed sample was captured before diagnostic output. All existing slots drained before the quiet matrix, and the train reservation excluded new unrelated jobs.

Condition/product Original test passes / attempts Original observation median / max, ms
quiet dev 15/15 587 / 860
quiet BM12 15/15 573 / 873
one build, dev 15/15 839 / 1126
one build, BM12 15/15 817 / 1142

The single controlled load was an ordinary cargo build --release --workspace --locked -j4 in one slot, with its own target directory. It passed and overlapped every loaded sample, from 07:01:39 to 07:07:20 IST. There was no second browser job during the matched matrix. Matrix checkpoint and browser-event statistics.

The historical 1654 ms observation did not reproduce in these 60 attempts. These samples do not establish that BM12 is slower or that load alone caused its failure. No first-bad PR is justified, so no speculative bisect or product repair was made.

A separate controlled probe suppresses the wheel's default action and schedules a real scroll at 1300 ms. Against the original polling test, the final diagnostic-safe baseline failed 3/3 even though the actual scroll events occurred at 1305, 1289 and 1295 ms. It captures the original elapsed sample before any diagnostic await, and collects event evidence after a failed assertion without changing its result. This demonstrates that the polling schedule can miss a valid response within the unchanged budget; it does not claim to reconstruct BM12's exact host scheduling.

The repair arms a browser-local promise before sending input, resolves it on the first scroll beyond the existing 100 px threshold, and checks its browser-recorded elapsed time. A timer enforces the same existing deadline when no qualifying event arrives. Controller scheduling and polling delays cannot inflate an already recorded response. App work or a main-thread stall during the armed interval still delays the event and fails the same budget; no CPU time is subtracted. Trains also exclude unrelated new slot jobs through the VM gate below.

Final qualification: 15/15 clean original chromium-1440 attempts, 3/3 controlled below-budget responses, and 1/1 WebKit-390 shared-path check. A 1700 ms response was rejected 3/3, preserving a useful negative control. Chromium's 1500 ms deadline, WebKit's 6000 ms deadline, the >100 px requirement and all existing frame-gap assertions remain unchanged. No settling sleep was added.

2. chromium-1440 refusal-performance: navigation/setup race repaired.

Before repair, 15/15 ordinary cold-cache attempts passed. To reproduce the actual failing mechanism, an outside-repo Vite probe sends one real full-reload while the first refusal module is being transformed. Before repair: 3/3 failed with the exact page.evaluate: Execution context was destroyed, most likely because of a navigation error, before timing assertions. The logs contain the Vite reload message and document navigation. This is a controlled reproduction of the race, not proof of what triggered BM12's particular reload.

performance.html now statically imports the measured module and exposes its exports only once that graph is loaded. The test waits for the requested exported function in the current document before starting its long evaluation. Navigation uses the commit signal so this readiness wait can survive a setup reload. There is no network-idle heuristic, sleep, evaluation retry, longer test timeout or change to the 16 ms assertions.

With the exact same injected reload: 3/3 passed. Then 15/15 clean fresh-cache listed-project attempts passed. Those independent runs use an outside-repo config that imports the native config, retains the native chromium-1440 project/options and source test, and omits unrelated app servers; the integration's own real Vite server still runs. A subsequent native-config full refusal spec passed 2/2, covering both refusedPaste and offlineBacklog. The four relevant source files were SHA256-compared between the tested BM12 overlay and the final pushed source head, including the unchanged missing-links spec; all matched.

3. webkit-390 renamed-link: still unresolved; no speculative readiness or product patch.

The original target passed 15/15 alongside the ordinary build, 15/15 quiet, 3/3 in three complete missing-links sequences, and 15/15 final clean original-case attempts: 48/48 target passes. The complete-spec sequence was 54/54 total, including those three target passes.

The first 33 target attempts captured mouse-down/up targets and the live editor text. In all observed attempts the press and release used the fresh renamed href, and the page reached the intended note. They do not demonstrate a click before the new target was ready. BM12's original source-note URL failure remains real and unattributed. Neither the test nor product was changed for this case. A failing trace with pointer and readiness evidence is still needed before selecting a cause or culprit PR.

VM train gate and control brief.

Installed paths:

  • claude@calternal-dev-01-compute02:~/bin/slot
  • cua@calternal-dev-02-std:~/bin/slot

Each original script remains at ~/bin/slot.pre-flake-wall. New non-train jobs check ~/.train.lock before finding a slot and again after acquiring one. The file contains <train-name> <owner-pid>; a gone PID is stale, including when a writer omits the trailing newline. SLOT_TRAIN=1 admits train work. Existing jobs finish normally. No other job was killed.

Final behavioral verification: 6/6 checks on each VM: live lock blocks a new ordinary job; train bypass works; an already-running ordinary job finishes; removing the lock releases a waiter; a stale PID does not block; an unterminated live lock still blocks. The last case failed against the initial parser and passed after correction. Shell syntax checks passed. Installed script and behavior evidence.

On control host code, /home/kayg/Developer/Worktrees/BRIEF-COMMON.md now has the short Trains rule, including waiting for existing jobs before quiet gates. kayg: re-copy that file to minipc.

Commands and gate accounting.

Builds were VM-only through ~/bin/slot, under the pinned Nix shell. Each independent product ran bun install --frozen-lockfile && cargo build --locked -j4 && cd apps/web && bun run build. Only the baseline needed the #779 flake tooling overlay; none of it was committed. Matrix/final tests used CI=1, workers=1, retries=0 and separate disk-backed TMPDIRs. Binaries, heads and complete commands are in the attachment.

The common native invocation, executed from the corresponding apps/web directory, was:

CALTERNAL_BIN="$work/target/debug/calternal" E2E_DIST="$work/apps/web/dist" \
PLAYWRIGHT_JSON_OUTPUT_FILE="$out/result.json" TMPDIR="$out/tmp" \
~/bin/slot nix develop "$work" -c bash -c \
'bunx playwright test "$1" --project="$2" --no-deps --workers=1 --retries=0 --repeat-each="$3" --reporter=line,json --output="$4/results"' \
_ "$spec" "$project" "$count" "$out"

For final gates $work=/home/claude/Developer/Worktrees/flake-wall-train and $out=/home/claude/flake-wall/<label>-train:

label spec project repeat-each result
list-clean list-fit.spec.ts:67 chromium-1440 15 15 passed
links-clean missing-links.spec.ts:245 webkit-390 15 15 passed, original test unchanged
refusal-native refusal-performance.spec.ts chromium-1440 1 2 passed
list-webkit list-fit.spec.ts:67 webkit-390 1 1 passed

Cold-cache refusal invocations and controlled probes have their exact outside-repo configs, scripts and commands attached. git diff --check, node ci/check-case-paths.mjs and node ci/check-no-evidence.mjs passed on the repaired train overlay; source worktree diff check passed and the pushed tree is clean. No full train rerun, merge, repository-setting change or new performance qualification is claimed. Linux frame checks are functional diagnostics; no Mac/GPU performance result is claimed. No assertion, limit, retry setting or timeout was loosened.

Complete attempt ledger: 228 actual attempts, 216 passes, 12 controlled-probe failures, zero nonzero retries and zero skips. Natural product gates were 207/207; the remaining 21 attempts were controlled probes.

Additional controlled attempt group pass / fail Treatment
initial scroll-probe-red 1 / 2 retained; polling failures before failure-event collection was added
scroll-probe-confirm 2 / 1 retained but excluded from causal proof: its diagnostic finally block could inflate the final wall-clock assertion
scroll-probe-timestamped-red 0 / 3 primary red evidence; captures elapsed before diagnostics and collects actual event times
scroll-probe-green 3 / 0 below-budget positive control after repair
scroll-probe-over-budget 0 / 3 deliberate above-budget negative control after repair
navigation-probe-red 0 / 3 exact context-destruction mechanism before repair
navigation-probe-green 3 / 0 same actual Vite reload after repair

There were also 20 zero-case setup failures, excluded from executed-case counts but preserved: 17 empty selections (two stale line selectors after instrumentation and 15 over-anchored title selectors), one external-config web-server cwd error, and two external-probe dependency-resolution errors. The external dependency link and server cwd were corrected; --list then selected one intended test for each probe. No attempted failing browser case was hidden or changed into a pass by retries.

Evidence attachment: flake-wall-artifacts.tar.gz, 88,860 bytes, SHA256 a0370d927fd24de34af8502120f17cade41c5345baedd4f09d29b13d8efcbf8c. It contains all result JSON and raw text logs, failure accounting, exact probe/config scripts, build provenance and the final installed slot script. No repository artifacts were added. The diagnostic-contaminated preliminary probe is expressly retained rather than used as evidence for the repair.

All owned jobs finished before handback: both owned dev-01 units inactive, reservation removed, owned VM worktrees, targets, dependencies, seeded Homes, traces and temporary reports removed after attachment upload. Free root space afterward: dev-01 295 GiB, dev-02 325 GiB. Installed slot scripts and their small backups remain. No other worktree or job was touched. Forgejo had no commit statuses or reviews published for the final head when checked.

Remaining: the historical renamed-link failure is not fixed or attributed; the exact historical 1654 ms cause and BM12 reload trigger are not established. The two independently demonstrated test faults and train admission tooling are repaired, with the requested clean targeted counts. A new train is required to qualify the aggregate; this report does not relabel BM12 green.

Flake-wall report: two source repairs pushed; the historical renamed-link failure remains unresolved. This is not a green replacement train and no merge is authorized. PR #856, branch `fix/playwright-train-flakes`, final source head `cf14dc43c1b2cd7ffdd61cb188346e26b2a40f65`. The owned control worktree is `/home/kayg/Developer/Worktrees/flake-wall`. The similarly named `fix/parallel-test-flakes` branch is a separate PR's work and was untouched. | Repair | Commit | |---|---| | Refusal integration waits for its document's loaded module graph across Vite reloads | `63faf32c81e637ea6f8d9e541e69c27cd59eefc6` | | List-fit records the actual qualifying browser scroll under the existing deadline | `cf14dc43c1b2cd7ffdd61cb188346e26b2a40f65` | **1. chromium-1440 list-fit: no demonstrated train-specific slowdown; a measurement fault was reproduced and repaired.** Independent products were built from BM12 `1e776f9e810c46c91bc752a117ed4f43ebd3938f` and origin/dev `55f35fcde3bc91ca3781055577b93e93819f6cd4`, with separate binaries and production distributions. The original test definition was identical on both products. A read-only browser scroll listener captured the event timestamp; the original elapsed sample was captured before diagnostic output. All existing slots drained before the quiet matrix, and the train reservation excluded new unrelated jobs. | Condition/product | Original test passes / attempts | Original observation median / max, ms | |---|---:|---:| | quiet dev | 15/15 | 587 / 860 | | quiet BM12 | 15/15 | 573 / 873 | | one build, dev | 15/15 | 839 / 1126 | | one build, BM12 | 15/15 | 817 / 1142 | The single controlled load was an ordinary `cargo build --release --workspace --locked -j4` in one slot, with its own target directory. It passed and overlapped every loaded sample, from 07:01:39 to 07:07:20 IST. There was no second browser job during the matched matrix. [Matrix checkpoint and browser-event statistics](https://git.kayg.org/kayg/calternal/pulls/856#issuecomment-76211). The historical 1654 ms observation did not reproduce in these 60 attempts. These samples do not establish that BM12 is slower or that load alone caused its failure. No first-bad PR is justified, so no speculative bisect or product repair was made. A separate controlled probe suppresses the wheel's default action and schedules a real scroll at 1300 ms. Against the original polling test, the final diagnostic-safe baseline failed **3/3** even though the actual scroll events occurred at **1305, 1289 and 1295 ms**. It captures the original elapsed sample before any diagnostic await, and collects event evidence after a failed assertion without changing its result. This demonstrates that the polling schedule can miss a valid response within the unchanged budget; it does not claim to reconstruct BM12's exact host scheduling. The repair arms a browser-local promise before sending input, resolves it on the first scroll beyond the existing 100 px threshold, and checks its browser-recorded elapsed time. A timer enforces the same existing deadline when no qualifying event arrives. Controller scheduling and polling delays cannot inflate an already recorded response. App work or a main-thread stall during the armed interval still delays the event and fails the same budget; no CPU time is subtracted. Trains also exclude unrelated new slot jobs through the VM gate below. Final qualification: **15/15 clean original chromium-1440 attempts**, **3/3** controlled below-budget responses, and **1/1** WebKit-390 shared-path check. A **1700 ms response was rejected 3/3**, preserving a useful negative control. Chromium's 1500 ms deadline, WebKit's 6000 ms deadline, the >100 px requirement and all existing frame-gap assertions remain unchanged. No settling sleep was added. **2. chromium-1440 refusal-performance: navigation/setup race repaired.** Before repair, **15/15 ordinary cold-cache attempts passed**. To reproduce the actual failing mechanism, an outside-repo Vite probe sends one real `full-reload` while the first refusal module is being transformed. Before repair: **3/3 failed with the exact `page.evaluate: Execution context was destroyed, most likely because of a navigation` error**, before timing assertions. The logs contain the Vite reload message and document navigation. This is a controlled reproduction of the race, not proof of what triggered BM12's particular reload. `performance.html` now statically imports the measured module and exposes its exports only once that graph is loaded. The test waits for the requested exported function in the current document before starting its long evaluation. Navigation uses the commit signal so this readiness wait can survive a setup reload. There is no network-idle heuristic, sleep, evaluation retry, longer test timeout or change to the 16 ms assertions. With the exact same injected reload: **3/3 passed**. Then **15/15 clean fresh-cache listed-project attempts passed**. Those independent runs use an outside-repo config that imports the native config, retains the native chromium-1440 project/options and source test, and omits unrelated app servers; the integration's own real Vite server still runs. A subsequent **native-config full refusal spec passed 2/2**, covering both refusedPaste and offlineBacklog. The four relevant source files were SHA256-compared between the tested BM12 overlay and the final pushed source head, including the unchanged missing-links spec; all matched. **3. webkit-390 renamed-link: still unresolved; no speculative readiness or product patch.** The original target passed **15/15 alongside the ordinary build**, **15/15 quiet**, **3/3 in three complete missing-links sequences**, and **15/15 final clean original-case attempts**: **48/48 target passes**. The complete-spec sequence was **54/54** total, including those three target passes. The first 33 target attempts captured mouse-down/up targets and the live editor text. In all observed attempts the press and release used the fresh renamed href, and the page reached the intended note. They do not demonstrate a click before the new target was ready. BM12's original source-note URL failure remains real and unattributed. Neither the test nor product was changed for this case. A failing trace with pointer and readiness evidence is still needed before selecting a cause or culprit PR. **VM train gate and control brief.** Installed paths: - `claude@calternal-dev-01-compute02:~/bin/slot` - `cua@calternal-dev-02-std:~/bin/slot` Each original script remains at `~/bin/slot.pre-flake-wall`. New non-train jobs check `~/.train.lock` before finding a slot and again after acquiring one. The file contains `<train-name> <owner-pid>`; a gone PID is stale, including when a writer omits the trailing newline. `SLOT_TRAIN=1` admits train work. Existing jobs finish normally. No other job was killed. Final behavioral verification: **6/6 checks on each VM**: live lock blocks a new ordinary job; train bypass works; an already-running ordinary job finishes; removing the lock releases a waiter; a stale PID does not block; an unterminated live lock still blocks. The last case failed against the initial parser and passed after correction. Shell syntax checks passed. [Installed script and behavior evidence](https://git.kayg.org/kayg/calternal/pulls/856#issuecomment-76193). On control host `code`, `/home/kayg/Developer/Worktrees/BRIEF-COMMON.md` now has the short **Trains** rule, including waiting for existing jobs before quiet gates. **kayg: re-copy that file to minipc.** **Commands and gate accounting.** Builds were VM-only through `~/bin/slot`, under the pinned Nix shell. Each independent product ran `bun install --frozen-lockfile && cargo build --locked -j4 && cd apps/web && bun run build`. Only the baseline needed the #779 flake tooling overlay; none of it was committed. Matrix/final tests used CI=1, workers=1, retries=0 and separate disk-backed TMPDIRs. Binaries, heads and complete commands are in the attachment. The common native invocation, executed from the corresponding apps/web directory, was: ```sh CALTERNAL_BIN="$work/target/debug/calternal" E2E_DIST="$work/apps/web/dist" \ PLAYWRIGHT_JSON_OUTPUT_FILE="$out/result.json" TMPDIR="$out/tmp" \ ~/bin/slot nix develop "$work" -c bash -c \ 'bunx playwright test "$1" --project="$2" --no-deps --workers=1 --retries=0 --repeat-each="$3" --reporter=line,json --output="$4/results"' \ _ "$spec" "$project" "$count" "$out" ``` For final gates `$work=/home/claude/Developer/Worktrees/flake-wall-train` and `$out=/home/claude/flake-wall/<label>-train`: | label | spec | project | repeat-each | result | |---|---|---|---:|---:| | list-clean | list-fit.spec.ts:67 | chromium-1440 | 15 | 15 passed | | links-clean | missing-links.spec.ts:245 | webkit-390 | 15 | 15 passed, original test unchanged | | refusal-native | refusal-performance.spec.ts | chromium-1440 | 1 | 2 passed | | list-webkit | list-fit.spec.ts:67 | webkit-390 | 1 | 1 passed | Cold-cache refusal invocations and controlled probes have their exact outside-repo configs, scripts and commands attached. `git diff --check`, `node ci/check-case-paths.mjs` and `node ci/check-no-evidence.mjs` passed on the repaired train overlay; source worktree diff check passed and the pushed tree is clean. No full train rerun, merge, repository-setting change or new performance qualification is claimed. Linux frame checks are functional diagnostics; no Mac/GPU performance result is claimed. No assertion, limit, retry setting or timeout was loosened. Complete attempt ledger: **228 actual attempts, 216 passes, 12 controlled-probe failures, zero nonzero retries and zero skips**. Natural product gates were **207/207**; the remaining 21 attempts were controlled probes. | Additional controlled attempt group | pass / fail | Treatment | |---|---:|---| | initial scroll-probe-red | 1 / 2 | retained; polling failures before failure-event collection was added | | scroll-probe-confirm | 2 / 1 | retained but excluded from causal proof: its diagnostic finally block could inflate the final wall-clock assertion | | scroll-probe-timestamped-red | 0 / 3 | primary red evidence; captures elapsed before diagnostics and collects actual event times | | scroll-probe-green | 3 / 0 | below-budget positive control after repair | | scroll-probe-over-budget | 0 / 3 | deliberate above-budget negative control after repair | | navigation-probe-red | 0 / 3 | exact context-destruction mechanism before repair | | navigation-probe-green | 3 / 0 | same actual Vite reload after repair | There were also **20 zero-case setup failures**, excluded from executed-case counts but preserved: 17 empty selections (two stale line selectors after instrumentation and 15 over-anchored title selectors), one external-config web-server cwd error, and two external-probe dependency-resolution errors. The external dependency link and server cwd were corrected; `--list` then selected one intended test for each probe. No attempted failing browser case was hidden or changed into a pass by retries. Evidence attachment: [flake-wall-artifacts.tar.gz](https://git.kayg.org/attachments/69e6215b-fd71-4e56-9f0d-15f2c562e251), 88,860 bytes, SHA256 `a0370d927fd24de34af8502120f17cade41c5345baedd4f09d29b13d8efcbf8c`. It contains all result JSON and raw text logs, failure accounting, exact probe/config scripts, build provenance and the final installed slot script. No repository artifacts were added. The diagnostic-contaminated preliminary probe is expressly retained rather than used as evidence for the repair. All owned jobs finished before handback: both owned dev-01 units inactive, reservation removed, owned VM worktrees, targets, dependencies, seeded Homes, traces and temporary reports removed after attachment upload. Free root space afterward: dev-01 **295 GiB**, dev-02 **325 GiB**. Installed slot scripts and their small backups remain. No other worktree or job was touched. Forgejo had no commit statuses or reviews published for the final head when checked. Remaining: the historical renamed-link failure is not fixed or attributed; the exact historical 1654 ms cause and BM12 reload trigger are not established. The two independently demonstrated test faults and train admission tooling are repaired, with the requested clean targeted counts. A new train is required to qualify the aggregate; this report does not relabel BM12 green.
kayg commented 2026-11-03 02:03:54 +00:00
Author
Owner
Copy link

Earlier PR description, preserved before updating the description for the final source scope. These are historical qualification results, not the new head's results. Current report: #856 (comment)

Four Playwright flakes that failed train BM6 (causes at #834 issuecomment-74246). No retries, no longer timeouts: each wait is on a real signal.

Spec Cause Fix Runs (0 failures)
search-corpus (chromium-1440) earlier corpus notes on a shared server crowd the top 8 deletes every leftover corpus-* note first (and its own at the end), waits until the engine has let them go; assertions unchanged --repeat-each=20 dev-01: 40 passed. Shared server: trash-search + list-search + corpus --repeat-each=10 in one run: 40 passed
missing-links (chromium-390) geometry sampled mid card animation waits for the card's finite animations (getAnimations({subtree}).finished) before measuring; tolerance unchanged dev-01 --repeat-each=20: 360 passed
review4-signout (webkit-1440) 5 s wait for a save state while the test holds the uplink/receipts counts the server's save receipts in the live-socket route and waits on them (no timer); writeTwoNotes waits until the server holds the typed edit dev-01 200 passed, dev-02 200 passed (--repeat-each=20, 20 skipped by design)
backdrop-page (look-webkit-1440) 5 s poll missed the Undo write waits for the app's own prefs.set reply (waitForResponse) before reading it back dev-01 80 passed, dev-02 80 passed (--repeat-each=20)

Extra flakes found while repeating the signout spec, fixed the same way: the fixture's 30 s server and proxy start waits now wait for the health reply (fail at once if the process exited), and the logout reply wait accepts the server saying the old session is gone when the reloaded page cannot report it.

Decisions: an attempt to wait on a receipt in the refusal-recovery step was dropped (the receipt tag does not fire on rejoin); that step keeps its existing 20 s wait. Commands: bunx playwright test e2e/<spec> --project=<p> --no-deps --workers=1 --repeat-each=20 via ~/bin/slot, binary built at the PR head's parent dev (55f35fcd); head 85845e3d. missing-links, search-corpus and backdrop-page were run at e8198af0 (those files are unchanged since).

🤖 Generated with Claude Code

Earlier PR description, preserved before updating the description for the final source scope. These are historical qualification results, not the new head's results. Current report: https://git.kayg.org/kayg/calternal/pulls/856#issuecomment-76224 Four Playwright flakes that failed train BM6 (causes at #834 issuecomment-74246). No retries, no longer timeouts: each wait is on a real signal. | Spec | Cause | Fix | Runs (0 failures) | |---|---|---|---| | `search-corpus` (chromium-1440) | earlier corpus notes on a shared server crowd the top 8 | deletes every leftover `corpus-*` note first (and its own at the end), waits until the engine has let them go; assertions unchanged | `--repeat-each=20` dev-01: 40 passed. Shared server: trash-search + list-search + corpus `--repeat-each=10` in one run: 40 passed | | `missing-links` (chromium-390) | geometry sampled mid card animation | waits for the card's finite animations (`getAnimations({subtree}).finished`) before measuring; tolerance unchanged | dev-01 `--repeat-each=20`: 360 passed | | `review4-signout` (webkit-1440) | 5 s wait for a save state while the test holds the uplink/receipts | counts the server's save receipts in the live-socket route and waits on them (no timer); `writeTwoNotes` waits until the server holds the typed edit | dev-01 200 passed, dev-02 200 passed (`--repeat-each=20`, 20 skipped by design) | | `backdrop-page` (look-webkit-1440) | 5 s poll missed the Undo write | waits for the app's own `prefs.set` reply (`waitForResponse`) before reading it back | dev-01 80 passed, dev-02 80 passed (`--repeat-each=20`) | Extra flakes found while repeating the signout spec, fixed the same way: the fixture's 30 s server and proxy start waits now wait for the health reply (fail at once if the process exited), and the logout reply wait accepts the server saying the old session is gone when the reloaded page cannot report it. Decisions: an attempt to wait on a receipt in the refusal-recovery step was dropped (the receipt tag does not fire on rejoin); that step keeps its existing 20 s wait. Commands: `bunx playwright test e2e/<spec> --project=<p> --no-deps --workers=1 --repeat-each=20` via `~/bin/slot`, binary built at the PR head's parent dev (`55f35fcd`); head `85845e3d`. missing-links, search-corpus and backdrop-page were run at `e8198af0` (those files are unchanged since). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
kayg changed title from e2e: wait on real signals in four train flakes to test: stabilize Playwright readiness and scroll observation 2026-11-03 02:03:54 +00:00
kayg commented 2026-11-04 13:04:59 +00:00
Author
Owner
Copy link

Closing: head cf14dc43 is already in dev (reached via 82549a24); nothing left to merge. Branch kept.

Closing: head cf14dc43 is already in dev (reached via 82549a24); nothing left to merge. Branch kept.
kayg closed this pull request 2026-11-04 13:05:00 +00:00
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!856
No description provided.