PROCESS: browser smoke suite of real User flows over a throttled network (deploy gate)

kayg commented

2026-10-05 07:59:11 +00:00

Owner

Owner decisions (2026-10-05 process grill) + "more things need to be tested with browser-use"

Build the per-deploy browser smoke suite: real flows driven in a real browser like a User (Playwright, Chromium + WebKit, iPhone-size and desktop), against a production build with realistic data, over a throttled network path (e.g. 2 MB/s, 80 ms latency proxy) so uploads and slow responses behave like the internet + edge.

First ~15 flows (each asserts what the User sees, with a screenshot kept as a baseline):

Log entry from the Composer (Send all with 3 queued) appears in Day view and Agenda.
Agenda newest first; scroll to the bottom without jumps; completed Tasks do not repeat.
Tasks Board: drag a Task through every column, including Blocked and Waiting; persists after reload.
Money: import a realistic ~15 MB Actual ZIP through the UI over the throttled path; review; import; budget shows.
Note: create, type, paste an image, reload, content intact.
Draw: insert a Drawing, draw, Done, reload.
Share a Note with a second User; recipient sees it under Shared.
Mail columns: open a message; switch messages; sidebar collapse leaves nothing behind.
Search palette: rows have readable labels (contrast check), open a result.
Settings sidebar hierarchy; Apps & Devices sheet title while scrolling.
Mode tray: every Tab icon at the icon size token, selected label not clipped (390/1440, light/dark).
Files: upload a 50 MB file over the throttled path, preview, delete with Undo.
Photos: an extensionless HEIC appears with a thumbnail.
Tag click filters the current view; tag rename/merge with Undo.
Long pull to update on phone size; update toast after a server build change.

Run: before every production deploy (gate, ~20–30 min) and nightly on dev. Output: pass/fail per flow plus screenshots; failures block the deploy. Keep it fast and stable (no fixed sleeps; wait on visible state).

## Owner decisions (2026-10-05 process grill) + "more things need to be tested with browser-use" Build the per-deploy browser smoke suite: real flows driven in a real browser like a User (Playwright, Chromium + WebKit, iPhone-size and desktop), against a production build with realistic data, over a throttled network path (e.g. 2 MB/s, 80 ms latency proxy) so uploads and slow responses behave like the internet + edge. First ~15 flows (each asserts what the User sees, with a screenshot kept as a baseline): 1. Log entry from the Composer (Send all with 3 queued) appears in Day view and Agenda. 2. Agenda newest first; scroll to the bottom without jumps; completed Tasks do not repeat. 3. Tasks Board: drag a Task through every column, including Blocked and Waiting; persists after reload. 4. Money: import a realistic ~15 MB Actual ZIP through the UI over the throttled path; review; import; budget shows. 5. Note: create, type, paste an image, reload, content intact. 6. Draw: insert a Drawing, draw, Done, reload. 7. Share a Note with a second User; recipient sees it under Shared. 8. Mail columns: open a message; switch messages; sidebar collapse leaves nothing behind. 9. Search palette: rows have readable labels (contrast check), open a result. 10. Settings sidebar hierarchy; Apps & Devices sheet title while scrolling. 11. Mode tray: every Tab icon at the icon size token, selected label not clipped (390/1440, light/dark). 12. Files: upload a 50 MB file over the throttled path, preview, delete with Undo. 13. Photos: an extensionless HEIC appears with a thumbnail. 14. Tag click filters the current view; tag rename/merge with Undo. 15. Long pull to update on phone size; update toast after a server build change. Run: before every production deploy (gate, ~20–30 min) and nightly on dev. Output: pass/fail per flow plus screenshots; failures block the deploy. Keep it fast and stable (no fixed sleeps; wait on visible state).

01-composer-batch.zip

5.5 MiB

02-agenda-scroll.zip

6.5 MiB

03-task-board.zip

5.9 MiB

04-money-actual.zip

4.2 MiB

05-note-paste-image.zip

4.6 MiB

06-drawing-insert.zip

3.5 MiB

07-share-note.zip

4.7 MiB

08-mail-columns.zip

4.8 MiB

09-search-result.zip

3.6 MiB

10-settings-devices.zip

3.9 MiB

11-tab-bar.zip

5 MiB

12-files-upload.zip

4.7 MiB

13-photos-heic.zip

5.7 MiB

14-tag-actions.zip

6.4 MiB

15-app-update.zip

7.7 MiB

kayg commented

2026-10-05 08:00:05 +00:00

Author

Owner

Starting smoke suite work on branch job/smoke-1122, based on dev at 4a8e22609e. I’m checking the existing e2e harness and design decisions before adding the runner.

Starting smoke suite work on branch job/smoke-1122, based on dev at 4a8e22609e51f53b037d1296dc757e8f0e2b833a. I’m checking the existing e2e harness and design decisions before adding the runner.

kayg commented

2026-10-05 08:53:33 +00:00

Author

Owner

Tags finding for #1122: the server already provides POST /api/v1/tags/rename/preview and POST /api/v1/tags/rename (crates/calternal-tags/src/lib.rs), but the production Tags route currently exposes Tag links and Copy link actions only (apps/web/src/routes/tags/+page.svelte). The requested rename/merge Undo has no visible UI action. The smoke flow verifies the server merge and restores its fixture through the same writer, then reports this missing UI action as a failure.

Tags finding for #1122: the server already provides `POST /api/v1/tags/rename/preview` and `POST /api/v1/tags/rename` (`crates/calternal-tags/src/lib.rs`), but the production Tags route currently exposes Tag links and Copy link actions only (`apps/web/src/routes/tags/+page.svelte`). The requested rename/merge Undo has no visible UI action. The smoke flow verifies the server merge and restores its fixture through the same writer, then reports this missing UI action as a failure.

kayg commented

2026-10-05 09:22:51 +00:00

Author

Owner

After merging origin/dev as required, bun run check reaches perf-lint and fails with this finding:

perf-lint: INVALID: ('render.bound', 'apps/web/src/lib/components/app-sidebar.svelte', 'apps/web/src/lib/components/app-sidebar.svelte#each:10a1aaf3137a2426:1'): unused or changed exception

The error is in the merged #1094 sidebar change and its contracts/perf/exceptions.json entry. The gate stops before the storage, token, and Svelte checks. The smoke work does not change the sidebar or performance contracts.

After merging `origin/dev` as required, `bun run check` reaches perf-lint and fails with this finding: ``` perf-lint: INVALID: ('render.bound', 'apps/web/src/lib/components/app-sidebar.svelte', 'apps/web/src/lib/components/app-sidebar.svelte#each:10a1aaf3137a2426:1'): unused or changed exception ``` The error is in the merged #1094 sidebar change and its `contracts/perf/exceptions.json` entry. The gate stops before the storage, token, and Svelte checks. The smoke work does not change the sidebar or performance contracts.

kayg commented

2026-10-05 09:27:41 +00:00

Author

Owner

The first bun run test:smoke attempt exited before running flows because the shared smoke harness did not re-export playwrightEntry, which the runner imports. I added the re-export in commit 5e000669c; syntax checks, the runner helper export check, and imports for all 15 flow files now pass. I am rerunning the suite after this startup fix.

The first `bun run test:smoke` attempt exited before running flows because the shared smoke harness did not re-export `playwrightEntry`, which the runner imports. I added the re-export in commit `5e000669c`; syntax checks, the runner helper export check, and imports for all 15 flow files now pass. I am rerunning the suite after this startup fix.

kayg commented

2026-10-05 09:34:16 +00:00

Author

Owner

The next setup attempt reached the real signed-in /today route but timed out because smoke readiness accepted only light or dark. Diagnostic state was rootMode=system, rootTheme=mono, route content mounted, with no page or console errors. system is a valid preference, so commit 451629cd5 now accepts it when a theme is present; screenshot capture still sets explicit light and dark themes.

The next setup attempt reached the real signed-in `/today` route but timed out because smoke readiness accepted only `light` or `dark`. Diagnostic state was `rootMode=system`, `rootTheme=mono`, route content mounted, with no page or console errors. `system` is a valid preference, so commit `451629cd5` now accepts it when a theme is present; screenshot capture still sets explicit light and dark themes.

kayg commented

2026-10-05 09:43:14 +00:00

Author

Owner

The real Chromium Agenda flow loaded the next history page and measured the visible boundary day at 623.4 px before and 745.4 px after the load. The 122.0 px shift violates the flow's stable-scroll assertion; the smoke check remains strict and will report this failure.

kayg commented

2026-10-05 09:45:14 +00:00

Author

Owner

The first real flow completed its Composer actions, then screenshot capture failed because waiting for fonts after a local-only theme change let the mounted System preference repaint. Commit 36ee86b47 now waits for fonts before the explicit screenshot theme. A focused production capture passed at 390, 820, and 1440 px in both light and dark on Chromium.

The first real flow completed its Composer actions, then screenshot capture failed because waiting for fonts after a local-only theme change let the mounted System preference repaint. Commit `36ee86b47` now waits for fonts before the explicit screenshot theme. A focused production capture passed at 390, 820, and 1440 px in both light and dark on Chromium.

kayg commented

2026-10-05 10:17:05 +00:00

Author

Owner

Full-run evidence (Chromium + WebKit, 1,428 s, exit 1): the runner omitted its session object when calling each flow. That left Share and Mail without their fixture state and caused TypeErrors in both engines. The first pass also exposed fixture/navigation defects: current-date data is Oct 5 while Agenda waits for history dates, Money checks proxy bytes before the upload finishes, and the Files/Photos selectors did not become visible. I am fixing harness defects and will retain explicit failures for product behavior.

Full-run evidence (Chromium + WebKit, 1,428 s, exit 1): the runner omitted its `session` object when calling each flow. That left Share and Mail without their fixture state and caused TypeErrors in both engines. The first pass also exposed fixture/navigation defects: current-date data is Oct 5 while Agenda waits for history dates, Money checks proxy bytes before the upload finishes, and the Files/Photos selectors did not become visible. I am fixing harness defects and will retain explicit failures for product behavior.

kayg commented

2026-10-05 10:45:59 +00:00

Author

Owner

Diagnostic subset evidence: after passing the runner's session context, Share and Mail passed in Chromium (29,347 ms and 27,363 ms). Files accepted the 50 MiB upload and rendered it as chromium-50mb.png; the fixture had been saved under that basename while the assertion expected Smoke chromium 50 MB.png, so the fixture path now uses the expected name. Money showed non-zero receive progress but did not reach its review dialog within 180 s, so the flow now reports this as #1121 with received-byte and status details and uses a 30 s completion bound to protect the suite budget.

Diagnostic subset evidence: after passing the runner's session context, Share and Mail passed in Chromium (29,347 ms and 27,363 ms). Files accepted the 50 MiB upload and rendered it as `chromium-50mb.png`; the fixture had been saved under that basename while the assertion expected `Smoke chromium 50 MB.png`, so the fixture path now uses the expected name. Money showed non-zero receive progress but did not reach its review dialog within 180 s, so the flow now reports this as `#1121` with received-byte and status details and uses a 30 s completion bound to protect the suite budget.

kayg commented

2026-10-05 11:05:54 +00:00

Author

Owner

Production smoke finding for #06 Drawing insert: Chromium shows the Draw alert “The Draw could not be saved. Your drawing is still here; try again.” after the User draws a rectangle and presses Done. The Drawing stays in Draw, and the Note has no Canvas preview. WebKit closes Draw but also does not show the preview. The flow now reports this as #1102 in both engines; I will attach the failure screenshots with the final result.

kayg commented

2026-10-05 11:08:30 +00:00

Author

Owner

Focused production smoke findings:

#02 Agenda scroll: the oldest seeded Log is present in the Calendar range. Loading the next history page moved the visible anchor from 623.5 px to 945.6 px; the smoke threshold is under 48 px. This reproduces the history-load jump in Chromium.
#03 Task Board: the status menu changes the Task through Done and the no-cache API confirms the persisted state. After a page reload, the Board does not show that Task in the Done column within 15 seconds.

Both checks stay strict in the final suite. The screenshots will be attached with the final report.

Focused production smoke findings: - #02 Agenda scroll: the oldest seeded Log is present in the Calendar range. Loading the next history page moved the visible anchor from 623.5 px to 945.6 px; the smoke threshold is under 48 px. This reproduces the history-load jump in Chromium. - #03 Task Board: the status menu changes the Task through Done and the no-cache API confirms the persisted state. After a page reload, the Board does not show that Task in the Done column within 15 seconds. Both checks stay strict in the final suite. The screenshots will be attached with the final report.

kayg commented

2026-10-05 11:12:15 +00:00

Author

Owner

The completed focused matrix adds these results:

#01 Composer batch: WebKit saved all three Journal entries and showed them in Agenda, but the Calendar Day view did not render “Smoke webkit-01 batch one” within 15 seconds.
#02 Agenda scroll: after the seeded Log appears in the Calendar API, WebKit does not show the 14-day history boundary after the User selects More days.
#03 Task Board: Chromium confirms each status in the no-cache Tasks API, but the persisted Done Task is missing from the Done lane after reload. WebKit does not persist the keyboard status change within 15 seconds.
#13 Photos: Chromium and WebKit both fail to find the uploaded extensionless HEIC in the Photos timeline after 60 seconds.

Tab Bar passed its shared-size and selected-label checks in both engines. The full smoke run will report every flow and engine.

The completed focused matrix adds these results: - #01 Composer batch: WebKit saved all three Journal entries and showed them in Agenda, but the Calendar Day view did not render “Smoke webkit-01 batch one” within 15 seconds. - #02 Agenda scroll: after the seeded Log appears in the Calendar API, WebKit does not show the 14-day history boundary after the User selects More days. - #03 Task Board: Chromium confirms each status in the no-cache Tasks API, but the persisted Done Task is missing from the Done lane after reload. WebKit does not persist the keyboard status change within 15 seconds. - #13 Photos: Chromium and WebKit both fail to find the uploaded extensionless HEIC in the Photos timeline after 60 seconds. Tab Bar passed its shared-size and selected-label checks in both engines. The full smoke run will report every flow and engine.

kayg commented

2026-10-05 11:26:43 +00:00

Author

Owner

Production smoke finding for #04 Money import: Chromium uploaded the full generated archive through the throttled proxy (15,055,637 bytes observed). After 30 seconds, import remained at “Aggregating Actual source totals · 189,440 of 560,000 transactions” and never reached the review dialog. The flow fails with #1121 and keeps the real progress check intact.

kayg commented

2026-10-05 11:38:06 +00:00

Author

Owner

The first unfiltered run stopped in WebKit after flow #05, before the runner could print its full table. Flow #06's Draw UI creates a blob: preview; the shared routeCurrentBuild handler attempted route.fetch() for that browser-owned URL, and Playwright failed with Protocol "blob:" not supported. I added an HTTP(S)-only route matcher and scoped the Drawing open action to its dedicated preview button. I will rerun the complete matrix after this fix.

The first unfiltered run stopped in WebKit after flow #05, before the runner could print its full table. Flow #06's Draw UI creates a `blob:` preview; the shared `routeCurrentBuild` handler attempted `route.fetch()` for that browser-owned URL, and Playwright failed with `Protocol "blob:" not supported`. I added an HTTP(S)-only route matcher and scoped the Drawing open action to its dedicated preview button. I will rerun the complete matrix after this fix.

kayg commented

2026-10-05 12:26:31 +00:00

Author

Owner

Finished — Forgejo #1122

Branch: job/smoke-1122
Head: 961eed317ab50348e6206b7f1097cc7e1cca3fbb
READY FOR MERGE: no. The complete suite found production-flow failures, and bun run check stops on an existing perf-lint exception in app-sidebar.svelte.

What I built

Added 15 production browser flows under apps/web/e2e/smoke/, with a real local server, production web build, realistic disposable data, Chromium and WebKit, Mac platform emulation, and screenshots at 390×844, 820×900 and 1440×900 in light and dark themes.
Added a streaming upload proxy capped at 2 MiB/s with 80 ms added latency. The deterministic Actual generator creates a 560,000-transaction archive of about 15 MB without real User data.
Added the root bun run test:smoke command and deploy/smoke-gate.sh <server-binary>. The normal runner executes all flows and exits non-zero if any flow fails; CALTERNAL_SMOKE_FLOWS is available for focused diagnosis.
Updated apps/web/e2e/harness.mjs so browser-owned blob: URLs bypass HTTP interception. The Drawing flow scopes its Open action to the preview button. Added a failure screenshot at the App Update stale-notice assertion.

Files: apps/web/e2e/smoke/01-composer-batch.mjs through 15-app-update.mjs, flow-utils.mjs, harness.mjs, run.mjs, generate-actual-fixture.py; apps/web/e2e/harness.mjs; apps/web/e2e/app-update-1059.mjs; deploy/smoke-gate.sh; root package.json.

Full-suite result

The complete run took 1,423 seconds (23m 43s), within the 25-minute target. The runner returned exit code 1 because it preserves and reports failed flows. Output:

SMOKE RESULTS
  PASS Chromium 36653 ms
  FAIL WebKit 45664 ms — waitFor: Timeout 15000ms exceeded.
FAIL 01-composer-batch 82317 ms — WebKit: waitFor: Timeout 15000ms exceeded.
  FAIL Chromium 52746 ms — waitFor: Timeout 15000ms exceeded.
  FAIL WebKit 86353 ms — waitFor: Timeout 15000ms exceeded.
FAIL 02-agenda-scroll 139099 ms — Chromium: waitFor: Timeout 15000ms exceeded.
  FAIL Chromium 40590 ms — waitFor: Timeout 15000ms exceeded.
  FAIL WebKit 40374 ms — waitFor: Timeout 15000ms exceeded.
FAIL 03-task-board 80964 ms — Chromium: waitFor: Timeout 15000ms exceeded.
  FAIL Chromium 57815 ms — #1121: the 15 MB Actual import did not reach review in 30 s (received 15055637 bytes; status: Aggregating Actual source totals · 114,688 of 560,000 transactions)
  FAIL WebKit 64254 ms — #1121: the 15 MB Actual import did not reach review in 30 s (received 15055635 bytes; status: Aggregating Actual source totals · 136,192 of 560,000 transactions)
FAIL 04-money-actual 122069 ms — Chromium: #1121: the 15 MB Actual import did not reach review in 30 s (received 15055637 bytes; status: Aggregating Actual source totals · 114,688 of 560,000 transactions) | WebKit: #1121: the 15 MB Actual import did not reach review in 30 s (received 15055635 bytes; status: Aggregating Actual source totals · 136,192 of 560,000 transactions)
  PASS Chromium 40707 ms
  PASS WebKit 22291 ms
PASS 05-note-paste-image 62998 ms
  FAIL Chromium 59055 ms — #1102: production Draw could not save the Drawing: The Draw could not be saved. Your drawing is still here; try again.
  FAIL WebKit 60288 ms — waitFor: Timeout 20000ms exceeded.
FAIL 06-drawing-insert 119343 ms — Chromium: #1102: production Draw could not save the Drawing: The Draw could not be saved. Your drawing is still here; try again. | WebKit: waitFor: Timeout 20000ms exceeded.
  PASS Chromium 47094 ms
  PASS WebKit 25884 ms
PASS 07-share-note 72978 ms
  PASS Chromium 23874 ms
  PASS WebKit 13789 ms
PASS 08-mail-columns 37663 ms
  FAIL Chromium 30752 ms — the result label has at least 4.5:1 text contrast
  PASS WebKit 15931 ms
FAIL 09-search-result 46683 ms — Chromium: the result label has at least 4.5:1 text contrast
  FAIL Chromium 31521 ms — the anchored phone sheet keeps its Apps & Devices title while scrolling
  PASS WebKit 36381 ms
FAIL 10-settings-devices 67902 ms — Chromium: the anchored phone sheet keeps its Apps & Devices title while scrolling
  PASS Chromium 21291 ms
  PASS WebKit 23753 ms
PASS 11-tab-bar 45044 ms
  PASS Chromium 68701 ms
  PASS WebKit 48247 ms
PASS 12-files-upload 116948 ms
  FAIL Chromium 49391 ms — Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms
  FAIL WebKit 38916 ms — Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms
FAIL 13-photos-heic 88307 ms — Chromium: Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms | WebKit: Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms
  FAIL Chromium 21501 ms — Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance.
  FAIL WebKit 26953 ms — Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance.
FAIL 14-tag-actions 48454 ms — Chromium: Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance. | WebKit: Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance.
  FAIL WebKit 45174 ms — #1059: the stale-build update flow clears the reload notice after the production refresh
FAIL 15-app-update 45174 ms — WebKit: #1059: the stale-build update flow clears the reload notice after the production refresh
TOTAL 1423 s; upload proxy 2 MiB/s + 80 ms; screenshots /home/kayg/Developer/calternal-wt/smoke-1122/artifacts/smoke

The full run's App Update result came from a stale local server binary: its build ID was 0582c41e0b6ce4a1f2807de0e527ffc899bcb68a, while the production web build was 05e0ecb863eb5139638f94ab43f1650223934b0e. After rebuilding calternal-server from this head, the focused App Update flow passed in 75,222 ms and captured all 18 responsive/theme screenshots:

BUILD_IDS old=0000000 current=05e0ecb863eb5139638f94ab43f1650223934b0e
REPLAY previous Log=201 Event=201
LONG_PULL {"oldRequests":131,"latestRequests":20}
SCREENSHOTS phone-390-light
SCREENSHOTS phone-390-dark
SCREENSHOTS tablet-820-light
SCREENSHOTS tablet-820-dark
SCREENSHOTS desktop-1440-light
SCREENSHOTS desktop-1440-dark
PASS 15-app-update [WebKit] 75222 ms
SMOKE RESULTS
PASS 15-app-update 75222 ms
TOTAL 167 s; upload proxy 2 MiB/s + 80 ms; screenshots /home/kayg/Developer/calternal-wt/smoke-1122/artifacts/smoke

Other gates

cargo fmt --check passed with no output. cargo build -p calternal-server passed after refreshing the build ID. bash -n deploy/smoke-gate.sh passed with no output. No Rust source changed, so Rust clippy/test gates were not run.

bun run check returned exit code 2. Verbatim output:

$ ../../scripts/perf-lint --check && node scripts/check-user-storage.mjs && node scripts/check-glass-tokens.mjs && node scripts/check-type-tokens.mjs && node scripts/check-focus-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
perf-lint: parse Rust product sources
perf-lint: parse browser product sources
perf-lint: validate coverage and architecture
perf-lint: INVALID: ('render.bound', 'apps/web/src/lib/components/app-sidebar.svelte', 'apps/web/src/lib/components/app-sidebar.svelte#each:10a1aaf3137a2426:1'): unused or changed exception

UX gaps

UX gaps closed: the suite exercises real primary actions and state persistence in the covered flows, keeps known-bug cases as hard failures, and captures the Mac rendering across the three widths and two themes. No production UI code changed.

UX gaps left: the full run found Composer batch visibility in WebKit; Agenda targets missing in both engines; Board Done cards missing in both engines; Actual import #1121 stuck while aggregating after the complete upload; Drawing save #1102 in Chromium and missing Canvas preview in WebKit; Search result contrast below 4.5:1 in Chromium; the Apps & Devices phone sheet title not staying visible in Chromium; extensionless HEIC indexing in both engines; and no visible Tag Undo in either engine. The focused App Update rerun passed with a current matching server. The whole matrix should be rerun on a matching server during the merge round.

Decisions not covered by DESIGN.md

Reuse the production E2E harness and the real local server, with one seeded User shared by Chromium and WebKit.
Keep all generated files below target/tmp; generate the 560,000-row Actual archive from invented deterministic values.
Capture six Mac-profile screenshot states per regular flow (three widths × two themes); the dedicated App Update flow keeps its existing WebKit touch path.
Keep a focused-flow environment variable for diagnosis while leaving the default command unfiltered.

Screenshot bundles

01 Composer batch · 02 Agenda scroll · 03 Task Board · 04 Money Actual · 05 Note paste image · 06 Drawing insert · 07 Share Note · 08 Mail columns · 09 Search result · 10 Settings devices · 11 Tab Bar · 12 Files upload · 13 Photos HEIC · 14 Tag actions · 15 App Update

## Finished — Forgejo #1122 Branch: `job/smoke-1122` Head: `961eed317ab50348e6206b7f1097cc7e1cca3fbb` READY FOR MERGE: **no**. The complete suite found production-flow failures, and `bun run check` stops on an existing perf-lint exception in `app-sidebar.svelte`. ### What I built - Added 15 production browser flows under `apps/web/e2e/smoke/`, with a real local server, production web build, realistic disposable data, Chromium and WebKit, Mac platform emulation, and screenshots at 390×844, 820×900 and 1440×900 in light and dark themes. - Added a streaming upload proxy capped at 2 MiB/s with 80 ms added latency. The deterministic Actual generator creates a 560,000-transaction archive of about 15 MB without real User data. - Added the root `bun run test:smoke` command and `deploy/smoke-gate.sh <server-binary>`. The normal runner executes all flows and exits non-zero if any flow fails; `CALTERNAL_SMOKE_FLOWS` is available for focused diagnosis. - Updated `apps/web/e2e/harness.mjs` so browser-owned `blob:` URLs bypass HTTP interception. The Drawing flow scopes its Open action to the preview button. Added a failure screenshot at the App Update stale-notice assertion. Files: `apps/web/e2e/smoke/01-composer-batch.mjs` through `15-app-update.mjs`, `flow-utils.mjs`, `harness.mjs`, `run.mjs`, `generate-actual-fixture.py`; `apps/web/e2e/harness.mjs`; `apps/web/e2e/app-update-1059.mjs`; `deploy/smoke-gate.sh`; root `package.json`. ### Full-suite result The complete run took 1,423 seconds (23m 43s), within the 25-minute target. The runner returned exit code 1 because it preserves and reports failed flows. Output: ```text SMOKE RESULTS PASS Chromium 36653 ms FAIL WebKit 45664 ms — waitFor: Timeout 15000ms exceeded. FAIL 01-composer-batch 82317 ms — WebKit: waitFor: Timeout 15000ms exceeded. FAIL Chromium 52746 ms — waitFor: Timeout 15000ms exceeded. FAIL WebKit 86353 ms — waitFor: Timeout 15000ms exceeded. FAIL 02-agenda-scroll 139099 ms — Chromium: waitFor: Timeout 15000ms exceeded. FAIL Chromium 40590 ms — waitFor: Timeout 15000ms exceeded. FAIL WebKit 40374 ms — waitFor: Timeout 15000ms exceeded. FAIL 03-task-board 80964 ms — Chromium: waitFor: Timeout 15000ms exceeded. FAIL Chromium 57815 ms — #1121: the 15 MB Actual import did not reach review in 30 s (received 15055637 bytes; status: Aggregating Actual source totals · 114,688 of 560,000 transactions) FAIL WebKit 64254 ms — #1121: the 15 MB Actual import did not reach review in 30 s (received 15055635 bytes; status: Aggregating Actual source totals · 136,192 of 560,000 transactions) FAIL 04-money-actual 122069 ms — Chromium: #1121: the 15 MB Actual import did not reach review in 30 s (received 15055637 bytes; status: Aggregating Actual source totals · 114,688 of 560,000 transactions) | WebKit: #1121: the 15 MB Actual import did not reach review in 30 s (received 15055635 bytes; status: Aggregating Actual source totals · 136,192 of 560,000 transactions) PASS Chromium 40707 ms PASS WebKit 22291 ms PASS 05-note-paste-image 62998 ms FAIL Chromium 59055 ms — #1102: production Draw could not save the Drawing: The Draw could not be saved. Your drawing is still here; try again. FAIL WebKit 60288 ms — waitFor: Timeout 20000ms exceeded. FAIL 06-drawing-insert 119343 ms — Chromium: #1102: production Draw could not save the Drawing: The Draw could not be saved. Your drawing is still here; try again. | WebKit: waitFor: Timeout 20000ms exceeded. PASS Chromium 47094 ms PASS WebKit 25884 ms PASS 07-share-note 72978 ms PASS Chromium 23874 ms PASS WebKit 13789 ms PASS 08-mail-columns 37663 ms FAIL Chromium 30752 ms — the result label has at least 4.5:1 text contrast PASS WebKit 15931 ms FAIL 09-search-result 46683 ms — Chromium: the result label has at least 4.5:1 text contrast FAIL Chromium 31521 ms — the anchored phone sheet keeps its Apps & Devices title while scrolling PASS WebKit 36381 ms FAIL 10-settings-devices 67902 ms — Chromium: the anchored phone sheet keeps its Apps & Devices title while scrolling PASS Chromium 21291 ms PASS WebKit 23753 ms PASS 11-tab-bar 45044 ms PASS Chromium 68701 ms PASS WebKit 48247 ms PASS 12-files-upload 116948 ms FAIL Chromium 49391 ms — Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms FAIL WebKit 38916 ms — Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms FAIL 13-photos-heic 88307 ms — Chromium: Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms | WebKit: Photos indexes the extensionless HEIC in its timeline did not reach its expected state within 30000 ms FAIL Chromium 21501 ms — Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance. FAIL WebKit 26953 ms — Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance. FAIL 14-tag-actions 48454 ms — Chromium: Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance. | WebKit: Tag rename/merge has no visible Undo action in the production UI; the smoke flow cannot verify the requested Undo affordance. FAIL WebKit 45174 ms — #1059: the stale-build update flow clears the reload notice after the production refresh FAIL 15-app-update 45174 ms — WebKit: #1059: the stale-build update flow clears the reload notice after the production refresh TOTAL 1423 s; upload proxy 2 MiB/s + 80 ms; screenshots /home/kayg/Developer/calternal-wt/smoke-1122/artifacts/smoke ``` The full run's App Update result came from a stale local server binary: its build ID was `0582c41e0b6ce4a1f2807de0e527ffc899bcb68a`, while the production web build was `05e0ecb863eb5139638f94ab43f1650223934b0e`. After rebuilding `calternal-server` from this head, the focused App Update flow passed in 75,222 ms and captured all 18 responsive/theme screenshots: ```text BUILD_IDS old=0000000 current=05e0ecb863eb5139638f94ab43f1650223934b0e REPLAY previous Log=201 Event=201 LONG_PULL {"oldRequests":131,"latestRequests":20} SCREENSHOTS phone-390-light SCREENSHOTS phone-390-dark SCREENSHOTS tablet-820-light SCREENSHOTS tablet-820-dark SCREENSHOTS desktop-1440-light SCREENSHOTS desktop-1440-dark PASS 15-app-update [WebKit] 75222 ms SMOKE RESULTS PASS 15-app-update 75222 ms TOTAL 167 s; upload proxy 2 MiB/s + 80 ms; screenshots /home/kayg/Developer/calternal-wt/smoke-1122/artifacts/smoke ``` ### Other gates `cargo fmt --check` passed with no output. `cargo build -p calternal-server` passed after refreshing the build ID. `bash -n deploy/smoke-gate.sh` passed with no output. No Rust source changed, so Rust clippy/test gates were not run. `bun run check` returned exit code 2. Verbatim output: ```text $ ../../scripts/perf-lint --check && node scripts/check-user-storage.mjs && node scripts/check-glass-tokens.mjs && node scripts/check-type-tokens.mjs && node scripts/check-focus-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json perf-lint: parse Rust product sources perf-lint: parse browser product sources perf-lint: validate coverage and architecture perf-lint: INVALID: ('render.bound', 'apps/web/src/lib/components/app-sidebar.svelte', 'apps/web/src/lib/components/app-sidebar.svelte#each:10a1aaf3137a2426:1'): unused or changed exception ``` ### UX gaps UX gaps closed: the suite exercises real primary actions and state persistence in the covered flows, keeps known-bug cases as hard failures, and captures the Mac rendering across the three widths and two themes. No production UI code changed. UX gaps left: the full run found Composer batch visibility in WebKit; Agenda targets missing in both engines; Board Done cards missing in both engines; Actual import #1121 stuck while aggregating after the complete upload; Drawing save #1102 in Chromium and missing Canvas preview in WebKit; Search result contrast below 4.5:1 in Chromium; the Apps & Devices phone sheet title not staying visible in Chromium; extensionless HEIC indexing in both engines; and no visible Tag Undo in either engine. The focused App Update rerun passed with a current matching server. The whole matrix should be rerun on a matching server during the merge round. ### Decisions not covered by DESIGN.md - Reuse the production E2E harness and the real local server, with one seeded User shared by Chromium and WebKit. - Keep all generated files below `target/tmp`; generate the 560,000-row Actual archive from invented deterministic values. - Capture six Mac-profile screenshot states per regular flow (three widths × two themes); the dedicated App Update flow keeps its existing WebKit touch path. - Keep a focused-flow environment variable for diagnosis while leaving the default command unfiltered. ### Screenshot bundles [01 Composer batch](https://git.kayg.org/attachments/6e6ccced-7e9a-4e2b-97db-13fb1faaba93) · [02 Agenda scroll](https://git.kayg.org/attachments/b576cdaf-0181-45e7-8ca7-39be4930c0b4) · [03 Task Board](https://git.kayg.org/attachments/0278c2e2-bfdd-4170-92c2-c33b9c8a9e18) · [04 Money Actual](https://git.kayg.org/attachments/bece17b6-ea19-4042-a060-e217c39c6817) · [05 Note paste image](https://git.kayg.org/attachments/ebcb8cec-ad29-4300-b886-a3c72de1e242) · [06 Drawing insert](https://git.kayg.org/attachments/cfcba5f2-c7d3-4979-a5b0-47f94dfeb110) · [07 Share Note](https://git.kayg.org/attachments/0d94dd9b-7523-49a5-890e-9113dad013e6) · [08 Mail columns](https://git.kayg.org/attachments/7bbf077a-7350-43f1-b200-14d46503671e) · [09 Search result](https://git.kayg.org/attachments/e000c10e-da27-48e7-b3c1-61bf681ca1a1) · [10 Settings devices](https://git.kayg.org/attachments/fc9e82b9-6250-41cc-b89f-21c072a0a938) · [11 Tab Bar](https://git.kayg.org/attachments/2a50daf2-c48f-4ecf-92de-d7f36f0d3668) · [12 Files upload](https://git.kayg.org/attachments/71d44385-8324-41b7-b5d3-b4dcfc4cc45b) · [13 Photos HEIC](https://git.kayg.org/attachments/810f20c5-6d41-4880-ba30-d440c20c5862) · [14 Tag actions](https://git.kayg.org/attachments/325a2992-90f5-4862-82a8-83815cf63e99) · [15 App Update](https://git.kayg.org/attachments/3913686a-0cff-4896-b028-68cac3d653a9)

Rows
Columns

PROCESS: browser smoke suite of real User flows over a throttled network (deploy gate) #1122

Owner decisions (2026-10-05 process grill) + "more things need to be tested with browser-use"

Finished — Forgejo #1122

What I built

Full-suite result

Other gates

UX gaps

Decisions not covered by DESIGN.md

Screenshot bundles