PERF: Reduce Search palette first-result latency and typing jank #434

Open
opened 2026-09-29 12:32:53 +00:00 by kayg · 3 comments
Owner

Parent: #367, Phase 2 split. This issue tracks the Search palette typing path and its main-thread long tasks. It is separate from Calendar Event FTS server latency, tracked in #433.

Evidence

The 2026-09-28 production palette profile ran on the shared host at load about 20, so treat it as diagnostic only. The first non-empty result frame took p50 92 ms and p95 425.8 ms over 48 samples. The server-result frame p95 was 513.6 ms. Typing produced 24 main-thread long tasks over 50 ms; the longest was 339 ms. The target is a first result frame within 50 ms and no main-thread long task over 50 ms while typing.

Work

Use the existing production palette probe and Chrome long-task/long-animation-frame attribution to split request time from client work and identify the longest main-thread task. Reduce the client-side contribution and record before/after values on the quiet perf-test VM. Keep the 2026-09-28 values labelled as busy-host diagnostics until the remeasurement is complete.

Parent: #367, Phase 2 split. This issue tracks the Search palette typing path and its main-thread long tasks. It is separate from Calendar Event FTS server latency, tracked in #433. ## Evidence The 2026-09-28 production palette profile ran on the shared host at load about 20, so treat it as diagnostic only. The first non-empty result frame took p50 92 ms and p95 425.8 ms over 48 samples. The server-result frame p95 was 513.6 ms. Typing produced 24 main-thread long tasks over 50 ms; the longest was 339 ms. The target is a first result frame within 50 ms and no main-thread long task over 50 ms while typing. ## Work Use the existing production palette probe and Chrome long-task/long-animation-frame attribution to split request time from client work and identify the longest main-thread task. Reduce the client-side contribution and record before/after values on the quiet perf-test VM. Keep the 2026-09-28 values labelled as busy-host diagnostics until the remeasurement is complete.
Author
Owner

Round 2 update (2026-09-29): this run collected no quiet-VM palette measurements. The release server rebuild did not finish before the four-hour job limit, so the first-result-frame probe and browser soak did not run under /root/perf.lock. Keep the 2026-09-28 busy-host values above marked as diagnostic; this update adds no new frame or long-task numbers. Parent issue #367 records the exact gates and remaining work.

Round 2 update (2026-09-29): this run collected no quiet-VM palette measurements. The release server rebuild did not finish before the four-hour job limit, so the first-result-frame probe and browser soak did not run under `/root/perf.lock`. Keep the 2026-09-28 busy-host values above marked as diagnostic; this update adds no new frame or long-task numbers. Parent issue #367 records the exact gates and remaining work.
Author
Owner

#367 quiet-VM remeasure (perf-test VM, 4 vCPU; release 369ab6a2f; production SPA; 4× CPU throttle; load < 1 at start under /root/perf.lock).

Probe p50 p95 Other
Palette keystroke → first result frame, 100k corpus 100.5 ms 218.4 ms 46 samples; 12 tasks > 50 ms, max 163 ms
Palette keystroke → server result frame, 100k 110.7 ms 218.4 ms 82 samples
Palette keystroke → first frame, 1M corpus 53.8 ms 116 ms 2 tasks > 50 ms, max 75 ms
open_palette interaction (route probe, 15 samples) 191 ms visible 405 ms visible INP 328 ms

Server round trip is not the cost: keyword Search is 6.2 ms p50 at 100k (see #433). The 50 ms p95 budget is missed by about 4× at 100k under 4× throttle. Long-animation-frame traces point at the #document.oninput handler in chunks/Dpg5AStj.js (46 ms script in one 69 ms frame) and Response.json continuations in chunks/O4MrJCmS.js.

Side note: apps/web/e2e/search.mjs fails on this build at "result growth stays centred" (fewCentre 450.5 vs manyCentre 442.2), so the search-ui profile had no result this round.

Raw: docs/perf/runs/2026-09-29T224501Z-369ab6a2/raw/{search-mixed,routes}.json.

#367 quiet-VM remeasure (perf-test VM, 4 vCPU; release `369ab6a2f`; production SPA; 4× CPU throttle; load < 1 at start under `/root/perf.lock`). | Probe | p50 | p95 | Other | |---|---:|---:|---| | Palette keystroke → first result frame, 100k corpus | 100.5 ms | 218.4 ms | 46 samples; 12 tasks > 50 ms, max 163 ms | | Palette keystroke → server result frame, 100k | 110.7 ms | 218.4 ms | 82 samples | | Palette keystroke → first frame, 1M corpus | 53.8 ms | 116 ms | 2 tasks > 50 ms, max 75 ms | | `open_palette` interaction (route probe, 15 samples) | 191 ms visible | 405 ms visible | INP 328 ms | Server round trip is not the cost: keyword Search is 6.2 ms p50 at 100k (see #433). The 50 ms p95 budget is missed by about 4× at 100k under 4× throttle. Long-animation-frame traces point at the `#document.oninput` handler in `chunks/Dpg5AStj.js` (46 ms script in one 69 ms frame) and `Response.json` continuations in `chunks/O4MrJCmS.js`. Side note: `apps/web/e2e/search.mjs` fails on this build at "result growth stays centred" (`fewCentre` 450.5 vs `manyCentre` 442.2), so the `search-ui` profile had no result this round. Raw: `docs/perf/runs/2026-09-29T224501Z-369ab6a2/raw/{search-mixed,routes}.json`.
Author
Owner

The round-3 Search e2e recorded another local shared-host diagnostic. It passed its functional Search assertions, then failed the 50 ms first-result budget:

keystroke to first result frame: p50=175.8ms p95=359.0ms (n=39)
main-thread long tasks: 19 over 50ms, max=724.0ms

The local host was heavily loaded during this round; the adjacent local attachment benchmark recorded load average 23.69 / 24.80 / 26.29 before measurement. This is one run only, not a quiet-host regression measurement. The e2e result log is target/tmp/search-e2e.log in the job worktree.

The round-3 Search e2e recorded another local shared-host diagnostic. It passed its functional Search assertions, then failed the 50 ms first-result budget: ``` keystroke to first result frame: p50=175.8ms p95=359.0ms (n=39) main-thread long tasks: 19 over 50ms, max=724.0ms ``` The local host was heavily loaded during this round; the adjacent local attachment benchmark recorded load average 23.69 / 24.80 / 26.29 before measurement. This is one run only, not a quiet-host regression measurement. The e2e result log is `target/tmp/search-e2e.log` in the job worktree.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#434
No description provided.