Bug bash: perf

kayg commented

2026-10-06 19:02:25 +00:00

Owner

AREA: Worst-case performance audit (start after perf2-1124 finishes; read its report on #1124 first and do not repeat it). Use the perf VM through perf-run only (netbird ssh root@10.69.69.63; perf-run calternal bash-perf <minutes> -- cmd; no builds on that box; build here and copy the binary). Profiles: a large Home (100k files, 20k photos, 10k notes, 50k transactions, 10k mails), cold and warm on HDD; request storm (200 concurrent clients mixing reads and writes); first load and every Tab switch to fully painted; typing latency in a 5k-block note; tag rename over 5k notes; search-as-you-type over the large Home; idle RSS and CPU after 10 minutes. Find the top slow paths with flamegraphs (cargo flamegraph / perf) and Chrome traces. Fix the clear ones (batched transactions, no awaited loops over independent IO, no full scans per operation, short locks, caching) with before/after medians over at least 3 interleaved runs. File the rest as issues with numbers.

Tracking issue for the 2026-10-07 bug bash.

AREA: Worst-case performance audit (start after perf2-1124 finishes; read its report on #1124 first and do not repeat it). Use the perf VM through perf-run only (netbird ssh root@10.69.69.63; `perf-run calternal bash-perf <minutes> -- cmd`; no builds on that box; build here and copy the binary). Profiles: a large Home (100k files, 20k photos, 10k notes, 50k transactions, 10k mails), cold and warm on HDD; request storm (200 concurrent clients mixing reads and writes); first load and every Tab switch to fully painted; typing latency in a 5k-block note; tag rename over 5k notes; search-as-you-type over the large Home; idle RSS and CPU after 10 minutes. Find the top slow paths with flamegraphs (cargo flamegraph / perf) and Chrome traces. Fix the clear ones (batched transactions, no awaited loops over independent IO, no full scans per operation, short locks, caching) with before/after medians over at least 3 interleaved runs. File the rest as issues with numbers. Tracking issue for the 2026-10-07 bug bash.

kayg referenced this issue

2026-10-06 22:16:42 +00:00

Files reconcile-storm gate panics when a completed JoinHandle is polled again #1213

kayg commented

2026-10-07 04:01:02 +00:00

Author

Owner

Starting bash-perf on branch job/bash-perf, based at ff53d7b8b4da5a21f862d57dcc3496f8c019f647 (current ancestor of origin/dev). I am reviewing the completed #1124 findings before selecting distinct worst-case paths.

Starting bash-perf on branch `job/bash-perf`, based at `ff53d7b8b4da5a21f862d57dcc3496f8c019f647` (current ancestor of `origin/dev`). I am reviewing the completed #1124 findings before selecting distinct worst-case paths.

kayg commented

2026-10-07 04:03:49 +00:00

Author

Owner

Finding: Mail's 10,000-Message initial backfill is incomplete within the 180-second profile window. The #1218 production-build run reported 8,880/10,000 Messages at 180 s (49.3 Messages/s), and a second scaled run reported 8,440 at 350 s. First content appeared at 16.517 s. I filed the symptom as #1311 and am profiling fetch versus Index commit work before selecting a fix.

kayg commented

2026-10-07 04:05:51 +00:00

Author

Owner

Source finding for #1311: Mail issues one bounded UID FETCH command per 80-UID window. In cache/store.rs::commit_window, it then performs a serial database lookup and multiple awaited writes for each Message inside the window transaction (membership lookup, provider identity/thread lookups, message projection, attachments, flags and membership). SQLite's ordinary writer uses WAL with synchronous=NORMAL. This rules out one IMAP round trip per Message and narrows the likely cost to per-Message Index work; I am measuring before changing it. The FTS insert trigger also runs for each new mail_messages row and remains a candidate to quantify.

Source finding for #1311: Mail issues one bounded `UID FETCH` command per 80-UID window. In `cache/store.rs::commit_window`, it then performs a serial database lookup and multiple awaited writes for each Message inside the window transaction (membership lookup, provider identity/thread lookups, message projection, attachments, flags and membership). SQLite's ordinary writer uses WAL with `synchronous=NORMAL`. This rules out one IMAP round trip per Message and narrows the likely cost to per-Message Index work; I am measuring before changing it. The FTS insert trigger also runs for each new `mail_messages` row and remains a candidate to quantify.

kayg commented

2026-10-07 04:15:47 +00:00

Author

Owner

Base refresh: I fetched and fast-forwarded this worktree from origin/dev once, after reading #1124. The branch is now at 3601719b80a39f712daf07b37879b22929dfffa0; there were no conflicts. No additional origin/dev merge is planned before final gates.

Base refresh: I fetched and fast-forwarded this worktree from `origin/dev` once, after reading #1124. The branch is now at `3601719b80a39f712daf07b37879b22929dfffa0`; there were no conflicts. No additional origin/dev merge is planned before final gates.

kayg commented

2026-10-07 04:19:04 +00:00

Author

Owner

Before/after rule for #1311: keep an Index write-path change only if three interleaved 10,000-message history pairs on the perf VM improve median wall time by at least 10% and do not regress p95. Keep all six samples. Each pair uses the same prebuilt release test binaries and the same 10k data size; the production-build Dovecot browser run separately checks first content, first paint, and full completion. No comparable Mail sync profile exists in docs/perf/baseline.json.

Before/after rule for #1311: keep an Index write-path change only if three interleaved 10,000-message history pairs on the perf VM improve median wall time by at least 10% and do not regress p95. Keep all six samples. Each pair uses the same prebuilt release test binaries and the same 10k data size; the production-build Dovecot browser run separately checks first content, first paint, and full completion. No comparable Mail sync profile exists in `docs/perf/baseline.json`.

kayg commented

2026-10-07 05:07:51 +00:00

Author

Owner

10k Mail baseline evidence: one optimized profile_full_history run on the local build host completed 10,000 messages in 27,106 ms (369 messages/s) across 125 UID FETCH windows. Index page latency was p50 84.6 ms and p95 104.2 ms; peak RSS was 28,073,984 bytes (26.8 MiB). This scripted IMAP stream excludes TLS and provider network time. Load average before/after was 47.22/44.77/45.33 and 44.92/44.45/45.21. The #1218 TLS fixture reached 8,880/10,000 in 180 seconds (~49.3 messages/s), so this synthetic profile does not reproduce the end-to-end delay. I will profile fetch versus commit on the real TLS fixture before choosing a product fix. No comparable Mail sync row exists in docs/perf/baseline.json.

10k Mail baseline evidence: one optimized `profile_full_history` run on the local build host completed 10,000 messages in 27,106 ms (369 messages/s) across 125 UID FETCH windows. Index page latency was p50 84.6 ms and p95 104.2 ms; peak RSS was 28,073,984 bytes (26.8 MiB). This scripted IMAP stream excludes TLS and provider network time. Load average before/after was 47.22/44.77/45.33 and 44.92/44.45/45.21. The #1218 TLS fixture reached 8,880/10,000 in 180 seconds (~49.3 messages/s), so this synthetic profile does not reproduce the end-to-end delay. I will profile fetch versus commit on the real TLS fixture before choosing a product fix. No comparable Mail sync row exists in docs/perf/baseline.json.

kayg commented

2026-10-07 05:29:48 +00:00

Author

Owner

Local interleaved release A/B evidence for #1311 (three profile_full_history samples per binary, 10,000 messages each): baseline wall times were 26,177 / 21,814 / 12,438 ms; candidate commit e4eea7644 was 22,846 / 13,955 / 14,350 ms. Median wall time fell 34.2% (21,814 → 14,350 ms); maximum sample time fell 12.7% (26,177 → 22,846 ms). Median worker CPU fell 9.5% (10.45 → 9.46 s). Median peak RSS was 27,774,976 B baseline vs 28,073,984 B candidate. Page-read p95 medians were 8.15 vs 7.47 ms, with one candidate run at 20.64 ms. These are local-only samples: the shared perf VM was held by tasksperf-1286-before-5000-final for its 60-minute reservation. Host load fell from roughly 48 to 37 during the sequence, so I treat the result as directional and have not made the keep decision; I will repeat under perf-run if the VM is free and compare the real TLS run.

Local interleaved release A/B evidence for #1311 (three `profile_full_history` samples per binary, 10,000 messages each): baseline wall times were 26,177 / 21,814 / 12,438 ms; candidate commit `e4eea7644` was 22,846 / 13,955 / 14,350 ms. Median wall time fell 34.2% (21,814 → 14,350 ms); maximum sample time fell 12.7% (26,177 → 22,846 ms). Median worker CPU fell 9.5% (10.45 → 9.46 s). Median peak RSS was 27,774,976 B baseline vs 28,073,984 B candidate. Page-read p95 medians were 8.15 vs 7.47 ms, with one candidate run at 20.64 ms. These are local-only samples: the shared perf VM was held by `tasksperf-1286-before-5000-final` for its 60-minute reservation. Host load fell from roughly 48 to 37 during the sequence, so I treat the result as directional and have not made the keep decision; I will repeat under `perf-run` if the VM is free and compare the real TLS run.

kayg commented

2026-10-07 05:42:20 +00:00

Author

Owner

#1213 local profile attempt: the focused Files storm binary compiled, but the test did not reach the reconcile phase. The first atomic write hit the existing calternal-fs free-space reserve guard: reserve 31,695,649,792 bytes; available 14,973,407,232 bytes (17 GB at df). I did not change the guard or any test expectation. The storm timings will be collected on the perf VM after its current perf-run reservation ends. Commit 86c441380 also makes the test print completed scan timings if it later reaches the five-minute timeout.

#1213 local profile attempt: the focused Files storm binary compiled, but the test did not reach the reconcile phase. The first atomic write hit the existing `calternal-fs` free-space reserve guard: reserve 31,695,649,792 bytes; available 14,973,407,232 bytes (17 GB at `df`). I did not change the guard or any test expectation. The storm timings will be collected on the perf VM after its current `perf-run` reservation ends. Commit `86c441380` also makes the test print completed scan timings if it later reaches the five-minute timeout.

kayg commented

2026-10-07 06:20:35 +00:00

Author

Owner

Filed test-fixture issue #1328 after the 10k real TLS profile completed in 39.292 s (254.50 messages/s): its UI follow-up could not find the Northstar thread because the fixed fixture messages were around UID 1999, outside the newest-first first page of a 10k mailbox. I moved the two scripted thread messages to the final two generated message positions for scaled fixtures and added a 10k fixture regression test. I will rerun the full UI flow after the focused fixture check.

kayg commented

2026-10-07 06:31:59 +00:00

Author

Owner

Correction to my initial #1328 diagnosis: the default 2k run showed the Northstar row was absent because DESIGN §45 classifies the invoice into Receipts and says Inbox displays People only. I corrected the E2E to open /mail/c/receipts before the existing thread/attachment/sender-category assertions, and kept that two-message fixture at the newest provider dates so it stays on the first page at 10k. #1328 now tracks this corrected symptom.

Correction to my initial #1328 diagnosis: the default 2k run showed the Northstar row was absent because DESIGN §45 classifies the invoice into Receipts and says Inbox displays People only. I corrected the E2E to open `/mail/c/receipts` before the existing thread/attachment/sender-category assertions, and kept that two-message fixture at the newest provider dates so it stays on the first page at 10k. #1328 now tracks this corrected symptom.

kayg commented

2026-10-07 06:33:59 +00:00

Author

Owner

Filed #1330 for the residual thread UI timeout observed after the 2k sync: opening the Northstar thread succeeded, but its expected body did not render within 30 s under local host load (~21 / 23 / 30). Server logs recorded a 2.15 s membership INSERT and a 1.62 s passive WAL checkpoint. The test did not reach screenshot capture. I stopped retries and kept the evidence separate from the successful backfill measurements.

kayg commented

2026-10-07 06:49:25 +00:00

Author

Owner

#1213 profile complete: local 1,000 writes + four scans passed in 151.654 s. Reconcile scans were 20/11/5/4 ms before and 22/8/8/7 ms during the write burst; watcher/feed/journal each processed 1,001 rows. Inference: scan/reconcile was not the slow part in this sample; writes and watcher adoption dominated. The earlier HDD-emulated perf-VM run exceeded 300 s under a low recorded load and hit the test cleanup double-await; that harness issue is fixed in 5e87adf1a. The crate gate passed 260/0/4 ignored.

#1213 profile complete: local 1,000 writes + four scans passed in 151.654 s. Reconcile scans were 20/11/5/4 ms before and 22/8/8/7 ms during the write burst; watcher/feed/journal each processed 1,001 rows. Inference: scan/reconcile was not the slow part in this sample; writes and watcher adoption dominated. The earlier HDD-emulated perf-VM run exceeded 300 s under a low recorded load and hit the test cleanup double-await; that harness issue is fixed in 5e87adf1a. The crate gate passed 260/0/4 ignored.

kayg commented

2026-10-07 06:49:50 +00:00

Author

Owner

Mail result: commit e4eea7644 reduces the local synthetic 10k history profile median from 21.814 s to 14.350 s across three interleaved samples (-34.2%; directional under falling host load). A real TLS 10k run completed in 39.292 s (254.50 messages/s) versus the historical #1218 result of 8,880/10,000 in 180 s (~49.3/s). Page timing from 2k TLS runs puts median Index commit at about 20 ms/page versus provider fetch at about 4 ms/page; p95 medians were 204 ms and 22 ms. The second 10k attempt hit a 22.77 s passive WAL checkpoint and storage error on the high-load local host; #1330 tracks a separate thread-body timeout. No controlled perf-VM Mail flamegraph was captured because perf-run was busy.

Mail result: commit `e4eea7644` reduces the local synthetic 10k history profile median from 21.814 s to 14.350 s across three interleaved samples (-34.2%; directional under falling host load). A real TLS 10k run completed in 39.292 s (254.50 messages/s) versus the historical #1218 result of 8,880/10,000 in 180 s (~49.3/s). Page timing from 2k TLS runs puts median Index commit at about 20 ms/page versus provider fetch at about 4 ms/page; p95 medians were 204 ms and 22 ms. The second 10k attempt hit a 22.77 s passive WAL checkpoint and storage error on the high-load local host; #1330 tracks a separate thread-body timeout. No controlled perf-VM Mail flamegraph was captured because `perf-run` was busy.

kayg commented

2026-10-07 07:05:46 +00:00

Author

Owner

Filed #1337 for the single 10k TLS attempt that failed after a passive WAL checkpoint took 22.768 s: the cache step returned a storage error and Mail surfaced a generic IMAP connection error. The run followed one 39.292 s successful 10k backfill and happened under a local host load of roughly 19 / 27 / 36. I stopped retries; this needs a controlled low-load reproduction.

kayg commented

2026-10-07 07:32:15 +00:00

Author

Owner

Result

Head: 88ce7a765dcb166940621ed35829483aa06aaaaa.

Implemented page-batched Mail thread-key resolution and writes, content-free fetch/Index timing, a 10k Mail profile option, Files reconcile-storm phase timings, timeout evidence, and a single-join watcher fix. The Mail cache work scales with the existing bounded page: one indexed lookup over its unique thread keys, O(keys in that page) memory, and writes in chunks of at most 300 rows. It adds no provider round trips.

Findings

Issue	Symptom	Severity	Outcome
#1311	10k Mail backfill reached only 8,880 messages after 180 s; per-message thread-key SQL amplified Index work.	Major	Fixed on branch in `e4eea7644`; profile changes are in `ccb788673`.
#1213	Files reconcile-storm test exceeded five minutes under contention.	Minor	Added scan/write phase timings and timeout evidence; scans measured 4–22 ms locally. Fixed the test's double-await panic in `5e87adf1a`. No production reconcile code changed.
#1328	Mail E2E expected a Receipt thread from the People-only Inbox.	Minor	Fixed the fixture to open Receipts in `ad5de1b04`; provider fixture regression test passed.
#1330	Mail thread body did not appear within 30 s in a 2k TLS E2E run under high host load.	Minor	Filed only; no product source change.
#1337	One 10k Mail run returned a cache storage error after a 22.768 s WAL checkpoint.	Minor	Filed only with the trace at `target/tmp/1329-mail-checkpoint-evidence.txt`; no retry under load.

Performance

Local synthetic 10k history A/B, three interleaved runs: wall median 21.814 s → 14.350 s (-34.2%); maximum 26.177 s → 22.846 s (-12.7%); CPU median 10.45 s → 9.46 s (-9.5%); peak RSS increased by about 0.3 MiB. The host load changed during sampling, so treat this as directional.
One real TLS 10k backfill completed in 39.292 s (254.50 messages/s); first content was 16.046 s and first paint 16.845 s. The historical #1218 run reached 8,880/10,000 in 180 s (~49.3 messages/s); this is not a controlled same-host comparison.
Three local 2k TLS samples took 28.696 / 25.824 / 18.195 s. Representative provider-fetch page p50/p95 was 4/22 ms; Index-commit p50/p95 was 20/204 ms, with spikes up to about 2.2 s under load.
Files storm profile: 1,000 writes and four concurrent scans; scans before writes were 20/11/5/4 ms, scans during the burst were 22/8/8/7 ms, and writes plus scans took 150.962 s (151.654 s total). Index/change-feed/journal rows were 1,000/1,001/1,001. The perf-VM run took 310.80 s and then hit the test's double-await panic. Later perf-run lock attempts found the VM busy, so there is no clean perf-VM A/B or flamegraph.
The Mail change costs O(K) page-local memory and processing for K unique thread keys, one indexed key lookup per non-empty page, and ceil(K/300) batched key writes inside the existing transaction.

Gates (output)

cargo fmt --check: exit 0; no output.
cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings: Finished dev profile [unoptimized + debuginfo] target(s) in 2m 28s.
cargo test -p calternal-plugin-mail: test result: ok. 128 passed; 0 failed; 6 ignored; 0 measured; 0 filtered out; finished in 22.48s; doc tests: test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s.
cargo clippy -p calternal-plugin-files --all-targets -- -D warnings: Finished dev profile [unoptimized + debuginfo] target(s) in 16.88s.
cargo test -p calternal-plugin-files: test result: ok. 260 passed; 0 failed; 4 ignored; 0 measured; 0 filtered out; finished in 214.01s; doc tests: test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s.
bun run check: perf-lint: PASS; 0 violations; 22548 scoped exceptions; svelte-check found 0 errors and 1 warning in 1 file. The warning is the existing empty ruleset at packages/ui/src/components/calendar/AttachmentDeck.svelte:1055.

bun run test (exit 1):

FAIL  |unit| src/lib/components/analytics/BklitTooltipMaterial.test.ts > analytics tooltip material > styles the readout content after it is portaled outside the chart
FAIL  |component| src/lib/components/KeyboardShortcutsCard.svelte.test.ts > KeyboardShortcutsCard > puts the current view group first and closes with Escape
FAIL  |component| src/lib/components/SidebarLinks.svelte.test.ts > SidebarLinks view identity > keeps the row and focus when its date destination changes
FAIL  |component| src/lib/files/InviteLinkSection.svelte.test.ts > Share dialog invite controls > hides access details while off, then reveals them after a keyboard enable
FAIL  |component| src/lib/files/RecipientPicker.svelte.test.ts > Share recipients > starts empty and selects a searched Group by keyboard
Error: Test timed out in 5000ms.
Test Files  5 failed | 282 passed (287)
     Tests  5 failed | 2030 passed (2035)
   Duration  2004.45s (transform 29%, environment 26%, import 26%, tests 12%, setup 6%, worker 1%)
error: script "test" exited with code 1

The run lasted 33m 24s while host load was roughly 45–63. These failures are test timeouts across unrelated UI areas; no expectations or unrelated code were changed.

bun run build passed. cargo build -p calternal-server --release --features mail-test-provider passed. Focused fixture suite: Ran 7 tests in 2.989s / OK.

UX gaps

Closed: the real Mail fixture now opens the Receipts view before checking its thread/attachment flow.
Left: the 2k TLS E2E body assertion timed out before screenshot capture. No visual fix was made, so this job has no before/after screenshots.

Gaps and decisions

Not measured: large Home cold/warm HDD load, 200-client mixed request storm, 5k-block typing latency, tag rename across 5k notes, large-Home search, 10-minute idle RSS/CPU, or first load/Tab-switch paint profiles. No Chrome trace or clean perf-VM flamegraph was captured because the VM lock was unavailable. Schedule those profiles through perf-run with prebuilt binaries; do not infer their results from these Mail/Files samples.
The thread-key batch uses 300 rows so its three bind values per row stay below SQLite's historical 999-variable limit. Page timing logs contain counts and durations only, not message data.
Protocol/storage references reviewed: RFC 9051 FETCH supports sequence-set fetches; SQLite WAL documentation notes that long readers can stop checkpoint progress. The local checkpoint sample is consistent with that risk but does not prove a reader blocked this specific checkpoint.
Files touched: apps/web/e2e/mail-sync-613.mjs, bench/mail-sync.py, crates/plugins/files/src/lib.rs, crates/plugins/mail/src/cache/store.rs, crates/plugins/mail/src/sync.rs, tests/adversarial/mail_sync_provider.py, tests/adversarial/test_mail_sync_provider.py.
apps/web/build and the Cargo target were removed after gates. No push, deploy, or merge was run.

For the merge round

Re-run cd apps/web && bun run test once on the combined branch and require a passing result. Do not change the five existing timeout expectations to make it pass; verify whether the same 5-second timeouts recur under the merge-round host conditions.
The scheduled performance follow-up should use prebuilt test binaries via perf-run only, capture controlled baseline/candidate samples and flamegraphs, then run the deferred large-Home and UI hot paths listed above.

## Result Head: `88ce7a765dcb166940621ed35829483aa06aaaaa`. Implemented page-batched Mail thread-key resolution and writes, content-free fetch/Index timing, a 10k Mail profile option, Files reconcile-storm phase timings, timeout evidence, and a single-join watcher fix. The Mail cache work scales with the existing bounded page: one indexed lookup over its unique thread keys, O(keys in that page) memory, and writes in chunks of at most 300 rows. It adds no provider round trips. ## Findings | Issue | Symptom | Severity | Outcome | |---|---|---|---| | #1311 | 10k Mail backfill reached only 8,880 messages after 180 s; per-message thread-key SQL amplified Index work. | Major | Fixed on branch in `e4eea7644`; profile changes are in `ccb788673`. | | #1213 | Files reconcile-storm test exceeded five minutes under contention. | Minor | Added scan/write phase timings and timeout evidence; scans measured 4–22 ms locally. Fixed the test's double-await panic in `5e87adf1a`. No production reconcile code changed. | | #1328 | Mail E2E expected a Receipt thread from the People-only Inbox. | Minor | Fixed the fixture to open Receipts in `ad5de1b04`; provider fixture regression test passed. | | #1330 | Mail thread body did not appear within 30 s in a 2k TLS E2E run under high host load. | Minor | Filed only; no product source change. | | #1337 | One 10k Mail run returned a cache storage error after a 22.768 s WAL checkpoint. | Minor | Filed only with the trace at `target/tmp/1329-mail-checkpoint-evidence.txt`; no retry under load. | ## Performance - Local synthetic 10k history A/B, three interleaved runs: wall median 21.814 s → 14.350 s (-34.2%); maximum 26.177 s → 22.846 s (-12.7%); CPU median 10.45 s → 9.46 s (-9.5%); peak RSS increased by about 0.3 MiB. The host load changed during sampling, so treat this as directional. - One real TLS 10k backfill completed in 39.292 s (254.50 messages/s); first content was 16.046 s and first paint 16.845 s. The historical #1218 run reached 8,880/10,000 in 180 s (~49.3 messages/s); this is not a controlled same-host comparison. - Three local 2k TLS samples took 28.696 / 25.824 / 18.195 s. Representative provider-fetch page p50/p95 was 4/22 ms; Index-commit p50/p95 was 20/204 ms, with spikes up to about 2.2 s under load. - Files storm profile: 1,000 writes and four concurrent scans; scans before writes were 20/11/5/4 ms, scans during the burst were 22/8/8/7 ms, and writes plus scans took 150.962 s (151.654 s total). Index/change-feed/journal rows were 1,000/1,001/1,001. The perf-VM run took 310.80 s and then hit the test's double-await panic. Later `perf-run` lock attempts found the VM busy, so there is no clean perf-VM A/B or flamegraph. - The Mail change costs O(K) page-local memory and processing for K unique thread keys, one indexed key lookup per non-empty page, and `ceil(K/300)` batched key writes inside the existing transaction. ## Gates (output) - `cargo fmt --check`: exit 0; no output. - `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings`: `Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 28s`. - `cargo test -p calternal-plugin-mail`: `test result: ok. 128 passed; 0 failed; 6 ignored; 0 measured; 0 filtered out; finished in 22.48s`; doc tests: `test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s`. - `cargo clippy -p calternal-plugin-files --all-targets -- -D warnings`: `Finished `dev` profile [unoptimized + debuginfo] target(s) in 16.88s`. - `cargo test -p calternal-plugin-files`: `test result: ok. 260 passed; 0 failed; 4 ignored; 0 measured; 0 filtered out; finished in 214.01s`; doc tests: `test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s`. - `bun run check`: `perf-lint: PASS; 0 violations; 22548 scoped exceptions`; `svelte-check found 0 errors and 1 warning in 1 file`. The warning is the existing empty ruleset at `packages/ui/src/components/calendar/AttachmentDeck.svelte:1055`. - `bun run test` (exit 1): ```text FAIL |unit| src/lib/components/analytics/BklitTooltipMaterial.test.ts > analytics tooltip material > styles the readout content after it is portaled outside the chart FAIL |component| src/lib/components/KeyboardShortcutsCard.svelte.test.ts > KeyboardShortcutsCard > puts the current view group first and closes with Escape FAIL |component| src/lib/components/SidebarLinks.svelte.test.ts > SidebarLinks view identity > keeps the row and focus when its date destination changes FAIL |component| src/lib/files/InviteLinkSection.svelte.test.ts > Share dialog invite controls > hides access details while off, then reveals them after a keyboard enable FAIL |component| src/lib/files/RecipientPicker.svelte.test.ts > Share recipients > starts empty and selects a searched Group by keyboard Error: Test timed out in 5000ms. Test Files 5 failed | 282 passed (287) Tests 5 failed | 2030 passed (2035) Duration 2004.45s (transform 29%, environment 26%, import 26%, tests 12%, setup 6%, worker 1%) error: script "test" exited with code 1 ``` The run lasted 33m 24s while host load was roughly 45–63. These failures are test timeouts across unrelated UI areas; no expectations or unrelated code were changed. - `bun run build` passed. `cargo build -p calternal-server --release --features mail-test-provider` passed. Focused fixture suite: `Ran 7 tests in 2.989s` / `OK`. ## UX gaps - Closed: the real Mail fixture now opens the Receipts view before checking its thread/attachment flow. - Left: the 2k TLS E2E body assertion timed out before screenshot capture. No visual fix was made, so this job has no before/after screenshots. ## Gaps and decisions - Not measured: large Home cold/warm HDD load, 200-client mixed request storm, 5k-block typing latency, tag rename across 5k notes, large-Home search, 10-minute idle RSS/CPU, or first load/Tab-switch paint profiles. No Chrome trace or clean perf-VM flamegraph was captured because the VM lock was unavailable. Schedule those profiles through `perf-run` with prebuilt binaries; do not infer their results from these Mail/Files samples. - The thread-key batch uses 300 rows so its three bind values per row stay below SQLite's historical 999-variable limit. Page timing logs contain counts and durations only, not message data. - Protocol/storage references reviewed: [RFC 9051 FETCH](https://datatracker.ietf.org/doc/html/rfc9051#section-6.4.5) supports sequence-set fetches; [SQLite WAL documentation](https://www.sqlite.org/wal.html) notes that long readers can stop checkpoint progress. The local checkpoint sample is consistent with that risk but does not prove a reader blocked this specific checkpoint. - Files touched: `apps/web/e2e/mail-sync-613.mjs`, `bench/mail-sync.py`, `crates/plugins/files/src/lib.rs`, `crates/plugins/mail/src/cache/store.rs`, `crates/plugins/mail/src/sync.rs`, `tests/adversarial/mail_sync_provider.py`, `tests/adversarial/test_mail_sync_provider.py`. - `apps/web/build` and the Cargo target were removed after gates. No push, deploy, or merge was run. ## For the merge round - Re-run `cd apps/web && bun run test` once on the combined branch and require a passing result. Do not change the five existing timeout expectations to make it pass; verify whether the same 5-second timeouts recur under the merge-round host conditions. - The scheduled performance follow-up should use prebuilt test binaries via `perf-run` only, capture controlled baseline/candidate samples and flamegraphs, then run the deferred large-Home and UI hot paths listed above.

kayg referenced this issue

2026-10-07 07:45:31 +00:00

Mail: first sync of 10k messages under 7 s #1340

Rows
Columns

Bug bash: perf #1224

Result

Findings

Performance

Gates (output)

UX gaps

Gaps and decisions

For the merge round