Mail: first sync of 10k messages under 7 s #1340

Closed

opened 2026-10-07 07:45:31 +00:00 by kayg · 61 comments

kayg commented 2026-10-07 07:45:31 +00:00

Owner

Copy link

Owner target 2026-10-07: Mail first sync of a 10,000-message history must finish in under 7 s (currently 14.35 s median after #1311 batching, from 21.8 s; local synthetic 10k profile in bench/mail-sync.py). Keep correctness exactly: same messages, threads, flags and folders as the current sync; no lost or duplicated messages after interruption and resume; memory bounded. Follow-up of #1311 / #1224.

Owner target 2026-10-07: Mail first sync of a 10,000-message history must finish in under 7 s (currently 14.35 s median after #1311 batching, from 21.8 s; local synthetic 10k profile in bench/mail-sync.py). Keep correctness exactly: same messages, threads, flags and folders as the current sync; no lost or duplicated messages after interruption and resume; memory bounded. Follow-up of #1311 / #1224.

kayg commented 2026-10-07 07:46:06 +00:00

Author

Owner

Copy link

Starting Phase 1 on branch job/mail7-1340 at base 88ce7a765d. I will fetch and merge origin/dev, then profile the existing 10k sync before changing code.

Starting Phase 1 on branch job/mail7-1340 at base 88ce7a765dcb166940621ed35829483aa06aaaaa. I will fetch and merge origin/dev, then profile the existing 10k sync before changing code.

kayg commented 2026-10-07 08:28:02 +00:00

Author

Owner

Copy link

Phase 1 profile complete, before sync algorithm changes.

I ran the release test binary with the existing synthetic 10k profile. It completed 10,000 UIDs in 16.080 s wall / 8.334 s process CPU; the host load average was 48.43 before and 49.61 after (1 min). This host contention makes the wall number diagnostic, not a clean comparison to the 14.35 s reference.

Phase	Wall s	CPU s	Wall share
Session greeting/login	0.000	0.000	0.0%
Folder discovery/Index setup	0.048	0.026	0.3%
SELECT/EXAMINE commands	0.000	0.000	0.0%
FETCH protocol and reply waits	0.241	0.176	1.5%
MIME/header/body projection	0.677	0.459	4.2%
Thread-key resolution	0.008	0.008	0.0%
SQLite Index commits, including FTS and invalidations	15.042	7.628	93.5%
Other sync work	0.061	0.035	0.4%
Total	16.080	8.334	100%

Evidence: 125 UID FETCH windows of 80 UIDs each; 3.49 MB scripted FETCH replies; 114 ms total FETCH reply wait; 186,904 SQLite statements (18.69 per message); 129 writes transactions; 9,999 FTS rows; 58.4 MB WAL; 16 checkpoint runs during sync (6,523 frames checkpointed). The test confirms 9,999 deduplicated messages and 10,000 memberships. Post-sync first-page cache reads had p50 5.47 ms and p95 14.32 ms (20 samples); this is not API latency during sync or time-to-Inbox-usable.

A sampled DEBUG flamegraph from the same scripted workload also put the SQLx SQLite worker at 42.8% inclusive samples, with commit_window at 20.9%, HTML resource sanitization at 9.8%, and external-link processing at 6.6%. The DEBUG sample is qualitative; I will use the release phase timers for decisions.

The current evidence points to repeated Index write/FTS work and WAL checkpointing as the first optimization target. Provider FETCH waiting, MIME projection and thread-key resolution do not explain the 14 seconds. Provider TLS and real network time are absent from this in-memory harness. Before calling Phase 1 complete I will add a bounded concurrent list/API sampler to measure first usable Inbox data and list latency while backfill continues, then test Index changes against the state-equivalence and resume cases.

Phase 1 profile complete, before sync algorithm changes. I ran the release test binary with the existing synthetic 10k profile. It completed 10,000 UIDs in 16.080 s wall / 8.334 s process CPU; the host load average was 48.43 before and 49.61 after (1 min). This host contention makes the wall number diagnostic, not a clean comparison to the 14.35 s reference. | Phase | Wall s | CPU s | Wall share | | --- | ---: | ---: | ---: | | Session greeting/login | 0.000 | 0.000 | 0.0% | | Folder discovery/Index setup | 0.048 | 0.026 | 0.3% | | SELECT/EXAMINE commands | 0.000 | 0.000 | 0.0% | | FETCH protocol and reply waits | 0.241 | 0.176 | 1.5% | | MIME/header/body projection | 0.677 | 0.459 | 4.2% | | Thread-key resolution | 0.008 | 0.008 | 0.0% | | SQLite Index commits, including FTS and invalidations | 15.042 | 7.628 | 93.5% | | Other sync work | 0.061 | 0.035 | 0.4% | | **Total** | **16.080** | **8.334** | **100%** | Evidence: 125 UID FETCH windows of 80 UIDs each; 3.49 MB scripted FETCH replies; 114 ms total FETCH reply wait; 186,904 SQLite statements (18.69 per message); 129 writes transactions; 9,999 FTS rows; 58.4 MB WAL; 16 checkpoint runs during sync (6,523 frames checkpointed). The test confirms 9,999 deduplicated messages and 10,000 memberships. Post-sync first-page cache reads had p50 5.47 ms and p95 14.32 ms (20 samples); this is not API latency during sync or time-to-Inbox-usable. A sampled DEBUG flamegraph from the same scripted workload also put the SQLx SQLite worker at 42.8% inclusive samples, with `commit_window` at 20.9%, HTML resource sanitization at 9.8%, and external-link processing at 6.6%. The DEBUG sample is qualitative; I will use the release phase timers for decisions. The current evidence points to repeated Index write/FTS work and WAL checkpointing as the first optimization target. Provider FETCH waiting, MIME projection and thread-key resolution do not explain the 14 seconds. Provider TLS and real network time are absent from this in-memory harness. Before calling Phase 1 complete I will add a bounded concurrent list/API sampler to measure first usable Inbox data and list latency while backfill continues, then test Index changes against the state-equivalence and resume cases.

kayg commented 2026-10-07 08:36:27 +00:00

Author

Owner

Copy link

Profile addendum: the message-list API sampler is now authenticated with the same UUID User identity the route requires. Against the valid profile fixture, the real /folders/{id}/messages route returned the first nonempty page 70.5 ms after profile start; it had 216 successful samples, zero errors, p50 27.8 ms, p95 125.3 ms. The host load average was 53.78 before and 50.91 after.

The probe issued another request 50 ms after each completed response. That produced about 10.5 requests/s and increased measured time to 20.64 s, so the measurement itself perturbed the sync. I will cap the sampler to four requests/s (one request every 250 ms after each response) for the before/after runs. The first-page measurement remains immediate; its result above is useful for timing but its list-latency percentiles are not a low-overhead baseline.

Profile addendum: the message-list API sampler is now authenticated with the same UUID User identity the route requires. Against the valid profile fixture, the real `/folders/{id}/messages` route returned the first nonempty page 70.5 ms after profile start; it had 216 successful samples, zero errors, p50 27.8 ms, p95 125.3 ms. The host load average was 53.78 before and 50.91 after. The probe issued another request 50 ms after each completed response. That produced about 10.5 requests/s and increased measured time to 20.64 s, so the measurement itself perturbed the sync. I will cap the sampler to four requests/s (one request every 250 ms after each response) for the before/after runs. The first-page measurement remains immediate; its result above is useful for timing but its list-latency percentiles are not a low-overhead baseline.

kayg commented 2026-10-07 08:46:19 +00:00

Author

Owner

Copy link

One baseline state-dump attempt failed before producing a digest: profile_full_history returned Transport from sync_folder after 13.69 s. The same 10k profile had completed on the prior run, including 12.3 s with 41 authenticated message-list samples and no route errors. This failure occurred before any sync algorithm changes. I will retry once with the API sampler disabled for state-snapshot runs; that keeps the baseline/candidate state comparison from adding concurrent reads.

One baseline state-dump attempt failed before producing a digest: `profile_full_history` returned `Transport` from `sync_folder` after 13.69 s. The same 10k profile had completed on the prior run, including 12.3 s with 41 authenticated message-list samples and no route errors. This failure occurred before any sync algorithm changes. I will retry once with the API sampler disabled for state-snapshot runs; that keeps the baseline/candidate state comparison from adding concurrent reads.

kayg commented 2026-10-07 08:52:51 +00:00

Author

Owner

Copy link

The 10k pre-optimization normalized state dump completed. Its BLAKE3 is 922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56. The dump covers sorted message content and categories, canonicalized thread groups and keys, folder/generation state and counts, all membership UIDs/flags/labels/proxy metadata, attachments, and FTS row count. It omits only random row IDs and wall-clock sync timestamps. I have added this as the expected digest for the 10k comparison run.

This run took 11.245 s with host load 50.67 before / 50.18 after; Index commit spans were 10.339 s wall / 8.726 s process CPU. It completed 125 windows and reported 186,904 SQLite statements. TLS 2k state capture is next; algorithm changes have not started.

The 10k pre-optimization normalized state dump completed. Its BLAKE3 is `922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56`. The dump covers sorted message content and categories, canonicalized thread groups and keys, folder/generation state and counts, all membership UIDs/flags/labels/proxy metadata, attachments, and FTS row count. It omits only random row IDs and wall-clock sync timestamps. I have added this as the expected digest for the 10k comparison run. This run took 11.245 s with host load 50.67 before / 50.18 after; Index commit spans were 10.339 s wall / 8.726 s process CPU. It completed 125 windows and reported 186,904 SQLite statements. TLS 2k state capture is next; algorithm changes have not started.

kayg commented 2026-10-07 08:53:49 +00:00

Author

Owner

Copy link

The pre-optimization 2k local TLS Dovecot fixture completed with 2,000 messages across 159 sparse-UID windows, first 100 memberships in 258 ms, 10.314 s wall and 2.89 s worker CPU. Host load was 47.65 before and 48.14 after. The normalized state dump digest is c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8; I added it as the expected 2k TLS state comparison. The profile also confirmed the thread and attachment fixture data is present.

The pre-optimization 2k local TLS Dovecot fixture completed with 2,000 messages across 159 sparse-UID windows, first 100 memberships in 258 ms, 10.314 s wall and 2.89 s worker CPU. Host load was 47.65 before and 48.14 after. The normalized state dump digest is `c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8`; I added it as the expected 2k TLS state comparison. The profile also confirmed the thread and attachment fixture data is present.

kayg referenced this issue from a commit 2026-10-07 09:03:33 +00:00

Instrument full-history Mail sync for #1340

kayg commented 2026-10-07 09:07:26 +00:00

Author

Owner

Copy link

The first optimized 10k run retained the baseline state digest 922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56; the TLS 2k run retained c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8. This validates messages, canonical thread grouping/keys, folders/generations/counts, memberships/flags/labels/proxy metadata, attachments, and FTS row count against the pre-change snapshots.

One candidate run with the actual message-list route sampled every 250 ms completed 10,000 messages in 6.906 s; process CPU during the timed sync was 6.806 s. The first Inbox page was nonempty at 276.9 ms. The route p95 was 82.6 ms over 24 successful samples with zero errors. Worker peak RSS was 33.86 MB. Host load was 42.21 before and 41.50 after. SQLite statement events fell from 186,904 to 127,530 (-31.8%); FTS rows remained 9,999. This is one sample, not the required three-run median; interleaved measurements remain.

The first optimized 10k run retained the baseline state digest `922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56`; the TLS 2k run retained `c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8`. This validates messages, canonical thread grouping/keys, folders/generations/counts, memberships/flags/labels/proxy metadata, attachments, and FTS row count against the pre-change snapshots. One candidate run with the actual message-list route sampled every 250 ms completed 10,000 messages in 6.906 s; process CPU during the timed sync was 6.806 s. The first Inbox page was nonempty at 276.9 ms. The route p95 was 82.6 ms over 24 successful samples with zero errors. Worker peak RSS was 33.86 MB. Host load was 42.21 before and 41.50 after. SQLite statement events fell from 186,904 to 127,530 (-31.8%); FTS rows remained 9,999. This is one sample, not the required three-run median; interleaved measurements remain.

kayg commented 2026-10-07 09:44:03 +00:00

Author

Owner

Copy link

#1340 profile and candidate update (7 Oct)

Implemented SQLite window-write batching and added a disconnect/reopen resume regression. The 10k and 2k TLS state dumps still match their pre-change BLAKE3 digests (922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56 and c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8). The recovery test commits one window, drops FETCH replies at UIDs 257 and 173, closes and reopens the file-backed Index after each interruption, then checks 399 distinct messages/threads and 400 memberships.

Three interleaved 10k runs on this host (baseline/candidate wall ms): 8179/9878, 15011/7553, 9641/8002. Medians: 9641 ms baseline, 8002 ms candidate. Candidate median process CPU was 5880 ms; candidate peak RSS max was 34.1 MB. Candidate Inbox-usable median was 270 ms. The three candidate message-list p95 samples were 66.9, 64.4 and 93.1 ms, with no API errors. Candidate SQL statements fell from 186,904 to 127,530 per profile; FTS still contains all 9,999 rows. The 7-second target is not met in this host round. Candidate SQLite wall time remains the dominant phase (median 7.17 s); host load averages were 31–44 during measurements.

The perf VM was held by another calternal tabswitch-1233 run requesting 60 minutes when checked at 11:39 CEST. I have not measured there while its lock is held. Clippy initially identified two unnecessary casts in the test profiler; both are removed and the Mail crate Clippy gate now passes.

#1340 profile and candidate update (7 Oct) Implemented SQLite window-write batching and added a disconnect/reopen resume regression. The 10k and 2k TLS state dumps still match their pre-change BLAKE3 digests (`922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56` and `c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8`). The recovery test commits one window, drops FETCH replies at UIDs 257 and 173, closes and reopens the file-backed Index after each interruption, then checks 399 distinct messages/threads and 400 memberships. Three interleaved 10k runs on this host (baseline/candidate wall ms): 8179/9878, 15011/7553, 9641/8002. Medians: 9641 ms baseline, 8002 ms candidate. Candidate median process CPU was 5880 ms; candidate peak RSS max was 34.1 MB. Candidate Inbox-usable median was 270 ms. The three candidate message-list p95 samples were 66.9, 64.4 and 93.1 ms, with no API errors. Candidate SQL statements fell from 186,904 to 127,530 per profile; FTS still contains all 9,999 rows. The 7-second target is not met in this host round. Candidate SQLite wall time remains the dominant phase (median 7.17 s); host load averages were 31–44 during measurements. The perf VM was held by another `calternal tabswitch-1233` run requesting 60 minutes when checked at 11:39 CEST. I have not measured there while its lock is held. Clippy initially identified two unnecessary casts in the test profiler; both are removed and the Mail crate Clippy gate now passes.

kayg commented 2026-10-07 10:07:45 +00:00

Author

Owner

Copy link

The recovery regression now kills a child sync process while its scripted provider holds UID 257's FETCH literal open. After the child exits, the test reopens the file-backed Index, checks that the incomplete window added no messages or memberships, resumes through another provider disconnect at UID 173, and verifies 399 messages/threads plus 400 memberships. The focused test passes. The final Mail crate run reports 129 passed, 7 ignored; Clippy passes.

The recovery regression now kills a child sync process while its scripted provider holds UID 257's FETCH literal open. After the child exits, the test reopens the file-backed Index, checks that the incomplete window added no messages or memberships, resumes through another provider disconnect at UID 173, and verifies 399 messages/threads plus 400 memberships. The focused test passes. The final Mail crate run reports 129 passed, 7 ignored; Clippy passes.

kayg commented 2026-10-07 10:23:46 +00:00

Author

Owner

Copy link

#1340 final report

Branch: job/mail7-1340
Head: 14d5e8e606e7873e1bccfb09c395a2c93c27f10b
Base: 88ce7a765 (job/bash-perf)

Built

• Added content-free phase, SQLite statement, and request sampler instrumentation for the ignored 10k profile.
• Batched existing projection and pending-flag reads, membership upserts, preferred-ID writes, and safe new-message cleanup in each existing 80-UID SQLite transaction. SQL statements fell from 186,904 to 127,530 per 10k run (-31.8%); FTS still contains 9,999 rows.
• Added state-dump comparisons for the 10k and 2k TLS fixtures, plus a recovery test that kills a child sync process while UID 257's FETCH literal is incomplete, reopens the file-backed Index, survives another provider disconnect at UID 173, and resumes without lost/duplicate messages or threads.

Files: crates/plugins/mail/src/sync.rs, crates/plugins/mail/src/cache/store.rs, crates/plugins/mail/Cargo.toml, Cargo.lock.

Local performance

Three interleaved 10k synthetic history runs (baseline/candidate wall ms): 8179/9878, 15011/7553, 9641/8002. Median: 9641 ms baseline, 8002 ms candidate. Host load averages ranged from 31 to 44.

Candidate median process CPU: 5880 ms; maximum candidate peak RSS: 34,099,200 bytes; Inbox-usable median: 270.2 ms. Candidate message-list API p95 samples were 66.9, 64.4, 93.1 ms, with zero errors. One earlier unpaired candidate run measured 6906 ms; the required interleaved median did not meet the 7-second target. The SQLite phase remained dominant (candidate median wall 7173 ms).

Final 10k state digest: 922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56 (matches baseline). Final 2k TLS state digest: c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8 (matches baseline); TLS run 4491 ms, first 100 messages 157 ms.

Perf VM

The queued candidate ran under perf-run calternal mail7-1340 5 on the shared VM with load average 5.38 → 5.02. Wall time was 5627 ms, process CPU 5613 ms, peak RSS 30,859,264 bytes, Inbox usable 265.2 ms, and message-list API p95 24.40 ms across 22 samples, with zero API errors. SQLite phase was 4984 ms wall / 4951 ms CPU; all 9,999 FTS rows were present. This single confirmation run meets 7 seconds; the VM baseline was not measured.

Gates

• bun run build: Compressed 917 static variants; saved 21487171 bytes. (The first attempt before bun install --frozen-lockfile stopped because the worktree had no @excalidraw/excalidraw install.)
• bun run check: perf-lint: PASS; 0 violations; 22548 scoped exceptions; svelte-check found 0 errors and 1 warning in 1 file (existing empty CSS ruleset warning in AttachmentDeck.svelte).
• bun run test -- --maxWorkers=2: Test Files 287 passed (287); Tests 2035 passed (2035); Duration 355.22s.
• cargo fmt --check: exit 0, no output.
• cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings:

  Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 12s
  

• cargo test -p calternal-plugin-mail:

  test result: ok. 129 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 27.56s
  test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
  

• Focused crash-resume test:

  test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
  test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 135 filtered out; finished in 1.31s
  

• Final cleanup: Removed 0 files (the prior cargo clean had already removed the build artifacts).

Decisions and remaining gaps

• Kept one IMAP connection: FETCH protocol/wait work was about 0.2 seconds in the initial 16.08-second profile, while SQLite was 93.5% of wall time. RFC 2683 also warns against multiple concurrent connections to the same mailbox: https://www.rfc-editor.org/info/rfc2683/ .
• Kept FTS updates synchronous and the existing durability/checkpoint configuration. This preserves complete search and acknowledged-state durability; the design does not define deferred search visibility or weaker sync guarantees.
• The local interleaved median remains 1.00 seconds above target under load averages 31–44. The single perf VM candidate run reached 5.627 seconds under load 5.38; no three-run VM median or VM baseline was measured. No API route or UI changed; visual and UX checks do not apply.
• Adversarial matrices and full workspace gates remain for the merge round; this change did not alter an HTTP route.

#1340 final report Branch: `job/mail7-1340` Head: `14d5e8e606e7873e1bccfb09c395a2c93c27f10b` Base: `88ce7a765` (`job/bash-perf`) ## Built - Added content-free phase, SQLite statement, and request sampler instrumentation for the ignored 10k profile. - Batched existing projection and pending-flag reads, membership upserts, preferred-ID writes, and safe new-message cleanup in each existing 80-UID SQLite transaction. SQL statements fell from 186,904 to 127,530 per 10k run (-31.8%); FTS still contains 9,999 rows. - Added state-dump comparisons for the 10k and 2k TLS fixtures, plus a recovery test that kills a child sync process while UID 257's FETCH literal is incomplete, reopens the file-backed Index, survives another provider disconnect at UID 173, and resumes without lost/duplicate messages or threads. Files: `crates/plugins/mail/src/sync.rs`, `crates/plugins/mail/src/cache/store.rs`, `crates/plugins/mail/Cargo.toml`, `Cargo.lock`. ## Local performance Three interleaved 10k synthetic history runs (baseline/candidate wall ms): `8179/9878`, `15011/7553`, `9641/8002`. Median: `9641 ms` baseline, `8002 ms` candidate. Host load averages ranged from 31 to 44. Candidate median process CPU: `5880 ms`; maximum candidate peak RSS: `34,099,200 bytes`; Inbox-usable median: `270.2 ms`. Candidate message-list API p95 samples were `66.9`, `64.4`, `93.1 ms`, with zero errors. One earlier unpaired candidate run measured `6906 ms`; the required interleaved median did not meet the 7-second target. The SQLite phase remained dominant (candidate median wall `7173 ms`). Final 10k state digest: `922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56` (matches baseline). Final 2k TLS state digest: `c909f456c377c1c96ffdb1277298b4dfd308fa245b820cd132b1a56f806ee7f8` (matches baseline); TLS run `4491 ms`, first 100 messages `157 ms`. ## Perf VM The queued candidate ran under `perf-run calternal mail7-1340 5` on the shared VM with load average `5.38 → 5.02`. Wall time was `5627 ms`, process CPU `5613 ms`, peak RSS `30,859,264 bytes`, Inbox usable `265.2 ms`, and message-list API p95 `24.40 ms` across 22 samples, with zero API errors. SQLite phase was `4984 ms` wall / `4951 ms` CPU; all 9,999 FTS rows were present. This single confirmation run meets 7 seconds; the VM baseline was not measured. ## Gates - `bun run build`: `Compressed 917 static variants; saved 21487171 bytes.` (The first attempt before `bun install --frozen-lockfile` stopped because the worktree had no `@excalidraw/excalidraw` install.) - `bun run check`: `perf-lint: PASS; 0 violations; 22548 scoped exceptions`; `svelte-check found 0 errors and 1 warning in 1 file` (existing empty CSS ruleset warning in `AttachmentDeck.svelte`). - `bun run test -- --maxWorkers=2`: `Test Files 287 passed (287)`; `Tests 2035 passed (2035)`; `Duration 355.22s`. - `cargo fmt --check`: exit 0, no output. - `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 12s ``` - `cargo test -p calternal-plugin-mail`: ```text test result: ok. 129 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 27.56s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` - Focused crash-resume test: ```text test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 135 filtered out; finished in 1.31s ``` - Final cleanup: `Removed 0 files` (the prior `cargo clean` had already removed the build artifacts). ## Decisions and remaining gaps - Kept one IMAP connection: FETCH protocol/wait work was about 0.2 seconds in the initial 16.08-second profile, while SQLite was 93.5% of wall time. RFC 2683 also warns against multiple concurrent connections to the same mailbox: https://www.rfc-editor.org/info/rfc2683/ . - Kept FTS updates synchronous and the existing durability/checkpoint configuration. This preserves complete search and acknowledged-state durability; the design does not define deferred search visibility or weaker sync guarantees. - The local interleaved median remains 1.00 seconds above target under load averages 31–44. The single perf VM candidate run reached 5.627 seconds under load 5.38; no three-run VM median or VM baseline was measured. No API route or UI changed; visual and UX checks do not apply. - Adversarial matrices and full workspace gates remain for the merge round; this change did not alter an HTTP route.

kayg commented 2026-10-07 10:31:32 +00:00

Author

Owner

Copy link

Addendum: refreshed Rust gates after the test-only interruption-point update.

Head: 030f03f7670924f9e28c6b1fe5d4ccf67ee5b01c.

Gate output:

• cargo fmt --check produced no output and exited 0.
• cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 31s

• cargo test -p calternal-plugin-mail:

test result: ok. 129 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 33.23s

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The second result is the doc-test result. No production code changed after the previously reported performance measurements.

Addendum: refreshed Rust gates after the test-only interruption-point update. Head: `030f03f7670924f9e28c6b1fe5d4ccf67ee5b01c`. Gate output: - `cargo fmt --check` produced no output and exited 0. - `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 31s ``` - `cargo test -p calternal-plugin-mail`: ```text test result: ok. 129 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 33.23s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` The second result is the doc-test result. No production code changed after the previously reported performance measurements.

kayg commented 2026-10-07 10:34:45 +00:00

Author

Owner

Copy link

Continuing #1340 on job/mail7-1340 at 030f03f76 (base 88ce7a765). I’m preparing release test binaries for that baseline and this candidate, then I’ll run three interleaved 10k samples under perf-run with wall, CPU, peak RSS, Inbox-usable time and message API p95; state-dump and interruption/recovery checks remain in scope.

Continuing #1340 on job/mail7-1340 at 030f03f76 (base 88ce7a765). I’m preparing release test binaries for that baseline and this candidate, then I’ll run three interleaved 10k samples under perf-run with wall, CPU, peak RSS, Inbox-usable time and message API p95; state-dump and interruption/recovery checks remain in scope.

kayg commented 2026-10-07 11:30:03 +00:00

Author

Owner

Copy link

Perf VM round 1 completed all six interleaved 10k sync samples under perf-run calternal mail7-1340 5.

Variant	Median sync wall	Median process CPU	Median sampled peak RSS
Baseline 88ce7a765	9,247 ms	8.94 s	26,308,608 B
Candidate 030f03f76	6,688 ms	7.12 s	31,756,288 B

Candidate median peak RSS is +20.7% versus baseline. The candidate median is also at least the 6.5 s margin threshold. This triggers the issue's next step: move full-text FTS population to bounded background batches after message rows commit, keep durability settings unchanged, and report incomplete search results until indexing completes.

The runner then exited 1 in its separate state-dump phase because baseline 88ce7a765 does not emit state_dump_blake3; all six A/B samples had already completed. I confirmed the field is absent from that baseline source. I will run the candidate's state-dump assertion against the pre-optimization digest, which is the supported equivalence check.

Perf VM round 1 completed all six interleaved 10k sync samples under `perf-run calternal mail7-1340 5`. | Variant | Median sync wall | Median process CPU | Median sampled peak RSS | | --- | ---: | ---: | ---: | | Baseline `88ce7a765` | 9,247 ms | 8.94 s | 26,308,608 B | | Candidate `030f03f76` | 6,688 ms | 7.12 s | 31,756,288 B | Candidate median peak RSS is +20.7% versus baseline. The candidate median is also at least the 6.5 s margin threshold. This triggers the issue's next step: move full-text FTS population to bounded background batches after message rows commit, keep durability settings unchanged, and report incomplete search results until indexing completes. The runner then exited 1 in its separate state-dump phase because baseline `88ce7a765` does not emit `state_dump_blake3`; all six A/B samples had already completed. I confirmed the field is absent from that baseline source. I will run the candidate's state-dump assertion against the pre-optimization digest, which is the supported equivalence check.

kayg commented 2026-10-07 12:43:29 +00:00

Author

Owner

Copy link

Merge checkpoint: origin/dev at 0419170fe456b58614befcb12d9c9d255118f1c0 is merged as 9abca974a on job/mail7-1340. cargo fmt --check and Files clippy passed. The full Files test run reported two failures under parallel load: the password-rejection lock test exceeded its 2 s bound, and the download-limit test saw 429 instead of 200. Each exact test passed when rerun alone (2.81 s and 3.98 s), so this is a test-load/rate-limit interaction, not a changed expectation. The merge resolution keeps both reconcile-storm timing output and dev's one-shot worker cleanup helper/test.

Merge checkpoint: `origin/dev` at `0419170fe456b58614befcb12d9c9d255118f1c0` is merged as `9abca974a` on `job/mail7-1340`. `cargo fmt --check` and Files clippy passed. The full Files test run reported two failures under parallel load: the password-rejection lock test exceeded its 2 s bound, and the download-limit test saw 429 instead of 200. Each exact test passed when rerun alone (2.81 s and 3.98 s), so this is a test-load/rate-limit interaction, not a changed expectation. The merge resolution keeps both reconcile-storm timing output and dev's one-shot worker cleanup helper/test.

kayg commented 2026-10-07 13:40:26 +00:00

Author

Owner

Copy link

Finding from the first full Mail test run: existing message upserts fired mail_search_queue_update while the message ID was already queued. SQLite applied the outer upsert conflict policy to the trigger's INSERT OR IGNORE, causing UNIQUE constraint failed: mail_search_queue.message_id in eight projection/proxy tests. Four sync recovery/expunge tests then failed with Transport after aborted commits. I will change the trigger to use INSERT ... SELECT ... WHERE NOT EXISTS, then rerun the Mail gate and confirm the seeded interruption/recovery test.

Finding from the first full Mail test run: existing message upserts fired `mail_search_queue_update` while the message ID was already queued. SQLite applied the outer upsert conflict policy to the trigger's `INSERT OR IGNORE`, causing `UNIQUE constraint failed: mail_search_queue.message_id` in eight projection/proxy tests. Four sync recovery/expunge tests then failed with `Transport` after aborted commits. I will change the trigger to use `INSERT ... SELECT ... WHERE NOT EXISTS`, then rerun the Mail gate and confirm the seeded interruption/recovery test.

kayg commented 2026-10-07 13:45:36 +00:00

Author

Owner

Copy link

Follow-up finding: after fixing duplicate queue inserts, expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids still failed alone with a Mail cache Transport error while deleting messages that had not yet been indexed. The AFTER DELETE trigger could observe the queue after its foreign-key cascade. I moved it to BEFORE DELETE, where it checks whether FTS needs a delete and removes any queue row before the parent row is removed. The queue test now covers deletion of pending work, and the sparse-UID expunge test passes alone.

Follow-up finding: after fixing duplicate queue inserts, `expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids` still failed alone with a Mail cache `Transport` error while deleting messages that had not yet been indexed. The `AFTER DELETE` trigger could observe the queue after its foreign-key cascade. I moved it to `BEFORE DELETE`, where it checks whether FTS needs a delete and removes any queue row before the parent row is removed. The queue test now covers deletion of pending work, and the sparse-UID expunge test passes alone.

kayg commented 2026-10-07 14:08:50 +00:00

Author

Owner

Copy link

Server gate finding: the first cargo test -p calternal-server run reported that its deployed-schema tests expected Mail migration 20 as the latest receipt. The new deferred-search migration is 21, so I updated the round-9 and 7b/7c pending-receipt lists and counts to require 21. The same run's nested large-Home responsiveness case missed its 20 s Calendar write deadline while 285 server tests ran in parallel. I also changed the idle Mail indexer to check its durable queue through a reader before opening a writer transaction; an empty queue now takes no writer lock. I will rerun the suite with four test threads and report that result.

Server gate finding: the first `cargo test -p calternal-server` run reported that its deployed-schema tests expected Mail migration 20 as the latest receipt. The new deferred-search migration is 21, so I updated the round-9 and 7b/7c pending-receipt lists and counts to require 21. The same run's nested large-Home responsiveness case missed its 20 s Calendar write deadline while 285 server tests ran in parallel. I also changed the idle Mail indexer to check its durable queue through a reader before opening a writer transaction; an empty queue now takes no writer lock. I will rerun the suite with four test threads and report that result.

kayg commented 2026-10-07 14:13:24 +00:00

Author

Owner

Copy link

Finding #1340: the durable queue stores owner_id for content-free per-User progress. An ownership update while a message was queued left that value stale, so the new owner could receive no “results may be incomplete” state while their message awaited FTS. Added a trigger that updates queued ownership without reindexing text, plus a regression case that transfers a queued row and verifies the new owner sees only the generic indexing state.

Finding #1340: the durable queue stores owner_id for content-free per-User progress. An ownership update while a message was queued left that value stale, so the new owner could receive no “results may be incomplete” state while their message awaited FTS. Added a trigger that updates queued ownership without reindexing text, plus a regression case that transfers a queued row and verifies the new owner sees only the generic indexing state.

kayg commented 2026-10-07 14:30:01 +00:00

Author

Owner

Copy link

Server gate finding for #1340: the round-9 migration receipt assertion reported left: 157, right: 156 after adding Mail migration 21; I updated the expected count from +24 to +25 and named the new migration in the assertion. The 4-thread Server test run had 271 passed, 12 ignored, and 2 failed. The other failure was nested startup_reconciliation_indexes_photos_written_while_stopped: its 200-folder startup scan timed out at 119.18 s under host load. This is a SLOW-only startup test; no assertion or behavior was changed. I will run the final Server suite once after the fixes.

Server gate finding for #1340: the round-9 migration receipt assertion reported `left: 157`, `right: 156` after adding Mail migration 21; I updated the expected count from +24 to +25 and named the new migration in the assertion. The 4-thread Server test run had 271 passed, 12 ignored, and 2 failed. The other failure was nested `startup_reconciliation_indexes_photos_written_while_stopped`: its 200-folder startup scan timed out at 119.18 s under host load. This is a SLOW-only startup test; no assertion or behavior was changed. I will run the final Server suite once after the fixes.

kayg commented 2026-10-07 16:28:13 +00:00

Author

Owner

Copy link

Perf run finding for #1340: all three interleaved 10k runs per binary completed and are in /root/mail7-1340/results.jsonl, but the summary step failed with KeyError: 'end_to_end_wall_ms' because the 88ce7a765 baseline profile emits wall_ms only. I fixed the summary fallback. Inspection of 88ce7a765 also confirms its profile has no Inbox-usable or message-list API probe fields. I am adding only test-profile instrumentation in a temporary detached 88ce7a765 worktree so those baseline measurements use the same route probe without changing baseline product code.

Perf run finding for #1340: all three interleaved 10k runs per binary completed and are in `/root/mail7-1340/results.jsonl`, but the summary step failed with `KeyError: 'end_to_end_wall_ms'` because the 88ce7a765 baseline profile emits `wall_ms` only. I fixed the summary fallback. Inspection of 88ce7a765 also confirms its profile has no Inbox-usable or message-list API probe fields. I am adding only test-profile instrumentation in a temporary detached 88ce7a765 worktree so those baseline measurements use the same route probe without changing baseline product code.

kayg commented 2026-10-07 17:13:38 +00:00

Author

Owner

Copy link

#1340 final report

Branch: job/mail7-1340
Base: origin/dev merged at 9abca974a
Head: 950bc49272b72e453b48c3beda8b8f3c10902d8b
Commits: 46c9d6d5 defers Mail full-text indexing; 950bc492 documents the queue regression test. No push or deploy.

Built

Mail now commits an ID-only Search queue with each message transaction. A background worker drains at most 80 messages per SQLite transaction and yields between batches. The FTS write and queue acknowledgement commit together. A restart resumes from the durable queue. Search reads its results and queue status from one SQLite snapshot, so results that may be incomplete carry a content-free indexing state. Search status omits counts and percentages that could expose another User's mailbox activity. Existing Search UI copy already explains that some results may be missing while indexing runs; no web code changed. SQLite durability settings are unchanged.

The sync profile reports Inbox usability, message-list API latency, sync completion, and Search-index completion. The 10k profile has 10,000 memberships and 9,999 stored messages because two UIDs share one message identity.

Perf VM: interleaved 10k profile

All six timed runs used perf-run calternal mail7-1340 10 and the same profile. Load average was recorded inside the lock; 1-minute load rose from 0.00 to 0.71. The baseline is the job/bash-perf release at 88ce7a765; its temporary test-only profile instrumentation supplied the same Inbox/API probes without changing production code. Wall values are end-to-end milliseconds; CPU is process CPU seconds; RSS is peak bytes.

Build	Run	Wall ms	CPU s	Peak RSS bytes	Inbox usable ms	Message-list API p95 ms
Baseline	1	7196	7.13	30,781,440	262.569	25.036
Candidate	1	5426	5.45	33,222,656	259.957	17.646
Baseline	2	7330	7.34	31,199,232	263.632	21.211
Candidate	2	6110	6.41	34,160,640	267.748	20.021
Baseline	3	9093	9.20	31,129,600	264.154	20.921
Candidate	3	6095	6.40	33,464,320	269.871	21.752

Median	Wall ms	CPU s	Peak RSS bytes	Inbox usable ms	Message-list API p95 ms
Baseline	7330	7.34	31,129,600	263.632	21.211
Candidate	6095	6.40	33,464,320	267.748	20.021

Candidate wall median is below the 6.5 s margin. Peak RSS is +7.5% versus baseline, within the +20% limit. Candidate Search-index completion was 5426, 6133, and 6096 ms (median 6096 ms); one run needed 22 ms after sync for the queue to drain. The separate state-dump run completed indexing with 9,999 FTS rows and matched golden BLAKE3 digest 922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56.

Gates

cargo fmt --check exited 0 and produced no output.

cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings output:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 07s

cargo test -p calternal-plugin-mail output:

test result: FAILED. 126 passed; 3 failed; 7 ignored; 0 measured; 0 filtered out; finished in 108.99s

error: test failed, to rerun pass `-p calternal-plugin-mail --lib`

Failures were sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout (inactivity timer assertion), sync::tests::idle_waits_have_a_separate_pool_from_mail_sync (timeout), and sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes (cache write returned SQLite BUSY, “database is locked”). The delta test also failed once when run alone. The new deferred-queue test and backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads passed in the suite.

Other changed-crate gates: API clippy/test passed (23 passed, 1 ignored); Plugin clippy/test passed (62 passed); Server clippy passed. The Server suite had one SLOW startup reconciliation timeout while scanning 200 folders (106.72 s); 273 tests passed and 12 were ignored.

Files

• bench/mail-sync.py
• crates/calternal-api/src/lib.rs
• crates/calternal-plugin/src/lib.rs
• crates/calternal-server/src/main.rs
• crates/calternal-server/src/upgrade_tests.rs
• crates/plugins/mail/migrations/0021_deferred_search.sql
• crates/plugins/mail/src/cache/store.rs
• crates/plugins/mail/src/search.rs
• crates/plugins/mail/src/sync.rs

UX gaps

• Closed: Search reports incomplete results while a provider index is pending. State comes from the same snapshot as the results.
• Left: None in changed UI; no visual surface was modified.

Decisions not set by DESIGN.md

• Use a durable queue keyed by message ID and current owner ID; index in batches of 80 with a 15 ms pause between nonempty batches.
• Expose a boolean indexing state and omit progress counts/percentages because they can reveal cross-User activity.
• Keep the baseline instrumentation in a temporary release test build only; production baseline source remains 88ce7a765.

For the merge round

• cargo test -p calternal-plugin-mail -- --test-threads=4 — confirm the full Mail suite under the shared-host test-thread cap, including the transient SQLite lock and timeout cases above.
• cargo test -p calternal-server -- --test-threads=4 — confirm startup reconciliation and the migration/API status coverage.
• bash tests/adversarial/run.sh — attack the merged API on a real local server for authz isolation, malformed/oversized inputs, 5xx/crash behavior, and inconsistent Search completeness status.

### #1340 final report **Branch:** `job/mail7-1340` **Base:** `origin/dev` merged at `9abca974a` **Head:** `950bc49272b72e453b48c3beda8b8f3c10902d8b` **Commits:** `46c9d6d5` defers Mail full-text indexing; `950bc492` documents the queue regression test. No push or deploy. #### Built Mail now commits an ID-only Search queue with each message transaction. A background worker drains at most 80 messages per SQLite transaction and yields between batches. The FTS write and queue acknowledgement commit together. A restart resumes from the durable queue. Search reads its results and queue status from one SQLite snapshot, so results that may be incomplete carry a content-free `indexing` state. Search status omits counts and percentages that could expose another User's mailbox activity. Existing Search UI copy already explains that some results may be missing while indexing runs; no web code changed. SQLite durability settings are unchanged. The sync profile reports Inbox usability, message-list API latency, sync completion, and Search-index completion. The 10k profile has 10,000 memberships and 9,999 stored messages because two UIDs share one message identity. #### Perf VM: interleaved 10k profile All six timed runs used `perf-run calternal mail7-1340 10` and the same profile. Load average was recorded inside the lock; 1-minute load rose from 0.00 to 0.71. The baseline is the `job/bash-perf` release at `88ce7a765`; its temporary test-only profile instrumentation supplied the same Inbox/API probes without changing production code. Wall values are end-to-end milliseconds; CPU is process CPU seconds; RSS is peak bytes. | Build | Run | Wall ms | CPU s | Peak RSS bytes | Inbox usable ms | Message-list API p95 ms | |---|---:|---:|---:|---:|---:|---:| | Baseline | 1 | 7196 | 7.13 | 30,781,440 | 262.569 | 25.036 | | Candidate | 1 | 5426 | 5.45 | 33,222,656 | 259.957 | 17.646 | | Baseline | 2 | 7330 | 7.34 | 31,199,232 | 263.632 | 21.211 | | Candidate | 2 | 6110 | 6.41 | 34,160,640 | 267.748 | 20.021 | | Baseline | 3 | 9093 | 9.20 | 31,129,600 | 264.154 | 20.921 | | Candidate | 3 | 6095 | 6.40 | 33,464,320 | 269.871 | 21.752 | | Median | Wall ms | CPU s | Peak RSS bytes | Inbox usable ms | Message-list API p95 ms | |---|---:|---:|---:|---:|---:| | Baseline | 7330 | 7.34 | 31,129,600 | 263.632 | 21.211 | | Candidate | 6095 | 6.40 | 33,464,320 | 267.748 | 20.021 | Candidate wall median is below the 6.5 s margin. Peak RSS is **+7.5%** versus baseline, within the +20% limit. Candidate Search-index completion was 5426, 6133, and 6096 ms (median 6096 ms); one run needed 22 ms after sync for the queue to drain. The separate state-dump run completed indexing with 9,999 FTS rows and matched golden BLAKE3 digest `922fb40b919bcc8b9cbf24ef258460a6976507988569678abadbb625b8007d56`. #### Gates `cargo fmt --check` exited 0 and produced no output. `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings` output: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 07s ``` `cargo test -p calternal-plugin-mail` output: ```text test result: FAILED. 126 passed; 3 failed; 7 ignored; 0 measured; 0 filtered out; finished in 108.99s error: test failed, to rerun pass `-p calternal-plugin-mail --lib` ``` Failures were `sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout` (inactivity timer assertion), `sync::tests::idle_waits_have_a_separate_pool_from_mail_sync` (timeout), and `sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes` (cache write returned SQLite `BUSY`, “database is locked”). The delta test also failed once when run alone. The new deferred-queue test and `backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads` passed in the suite. Other changed-crate gates: API clippy/test passed (23 passed, 1 ignored); Plugin clippy/test passed (62 passed); Server clippy passed. The Server suite had one SLOW startup reconciliation timeout while scanning 200 folders (106.72 s); 273 tests passed and 12 were ignored. #### Files - `bench/mail-sync.py` - `crates/calternal-api/src/lib.rs` - `crates/calternal-plugin/src/lib.rs` - `crates/calternal-server/src/main.rs` - `crates/calternal-server/src/upgrade_tests.rs` - `crates/plugins/mail/migrations/0021_deferred_search.sql` - `crates/plugins/mail/src/cache/store.rs` - `crates/plugins/mail/src/search.rs` - `crates/plugins/mail/src/sync.rs` #### UX gaps - **Closed:** Search reports incomplete results while a provider index is pending. State comes from the same snapshot as the results. - **Left:** None in changed UI; no visual surface was modified. #### Decisions not set by DESIGN.md - Use a durable queue keyed by message ID and current owner ID; index in batches of 80 with a 15 ms pause between nonempty batches. - Expose a boolean indexing state and omit progress counts/percentages because they can reveal cross-User activity. - Keep the baseline instrumentation in a temporary release test build only; production baseline source remains `88ce7a765`. #### For the merge round - `cargo test -p calternal-plugin-mail -- --test-threads=4` — confirm the full Mail suite under the shared-host test-thread cap, including the transient SQLite lock and timeout cases above. - `cargo test -p calternal-server -- --test-threads=4` — confirm startup reconciliation and the migration/API status coverage. - `bash tests/adversarial/run.sh` — attack the merged API on a real local server for authz isolation, malformed/oversized inputs, 5xx/crash behavior, and inconsistent Search completeness status.

kayg commented 2026-10-07 20:57:20 +00:00

Author

Owner

Copy link

Starting #1340 work in job/mail7-1340 at 950bc4927 (base 0419170fe). I am tracing the deferred Mail Search writer against sync admission and will report evidence, gates, and final head SHA here.

Starting #1340 work in job/mail7-1340 at 950bc4927 (base 0419170fe). I am tracing the deferred Mail Search writer against sync admission and will report evidence, gates, and final head SHA here.

kayg commented 2026-10-07 21:14:12 +00:00

Author

Owner

Copy link

Isolated baseline reproduction at head 950bc4927:

Command: cargo test -p calternal-plugin-mail delta_catch_up_shares_the_bounded_window_budget_and_resumes -- --nocapture --test-threads=1

Result: exit 101. The first delta pass failed after 65.00 s with Transport; the captured classified log was WARN mail sync step failed step="cache" error_kind="storage". The underlying commit_window still opens a deferred transaction with begin(), reads the generation, then writes. That permits a concurrent SQLite writer to commit between the read and write, making the stale snapshot upgrade fail with BUSY. The previous gate report saw the same test fail with SQLite database is locked. Fetched origin/dev has the existing BEGIN IMMEDIATE admission fix and the focused regression test; I will verify after merging it.

Isolated baseline reproduction at head 950bc4927: Command: `cargo test -p calternal-plugin-mail delta_catch_up_shares_the_bounded_window_budget_and_resumes -- --nocapture --test-threads=1` Result: exit 101. The first delta pass failed after 65.00 s with `Transport`; the captured classified log was `WARN mail sync step failed step="cache" error_kind="storage"`. The underlying `commit_window` still opens a deferred transaction with `begin()`, reads the generation, then writes. That permits a concurrent SQLite writer to commit between the read and write, making the stale snapshot upgrade fail with BUSY. The previous gate report saw the same test fail with SQLite `database is locked`. Fetched `origin/dev` has the existing `BEGIN IMMEDIATE` admission fix and the focused regression test; I will verify after merging it.

kayg commented 2026-10-07 21:14:42 +00:00

Author

Owner

Copy link

The two reported IDLE timeout tests pass when run individually on the job branch at 950bc4927, with the same host and target:

• cargo test -p calternal-plugin-mail idle_waits_have_a_separate_pool_from_mail_sync -- --nocapture --test-threads=1: 1 passed, finished in 2.92 s.
• cargo test -p calternal-plugin-mail idle_keepalives_end_at_the_bounded_window_without_timeout -- --nocapture --test-threads=1: 1 passed, finished in 0.09 s.

No change is indicated by these isolated results. I will include them in the final Mail gate as well.

The two reported IDLE timeout tests pass when run individually on the job branch at 950bc4927, with the same host and target: - `cargo test -p calternal-plugin-mail idle_waits_have_a_separate_pool_from_mail_sync -- --nocapture --test-threads=1`: `1 passed`, finished in 2.92 s. - `cargo test -p calternal-plugin-mail idle_keepalives_end_at_the_bounded_window_without_timeout -- --nocapture --test-threads=1`: `1 passed`, finished in 0.09 s. No change is indicated by these isolated results. I will include them in the final Mail gate as well.

kayg commented 2026-10-07 21:50:59 +00:00

Author

Owner

Copy link

The requested Server suite reached its tests after I built the missing web assets. It reported 273 passed, 12 ignored, and one SLOW timing failure in wire::tests::startup_search_reconciliation_keeps_calendar_writes_responsive_on_large_home (run by live_apps_run_in_separate_processes). Evidence: search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12344 p95_ms=12800 p95_budget_ms=10000; the nested test panicked with Calendar write p95 12.800062521s exceeded 10s. No correctness assertion failed. This is an existing origin/dev large-Home timing case; I am reporting it as SLOW-only under the shared-host rule.

The requested Server suite reached its tests after I built the missing web assets. It reported 273 passed, 12 ignored, and one SLOW timing failure in `wire::tests::startup_search_reconciliation_keeps_calendar_writes_responsive_on_large_home` (run by `live_apps_run_in_separate_processes`). Evidence: `search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12344 p95_ms=12800 p95_budget_ms=10000`; the nested test panicked with `Calendar write p95 12.800062521s exceeded 10s`. No correctness assertion failed. This is an existing origin/dev large-Home timing case; I am reporting it as SLOW-only under the shared-host rule.

kayg commented 2026-10-07 22:01:35 +00:00

Author

Owner

Copy link

#1340 final report

Branch: job/mail7-1340
Base: 0419170fe456b58614befcb12d9c9d255118f1c0
Merged origin/dev once at ef0aa12b12e0047cf9a779827cc0ed32261b934e before final gates.
Head: 02a6d566505d3d6d56b0d63712a6d0e478500857
Commit: 02a6d566 — Merge origin/dev: reserve Mail writer before window reads. No push or deploy.

Built

• The delta catch-up failed on the job branch because commit_window began a deferred transaction, read the generation, then tried to write. A concurrent SQLite commit can make that WAL snapshot impossible to upgrade, returning BUSY; sync surfaced it as a cache Transport error. The branch still used begin() at that point.
• Merged the existing fix from origin/dev: commit_window now uses BEGIN IMMEDIATE to reserve the writer before its reads. This keeps SQLite's existing busy timeout and retry policy unchanged. The merge also adds window_waits_for_writer_before_reading_generation, which proves that a single window attempt waits before reading.
• The durable Mail Search queue still indexes bounded pages of 80 messages and releases the writer between batches. The existing 10k profile measured 125 windows, or one writer admission per window; this fix adds no per-message query.

Files relevant to this fix: crates/plugins/mail/src/cache/store.rs. Merge conflict resolutions kept the 1340 Mail profile schema and dev's Mail proxy schema v3 in bench/mail-sync.py, and kept both sides' test-cleanup documentation in crates/plugins/files/src/lib.rs. Cargo.lock includes the dev merge updates.

Focused checks

Before the merge, the exact delta test failed at 950bc4927 with Transport and classified error_kind="storage" after 65.00 s. After the merge it passed:

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 76.16s

The writer-admission regression passed:

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.81s

The two reported IDLE tests passed individually on the job branch: idle_waits_have_a_separate_pool_from_mail_sync finished in 2.92 s, and idle_keepalives_end_at_the_bounded_window_without_timeout finished in 0.09 s. Both also passed in each full Mail run.

Gates

cargo fmt --check: exit 0; no output.

cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 4m 57s

First cargo test -p calternal-plugin-mail -- --test-threads=4:

test result: ok. 131 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 77.87s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Second cargo test -p calternal-plugin-mail -- --test-threads=4:

test result: ok. 131 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 62.52s
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-server -- --test-threads=4: the first attempt could not compile because apps/web/build/ did not exist. bun run build succeeded; I removed its output after the Server rerun. The rerun had one SLOW timing failure in the existing large-Home test:

search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12344 p95_ms=12800 p95_budget_ms=10000
Calendar write p95 12.800062521s exceeded 10s
 test result: FAILED. 273 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 138.10s
error: test failed, to rerun pass `-p calternal-server --bin calternal-server`

No correctness assertion failed in that test. I report the timing result as SLOW under the shared-host rule. The web build emitted existing unused CSS selector warnings and completed with Compressed 914 static variants; saved 21490250 bytes.

10k sync timing

The local release-profile profile_full_history run completed with 10,000 memberships, 9,999 messages, and 9,999 FTS rows:

end_to_end_wall_ms=7222
end_to_end_process_cpu_ms=5952
search_indexing_completion_ms=7250
search_indexing_post_sync_wait_ms=27

The local dev-profile sample took 23,579 ms and is not comparable to the release result. The perf VM was held by calternal tabswitch4-paired (30-minute request), so I did not run there. The local release result and the prior perf-VM median of 6.095 s are not a like-for-like comparison; a fresh perf-VM comparison remains open.

Decisions and gaps

• Used the existing SQLite BEGIN IMMEDIATE admission path. I did not raise timeouts or add a separate writer lock.
• The design doc does not define a different writer-admission rule for deferred Search indexing; the fix uses the existing SQLite transaction invariant.
• No UI changed; UX and screenshot checks do not apply.
• Known gaps: the Server suite's one large-Home timing failure above, and no fresh perf-VM comparison while its lock was held.

Cleanup: cargo clean output was Removed 17783 files, 11.0GiB total. The web build output and temporary report files are removed. The worktree is clean.

#1340 final report Branch: `job/mail7-1340` Base: `0419170fe456b58614befcb12d9c9d255118f1c0` Merged `origin/dev` once at `ef0aa12b12e0047cf9a779827cc0ed32261b934e` before final gates. Head: `02a6d566505d3d6d56b0d63712a6d0e478500857` Commit: `02a6d566` — `Merge origin/dev: reserve Mail writer before window reads`. No push or deploy. ## Built - The delta catch-up failed on the job branch because `commit_window` began a deferred transaction, read the generation, then tried to write. A concurrent SQLite commit can make that WAL snapshot impossible to upgrade, returning BUSY; sync surfaced it as a cache `Transport` error. The branch still used `begin()` at that point. - Merged the existing fix from `origin/dev`: `commit_window` now uses `BEGIN IMMEDIATE` to reserve the writer before its reads. This keeps SQLite's existing busy timeout and retry policy unchanged. The merge also adds `window_waits_for_writer_before_reading_generation`, which proves that a single window attempt waits before reading. - The durable Mail Search queue still indexes bounded pages of 80 messages and releases the writer between batches. The existing 10k profile measured 125 windows, or one writer admission per window; this fix adds no per-message query. Files relevant to this fix: `crates/plugins/mail/src/cache/store.rs`. Merge conflict resolutions kept the 1340 Mail profile schema and dev's Mail proxy schema v3 in `bench/mail-sync.py`, and kept both sides' test-cleanup documentation in `crates/plugins/files/src/lib.rs`. `Cargo.lock` includes the dev merge updates. ## Focused checks Before the merge, the exact delta test failed at `950bc4927` with `Transport` and classified `error_kind="storage"` after 65.00 s. After the merge it passed: ```text test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 76.16s ``` The writer-admission regression passed: ```text test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.81s ``` The two reported IDLE tests passed individually on the job branch: `idle_waits_have_a_separate_pool_from_mail_sync` finished in 2.92 s, and `idle_keepalives_end_at_the_bounded_window_without_timeout` finished in 0.09 s. Both also passed in each full Mail run. ## Gates `cargo fmt --check`: exit 0; no output. `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 4m 57s ``` First `cargo test -p calternal-plugin-mail -- --test-threads=4`: ```text test result: ok. 131 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 77.87s running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Second `cargo test -p calternal-plugin-mail -- --test-threads=4`: ```text test result: ok. 131 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 62.52s running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo test -p calternal-server -- --test-threads=4`: the first attempt could not compile because `apps/web/build/` did not exist. `bun run build` succeeded; I removed its output after the Server rerun. The rerun had one SLOW timing failure in the existing large-Home test: ```text search_reconcile_calendar_write_profile files=20000 writes=32 p50_ms=12344 p95_ms=12800 p95_budget_ms=10000 Calendar write p95 12.800062521s exceeded 10s test result: FAILED. 273 passed; 1 failed; 12 ignored; 0 measured; 0 filtered out; finished in 138.10s error: test failed, to rerun pass `-p calternal-server --bin calternal-server` ``` No correctness assertion failed in that test. I report the timing result as SLOW under the shared-host rule. The web build emitted existing unused CSS selector warnings and completed with `Compressed 914 static variants; saved 21490250 bytes.` ## 10k sync timing The local release-profile `profile_full_history` run completed with 10,000 memberships, 9,999 messages, and 9,999 FTS rows: ```text end_to_end_wall_ms=7222 end_to_end_process_cpu_ms=5952 search_indexing_completion_ms=7250 search_indexing_post_sync_wait_ms=27 ``` The local dev-profile sample took 23,579 ms and is not comparable to the release result. The perf VM was held by `calternal tabswitch4-paired` (30-minute request), so I did not run there. The local release result and the prior perf-VM median of 6.095 s are not a like-for-like comparison; a fresh perf-VM comparison remains open. ## Decisions and gaps - Used the existing SQLite `BEGIN IMMEDIATE` admission path. I did not raise timeouts or add a separate writer lock. - The design doc does not define a different writer-admission rule for deferred Search indexing; the fix uses the existing SQLite transaction invariant. - No UI changed; UX and screenshot checks do not apply. - Known gaps: the Server suite's one large-Home timing failure above, and no fresh perf-VM comparison while its lock was held. Cleanup: `cargo clean` output was `Removed 17783 files, 11.0GiB total`. The web build output and temporary report files are removed. The worktree is clean.

kayg referenced this issue 2026-10-08 04:21:46 +00:00

Server test: Calendar write p95 over 10 s with 20k files #1441

kayg commented 2026-10-08 05:54:44 +00:00

Author

Owner

Copy link

Independent Mail reliability review started on job/mail7-1340, HEAD 02a6d56650, supplied base origin/dev ef0aa12b1. I will inspect code and run my own checks without reading prior reports. Review covers writer admission, sync and action races, crash recovery, User isolation, and writer latency.

Independent Mail reliability review started on job/mail7-1340, HEAD 02a6d566505d3d6d56b0d63712a6d0e478500857, supplied base origin/dev ef0aa12b1. I will inspect code and run my own checks without reading prior reports. Review covers writer admission, sync and action races, crash recovery, User isolation, and writer latency.

kayg commented 2026-10-08 06:03:15 +00:00

Author

Owner

Copy link

Independent Mail reliability review — #1340

Review starts at 02a6d566505d3d6d56b0d63712a6d0e478500857 on
job/mail7-1340. Supplied base: origin/dev ef0aa12b1.
Read CLAUDE.md, CONTEXT.md, DESIGN §§2,45,49,53,59, and the issue body.
No prior job reports or result files were read.

Writer admission inventory (code evidence; runnable probes pending)

• commit_window: BEGIN IMMEDIATE before owner and generation reads.
• Cache deferred writers that read first: ensure_generation (new epoch branch),
  expunge_uid_page, stage_expunge_page, validate_expunge_sweep,
  prune_expunge_page, cleanup_expunge_page.
• Cache deferred writers that write first: upsert_folders, restart_backfill,
  test-only set_message_flag, set_sender_category. Their first statement
  reserves SQLite's writer, so they do not upgrade a read snapshot.
• Cache read-only transaction: list_messages on the reader pool.
• Borrowed transactions: create_integrated_account, preferred-row helpers,
  activate_generation, apply_proxy_flags. Admission belongs to the caller.
• Proxy deferred read-first writers: mutate, queue_transfer, queue_draft,
  transfer finalization, non-expunge mutation receipt finalization.
• Proxy write-first transactions: writable SELECT snapshot, web UID mapping,
  exact body cache. Read-only SELECT uses the reader pool.
• Search index batches write first. The empty-queue check is a standalone read,
  outside the later write transaction.

Severity if reproduced: high reliability defect. An authority writer can make
these read-first transactions return SQLITE_BUSY without using busy_timeout;
SQLite can return SQLITE_BUSY_SNAPSHOT if another writer commits after the read.
Repro: new sync_stages_reserve_writer_before_reading and
user_intents_wait_for_writer_before_reading tests.

Independent review probes

Temporary source: artifacts/mailrev-probes.rs, injected as
search::mailrev_review for the review only. Passing probes are not committed.
It tests 25 stale sync replays concurrent with flags/mark read, MOVE and EXPUNGE;
public cache and journal cross-User isolation; 10k-row writer latency; and actual
SIGKILL at six SQLite window stages, followed by replay, UIDVALIDITY change,
expunge and real FTS integrity checks.

FTS verification drains the real background batch function and runs
integrity-check with rank=1, plus MATCH counts. An external-content FTS SELECT
count alone is insufficient: it reads content rows even before postings exist.

Confirmed defect: deferred writer upgrades (HIGH; fixed)

Original code fails both new regression tests. Six cache phases return
Database(SqliteError { code: 5, message: "database is locked" }) while the
FULL authority writer is held. User intents fail with Mail cache unavailable
(flags and draft) or Mailbox UID space exhausted (MOVE, a misleading mapping
of the lock failure). These tests hold the authority writer for only 100 ms.
They require admission to wait, then successful completion after its release.
No retry or provider I/O can conceal the failure.

Evidence: artifacts/admission-before.log, artifacts/user-admission-before.log.
The fix uses BEGIN IMMEDIATE in six cache stages, user mutation, transfer/draft
admission, and both journal receipt transactions. Existing write-first and
read-only transactions retain their current transaction mode. No DB settings,
provider behavior, or prior test expectations change.

# Independent Mail reliability review — #1340 Review starts at `02a6d566505d3d6d56b0d63712a6d0e478500857` on `job/mail7-1340`. Supplied base: `origin/dev ef0aa12b1`. Read CLAUDE.md, CONTEXT.md, DESIGN §§2,45,49,53,59, and the issue body. No prior job reports or result files were read. ## Writer admission inventory (code evidence; runnable probes pending) - `commit_window`: BEGIN IMMEDIATE before owner and generation reads. - Cache deferred writers that read first: `ensure_generation` (new epoch branch), `expunge_uid_page`, `stage_expunge_page`, `validate_expunge_sweep`, `prune_expunge_page`, `cleanup_expunge_page`. - Cache deferred writers that write first: `upsert_folders`, `restart_backfill`, test-only `set_message_flag`, `set_sender_category`. Their first statement reserves SQLite's writer, so they do not upgrade a read snapshot. - Cache read-only transaction: `list_messages` on the reader pool. - Borrowed transactions: `create_integrated_account`, preferred-row helpers, `activate_generation`, `apply_proxy_flags`. Admission belongs to the caller. - Proxy deferred read-first writers: `mutate`, `queue_transfer`, `queue_draft`, transfer finalization, non-expunge mutation receipt finalization. - Proxy write-first transactions: writable SELECT snapshot, web UID mapping, exact body cache. Read-only SELECT uses the reader pool. - Search index batches write first. The empty-queue check is a standalone read, outside the later write transaction. Severity if reproduced: high reliability defect. An authority writer can make these read-first transactions return SQLITE_BUSY without using busy_timeout; SQLite can return SQLITE_BUSY_SNAPSHOT if another writer commits after the read. Repro: new `sync_stages_reserve_writer_before_reading` and `user_intents_wait_for_writer_before_reading` tests. ## Independent review probes Temporary source: `artifacts/mailrev-probes.rs`, injected as `search::mailrev_review` for the review only. Passing probes are not committed. It tests 25 stale sync replays concurrent with flags/mark read, MOVE and EXPUNGE; public cache and journal cross-User isolation; 10k-row writer latency; and actual SIGKILL at six SQLite window stages, followed by replay, UIDVALIDITY change, expunge and real FTS integrity checks. FTS verification drains the real background batch function and runs `integrity-check` with rank=1, plus MATCH counts. An external-content FTS SELECT count alone is insufficient: it reads content rows even before postings exist. ## Confirmed defect: deferred writer upgrades (HIGH; fixed) Original code fails both new regression tests. Six cache phases return `Database(SqliteError { code: 5, message: "database is locked" })` while the FULL authority writer is held. User intents fail with `Mail cache unavailable` (flags and draft) or `Mailbox UID space exhausted` (MOVE, a misleading mapping of the lock failure). These tests hold the authority writer for only 100 ms. They require admission to wait, then successful completion after its release. No retry or provider I/O can conceal the failure. Evidence: `artifacts/admission-before.log`, `artifacts/user-admission-before.log`. The fix uses BEGIN IMMEDIATE in six cache stages, user mutation, transfer/draft admission, and both journal receipt transactions. Existing write-first and read-only transactions retain their current transaction mode. No DB settings, provider behavior, or prior test expectations change.

kayg commented 2026-10-08 06:10:08 +00:00

Author

Owner

Copy link

Independent Mail reliability review — #1340

Review starts at 02a6d566505d3d6d56b0d63712a6d0e478500857 on
job/mail7-1340. Supplied base: origin/dev ef0aa12b1.
Read CLAUDE.md, CONTEXT.md, DESIGN §§2,45,49,53,59, and the issue body.
No prior job reports or result files were read.

Writer admission inventory (code evidence; runnable probes pending)

• commit_window: BEGIN IMMEDIATE before owner and generation reads.
• Cache deferred writers that read first: ensure_generation (new epoch branch),
  expunge_uid_page, stage_expunge_page, validate_expunge_sweep,
  prune_expunge_page, cleanup_expunge_page.
• Cache deferred writers that write first: upsert_folders, restart_backfill,
  test-only set_message_flag, set_sender_category. Their first statement
  reserves SQLite's writer, so they do not upgrade a read snapshot.
• Cache read-only transaction: list_messages on the reader pool.
• Borrowed transactions: create_integrated_account, preferred-row helpers,
  activate_generation, apply_proxy_flags. Admission belongs to the caller.
• Proxy deferred read-first writers: mutate, queue_transfer, queue_draft,
  transfer finalization, non-expunge mutation receipt finalization.
• Proxy write-first transactions: writable SELECT snapshot, web UID mapping,
  exact body cache. Read-only SELECT uses the reader pool.
• Search index batches write first. The empty-queue check is a standalone read,
  outside the later write transaction.

Severity if reproduced: high reliability defect. An authority writer can make
these read-first transactions return SQLITE_BUSY without using busy_timeout;
SQLite can return SQLITE_BUSY_SNAPSHOT if another writer commits after the read.
Repro: new sync_stages_reserve_writer_before_reading and
user_intents_wait_for_writer_before_reading tests.

Independent review probes

Temporary source: artifacts/mailrev-probes.rs, injected as
search::mailrev_review for the review only. Passing probes are not committed.
It tests 25 stale sync replays concurrent with flags/mark read, MOVE and EXPUNGE;
public cache and journal cross-User isolation; 10k-row writer latency; and actual
SIGKILL at six SQLite window stages, followed by replay, UIDVALIDITY change,
expunge and real FTS integrity checks.

FTS verification drains the real background batch function and runs
integrity-check with rank=1, plus MATCH counts. An external-content FTS SELECT
count alone is insufficient: it reads content rows even before postings exist.

Confirmed defect: deferred writer upgrades (HIGH; fixed)

Original code fails both new regression tests. Six cache phases return
Database(SqliteError { code: 5, message: "database is locked" }) while the
FULL authority writer is held. User intents fail with Mail cache unavailable
(flags and draft) or Mailbox UID space exhausted (MOVE, a misleading mapping
of the lock failure). These tests hold the authority writer for only 100 ms.
They require admission to wait, then successful completion after its release.
No retry or provider I/O can conceal the failure.

Evidence: artifacts/admission-before.log, artifacts/user-admission-before.log.
The fix uses BEGIN IMMEDIATE in six cache stages, user mutation, transfer/draft
admission, and both journal receipt transactions. Existing write-first and
read-only transactions retain their current transaction mode. No DB settings,
provider behavior, or prior test expectations change.

Action races (PASS)

Independent probe runs 25 stale sync replays against the same UID, concurrent
with separate Seen and Flagged writes, MOVE, or Deleted plus EXPUNGE. It checks
accepted flags and stable UIDs, hidden source views, one pending MOVE destination,
no duplicate physical membership keys, an empty search queue after draining,
MATCH counts, and FTS rank=1 integrity. All three action cases pass.
Evidence: artifacts/review-probes.log; test
mailrev_actions_race_sync_without_lost_updates.

Crash recovery (PASS)

SIGKILL stops the real child window before message INSERT, batched membership
INSERT, cursor UPDATE, live-generation activation, COMMIT, and immediately after
COMMIT. Before COMMIT the next process sees one original membership; after
COMMIT it sees all three. Replaying the window, changing UIDVALIDITY 77 → 88,
and removing UID 10 by a validated expunge leaves exactly UID 11. Real FTS
postings and integrity pass in every case. No duplicate or missing UID survives.
Evidence: artifacts/review-probes.log; test
mailrev_kill_window_stages_then_epoch_and_expunge.

The first probe run had two harness faults: the reused fixture left both Users'
thread IDs empty, and the membership kill marker matched INSERT instead of the
actual WITH-prefixed batch statement. Both were corrected. The failed run is
kept at artifacts/review-probes-initial.log; it is not counted as a pass.

User isolation (PASS at the application boundary)

Two Users have the same remote UID 9 and the same account email. Public
account, folder, message, thread, attachment, unread and search reads refuse
foreign coordinates; malformed window owner/account/folder combinations fail;
foreign flag and transfer requests fail. Provider/sweep helpers accept an
internal FolderRecord or opaque folder ID; their callers first obtain that
identity through owner-scoped account/folder queries. They are not independent
actor authorization boundaries.
Evidence: mailrev_cache_queries_are_user_scoped and code audit of sync/IDLE
callers, plus the existing foreign-UID proxy tests.

Writer latency (PASS in the measured local scenarios; no hard bound)

Real window commits and journal writes share the production single writer.
125 windows × 80 new messages run with 100 Seen writes and the real FTS batch
function, paused 15 ms between batches as in production.

Local data	p50 write	p95 write	Maximum write	Sync duration
10k, 1040-byte bodies, concurrent FTS	39.099 ms	87.540 ms	197.413 ms	6418.575 ms
10k, 65533-byte bodies, concurrent FTS	151.858 ms	239.896 ms	598.182 ms	23514.341 ms

Every sample is retained. Both runs kept all 10,001 Alice memberships and passed
FTS integrity. This is a correctness/latency probe on the shared build host,
not the first-sync benchmark or a perf-VM baseline comparison.
Evidence: artifacts/latency-with-indexer.log,
artifacts/latency-large-bodies.log; test mailrev_long_sync_user_write_latency.

The default SQLite busy_timeout is 5000 ms. Fetch windows contain 80 messages;
expunge scan/cleanup pages contain at most 1024 UIDs; prune pages contain at most
500 (normal sync requests 80). Each job has 100 windows shared across backfill,
delta and expunge. No provider I/O holds the SQLite writer. Measured sync does
not starve writes for 1 s, but code does not enforce a universal 1 s deadline
under unrelated authority locks, slow disks or host scheduling delays.

Reproduction

python3 artifacts/run-mailrev-probes.py temporarily injects the independent
probes into search.rs, uses the existing target directory, runs the normal and
large-body cases, and restores the source in a finally block. Passing review
probes stay in artifacts. Only the two defect regressions and small fix are
intended for commit. No existing test expectation was changed.

# Independent Mail reliability review — #1340 Review starts at `02a6d566505d3d6d56b0d63712a6d0e478500857` on `job/mail7-1340`. Supplied base: `origin/dev ef0aa12b1`. Read CLAUDE.md, CONTEXT.md, DESIGN §§2,45,49,53,59, and the issue body. No prior job reports or result files were read. ## Writer admission inventory (code evidence; runnable probes pending) - `commit_window`: BEGIN IMMEDIATE before owner and generation reads. - Cache deferred writers that read first: `ensure_generation` (new epoch branch), `expunge_uid_page`, `stage_expunge_page`, `validate_expunge_sweep`, `prune_expunge_page`, `cleanup_expunge_page`. - Cache deferred writers that write first: `upsert_folders`, `restart_backfill`, test-only `set_message_flag`, `set_sender_category`. Their first statement reserves SQLite's writer, so they do not upgrade a read snapshot. - Cache read-only transaction: `list_messages` on the reader pool. - Borrowed transactions: `create_integrated_account`, preferred-row helpers, `activate_generation`, `apply_proxy_flags`. Admission belongs to the caller. - Proxy deferred read-first writers: `mutate`, `queue_transfer`, `queue_draft`, transfer finalization, non-expunge mutation receipt finalization. - Proxy write-first transactions: writable SELECT snapshot, web UID mapping, exact body cache. Read-only SELECT uses the reader pool. - Search index batches write first. The empty-queue check is a standalone read, outside the later write transaction. Severity if reproduced: high reliability defect. An authority writer can make these read-first transactions return SQLITE_BUSY without using busy_timeout; SQLite can return SQLITE_BUSY_SNAPSHOT if another writer commits after the read. Repro: new `sync_stages_reserve_writer_before_reading` and `user_intents_wait_for_writer_before_reading` tests. ## Independent review probes Temporary source: `artifacts/mailrev-probes.rs`, injected as `search::mailrev_review` for the review only. Passing probes are not committed. It tests 25 stale sync replays concurrent with flags/mark read, MOVE and EXPUNGE; public cache and journal cross-User isolation; 10k-row writer latency; and actual SIGKILL at six SQLite window stages, followed by replay, UIDVALIDITY change, expunge and real FTS integrity checks. FTS verification drains the real background batch function and runs `integrity-check` with rank=1, plus MATCH counts. An external-content FTS SELECT count alone is insufficient: it reads content rows even before postings exist. ## Confirmed defect: deferred writer upgrades (HIGH; fixed) Original code fails both new regression tests. Six cache phases return `Database(SqliteError { code: 5, message: "database is locked" })` while the FULL authority writer is held. User intents fail with `Mail cache unavailable` (flags and draft) or `Mailbox UID space exhausted` (MOVE, a misleading mapping of the lock failure). These tests hold the authority writer for only 100 ms. They require admission to wait, then successful completion after its release. No retry or provider I/O can conceal the failure. Evidence: `artifacts/admission-before.log`, `artifacts/user-admission-before.log`. The fix uses BEGIN IMMEDIATE in six cache stages, user mutation, transfer/draft admission, and both journal receipt transactions. Existing write-first and read-only transactions retain their current transaction mode. No DB settings, provider behavior, or prior test expectations change. ## Action races (PASS) Independent probe runs 25 stale sync replays against the same UID, concurrent with separate Seen and Flagged writes, MOVE, or Deleted plus EXPUNGE. It checks accepted flags and stable UIDs, hidden source views, one pending MOVE destination, no duplicate physical membership keys, an empty search queue after draining, MATCH counts, and FTS rank=1 integrity. All three action cases pass. Evidence: `artifacts/review-probes.log`; test `mailrev_actions_race_sync_without_lost_updates`. ## Crash recovery (PASS) SIGKILL stops the real child window before message INSERT, batched membership INSERT, cursor UPDATE, live-generation activation, COMMIT, and immediately after COMMIT. Before COMMIT the next process sees one original membership; after COMMIT it sees all three. Replaying the window, changing UIDVALIDITY 77 → 88, and removing UID 10 by a validated expunge leaves exactly UID 11. Real FTS postings and integrity pass in every case. No duplicate or missing UID survives. Evidence: `artifacts/review-probes.log`; test `mailrev_kill_window_stages_then_epoch_and_expunge`. The first probe run had two harness faults: the reused fixture left both Users' thread IDs empty, and the membership kill marker matched INSERT instead of the actual WITH-prefixed batch statement. Both were corrected. The failed run is kept at `artifacts/review-probes-initial.log`; it is not counted as a pass. ## User isolation (PASS at the application boundary) Two Users have the same remote UID 9 and the same account email. Public account, folder, message, thread, attachment, unread and search reads refuse foreign coordinates; malformed window owner/account/folder combinations fail; foreign flag and transfer requests fail. Provider/sweep helpers accept an internal FolderRecord or opaque folder ID; their callers first obtain that identity through owner-scoped account/folder queries. They are not independent actor authorization boundaries. Evidence: `mailrev_cache_queries_are_user_scoped` and code audit of sync/IDLE callers, plus the existing foreign-UID proxy tests. ## Writer latency (PASS in the measured local scenarios; no hard bound) Real window commits and journal writes share the production single writer. 125 windows × 80 new messages run with 100 Seen writes and the real FTS batch function, paused 15 ms between batches as in production. | Local data | p50 write | p95 write | Maximum write | Sync duration | | --- | --- | --- | --- | --- | | 10k, 1040-byte bodies, concurrent FTS | 39.099 ms | 87.540 ms | 197.413 ms | 6418.575 ms | | 10k, 65533-byte bodies, concurrent FTS | 151.858 ms | 239.896 ms | 598.182 ms | 23514.341 ms | Every sample is retained. Both runs kept all 10,001 Alice memberships and passed FTS integrity. This is a correctness/latency probe on the shared build host, not the first-sync benchmark or a perf-VM baseline comparison. Evidence: `artifacts/latency-with-indexer.log`, `artifacts/latency-large-bodies.log`; test `mailrev_long_sync_user_write_latency`. The default SQLite busy_timeout is 5000 ms. Fetch windows contain 80 messages; expunge scan/cleanup pages contain at most 1024 UIDs; prune pages contain at most 500 (normal sync requests 80). Each job has 100 windows shared across backfill, delta and expunge. No provider I/O holds the SQLite writer. Measured sync does not starve writes for 1 s, but code does not enforce a universal 1 s deadline under unrelated authority locks, slow disks or host scheduling delays. ## Reproduction `python3 artifacts/run-mailrev-probes.py` temporarily injects the independent probes into search.rs, uses the existing target directory, runs the normal and large-body cases, and restores the source in a finally block. Passing review probes stay in artifacts. Only the two defect regressions and small fix are intended for commit. No existing test expectation was changed.

kayg commented 2026-10-08 06:18:12 +00:00

Author

Owner

Copy link

Independent Mail reliability review — #1340

Review starts at 02a6d566505d3d6d56b0d63712a6d0e478500857 on
job/mail7-1340. Supplied base: origin/dev ef0aa12b1.
Read CLAUDE.md, CONTEXT.md, DESIGN §§2,45,49,53,59, and the issue body.
No prior job reports or result files were read.

Writer admission inventory (code evidence; runnable probes pending)

• commit_window: BEGIN IMMEDIATE before owner and generation reads.
• Cache deferred writers that read first: ensure_generation (new epoch branch),
  expunge_uid_page, stage_expunge_page, validate_expunge_sweep,
  prune_expunge_page, cleanup_expunge_page.
• Cache deferred writers that write first: upsert_folders, restart_backfill,
  test-only set_message_flag, set_sender_category. Their first statement
  reserves SQLite's writer, so they do not upgrade a read snapshot.
• Cache read-only transaction: list_messages on the reader pool.
• Borrowed transactions: create_integrated_account, preferred-row helpers,
  activate_generation, apply_proxy_flags. Admission belongs to the caller.
• Proxy deferred read-first writers: mutate, queue_transfer, queue_draft,
  transfer finalization, non-expunge mutation receipt finalization.
• Proxy write-first transactions: writable SELECT snapshot, web UID mapping,
  exact body cache. Read-only SELECT uses the reader pool.
• Search index batches write first. The empty-queue check is a standalone read,
  outside the later write transaction.

Severity if reproduced: high reliability defect. An authority writer can make
these read-first transactions return SQLITE_BUSY without using busy_timeout;
SQLite can return SQLITE_BUSY_SNAPSHOT if another writer commits after the read.
Repro: new sync_stages_reserve_writer_before_reading and
user_intents_wait_for_writer_before_reading tests.

Independent review probes

Temporary source: artifacts/mailrev-probes.rs, injected as
search::mailrev_review for the review only. Passing probes are not committed.
It tests 25 stale sync replays concurrent with flags/mark read, MOVE and EXPUNGE;
public cache and journal cross-User isolation; 10k-row writer latency; and actual
SIGKILL at six SQLite window stages, followed by replay, UIDVALIDITY change,
expunge and real FTS integrity checks.

FTS verification drains the real background batch function and runs
integrity-check with rank=1, plus MATCH counts. An external-content FTS SELECT
count alone is insufficient: it reads content rows even before postings exist.

Confirmed defect: deferred writer upgrades (HIGH; fixed)

Original code fails both new regression tests. Six cache phases return
Database(SqliteError { code: 5, message: "database is locked" }) while the
FULL authority writer is held. User intents fail with Mail cache unavailable
(flags and draft) or Mailbox UID space exhausted (MOVE, a misleading mapping
of the lock failure). These tests hold the authority writer for only 100 ms.
They require admission to wait, then successful completion after its release.
No retry or provider I/O can conceal the failure.

Evidence: artifacts/admission-before.log, artifacts/user-admission-before.log.
The fix uses BEGIN IMMEDIATE in six cache stages, user mutation, transfer/draft
admission, and both journal receipt transactions. Existing write-first and
read-only transactions retain their current transaction mode. No DB settings,
provider behavior, or prior test expectations change.

Action races (PASS)

Independent probe runs 25 stale sync replays against the same UID, concurrent
with separate Seen and Flagged writes, MOVE, or Deleted plus EXPUNGE. It checks
accepted flags and stable UIDs, hidden source views, one pending MOVE destination,
no duplicate physical membership keys, an empty search queue after draining,
MATCH counts, and FTS rank=1 integrity. All three action cases pass.
Evidence: artifacts/review-probes.log; test
mailrev_actions_race_sync_without_lost_updates.

Crash recovery (PASS)

SIGKILL stops the real child window before message INSERT, batched membership
INSERT, cursor UPDATE, live-generation activation, COMMIT, and immediately after
COMMIT. Before COMMIT the next process sees one original membership; after
COMMIT it sees all three. Replaying the window, changing UIDVALIDITY 77 → 88,
and removing UID 10 by a validated expunge leaves exactly UID 11. Real FTS
postings and integrity pass in every case. No duplicate or missing UID survives.
Evidence: artifacts/review-probes.log; test
mailrev_kill_window_stages_then_epoch_and_expunge.

The first probe run had two harness faults: the reused fixture left both Users'
thread IDs empty, and the membership kill marker matched INSERT instead of the
actual WITH-prefixed batch statement. Both were corrected. The failed run is
kept at artifacts/review-probes-initial.log; it is not counted as a pass.

User isolation (PASS at the application boundary)

Two Users have the same remote UID 9 and the same account email. Public
account, folder, message, thread, attachment, unread and search reads refuse
foreign coordinates; malformed window owner/account/folder combinations fail;
foreign flag and transfer requests fail. Provider/sweep helpers accept an
internal FolderRecord or opaque folder ID; their callers first obtain that
identity through owner-scoped account/folder queries. They are not independent
actor authorization boundaries.
Evidence: mailrev_cache_queries_are_user_scoped and code audit of sync/IDLE
callers, plus the existing foreign-UID proxy tests.

Writer latency (PASS in the measured local scenarios; no hard bound)

Real window commits and journal writes share the production single writer.
125 windows × 80 new messages run with 100 Seen writes and the real FTS batch
function, paused 15 ms between batches as in production.

Local data	p50 write	p95 write	Maximum write	Sync duration
10k, 1040-byte bodies, concurrent FTS	39.099 ms	87.540 ms	197.413 ms	6418.575 ms
10k, 65533-byte bodies, concurrent FTS	151.858 ms	239.896 ms	598.182 ms	23514.341 ms

Every sample is retained. Both runs kept all 10,001 Alice memberships and passed
FTS integrity. This is a correctness/latency probe on the shared build host,
not the first-sync benchmark or a perf-VM baseline comparison.
Evidence: artifacts/latency-with-indexer.log,
artifacts/latency-large-bodies.log; test mailrev_long_sync_user_write_latency.

The default SQLite busy_timeout is 5000 ms. Fetch windows contain 80 messages;
expunge scan/cleanup pages contain at most 1024 UIDs; prune pages contain at most
500 (normal sync requests 80). Each job has 100 windows shared across backfill,
delta and expunge. No provider I/O holds the SQLite writer. Measured sync does
not starve writes for 1 s, but code does not enforce a universal 1 s deadline
under unrelated authority locks, slow disks or host scheduling delays.

Reproduction

python3 artifacts/run-mailrev-probes.py temporarily injects the independent
probes into search.rs, uses the existing target directory, runs the normal and
large-body cases, and restores the source in a finally block. Passing review
probes stay in artifacts. Only the two defect regressions and small fix are
intended for commit. No existing test expectation was changed.

Confirmed defect: incomplete epoch replaces the live cache (HIGH; OPEN)

A real regression on committed head 8fbc1a3547864be033398e8d1a0c1cce6f6b90da
creates one verified live message in UIDVALIDITY 8, changes UIDVALIDITY to 9,
then commits only one of the two expected replacement messages. The generation
correctly has backfill_complete=false, but its live_generation becomes 2.
It must remain 1 until the replacement count passes. Activation deletes old
memberships and orphaned messages, so the old verified projection is lost.
This violates DESIGN §53's live-projection rule during an epoch rebuild.

Repro test: incomplete_replacement_epoch_keeps_live_messages_until_verified.
Source: artifacts/epoch-regression.rs. Output: artifacts/epoch-before.log.
Run: python3 artifacts/run-mailrev-probes.py --epoch-defect (expected failure;
the script restores the source even when the regression fails).

Exact failure:

assertion `left == right` failed: a failed count check must not activate the incomplete replacement
  left: 2
 right: 1

Cause: commit_window writes the count-checked completion flag, but calls
activate_generation based only on window.backfill_complete. The regression
also checks durable reopen and successful activation after the missing UID is
supplied. It fails before those later checks on the current implementation.

Do not apply only an activation guard: sync_folder's end == 0 resume branch
sets completion in memory without a window commit; its successful footer can
then report completion while the durable generation stays incomplete. Repair
must preserve the old live projection and arrange a fresh, durable count check
before activation on resume, including provider expunge/count drift. This is
filed for the sync owner instead of changing those semantics in a read-only
review. The failing regression is not committed, as the job requested.

Final state

Merged fetched origin/dev d395b6e25da20d8c67b8b9fc1a593b813939c8a7 once.
Merge commit: 18a80cc30d1bdeec60bdd5fc1f66707d322ef08d.
Fix commit/head: 8fbc1a3547864be033398e8d1a0c1cce6f6b90da.
Only the two proven writer-admission regressions and the small fix are committed.
The five changed files are cache/store.rs, proxy.rs, proxy_mutations.rs,
proxy_tests.rs, and proxy_transfers.rs under crates/plugins/mail/src/.
Comments were re-read. No dependency or migration was added. No push or deploy.

The Mail gates passed after the merge and before the atomic fix commit:
cargo fmt --check (no output),
cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings,
cargo test -p calternal-plugin-mail -- --test-threads=4.
Full verbatim output is in artifacts/admission-gate-{fmt,clippy,test}.log.
The later failing epoch probe was removed from the tracked source. The worktree
returns to the exact committed and gated source.

Known gap: the open epoch activation defect prevents a clean correctness sign-off.
The seven ignored upstream tests are manual profiles/real-provider checks and
the child worker that the normal recovery test spawns. The passing cache probes
are local synthetic tests; no external-provider interoperability claim is made.

Decisions: no new product decision. Retain deferred transactions when their
first statement writes, and reserve only read-first writers. Keep passing review
probes as artifacts; file the larger epoch/resume defect with its failing repro.
Expected cost of the fix: one writer reservation per existing transaction,
with no extra row query, batch growth or provider I/O under the writer.

Failing epoch regression (temporary review source)

    /// A short replacement epoch must keep the last verified live projection.
    /// Its cursor can commit for restart, but activation and old-row cleanup
    /// must wait for the same provider-count check (#1340, DESIGN §§45,53).
    #[tokio::test]
    async fn incomplete_replacement_epoch_keeps_live_messages_until_verified() {
        let (_directory, db) = test_db().await;
        insert_account(&db).await;
        let folder = upsert_folders(
            &db, "owner-1", "account-1",
            &[DiscoveredFolder {
                remote_name: "INBOX".into(), delimiter: Some("/".into()),
                special_use: "inbox".into(), selectable: true,
            }], 2,
        ).await.unwrap().remove(0);
        let first = ensure_generation(&db, &folder, 8, 2, 3).await.unwrap();
        store_window(&db, WindowCommit {
            owner_id: "owner-1", account_id: "account-1", folder_id: &folder.id,
            generation: first.generation, uid_validity: 8,
            messages: &[test_message("old-live", "old-provider", 1)],
            backfill_low_water: Some(0), backfill_complete: Some(true),
            expected_message_count: Some(1), delta_cursor: Some(1), now_ms: 4,
        }).await.unwrap();
        let folder = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap();
        let replacement = ensure_generation(&db, &folder, 9, 3, 5).await.unwrap();
        let partial = test_message("new-one", "new-provider-one", 1);
        store_window(&db, WindowCommit {
            owner_id: "owner-1", account_id: "account-1", folder_id: &folder.id,
            generation: replacement.generation, uid_validity: 9,
            messages: std::slice::from_ref(&partial), backfill_low_water: Some(0),
            backfill_complete: Some(true), expected_message_count: Some(2),
            delta_cursor: Some(2), now_ms: 6,
        }).await.unwrap();
        let pending = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap();
        assert!(!pending.backfill_complete);
        assert_eq!(pending.live_generation, first.generation,
            "a failed count check must not activate the incomplete replacement");
        let page = list_messages(&db, "owner-1", &folder.id, None, None, 10).await.unwrap();
        assert_eq!(page.messages.len(), 1);
        assert_eq!(page.messages[0].id, "old-live");
        assert!(get_message(&db, "owner-1", "old-live").await.unwrap().is_some());

        // Reopen the actual file to check the durable cursor/live split, then
        // repeat the partial UID and supply the missing provider identity.
        let path = _directory.path().join("index.sqlite");
        db.close().await;
        let db = Db::connect(path, DbOptions::default()).await.unwrap();
        let resumed = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap();
        assert_eq!(resumed.live_generation, first.generation);
        store_window(&db, WindowCommit {
            owner_id: "owner-1", account_id: "account-1", folder_id: &folder.id,
            generation: replacement.generation, uid_validity: 9,
            messages: &[partial, test_message("new-two", "new-provider-two", 2)],
            backfill_low_water: Some(0), backfill_complete: Some(true),
            expected_message_count: Some(2), delta_cursor: Some(2), now_ms: 7,
        }).await.unwrap();
        let completed = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap();
        assert!(completed.backfill_complete);
        assert_eq!(completed.live_generation, replacement.generation);
        let uids: Vec<i64> = sqlx::query_scalar("SELECT uid FROM mail_memberships WHERE folder_id=? ORDER BY uid")
            .bind(&folder.id).fetch_all(db.reader_pool()).await.unwrap();
        assert_eq!(uids, vec![1, 2]);
        assert!(get_message(&db, "owner-1", "old-live").await.unwrap().is_none());
        let count: i64 = sqlx::query_scalar("SELECT count(*) FROM mail_messages WHERE owner_id='owner-1'")
            .fetch_one(db.reader_pool()).await.unwrap();
        assert_eq!(count, 2);
        db.close().await;
    }

Gate output (verbatim)

cargo fmt --check: no output.

cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings

   Compiling libc v0.2.189
    Checking cfg-if v1.0.5
   Compiling siphasher v1.0.3
    Checking log v0.4.34
    Checking smallvec v1.16.1
    Checking serde_core v1.0.229
    Checking itoa v1.0.18
   Compiling jobserver v0.1.35
    Checking pin-project-lite v0.2.17
    Checking once_cell v1.21.4
    Checking memchr v2.8.3
   Compiling cc v1.4.7
    Checking futures-core v0.3.34
    Checking serde v1.0.229
    Checking scopeguard v1.2.0
   Compiling zerocopy v0.8.58
   Compiling getrandom v0.2.17
    Checking lock_api v0.4.14
   Compiling rand_core v0.6.4
    Checking stable_deref_trait v1.2.1
    Checking bytes v1.12.1
    Checking parking_lot_core v0.9.12
    Checking mio v1.2.3
    Checking parking_lot v0.12.5
    Checking errno v0.3.14
    Checking zerofrom v0.1.8
    Checking signal-hook-registry v1.4.8
    Checking socket2 v0.6.5
    Checking yoke v0.8.3
   Compiling phf_shared v0.11.3
    Checking futures-sink v0.3.34
    Checking equivalent v1.0.2
    Checking new_debug_unreachable v1.0.6
    Checking typenum v1.20.1
    Checking tokio v1.53.1
   Compiling ppv-lite86 v0.2.21
    Checking tracing-core v0.1.36
   Compiling rand_chacha v0.3.1
    Checking precomputed-hash v0.1.1
    Checking zerovec v0.11.8
    Checking bitflags v2.13.2
   Compiling rand v0.8.8
    Checking hashbrown v0.17.1
   Compiling phf_generator v0.11.3
    Checking indexmap v2.14.2
    Checking slab v0.4.12
    Checking num-traits v0.2.19
    Checking tracing v0.1.44
    Checking futures-channel v0.3.34
   Compiling phf_shared v0.13.1
    Checking percent-encoding v2.3.2
    Checking tinystr v0.8.4
    Checking futures-io v0.3.34
    Checking base64 v0.22.1
   Compiling phf_generator v0.13.1
    Checking futures-task v0.3.34
    Checking writeable v0.6.4
    Checking litemap v0.8.3
    Checking futures-util v0.3.34
    Checking potential_utf v0.1.6
    Checking icu_locale_core v2.3.0
    Checking zerotrie v0.2.5
    Checking utf8_iter v1.0.4
    Checking icu_collections v2.3.0
    Checking icu_provider v2.3.1
   Compiling cmake v0.1.58
   Compiling aws-lc-sys v0.45.0
    Checking icu_properties_data v2.3.0
    Checking icu_normalizer_data v2.3.0
    Checking rand_core v0.10.1
   Compiling siphasher v0.3.11
    Checking icu_properties v2.3.0
    Checking getrandom v0.4.3
   Compiling phf_shared v0.10.0
    Checking icu_normalizer v2.3.0
   Compiling generic-array v0.14.9
    Checking form_urlencoded v1.2.2
    Checking zmij v1.0.23
    Checking crossbeam-utils v0.8.23
    Checking hybrid-array v0.4.15
    Checking http v1.5.0
    Checking dtoa v1.0.11
    Checking zeroize v1.9.0
    Checking iana-time-zone v0.1.65
    Checking dtoa-short v0.3.5
    Checking chrono v0.4.45
    Checking serde_json v1.0.151
    Checking idna_adapter v1.2.2
   Compiling phf_generator v0.10.0
   Compiling phf_macros v0.13.1
   Compiling phf_macros v0.11.3
    Checking phf v0.13.1
    Checking phf v0.11.3
   Compiling phf_codegen v0.10.0
    Checking idna v1.1.0
    Checking thiserror v2.0.21
    Checking rustls-pki-types v1.15.1
    Checking http-body v1.1.0
   Compiling block-buffer v0.10.4
   Compiling crypto-common v0.1.6
   Compiling phf_shared v0.14.0
   Compiling allocator-api2 v0.2.21
   Compiling foldhash v0.2.0
    Checking parking v2.2.1
    Checking cpufeatures v0.3.1
    Checking event-listener v5.4.2
   Compiling hashbrown v0.16.1
   Compiling phf_generator v0.14.0
   Compiling digest v0.10.7
    Checking url v2.5.8
    Checking crypto-common v0.2.2
   Compiling libsqlite3-sys v0.37.0
   Compiling crc-catalog v2.5.0
   Compiling cpufeatures v0.2.17
    Checking simd-adler32 v0.3.10
    Checking tower-service v0.3.3
   Compiling crossbeam-queue v0.3.14
   Compiling sha2 v0.10.9
   Compiling crc v3.4.0
   Compiling tokio-stream v0.1.19
   Compiling either v1.18.0
   Compiling futures-intrusive v0.5.0
   Compiling hashlink v0.11.1
   Compiling spin v0.9.9
   Compiling phf_codegen v0.13.1
   Compiling string_cache_codegen v0.6.1
    Checking tokio-util v0.7.19
    Checking tendril v0.5.1
   Compiling blake3 v1.8.7
    Checking fnv v1.0.7
    Checking try-lock v0.2.5
    Checking atomic-waker v1.1.2
    Checking untrusted v0.9.0
    Checking adler2 v2.0.1
    Checking h2 v0.4.19
    Checking want v0.3.1
   Compiling sqlx-core v0.9.0
   Compiling web_atoms v0.2.6
   Compiling flume v0.12.0
   Compiling atoi v2.0.0
    Checking httparse v1.10.1
   Compiling futures-executor v0.3.34
   Compiling phf_codegen v0.14.0
   Compiling string_cache_codegen v0.11.2
    Checking inout v0.2.2
   Compiling string_cache_codegen v0.5.4
    Checking sync_wrapper v1.0.2
    Checking phf_shared v0.12.1
    Checking tower-layer v0.3.3
    Checking cmov v0.5.4
    Checking constant_time_eq v0.4.2
    Checking minimal-lexical v0.2.1
    Checking mac v0.1.1
    Checking ipnet v2.12.2
    Checking subtle v2.6.1
    Checking arrayvec v0.7.8
    Checking httpdate v1.0.3
    Checking ryu v1.0.23
    Checking hyper v1.11.1
    Checking futf v0.1.5
    Checking nom v7.1.3
   Compiling aws-lc-rs v1.18.1
   Compiling rustls v0.23.45
    Checking ctutils v0.4.2
    Checking phf v0.12.1
   Compiling markup5ever v0.11.0
   Compiling sqlx-sqlite v0.9.0
   Compiling web_atoms v0.3.0
    Checking crc32fast v1.5.2
    Checking rustls-webpki v0.103.15
    Checking phf v0.10.1
    Checking miniz_oxide v0.9.1
    Checking string_cache v0.9.0
    Checking block-buffer v0.12.1
    Checking linux-raw-sys v0.12.1
    Checking utf-8 v0.7.6
    Checking tinyvec v1.13.3
   Compiling sqlx-macros-core v0.9.0
    Checking rustix v1.1.5
    Checking unicode-normalization v0.1.25
    Checking tendril v0.4.3
    Checking cipher v0.5.2
    Checking flate2 v1.1.10
    Checking chrono-tz v0.10.4
    Checking universal-hash v0.6.1
    Checking hyper-util v0.1.20
    Checking getrandom v0.3.4
    Checking phf v0.14.0
    Checking string_cache v0.11.0
    Checking tower v0.5.3
    Checking miniz_oxide v0.8.9
    Checking fdeflate v0.3.7
    Checking http-body-util v0.1.5
   Compiling selectors v0.25.0
    Checking concurrent-queue v2.5.0
    Checking string_cache v0.8.9
   Compiling phf_codegen v0.11.3
    Checking zune-core v0.5.3
    Checking byteorder-lite v0.1.0
    Checking color_quant v1.1.0
    Checking weezl v0.1.12
    Checking byteorder v1.5.0
    Checking quick-error v2.0.1
    Checking pxfm v0.1.30
    Checking openssl-probe v0.2.1
    Checking calternal-path v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-path)
    Checking image-webp v0.2.4
    Checking rustls-native-certs v0.8.4
    Checking fxhash v0.2.1
    Checking gif v0.14.2
    Checking zune-jpeg v0.5.15
   Compiling parcel_selectors v0.28.3
    Checking cssparser v0.37.0
    Checking moxcms v0.8.1
    Checking cssparser v0.31.2
    Checking ahash v0.8.12
    Checking png v0.18.1
    Checking multiversion v0.9.0
    Checking poly1305 v0.9.1
    Checking markup5ever v0.39.0
    Checking chacha20 v0.10.2
   Compiling sqlx-macros v0.9.0
    Checking inotify-sys v0.1.8
    Checking unicode-joining-type v1.0.0
    Checking tokio-rustls v0.26.6
    Checking aead v0.6.1
    Checking servo_arc v0.3.0
    Checking bytemuck v1.25.2
    Checking mime v0.3.17
    Checking winnow v0.7.15
    Checking core_detect v1.0.0
    Checking unicode-casefold v0.2.0
    Checking unicase v2.9.0
    Checking same-file v1.0.6
    Checking unicode-width v0.2.2
    Checking lazy_static v1.5.0
    Checking simdutf8 v0.1.5
    Checking bounded-static v0.8.0
    Checking unicode-properties v0.1.4
    Checking event-listener v2.5.3
    Checking pulldown-cmark-escape v0.11.0
    Checking unsafe-libyaml v0.2.11
    Checking async-channel v1.9.0
    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-fs)
    Checking pulldown-cmark v0.13.4
    Checking cron v0.17.0
    Checking serde_yaml v0.9.34+deprecated
    Checking encoding_rs v0.8.41
    Checking imap-types v2.0.0-alpha.7
    Checking getopts v0.2.24
    Checking walkdir v2.5.0
    Checking axum-core v0.5.6
    Checking image v0.25.10
    Checking chacha20poly1305 v0.11.0
    Checking hyper-rustls v0.27.10
    Checking utoipa v6.0.0
    Checking sqlx v0.9.0
    Checking inotify v0.11.5
    Checking markup5ever v0.40.0
    Checking html5ever v0.26.0
    Checking rustls-platform-verifier v0.7.0
    Checking tower-http v0.6.11
    Checking abnf-core v0.6.0
    Checking serde_urlencoded v0.7.1
    Checking event-listener-strategy v0.5.4
   Compiling hashify v0.2.9
    Checking uuid v1.26.1
    Checking notify-types v2.1.0
    Checking quick-xml v0.42.0
    Checking serde_path_to_error v0.1.20
    Checking matchit v0.8.4
    Checking base64 v0.23.1
    Checking ego-tree v0.6.3
    Checking rustc-hash v2.1.3
    Checking calternal-notes-core v0.1.0 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-notes-core)
    Checking reqwest v0.13.5
    Checking scraper v0.18.1
    Checking axum v0.8.9
    Checking notify v8.2.0
    Checking thiserror v1.0.69
    Checking pin-project v1.1.13
    Checking calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-db)
    Checking calternal-api v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-api)
    Checking mail-parser v0.11.9
    Checking async-channel v2.5.0
    Checking imap-codec v2.0.0-alpha.9
    Checking linkme v0.3.37
    Checking html5ever v0.40.1
    Checking stop-token v0.7.0
    Checking xml5ever v0.39.0
    Checking html5ever v0.39.0
    Checking cssparser-color v0.5.0
    Checking imap-proto v0.16.7
    Checking futures v0.3.34
    Checking itertools v0.10.5
    Checking cssparser v0.38.0
    Checking const-str v1.1.0
    Checking similar v3.2.0
    Checking self_cell v1.3.0
    Checking pathdiff v0.2.3
    Checking data-encoding v2.11.1
    Checking maplit v1.0.2
    Checking pin-utils v0.1.0
    Checking ammonia v4.2.0
    Checking calternal-imap v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-imap)
    Checking lightningcss v1.0.0-alpha.72
    Checking async-imap v0.11.3 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail/vendor/async-imap)
    Checking markup5ever_rcdom v0.39.0+unofficial
    Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-plugin)
    Checking sharded-slab v0.1.7
    Checking webpki-roots v1.0.9
    Checking tracing-log v0.2.0
    Checking thread_local v1.1.10
    Checking fastrand v2.5.0
    Checking nu-ansi-term v0.50.3
    Checking tempfile v3.27.0
    Checking tracing-subscriber v0.3.23
    Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 46s

cargo test -p calternal-plugin-mail -- --test-threads=4

   Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 02s
     Running unittests src/lib.rs (/home/kayg/build/targets/mailrev/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 140 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 133 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 38.80s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

# Independent Mail reliability review — #1340 Review starts at `02a6d566505d3d6d56b0d63712a6d0e478500857` on `job/mail7-1340`. Supplied base: `origin/dev ef0aa12b1`. Read CLAUDE.md, CONTEXT.md, DESIGN §§2,45,49,53,59, and the issue body. No prior job reports or result files were read. ## Writer admission inventory (code evidence; runnable probes pending) - `commit_window`: BEGIN IMMEDIATE before owner and generation reads. - Cache deferred writers that read first: `ensure_generation` (new epoch branch), `expunge_uid_page`, `stage_expunge_page`, `validate_expunge_sweep`, `prune_expunge_page`, `cleanup_expunge_page`. - Cache deferred writers that write first: `upsert_folders`, `restart_backfill`, test-only `set_message_flag`, `set_sender_category`. Their first statement reserves SQLite's writer, so they do not upgrade a read snapshot. - Cache read-only transaction: `list_messages` on the reader pool. - Borrowed transactions: `create_integrated_account`, preferred-row helpers, `activate_generation`, `apply_proxy_flags`. Admission belongs to the caller. - Proxy deferred read-first writers: `mutate`, `queue_transfer`, `queue_draft`, transfer finalization, non-expunge mutation receipt finalization. - Proxy write-first transactions: writable SELECT snapshot, web UID mapping, exact body cache. Read-only SELECT uses the reader pool. - Search index batches write first. The empty-queue check is a standalone read, outside the later write transaction. Severity if reproduced: high reliability defect. An authority writer can make these read-first transactions return SQLITE_BUSY without using busy_timeout; SQLite can return SQLITE_BUSY_SNAPSHOT if another writer commits after the read. Repro: new `sync_stages_reserve_writer_before_reading` and `user_intents_wait_for_writer_before_reading` tests. ## Independent review probes Temporary source: `artifacts/mailrev-probes.rs`, injected as `search::mailrev_review` for the review only. Passing probes are not committed. It tests 25 stale sync replays concurrent with flags/mark read, MOVE and EXPUNGE; public cache and journal cross-User isolation; 10k-row writer latency; and actual SIGKILL at six SQLite window stages, followed by replay, UIDVALIDITY change, expunge and real FTS integrity checks. FTS verification drains the real background batch function and runs `integrity-check` with rank=1, plus MATCH counts. An external-content FTS SELECT count alone is insufficient: it reads content rows even before postings exist. ## Confirmed defect: deferred writer upgrades (HIGH; fixed) Original code fails both new regression tests. Six cache phases return `Database(SqliteError { code: 5, message: "database is locked" })` while the FULL authority writer is held. User intents fail with `Mail cache unavailable` (flags and draft) or `Mailbox UID space exhausted` (MOVE, a misleading mapping of the lock failure). These tests hold the authority writer for only 100 ms. They require admission to wait, then successful completion after its release. No retry or provider I/O can conceal the failure. Evidence: `artifacts/admission-before.log`, `artifacts/user-admission-before.log`. The fix uses BEGIN IMMEDIATE in six cache stages, user mutation, transfer/draft admission, and both journal receipt transactions. Existing write-first and read-only transactions retain their current transaction mode. No DB settings, provider behavior, or prior test expectations change. ## Action races (PASS) Independent probe runs 25 stale sync replays against the same UID, concurrent with separate Seen and Flagged writes, MOVE, or Deleted plus EXPUNGE. It checks accepted flags and stable UIDs, hidden source views, one pending MOVE destination, no duplicate physical membership keys, an empty search queue after draining, MATCH counts, and FTS rank=1 integrity. All three action cases pass. Evidence: `artifacts/review-probes.log`; test `mailrev_actions_race_sync_without_lost_updates`. ## Crash recovery (PASS) SIGKILL stops the real child window before message INSERT, batched membership INSERT, cursor UPDATE, live-generation activation, COMMIT, and immediately after COMMIT. Before COMMIT the next process sees one original membership; after COMMIT it sees all three. Replaying the window, changing UIDVALIDITY 77 → 88, and removing UID 10 by a validated expunge leaves exactly UID 11. Real FTS postings and integrity pass in every case. No duplicate or missing UID survives. Evidence: `artifacts/review-probes.log`; test `mailrev_kill_window_stages_then_epoch_and_expunge`. The first probe run had two harness faults: the reused fixture left both Users' thread IDs empty, and the membership kill marker matched INSERT instead of the actual WITH-prefixed batch statement. Both were corrected. The failed run is kept at `artifacts/review-probes-initial.log`; it is not counted as a pass. ## User isolation (PASS at the application boundary) Two Users have the same remote UID 9 and the same account email. Public account, folder, message, thread, attachment, unread and search reads refuse foreign coordinates; malformed window owner/account/folder combinations fail; foreign flag and transfer requests fail. Provider/sweep helpers accept an internal FolderRecord or opaque folder ID; their callers first obtain that identity through owner-scoped account/folder queries. They are not independent actor authorization boundaries. Evidence: `mailrev_cache_queries_are_user_scoped` and code audit of sync/IDLE callers, plus the existing foreign-UID proxy tests. ## Writer latency (PASS in the measured local scenarios; no hard bound) Real window commits and journal writes share the production single writer. 125 windows × 80 new messages run with 100 Seen writes and the real FTS batch function, paused 15 ms between batches as in production. | Local data | p50 write | p95 write | Maximum write | Sync duration | | --- | --- | --- | --- | --- | | 10k, 1040-byte bodies, concurrent FTS | 39.099 ms | 87.540 ms | 197.413 ms | 6418.575 ms | | 10k, 65533-byte bodies, concurrent FTS | 151.858 ms | 239.896 ms | 598.182 ms | 23514.341 ms | Every sample is retained. Both runs kept all 10,001 Alice memberships and passed FTS integrity. This is a correctness/latency probe on the shared build host, not the first-sync benchmark or a perf-VM baseline comparison. Evidence: `artifacts/latency-with-indexer.log`, `artifacts/latency-large-bodies.log`; test `mailrev_long_sync_user_write_latency`. The default SQLite busy_timeout is 5000 ms. Fetch windows contain 80 messages; expunge scan/cleanup pages contain at most 1024 UIDs; prune pages contain at most 500 (normal sync requests 80). Each job has 100 windows shared across backfill, delta and expunge. No provider I/O holds the SQLite writer. Measured sync does not starve writes for 1 s, but code does not enforce a universal 1 s deadline under unrelated authority locks, slow disks or host scheduling delays. ## Reproduction `python3 artifacts/run-mailrev-probes.py` temporarily injects the independent probes into search.rs, uses the existing target directory, runs the normal and large-body cases, and restores the source in a finally block. Passing review probes stay in artifacts. Only the two defect regressions and small fix are intended for commit. No existing test expectation was changed. ## Confirmed defect: incomplete epoch replaces the live cache (HIGH; OPEN) A real regression on committed head `8fbc1a3547864be033398e8d1a0c1cce6f6b90da` creates one verified live message in UIDVALIDITY 8, changes UIDVALIDITY to 9, then commits only one of the two expected replacement messages. The generation correctly has `backfill_complete=false`, but its `live_generation` becomes 2. It must remain 1 until the replacement count passes. Activation deletes old memberships and orphaned messages, so the old verified projection is lost. This violates DESIGN §53's live-projection rule during an epoch rebuild. Repro test: `incomplete_replacement_epoch_keeps_live_messages_until_verified`. Source: `artifacts/epoch-regression.rs`. Output: `artifacts/epoch-before.log`. Run: `python3 artifacts/run-mailrev-probes.py --epoch-defect` (expected failure; the script restores the source even when the regression fails). Exact failure: ``` assertion `left == right` failed: a failed count check must not activate the incomplete replacement left: 2 right: 1 ``` Cause: `commit_window` writes the count-checked completion flag, but calls `activate_generation` based only on `window.backfill_complete`. The regression also checks durable reopen and successful activation after the missing UID is supplied. It fails before those later checks on the current implementation. Do not apply only an activation guard: `sync_folder`'s `end == 0` resume branch sets completion in memory without a window commit; its successful footer can then report completion while the durable generation stays incomplete. Repair must preserve the old live projection and arrange a fresh, durable count check before activation on resume, including provider expunge/count drift. This is filed for the sync owner instead of changing those semantics in a read-only review. The failing regression is not committed, as the job requested. ## Final state Merged fetched `origin/dev d395b6e25da20d8c67b8b9fc1a593b813939c8a7` once. Merge commit: `18a80cc30d1bdeec60bdd5fc1f66707d322ef08d`. Fix commit/head: `8fbc1a3547864be033398e8d1a0c1cce6f6b90da`. Only the two proven writer-admission regressions and the small fix are committed. The five changed files are `cache/store.rs`, `proxy.rs`, `proxy_mutations.rs`, `proxy_tests.rs`, and `proxy_transfers.rs` under `crates/plugins/mail/src/`. Comments were re-read. No dependency or migration was added. No push or deploy. The Mail gates passed after the merge and before the atomic fix commit: `cargo fmt --check` (no output), `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings`, `cargo test -p calternal-plugin-mail -- --test-threads=4`. Full verbatim output is in `artifacts/admission-gate-{fmt,clippy,test}.log`. The later failing epoch probe was removed from the tracked source. The worktree returns to the exact committed and gated source. Known gap: the open epoch activation defect prevents a clean correctness sign-off. The seven ignored upstream tests are manual profiles/real-provider checks and the child worker that the normal recovery test spawns. The passing cache probes are local synthetic tests; no external-provider interoperability claim is made. Decisions: no new product decision. Retain deferred transactions when their first statement writes, and reserve only read-first writers. Keep passing review probes as artifacts; file the larger epoch/resume defect with its failing repro. Expected cost of the fix: one writer reservation per existing transaction, with no extra row query, batch growth or provider I/O under the writer. ## Failing epoch regression (temporary review source) ```rust /// A short replacement epoch must keep the last verified live projection. /// Its cursor can commit for restart, but activation and old-row cleanup /// must wait for the same provider-count check (#1340, DESIGN §§45,53). #[tokio::test] async fn incomplete_replacement_epoch_keeps_live_messages_until_verified() { let (_directory, db) = test_db().await; insert_account(&db).await; let folder = upsert_folders( &db, "owner-1", "account-1", &[DiscoveredFolder { remote_name: "INBOX".into(), delimiter: Some("/".into()), special_use: "inbox".into(), selectable: true, }], 2, ).await.unwrap().remove(0); let first = ensure_generation(&db, &folder, 8, 2, 3).await.unwrap(); store_window(&db, WindowCommit { owner_id: "owner-1", account_id: "account-1", folder_id: &folder.id, generation: first.generation, uid_validity: 8, messages: &[test_message("old-live", "old-provider", 1)], backfill_low_water: Some(0), backfill_complete: Some(true), expected_message_count: Some(1), delta_cursor: Some(1), now_ms: 4, }).await.unwrap(); let folder = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap(); let replacement = ensure_generation(&db, &folder, 9, 3, 5).await.unwrap(); let partial = test_message("new-one", "new-provider-one", 1); store_window(&db, WindowCommit { owner_id: "owner-1", account_id: "account-1", folder_id: &folder.id, generation: replacement.generation, uid_validity: 9, messages: std::slice::from_ref(&partial), backfill_low_water: Some(0), backfill_complete: Some(true), expected_message_count: Some(2), delta_cursor: Some(2), now_ms: 6, }).await.unwrap(); let pending = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap(); assert!(!pending.backfill_complete); assert_eq!(pending.live_generation, first.generation, "a failed count check must not activate the incomplete replacement"); let page = list_messages(&db, "owner-1", &folder.id, None, None, 10).await.unwrap(); assert_eq!(page.messages.len(), 1); assert_eq!(page.messages[0].id, "old-live"); assert!(get_message(&db, "owner-1", "old-live").await.unwrap().is_some()); // Reopen the actual file to check the durable cursor/live split, then // repeat the partial UID and supply the missing provider identity. let path = _directory.path().join("index.sqlite"); db.close().await; let db = Db::connect(path, DbOptions::default()).await.unwrap(); let resumed = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap(); assert_eq!(resumed.live_generation, first.generation); store_window(&db, WindowCommit { owner_id: "owner-1", account_id: "account-1", folder_id: &folder.id, generation: replacement.generation, uid_validity: 9, messages: &[partial, test_message("new-two", "new-provider-two", 2)], backfill_low_water: Some(0), backfill_complete: Some(true), expected_message_count: Some(2), delta_cursor: Some(2), now_ms: 7, }).await.unwrap(); let completed = get_folder(&db, "owner-1", &folder.id).await.unwrap().unwrap(); assert!(completed.backfill_complete); assert_eq!(completed.live_generation, replacement.generation); let uids: Vec<i64> = sqlx::query_scalar("SELECT uid FROM mail_memberships WHERE folder_id=? ORDER BY uid") .bind(&folder.id).fetch_all(db.reader_pool()).await.unwrap(); assert_eq!(uids, vec![1, 2]); assert!(get_message(&db, "owner-1", "old-live").await.unwrap().is_none()); let count: i64 = sqlx::query_scalar("SELECT count(*) FROM mail_messages WHERE owner_id='owner-1'") .fetch_one(db.reader_pool()).await.unwrap(); assert_eq!(count, 2); db.close().await; } ``` ## Gate output (verbatim) `cargo fmt --check`: no output. `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings` ```text Compiling libc v0.2.189 Checking cfg-if v1.0.5 Compiling siphasher v1.0.3 Checking log v0.4.34 Checking smallvec v1.16.1 Checking serde_core v1.0.229 Checking itoa v1.0.18 Compiling jobserver v0.1.35 Checking pin-project-lite v0.2.17 Checking once_cell v1.21.4 Checking memchr v2.8.3 Compiling cc v1.4.7 Checking futures-core v0.3.34 Checking serde v1.0.229 Checking scopeguard v1.2.0 Compiling zerocopy v0.8.58 Compiling getrandom v0.2.17 Checking lock_api v0.4.14 Compiling rand_core v0.6.4 Checking stable_deref_trait v1.2.1 Checking bytes v1.12.1 Checking parking_lot_core v0.9.12 Checking mio v1.2.3 Checking parking_lot v0.12.5 Checking errno v0.3.14 Checking zerofrom v0.1.8 Checking signal-hook-registry v1.4.8 Checking socket2 v0.6.5 Checking yoke v0.8.3 Compiling phf_shared v0.11.3 Checking futures-sink v0.3.34 Checking equivalent v1.0.2 Checking new_debug_unreachable v1.0.6 Checking typenum v1.20.1 Checking tokio v1.53.1 Compiling ppv-lite86 v0.2.21 Checking tracing-core v0.1.36 Compiling rand_chacha v0.3.1 Checking precomputed-hash v0.1.1 Checking zerovec v0.11.8 Checking bitflags v2.13.2 Compiling rand v0.8.8 Checking hashbrown v0.17.1 Compiling phf_generator v0.11.3 Checking indexmap v2.14.2 Checking slab v0.4.12 Checking num-traits v0.2.19 Checking tracing v0.1.44 Checking futures-channel v0.3.34 Compiling phf_shared v0.13.1 Checking percent-encoding v2.3.2 Checking tinystr v0.8.4 Checking futures-io v0.3.34 Checking base64 v0.22.1 Compiling phf_generator v0.13.1 Checking futures-task v0.3.34 Checking writeable v0.6.4 Checking litemap v0.8.3 Checking futures-util v0.3.34 Checking potential_utf v0.1.6 Checking icu_locale_core v2.3.0 Checking zerotrie v0.2.5 Checking utf8_iter v1.0.4 Checking icu_collections v2.3.0 Checking icu_provider v2.3.1 Compiling cmake v0.1.58 Compiling aws-lc-sys v0.45.0 Checking icu_properties_data v2.3.0 Checking icu_normalizer_data v2.3.0 Checking rand_core v0.10.1 Compiling siphasher v0.3.11 Checking icu_properties v2.3.0 Checking getrandom v0.4.3 Compiling phf_shared v0.10.0 Checking icu_normalizer v2.3.0 Compiling generic-array v0.14.9 Checking form_urlencoded v1.2.2 Checking zmij v1.0.23 Checking crossbeam-utils v0.8.23 Checking hybrid-array v0.4.15 Checking http v1.5.0 Checking dtoa v1.0.11 Checking zeroize v1.9.0 Checking iana-time-zone v0.1.65 Checking dtoa-short v0.3.5 Checking chrono v0.4.45 Checking serde_json v1.0.151 Checking idna_adapter v1.2.2 Compiling phf_generator v0.10.0 Compiling phf_macros v0.13.1 Compiling phf_macros v0.11.3 Checking phf v0.13.1 Checking phf v0.11.3 Compiling phf_codegen v0.10.0 Checking idna v1.1.0 Checking thiserror v2.0.21 Checking rustls-pki-types v1.15.1 Checking http-body v1.1.0 Compiling block-buffer v0.10.4 Compiling crypto-common v0.1.6 Compiling phf_shared v0.14.0 Compiling allocator-api2 v0.2.21 Compiling foldhash v0.2.0 Checking parking v2.2.1 Checking cpufeatures v0.3.1 Checking event-listener v5.4.2 Compiling hashbrown v0.16.1 Compiling phf_generator v0.14.0 Compiling digest v0.10.7 Checking url v2.5.8 Checking crypto-common v0.2.2 Compiling libsqlite3-sys v0.37.0 Compiling crc-catalog v2.5.0 Compiling cpufeatures v0.2.17 Checking simd-adler32 v0.3.10 Checking tower-service v0.3.3 Compiling crossbeam-queue v0.3.14 Compiling sha2 v0.10.9 Compiling crc v3.4.0 Compiling tokio-stream v0.1.19 Compiling either v1.18.0 Compiling futures-intrusive v0.5.0 Compiling hashlink v0.11.1 Compiling spin v0.9.9 Compiling phf_codegen v0.13.1 Compiling string_cache_codegen v0.6.1 Checking tokio-util v0.7.19 Checking tendril v0.5.1 Compiling blake3 v1.8.7 Checking fnv v1.0.7 Checking try-lock v0.2.5 Checking atomic-waker v1.1.2 Checking untrusted v0.9.0 Checking adler2 v2.0.1 Checking h2 v0.4.19 Checking want v0.3.1 Compiling sqlx-core v0.9.0 Compiling web_atoms v0.2.6 Compiling flume v0.12.0 Compiling atoi v2.0.0 Checking httparse v1.10.1 Compiling futures-executor v0.3.34 Compiling phf_codegen v0.14.0 Compiling string_cache_codegen v0.11.2 Checking inout v0.2.2 Compiling string_cache_codegen v0.5.4 Checking sync_wrapper v1.0.2 Checking phf_shared v0.12.1 Checking tower-layer v0.3.3 Checking cmov v0.5.4 Checking constant_time_eq v0.4.2 Checking minimal-lexical v0.2.1 Checking mac v0.1.1 Checking ipnet v2.12.2 Checking subtle v2.6.1 Checking arrayvec v0.7.8 Checking httpdate v1.0.3 Checking ryu v1.0.23 Checking hyper v1.11.1 Checking futf v0.1.5 Checking nom v7.1.3 Compiling aws-lc-rs v1.18.1 Compiling rustls v0.23.45 Checking ctutils v0.4.2 Checking phf v0.12.1 Compiling markup5ever v0.11.0 Compiling sqlx-sqlite v0.9.0 Compiling web_atoms v0.3.0 Checking crc32fast v1.5.2 Checking rustls-webpki v0.103.15 Checking phf v0.10.1 Checking miniz_oxide v0.9.1 Checking string_cache v0.9.0 Checking block-buffer v0.12.1 Checking linux-raw-sys v0.12.1 Checking utf-8 v0.7.6 Checking tinyvec v1.13.3 Compiling sqlx-macros-core v0.9.0 Checking rustix v1.1.5 Checking unicode-normalization v0.1.25 Checking tendril v0.4.3 Checking cipher v0.5.2 Checking flate2 v1.1.10 Checking chrono-tz v0.10.4 Checking universal-hash v0.6.1 Checking hyper-util v0.1.20 Checking getrandom v0.3.4 Checking phf v0.14.0 Checking string_cache v0.11.0 Checking tower v0.5.3 Checking miniz_oxide v0.8.9 Checking fdeflate v0.3.7 Checking http-body-util v0.1.5 Compiling selectors v0.25.0 Checking concurrent-queue v2.5.0 Checking string_cache v0.8.9 Compiling phf_codegen v0.11.3 Checking zune-core v0.5.3 Checking byteorder-lite v0.1.0 Checking color_quant v1.1.0 Checking weezl v0.1.12 Checking byteorder v1.5.0 Checking quick-error v2.0.1 Checking pxfm v0.1.30 Checking openssl-probe v0.2.1 Checking calternal-path v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-path) Checking image-webp v0.2.4 Checking rustls-native-certs v0.8.4 Checking fxhash v0.2.1 Checking gif v0.14.2 Checking zune-jpeg v0.5.15 Compiling parcel_selectors v0.28.3 Checking cssparser v0.37.0 Checking moxcms v0.8.1 Checking cssparser v0.31.2 Checking ahash v0.8.12 Checking png v0.18.1 Checking multiversion v0.9.0 Checking poly1305 v0.9.1 Checking markup5ever v0.39.0 Checking chacha20 v0.10.2 Compiling sqlx-macros v0.9.0 Checking inotify-sys v0.1.8 Checking unicode-joining-type v1.0.0 Checking tokio-rustls v0.26.6 Checking aead v0.6.1 Checking servo_arc v0.3.0 Checking bytemuck v1.25.2 Checking mime v0.3.17 Checking winnow v0.7.15 Checking core_detect v1.0.0 Checking unicode-casefold v0.2.0 Checking unicase v2.9.0 Checking same-file v1.0.6 Checking unicode-width v0.2.2 Checking lazy_static v1.5.0 Checking simdutf8 v0.1.5 Checking bounded-static v0.8.0 Checking unicode-properties v0.1.4 Checking event-listener v2.5.3 Checking pulldown-cmark-escape v0.11.0 Checking unsafe-libyaml v0.2.11 Checking async-channel v1.9.0 Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-fs) Checking pulldown-cmark v0.13.4 Checking cron v0.17.0 Checking serde_yaml v0.9.34+deprecated Checking encoding_rs v0.8.41 Checking imap-types v2.0.0-alpha.7 Checking getopts v0.2.24 Checking walkdir v2.5.0 Checking axum-core v0.5.6 Checking image v0.25.10 Checking chacha20poly1305 v0.11.0 Checking hyper-rustls v0.27.10 Checking utoipa v6.0.0 Checking sqlx v0.9.0 Checking inotify v0.11.5 Checking markup5ever v0.40.0 Checking html5ever v0.26.0 Checking rustls-platform-verifier v0.7.0 Checking tower-http v0.6.11 Checking abnf-core v0.6.0 Checking serde_urlencoded v0.7.1 Checking event-listener-strategy v0.5.4 Compiling hashify v0.2.9 Checking uuid v1.26.1 Checking notify-types v2.1.0 Checking quick-xml v0.42.0 Checking serde_path_to_error v0.1.20 Checking matchit v0.8.4 Checking base64 v0.23.1 Checking ego-tree v0.6.3 Checking rustc-hash v2.1.3 Checking calternal-notes-core v0.1.0 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-notes-core) Checking reqwest v0.13.5 Checking scraper v0.18.1 Checking axum v0.8.9 Checking notify v8.2.0 Checking thiserror v1.0.69 Checking pin-project v1.1.13 Checking calternal-db v0.1.0 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-db) Checking calternal-api v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-api) Checking mail-parser v0.11.9 Checking async-channel v2.5.0 Checking imap-codec v2.0.0-alpha.9 Checking linkme v0.3.37 Checking html5ever v0.40.1 Checking stop-token v0.7.0 Checking xml5ever v0.39.0 Checking html5ever v0.39.0 Checking cssparser-color v0.5.0 Checking imap-proto v0.16.7 Checking futures v0.3.34 Checking itertools v0.10.5 Checking cssparser v0.38.0 Checking const-str v1.1.0 Checking similar v3.2.0 Checking self_cell v1.3.0 Checking pathdiff v0.2.3 Checking data-encoding v2.11.1 Checking maplit v1.0.2 Checking pin-utils v0.1.0 Checking ammonia v4.2.0 Checking calternal-imap v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-imap) Checking lightningcss v1.0.0-alpha.72 Checking async-imap v0.11.3 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail/vendor/async-imap) Checking markup5ever_rcdom v0.39.0+unofficial Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-plugin) Checking sharded-slab v0.1.7 Checking webpki-roots v1.0.9 Checking tracing-log v0.2.0 Checking thread_local v1.1.10 Checking fastrand v2.5.0 Checking nu-ansi-term v0.50.3 Checking tempfile v3.27.0 Checking tracing-subscriber v0.3.23 Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 46s ``` `cargo test -p calternal-plugin-mail -- --test-threads=4` ```text Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 02s Running unittests src/lib.rs (/home/kayg/build/targets/mailrev/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 140 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test routes::tests::read_state_is_journaled_without_provider_access ... ok test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 133 passed; 0 failed; 7 ignored; 0 measured; 0 filtered out; finished in 38.80s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ```

kayg referenced this issue from a commit 2026-10-08 06:26:58 +00:00

fix(mail): reserve cache and journal writers before snapshot reads

kayg commented 2026-10-08 15:17:00 +00:00

Author

Owner

Copy link

mail7d starts on job/mail7-1340 at 8fbc1a3547. I will repair verified epoch activation and durable exhausted-cursor resume, retain reserved-writer admission, and run the filed reproduction plus epoch crash/action/isolation probes.

mail7d starts on job/mail7-1340 at 8fbc1a3547864be033398e8d1a0c1cce6f6b90da. I will repair verified epoch activation and durable exhausted-cursor resume, retain reserved-writer admission, and run the filed reproduction plus epoch crash/action/isolation probes.

kayg commented 2026-10-08 15:29:08 +00:00

Author

Owner

Copy link

The filed incomplete_replacement_epoch_keeps_live_messages_until_verified reproduction now passes (1 passed; 0 failed). Activation is gated by the durable count result. Exhausted-cursor resume now runs fresh provider checks and commits completion instead of only changing memory. I also found that epoch cleanup can cascade-delete queued flag intent; the fix hands unambiguous matching identities to fresh virtual UIDs atomically and keeps unmatched journals for an explicit delivery failure. New epoch/action/SIGKILL regressions and crate gates are running; no final sign-off yet.

The filed incomplete_replacement_epoch_keeps_live_messages_until_verified reproduction now passes (1 passed; 0 failed). Activation is gated by the durable count result. Exhausted-cursor resume now runs fresh provider checks and commits completion instead of only changing memory. I also found that epoch cleanup can cascade-delete queued flag intent; the fix hands unambiguous matching identities to fresh virtual UIDs atomically and keeps unmatched journals for an explicit delivery failure. New epoch/action/SIGKILL regressions and crate gates are running; no final sign-off yet.

kayg commented 2026-10-08 15:51:45 +00:00

Author

Owner

Copy link

Committed ce12de23d (verified atomic switch, durable resume, and queued-intent handoff) and 0b90a4b53 (SIGKILL/action/isolation regressions). The corrected crate suite passed: 139 passed; 0 failed; 8 ignored. Existing stale-epoch mutation rejection remains intact: intent is accepted against an old live epoch only while a real replacement generation is pending. The only existing assertion expectation changed is the schema table inventory, which now includes the new mail_proxy_intent_redirects table; no status, flag, UID, or recovery expectation was weakened. Merged origin/dev af202e1514 once, without conflicts, at 1d2ac902db. Migration 0022 is free on origin/dev. Final gates and local release timing are in progress.

Committed ce12de23d (verified atomic switch, durable resume, and queued-intent handoff) and 0b90a4b53 (SIGKILL/action/isolation regressions). The corrected crate suite passed: 139 passed; 0 failed; 8 ignored. Existing stale-epoch mutation rejection remains intact: intent is accepted against an old live epoch only while a real replacement generation is pending. The only existing assertion expectation changed is the schema table inventory, which now includes the new mail_proxy_intent_redirects table; no status, flag, UID, or recovery expectation was weakened. Merged origin/dev af202e1514955ddf0f8b4b3d2503c780d807ed84 once, without conflicts, at 1d2ac902db8590b6c9e77d2deedbfeb77b145ee1. Migration 0022 is free on origin/dev. Final gates and local release timing are in progress.

kayg commented 2026-10-08 16:12:04 +00:00

Author

Owner

Copy link

mail7d — #1340 final report

Built and committed the epoch repair. Head: 6834ffce8510d39eb79dd6998c4c5407cb877e3d on job/mail7-1340.
The old complete epoch stays live while replacement windows and cursors commit.
One reserved writer transaction verifies completion, hands off safe intent,
switches the live epoch and removes old memberships. Empty replacements also
need a verified commit. A stale old window cannot switch the epoch back.
BEGIN IMMEDIATE and the existing writer-admission rules stay in place.

Commits: ce12de23d (epoch fix); 0b90a4b53 (crash/action regressions);
1d2ac902d (merge origin/dev once); 9724ab108 (replacement profile fixture);
6834ffce8 (single-row completion checkpoint).
Merged origin/dev af202e1514 without conflicts.
No push, deploy or merge into dev/main was performed. The issue stays open.

Files

• bench/mail-sync.py
• crates/plugins/mail/migrations/0022_epoch_intents.sql
• crates/plugins/mail/src/cache.rs
• crates/plugins/mail/src/cache/store.rs
• crates/plugins/mail/src/sync.rs
• crates/plugins/mail/src/proxy.rs
• crates/plugins/mail/src/proxy_mutations.rs
• crates/plugins/mail/src/proxy_transfers.rs
• crates/plugins/mail/src/search.rs
• crates/plugins/mail/src/epoch_tests.rs

Proof

• The filed incomplete_replacement_epoch_keeps_live_messages_until_verified test passes: failed count, old live message, durable reopen, verified switch, stale replay rejection and verified empty replacement.
• Exhausted pending cursors resume with fresh provider checks. Tests cover unchanged count, expunge and a new arrival. Completion is durable, not only an in-memory flag.
• Actual SIGKILL passes at generation creation, mid-copy, before the live-pointer switch, before COMMIT and after COMMIT. Before commit, reads show the old complete view; after commit, they show the new complete view. Replays converge to exactly UIDs 9, 10 and 11, without duplicates or missing messages. One queued Seen intent survives every kill stage.
• Real FTS batches drain the durable queue. Rank-1 integrity checks and MATCH counts pass. External-content SELECT counts alone are not used as proof.
• Flags and web read state accepted before and during replacement move to fresh virtual UIDs and changed remote UIDs (9 to 109). Seen and Flagged remain visible; accepted deletion stays hidden. A second reset flattens the original Job redirect. Original Jobs resolve the new coordinates.
• Per-User isolation holds, including overlapping remote UIDs. Existing foreign-UID and stale-epoch rejection assertions stay unchanged and pass.
• Missing identities keep their message and journal for a reported Source unavailable delivery failure. Cleanup cannot silently cascade-delete accepted intent. New COPY/MOVE and draft work into a pending destination fails before a receipt commits.

Focused output: artifacts/mail7d-epoch-proof.log (4 passed, 0 failed, 1 ignored child worker). The final full suite runs the recovery/action tests twice.
The only existing expectation changed is the exact schema inventory: it adds mail_proxy_intent_redirects. No status, flag, UID, authorization or recovery expectation was weakened.

Local 10k measurements

Release test binary, final source, scripted IMAP transport; TLS and provider network are excluded. The real Search worker and message-list API run during sync. These are single local samples, not medians or a controlled A/B comparison.

Final sample	Sync	Timed process CPU	Mean RSS	Peak RSS	API p50 / p95	Inbox usable	API errors
First sync	5.743 s	5.626 s	29.68 MB	34.48 MB	23.563 / 50.978 ms	269.723 ms	0
Replace a complete 10k epoch	6.808 s	6.880 s	33.22 MB	49.18 MB	43.376 / 68.407 ms	57.837 ms	0

Both final samples meet the 7 s local budget. Both have 10,000 memberships, 9,999 messages (one intentional duplicate identity), and 9,999 FTS rows. Search completion is 5.804 s for first sync and 6.808 s for replacement.
Load (1/5/15 min): first sync 22.27/24.37/27.28 to 21.93/24.26/27.23; replacement 21.93/24.26/27.23 to 21.01/23.94/27.08.

Raw final samples: artifacts/mail7d-local-first-sync-checkpoint.json and artifacts/mail7d-local-replacement-checkpoint.json.
docs/perf/baseline.json has no matching Mail sync baseline. The earlier #1340 controlled perf-VM median was 6.095 s, process CPU 6.40 s and peak RSS 33,464,320 bytes. That median and local single samples are not directly comparable. The perf VM was held by calternal notifread-1387-after (30-minute request). It was checked once; no measurement or wait occurred there.

Earlier local samples remain in artifacts: first sync 7.281 s (with the focused proof), then 8.346 s; replacement 7.554 s. These SLOW samples had greater shared-host load. Do not attribute the full timing difference to the checkpoint change. The change removes an unnecessary folder-count scan with the existing single-row reader.
The initial replacement profile failed before timing: its cache-only fixture had no enabled Connected Account. artifacts/mail7d-replacement-initial-failure.log records the failure; it is not a sample. The optional fixture was fixed and the full replacement profile passed.

Expected cost: 10k sync uses 125 windows of at most 80 UIDs across two worker runs. The fix adds one indexed epoch read per worker run and one indexed checkpoint read at final coverage, with no new per-message query. Intent handoff uses set-based writes and indexed stable-identity joins. No provider I/O runs under the writer. The replacement profile seeds a complete old 10k history and its virtual UIDs outside timing. The bench option also supports the existing average and three-worker burst phases.

UX gaps closed

• Web read-state edits work against the old complete epoch during resync.
• Accepted flags and deletion stay visible after the switch and restart.
• Pending destinations cannot accept invisible transfer or draft receipts.
• Empty-window finalization publishes a folder invalidation after commit.

No UI or visible control was authored by this job. The authorized origin/dev merge includes Notes changes. Shared UI components and screenshots do not apply to the epoch fix.

Known gaps / UX gaps left

• Missing or ambiguous identities cannot safely select new UIDs. Intent stays durable and delivery reports unavailable. Automatic handoff requires one unambiguous replacement membership and one old intent.
• Non-idempotent COPY/MOVE receipts keep their original epoch pins as DESIGN section 53 requires. They are not guessed across a reset.
• Measurements are local synthetic samples. No new real-provider or macOS interoperability claim is made. Eight ignored tests are existing manual profiles/provider checks plus two child workers; normal recovery tests spawn those workers.

Decisions

No new product decision. DESIGN section 53 does not specify these mechanics:

• Completion uses equality with the fresh provider count, not an unchecked request bit or lower bound.
• Restart reuses the pending generation and checkpoint. UIDNEXT=1 also waits for a verified empty commit.
• Safe absolute flag/deletion handoff allocates fresh virtual UIDs. An owner-bound Index redirect lets original durable Jobs follow them. Missing/ambiguous identity retains reported failed work instead of targeting a reused UID.
• New non-idempotent destination work is refused during rebuild. Existing receipts retain immutable coordinates.
• Migration 0022 is free on fetched origin/dev (last Mail migration 0020); branch migration 0021 already belongs to #1340. No dependency was added.

Module and function comments were re-read. The checkpoint helper is a minimal crate-private export of the existing query.

For the merge round

No requested epoch, SIGKILL, action, isolation or local 10k proof is deferred. The orchestrator owns the normal combined-branch workspace gates and broad Server adversarial matrix. Exact workspace commands: cargo clippy --workspace --all-targets -- -D warnings; cargo test --workspace -- --test-threads=4. They were not run in this job.

Gate output (verbatim)

cargo fmt --check: no output; exit 0.

cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings

    Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 15.15s

cargo test -p calternal-plugin-mail -- --test-threads=4 (run 1)

sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
   Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 47.71s
     Running unittests src/lib.rs (/home/kayg/build/targets/mail7d/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 147 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 139 passed; 0 failed; 8 ignored; 0 measured; 0 filtered out; finished in 39.00s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-plugin-mail -- --test-threads=4 (run 2)

sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.70s
     Running unittests src/lib.rs (/home/kayg/build/targets/mail7d/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 147 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok

test result: ok. 139 passed; 0 failed; 8 ignored; 0 measured; 0 filtered out; finished in 35.79s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Cleanup

Cargo cleanup output (verbatim):

     Removed 11411 files, 3.6GiB total

No web build directories were present. The worktree is clean. Review artifacts remain ignored and are not committed.

# mail7d — #1340 final report Built and committed the epoch repair. Head: `6834ffce8510d39eb79dd6998c4c5407cb877e3d` on `job/mail7-1340`. The old complete epoch stays live while replacement windows and cursors commit. One reserved writer transaction verifies completion, hands off safe intent, switches the live epoch and removes old memberships. Empty replacements also need a verified commit. A stale old window cannot switch the epoch back. BEGIN IMMEDIATE and the existing writer-admission rules stay in place. Commits: ce12de23d (epoch fix); 0b90a4b53 (crash/action regressions); 1d2ac902d (merge origin/dev once); 9724ab108 (replacement profile fixture); 6834ffce8 (single-row completion checkpoint). Merged origin/dev af202e1514955ddf0f8b4b3d2503c780d807ed84 without conflicts. No push, deploy or merge into dev/main was performed. The issue stays open. ## Files - `bench/mail-sync.py` - `crates/plugins/mail/migrations/0022_epoch_intents.sql` - `crates/plugins/mail/src/cache.rs` - `crates/plugins/mail/src/cache/store.rs` - `crates/plugins/mail/src/sync.rs` - `crates/plugins/mail/src/proxy.rs` - `crates/plugins/mail/src/proxy_mutations.rs` - `crates/plugins/mail/src/proxy_transfers.rs` - `crates/plugins/mail/src/search.rs` - `crates/plugins/mail/src/epoch_tests.rs` ## Proof - The filed incomplete_replacement_epoch_keeps_live_messages_until_verified test passes: failed count, old live message, durable reopen, verified switch, stale replay rejection and verified empty replacement. - Exhausted pending cursors resume with fresh provider checks. Tests cover unchanged count, expunge and a new arrival. Completion is durable, not only an in-memory flag. - Actual SIGKILL passes at generation creation, mid-copy, before the live-pointer switch, before COMMIT and after COMMIT. Before commit, reads show the old complete view; after commit, they show the new complete view. Replays converge to exactly UIDs 9, 10 and 11, without duplicates or missing messages. One queued Seen intent survives every kill stage. - Real FTS batches drain the durable queue. Rank-1 integrity checks and MATCH counts pass. External-content SELECT counts alone are not used as proof. - Flags and web read state accepted before and during replacement move to fresh virtual UIDs and changed remote UIDs (9 to 109). Seen and Flagged remain visible; accepted deletion stays hidden. A second reset flattens the original Job redirect. Original Jobs resolve the new coordinates. - Per-User isolation holds, including overlapping remote UIDs. Existing foreign-UID and stale-epoch rejection assertions stay unchanged and pass. - Missing identities keep their message and journal for a reported Source unavailable delivery failure. Cleanup cannot silently cascade-delete accepted intent. New COPY/MOVE and draft work into a pending destination fails before a receipt commits. Focused output: artifacts/mail7d-epoch-proof.log (4 passed, 0 failed, 1 ignored child worker). The final full suite runs the recovery/action tests twice. The only existing expectation changed is the exact schema inventory: it adds mail_proxy_intent_redirects. No status, flag, UID, authorization or recovery expectation was weakened. ## Local 10k measurements Release test binary, final source, scripted IMAP transport; TLS and provider network are excluded. The real Search worker and message-list API run during sync. These are single local samples, not medians or a controlled A/B comparison. | Final sample | Sync | Timed process CPU | Mean RSS | Peak RSS | API p50 / p95 | Inbox usable | API errors | | --- | --- | --- | --- | --- | --- | --- | --- | | First sync | 5.743 s | 5.626 s | 29.68 MB | 34.48 MB | 23.563 / 50.978 ms | 269.723 ms | 0 | | Replace a complete 10k epoch | 6.808 s | 6.880 s | 33.22 MB | 49.18 MB | 43.376 / 68.407 ms | 57.837 ms | 0 | Both final samples meet the 7 s local budget. Both have 10,000 memberships, 9,999 messages (one intentional duplicate identity), and 9,999 FTS rows. Search completion is 5.804 s for first sync and 6.808 s for replacement. Load (1/5/15 min): first sync 22.27/24.37/27.28 to 21.93/24.26/27.23; replacement 21.93/24.26/27.23 to 21.01/23.94/27.08. Raw final samples: artifacts/mail7d-local-first-sync-checkpoint.json and artifacts/mail7d-local-replacement-checkpoint.json. docs/perf/baseline.json has no matching Mail sync baseline. The earlier #1340 controlled perf-VM median was 6.095 s, process CPU 6.40 s and peak RSS 33,464,320 bytes. That median and local single samples are not directly comparable. The perf VM was held by calternal notifread-1387-after (30-minute request). It was checked once; no measurement or wait occurred there. Earlier local samples remain in artifacts: first sync 7.281 s (with the focused proof), then 8.346 s; replacement 7.554 s. These SLOW samples had greater shared-host load. Do not attribute the full timing difference to the checkpoint change. The change removes an unnecessary folder-count scan with the existing single-row reader. The initial replacement profile failed before timing: its cache-only fixture had no enabled Connected Account. artifacts/mail7d-replacement-initial-failure.log records the failure; it is not a sample. The optional fixture was fixed and the full replacement profile passed. Expected cost: 10k sync uses 125 windows of at most 80 UIDs across two worker runs. The fix adds one indexed epoch read per worker run and one indexed checkpoint read at final coverage, with no new per-message query. Intent handoff uses set-based writes and indexed stable-identity joins. No provider I/O runs under the writer. The replacement profile seeds a complete old 10k history and its virtual UIDs outside timing. The bench option also supports the existing average and three-worker burst phases. ## UX gaps closed - Web read-state edits work against the old complete epoch during resync. - Accepted flags and deletion stay visible after the switch and restart. - Pending destinations cannot accept invisible transfer or draft receipts. - Empty-window finalization publishes a folder invalidation after commit. No UI or visible control was authored by this job. The authorized origin/dev merge includes Notes changes. Shared UI components and screenshots do not apply to the epoch fix. ## Known gaps / UX gaps left - Missing or ambiguous identities cannot safely select new UIDs. Intent stays durable and delivery reports unavailable. Automatic handoff requires one unambiguous replacement membership and one old intent. - Non-idempotent COPY/MOVE receipts keep their original epoch pins as DESIGN section 53 requires. They are not guessed across a reset. - Measurements are local synthetic samples. No new real-provider or macOS interoperability claim is made. Eight ignored tests are existing manual profiles/provider checks plus two child workers; normal recovery tests spawn those workers. ## Decisions No new product decision. DESIGN section 53 does not specify these mechanics: - Completion uses equality with the fresh provider count, not an unchecked request bit or lower bound. - Restart reuses the pending generation and checkpoint. UIDNEXT=1 also waits for a verified empty commit. - Safe absolute flag/deletion handoff allocates fresh virtual UIDs. An owner-bound Index redirect lets original durable Jobs follow them. Missing/ambiguous identity retains reported failed work instead of targeting a reused UID. - New non-idempotent destination work is refused during rebuild. Existing receipts retain immutable coordinates. - Migration 0022 is free on fetched origin/dev (last Mail migration 0020); branch migration 0021 already belongs to #1340. No dependency was added. Module and function comments were re-read. The checkpoint helper is a minimal crate-private export of the existing query. ## For the merge round No requested epoch, SIGKILL, action, isolation or local 10k proof is deferred. The orchestrator owns the normal combined-branch workspace gates and broad Server adversarial matrix. Exact workspace commands: cargo clippy --workspace --all-targets -- -D warnings; cargo test --workspace -- --test-threads=4. They were not run in this job. ## Gate output (verbatim) cargo fmt --check: no output; exit 0. `cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings` ```text Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `dev` profile [unoptimized + debuginfo] target(s) in 15.15s ``` `cargo test -p calternal-plugin-mail -- --test-threads=4 (run 1)` ```text sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `test` profile [unoptimized + debuginfo] target(s) in 47.71s Running unittests src/lib.rs (/home/kayg/build/targets/mail7d/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 147 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test routes::tests::read_state_is_journaled_without_provider_access ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test search::epoch_tests::epoch_sigkill_stages_converge ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 139 passed; 0 failed; 8 ignored; 0 measured; 0 filtered out; finished in 39.00s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo test -p calternal-plugin-mail -- --test-threads=4 (run 2)` ```text sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead sccache: warning: The server looks like it shut down unexpectedly, compiling locally instead Finished `test` profile [unoptimized + debuginfo] target(s) in 0.70s Running unittests src/lib.rs (/home/kayg/build/targets/mail7d/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 147 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::read_state_is_journaled_without_provider_access ... ok test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test search::epoch_tests::epoch_sigkill_stages_converge ... ok test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test result: ok. 139 passed; 0 failed; 8 ignored; 0 measured; 0 filtered out; finished in 35.79s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` ## Cleanup Cargo cleanup output (verbatim): ```text Removed 11411 files, 3.6GiB total ``` No web build directories were present. The worktree is clean. Review artifacts remain ignored and are not committed.

kayg commented 2026-10-08 18:04:56 +00:00

Author

Owner

Copy link

Independent mailrev2 review started on job/mail7-1340 at 6834ffce85. Reviewing code and behavior without author reports; probes cover epoch reads, restart, queued actions, User isolation, migration 0022 and repeated UIDVALIDITY changes.

Independent mailrev2 review started on job/mail7-1340 at 6834ffce8510d39eb79dd6998c4c5407cb877e3d. Reviewing code and behavior without author reports; probes cover epoch reads, restart, queued actions, User isolation, migration 0022 and repeated UIDVALIDITY changes.

kayg commented 2026-10-08 18:17:49 +00:00

Author

Owner

Copy link

Independent mailrev2 findings — #1340

Reviewed head: 6834ffce85.
Branch: job/mail7-1340. Author reports and result files were not read.

Pending probes

• Reader consistency: pending shared message upserts can alter live text and FTS before activation. Test pending corrected content with the same normalized Message-ID identity.
• Move intent: activation does not hand off transfer receipts. Check whether a uniquely mapped source becomes visible again.
• Delete/Search consistency: Search uses raw memberships rather than the accepted-intent view. Check an expunged message after handoff.
• Crash stages: start, messages, memberships, cursor, activation, commit, after_commit; list, FTS and folder count after reopening.
• Migration: true pre-0022 schema with existing Users, UIDs and queued flags; upgrade, replacement and restart.
• Ambiguity: two new memberships share one stable identity; retain old intent and refuse delivery.
• Second reset: generation 2 pending, generation 3 starts, reject a late generation 2 window, restart and finalize generation 3.

No verdict until probes complete.

P1 / BLOCK — pending epochs alter the live projection before activation

Evidence: commit_window updates shared mail_messages and merges shared thread keys/aliases for every window (cache/store.rs:1868, 1904), including non-live generations. live_generation filters memberships only; it does not isolate text, attachments or thread state.

Repro A: commit live epoch 77 with Reviewneedle; start replacement 88; commit one non-final window with the same provider identity, UID 109 and Replacementneedle. The folder still reports live generation 1, but its list returns Replacementneedle. Drain Search's queue: the pending term is searchable through the old live membership.

Repro B (no changed existing content): commit two live messages in separate threads; start replacement; commit one new, non-live message whose References bridge both live threads. The old list changes from [(review-11, review-11), (alice-message, alice-message)] to [(review-11, alice-message), (alice-message, alice-message)] before the pending epoch activates.

Failing regressions: mailrev2_pending_window_keeps_live_text, mailrev2_pending_window_keeps_live_fts, mailrev2_pending_bridge_keeps_live_threads.

P1 / BLOCK — mapped queued Move loses its accepted source hiding

Evidence: handoff_epoch_intents excludes transfer-owned UIDs (cache/store.rs:2301). mail_visible_memberships hides a Move's source only at its original generation and remote UID (migrations/0017_proxy_projection.sql). The transfer worker requires the original source generation to remain live (proxy_transfers.rs:360).

Repro: queue an undispatched Move from INBOX to Archive; verify the source list is empty; replace INBOX with exactly one membership for that same stable message, at UID 109. After activation the source reappears in INBOX while the accepted destination remains in Archive. The receipt stays pinned to the retired source generation, so it cannot deliver to the uniquely mapped replacement. Retaining the receipt alone does not preserve the accepted action.

Failing regression: mailrev2_mapped_move_keeps_source_hidden.

P2 — Search ignores accepted deletion after intent handoff

Evidence: MAIL_SEARCH_SQL tests raw mail_memberships rather than the intent overlay (search.rs:137). This predicate predates this branch, but breaks the requested replacement consistency check.

Repro: accept Deleted + selective expunge; replace with one mapped message; drain FTS. The folder list is empty and message count is 0, but Search returns the deleted message.

Failing regression: mailrev2_mapped_delete_stays_out_of_search.

Passing independent probes

• Real SIGKILL at generation creation, message insert, membership insert, cursor update, activation, pre-COMMIT and post-COMMIT. Restart reads agree across list, folder count and FTS; resumed windows converge to exact UIDs and one accepted flag intent. This proves transaction boundaries for unchanged text, not isolation of shared projections (P1 above).
• Actual pre-0022 schema with old-format accounts, UIDs and queued Seen intent; migrate, replace, restart and apply migrations again. Accepted flags and User scoping survive.
• Two replacement memberships with one stable identity retain the old queue without redirects; delivery reports Source unavailable after restart.
• Two Users share account labels, provider identities, remote UID 9 and virtual UID 1; distinct flag intents map to their own UIDs 109/209. FTS returns only the requesting User's message.
• A second UIDVALIDITY change abandons a pending generation; late old windows are refused; restart resumes generation 3 and maps one intent to epoch 99, UID 209.

Focused probe output (verbatim):

test result: FAILED. 5 passed; 5 failed; 1 ignored; 0 measured; 147 filtered out; finished in 15.36s

error: test failed, to rerun pass `-p calternal-plugin-mail --lib`

Full probe source and complete output stay in artifacts. Only the five failing regressions will be committed. No production fix is in scope for this independent review.

# Independent mailrev2 findings — #1340 Reviewed head: 6834ffce8510d39eb79dd6998c4c5407cb877e3d. Branch: job/mail7-1340. Author reports and result files were not read. ## Pending probes - Reader consistency: pending shared message upserts can alter live text and FTS before activation. Test pending corrected content with the same normalized Message-ID identity. - Move intent: activation does not hand off transfer receipts. Check whether a uniquely mapped source becomes visible again. - Delete/Search consistency: Search uses raw memberships rather than the accepted-intent view. Check an expunged message after handoff. - Crash stages: start, messages, memberships, cursor, activation, commit, after_commit; list, FTS and folder count after reopening. - Migration: true pre-0022 schema with existing Users, UIDs and queued flags; upgrade, replacement and restart. - Ambiguity: two new memberships share one stable identity; retain old intent and refuse delivery. - Second reset: generation 2 pending, generation 3 starts, reject a late generation 2 window, restart and finalize generation 3. No verdict until probes complete. ## P1 / BLOCK — pending epochs alter the live projection before activation Evidence: `commit_window` updates shared `mail_messages` and merges shared thread keys/aliases for every window (`cache/store.rs:1868`, `1904`), including non-live generations. `live_generation` filters memberships only; it does not isolate text, attachments or thread state. Repro A: commit live epoch 77 with `Reviewneedle`; start replacement 88; commit one non-final window with the same provider identity, UID 109 and `Replacementneedle`. The folder still reports live generation 1, but its list returns `Replacementneedle`. Drain Search's queue: the pending term is searchable through the old live membership. Repro B (no changed existing content): commit two live messages in separate threads; start replacement; commit one new, non-live message whose References bridge both live threads. The old list changes from `[(review-11, review-11), (alice-message, alice-message)]` to `[(review-11, alice-message), (alice-message, alice-message)]` before the pending epoch activates. Failing regressions: `mailrev2_pending_window_keeps_live_text`, `mailrev2_pending_window_keeps_live_fts`, `mailrev2_pending_bridge_keeps_live_threads`. ## P1 / BLOCK — mapped queued Move loses its accepted source hiding Evidence: `handoff_epoch_intents` excludes transfer-owned UIDs (`cache/store.rs:2301`). `mail_visible_memberships` hides a Move's source only at its original generation and remote UID (`migrations/0017_proxy_projection.sql`). The transfer worker requires the original source generation to remain live (`proxy_transfers.rs:360`). Repro: queue an undispatched Move from INBOX to Archive; verify the source list is empty; replace INBOX with exactly one membership for that same stable message, at UID 109. After activation the source reappears in INBOX while the accepted destination remains in Archive. The receipt stays pinned to the retired source generation, so it cannot deliver to the uniquely mapped replacement. Retaining the receipt alone does not preserve the accepted action. Failing regression: `mailrev2_mapped_move_keeps_source_hidden`. ## P2 — Search ignores accepted deletion after intent handoff Evidence: `MAIL_SEARCH_SQL` tests raw `mail_memberships` rather than the intent overlay (`search.rs:137`). This predicate predates this branch, but breaks the requested replacement consistency check. Repro: accept Deleted + selective expunge; replace with one mapped message; drain FTS. The folder list is empty and message count is 0, but Search returns the deleted message. Failing regression: `mailrev2_mapped_delete_stays_out_of_search`. ## Passing independent probes - Real SIGKILL at generation creation, message insert, membership insert, cursor update, activation, pre-COMMIT and post-COMMIT. Restart reads agree across list, folder count and FTS; resumed windows converge to exact UIDs and one accepted flag intent. This proves transaction boundaries for unchanged text, not isolation of shared projections (P1 above). - Actual pre-0022 schema with old-format accounts, UIDs and queued Seen intent; migrate, replace, restart and apply migrations again. Accepted flags and User scoping survive. - Two replacement memberships with one stable identity retain the old queue without redirects; delivery reports `Source unavailable` after restart. - Two Users share account labels, provider identities, remote UID 9 and virtual UID 1; distinct flag intents map to their own UIDs 109/209. FTS returns only the requesting User's message. - A second UIDVALIDITY change abandons a pending generation; late old windows are refused; restart resumes generation 3 and maps one intent to epoch 99, UID 209. Focused probe output (verbatim): ``` test result: FAILED. 5 passed; 5 failed; 1 ignored; 0 measured; 147 filtered out; finished in 15.36s error: test failed, to rerun pass `-p calternal-plugin-mail --lib` ``` Full probe source and complete output stay in artifacts. Only the five failing regressions will be committed. No production fix is in scope for this independent review.

kayg commented 2026-10-08 18:21:59 +00:00

Author

Owner

Copy link

mailrev2 independent data-safety re-review — #1340

Verdict: BLOCK.

Reviewed starting head: 6834ffce85.
Final head: 989656f982, branch job/mail7-1340.
git fetch origin && git merge origin/dev returned Already up to date. No push, deploy or merge into another branch was done. Author reports and result files were not read.

Findings

1. P1 / BLOCK: pending epochs change live text, FTS and thread grouping. Membership activation is atomic, but pending windows upsert shared mail_messages and merge shared thread keys before activation. A non-final replacement changes the old list's subject to Replacementneedle; the new term becomes searchable through the old membership. A distinct pending message that bridges two live threads changes their grouping before activation even without changing existing message content.
2. P1 / BLOCK: a uniquely mapped queued Move resurrects its source. Queue an undispatched Move to Archive, then replace INBOX with the same stable message at UID 109. The source returns to INBOX while the accepted destination remains in Archive. The journal retains obsolete source coordinates; handoff excludes transfer-owned UIDs.
3. P2: Search returns an accepted deletion after handoff. List and count hide the mapped expunged message (0 items), but Search returns it. Search uses raw memberships. This predicate predates this branch; it still fails the requested consistency check.

Detailed severity, source locations and repro steps were posted in the earlier findings comment and remain in artifacts/mailrev2-findings.md.

Built and files

Added only five failing regression tests to crates/plugins/mail/src/epoch_tests.rs, in three commits: live projection isolation, queued Move, deletion/Search consistency. No existing expectation was changed. No production code was changed.

Review artifacts (not committed): artifacts/mailrev2-findings.md, artifacts/mailrev2-all-probes.rs, artifacts/mailrev2-probe-output-retry.txt, artifacts/mailrev2-cargo-test.txt, artifacts/mailrev2-cargo-fmt.txt, and this report.

Independent checks passed

• SIGKILL at seven stages: start, messages, memberships, cursor, activation, before COMMIT and after COMMIT. Reopened list, folder count and FTS agree for unchanged text; replay converges to exact UIDs and one accepted flag intent.
• Real pre-0022 schema with old-format queued Seen intent: upgrade, sync replacement, restart, apply migrations again; intent and owner remain intact.
• Duplicate new memberships for one stable identity: no arbitrary redirect; retain the old intent and return Source unavailable after restart.
• Two Users with identical account labels, provider identities, remote UIDs and virtual UIDs: distinct intents map only to their own targets; Search stays User-bound.
• Second UIDVALIDITY reset while replacement is pending: reject late generation-2 commits, resume generation 3 after restart, preserve one mapped intent.
• All 139 pre-existing unignored crate tests passed, including provider interruption recovery, exhausted pending checkpoint recovery and wire UID/epoch checks.

Gates (verbatim)

Command: cargo test -p calternal-plugin-mail.

cargo fmt --check exited 0 with no output. The initial focused build failed because shared sccache reset its connection; the retry completed. Complete stdout/stderr stays in the artifact files. Cargo commands used line-tables-only debug info, no incremental compilation, four build jobs and the worktree temporary directory. The preset target directory was not overridden.

Known gaps and decisions

• The two P1 defects are unfixed. This is an independent review, and the brief permits committing only defect-exposing tests.
• Passing scratch probes were removed from tracked source and saved under artifacts. Only failing regressions were committed.
• Fixtures use inert credentials. Handoff coordinates, retained revisions, visibility and wire epoch checks were verified; this review does not claim a real external provider delivery or Apple-client acceptance run.
• Crash checks establish committed membership visibility. They do not negate the separate pending shared-projection defects.
• No product design decision was made. The only review choice was to reuse the existing cache fixture and SQL trace/SIGKILL harness.
• No new UI was built. UX gaps closed: none. UX gaps left: Move source visibility and deletion/Search consistency.

BLOCK — pending epochs change live projections before activation; uniquely mapped queued Moves lose their accepted source hiding.

# mailrev2 independent data-safety re-review — #1340 Verdict: BLOCK. Reviewed starting head: 6834ffce8510d39eb79dd6998c4c5407cb877e3d. Final head: 989656f982221debbf895082190abf24250b3814, branch job/mail7-1340. `git fetch origin && git merge origin/dev` returned `Already up to date.` No push, deploy or merge into another branch was done. Author reports and result files were not read. ## Findings 1. **P1 / BLOCK: pending epochs change live text, FTS and thread grouping.** Membership activation is atomic, but pending windows upsert shared `mail_messages` and merge shared thread keys before activation. A non-final replacement changes the old list's subject to `Replacementneedle`; the new term becomes searchable through the old membership. A distinct pending message that bridges two live threads changes their grouping before activation even without changing existing message content. 2. **P1 / BLOCK: a uniquely mapped queued Move resurrects its source.** Queue an undispatched Move to Archive, then replace INBOX with the same stable message at UID 109. The source returns to INBOX while the accepted destination remains in Archive. The journal retains obsolete source coordinates; handoff excludes transfer-owned UIDs. 3. **P2: Search returns an accepted deletion after handoff.** List and count hide the mapped expunged message (0 items), but Search returns it. Search uses raw memberships. This predicate predates this branch; it still fails the requested consistency check. Detailed severity, source locations and repro steps were posted in the earlier findings comment and remain in `artifacts/mailrev2-findings.md`. ## Built and files Added only five failing regression tests to `crates/plugins/mail/src/epoch_tests.rs`, in three commits: live projection isolation, queued Move, deletion/Search consistency. No existing expectation was changed. No production code was changed. Review artifacts (not committed): `artifacts/mailrev2-findings.md`, `artifacts/mailrev2-all-probes.rs`, `artifacts/mailrev2-probe-output-retry.txt`, `artifacts/mailrev2-cargo-test.txt`, `artifacts/mailrev2-cargo-fmt.txt`, and this report. ## Independent checks passed - SIGKILL at seven stages: start, messages, memberships, cursor, activation, before COMMIT and after COMMIT. Reopened list, folder count and FTS agree for unchanged text; replay converges to exact UIDs and one accepted flag intent. - Real pre-0022 schema with old-format queued Seen intent: upgrade, sync replacement, restart, apply migrations again; intent and owner remain intact. - Duplicate new memberships for one stable identity: no arbitrary redirect; retain the old intent and return `Source unavailable` after restart. - Two Users with identical account labels, provider identities, remote UIDs and virtual UIDs: distinct intents map only to their own targets; Search stays User-bound. - Second UIDVALIDITY reset while replacement is pending: reject late generation-2 commits, resume generation 3 after restart, preserve one mapped intent. - All 139 pre-existing unignored crate tests passed, including provider interruption recovery, exhausted pending checkpoint recovery and wire UID/epoch checks. ## Gates (verbatim) Command: `cargo test -p calternal-plugin-mail`. ``` ``` `cargo fmt --check` exited 0 with no output. The initial focused build failed because shared sccache reset its connection; the retry completed. Complete stdout/stderr stays in the artifact files. Cargo commands used line-tables-only debug info, no incremental compilation, four build jobs and the worktree temporary directory. The preset target directory was not overridden. ## Known gaps and decisions - The two P1 defects are unfixed. This is an independent review, and the brief permits committing only defect-exposing tests. - Passing scratch probes were removed from tracked source and saved under artifacts. Only failing regressions were committed. - Fixtures use inert credentials. Handoff coordinates, retained revisions, visibility and wire epoch checks were verified; this review does not claim a real external provider delivery or Apple-client acceptance run. - Crash checks establish committed membership visibility. They do not negate the separate pending shared-projection defects. - No product design decision was made. The only review choice was to reuse the existing cache fixture and SQL trace/SIGKILL harness. - No new UI was built. UX gaps closed: none. UX gaps left: Move source visibility and deletion/Search consistency. BLOCK — pending epochs change live projections before activation; uniquely mapped queued Moves lose their accepted source hiding.

kayg commented 2026-10-08 18:22:42 +00:00

Author

Owner

Copy link

The final-report gate block was empty due to a report extraction error. The corrected full report is in artifacts/mailrev2-final-report.md. The actual gate output is below, verbatim.

Command: cargo test -p calternal-plugin-mail

failures:
    search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search
    search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden
    search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads
    search::epoch_tests::mailrev2_pending_window_keeps_live_fts
    search::epoch_tests::mailrev2_pending_window_keeps_live_text

test result: FAILED. 139 passed; 5 failed; 8 ignored; 0 measured; 0 filtered out; finished in 56.27s

error: test failed, to rerun pass `-p calternal-plugin-mail --lib`

cargo fmt --check exited 0 with no output. Build cleanup completed: Removed 3966 files, 1.9GiB total. Working tree is clean. Head: 989656f982.

BLOCK — pending epochs change live projections before activation; uniquely mapped queued Moves lose their accepted source hiding.

The final-report gate block was empty due to a report extraction error. The corrected full report is in artifacts/mailrev2-final-report.md. The actual gate output is below, verbatim. Command: `cargo test -p calternal-plugin-mail` ``` failures: search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads search::epoch_tests::mailrev2_pending_window_keeps_live_fts search::epoch_tests::mailrev2_pending_window_keeps_live_text test result: FAILED. 139 passed; 5 failed; 8 ignored; 0 measured; 0 filtered out; finished in 56.27s error: test failed, to rerun pass `-p calternal-plugin-mail --lib` ``` `cargo fmt --check` exited 0 with no output. Build cleanup completed: `Removed 3966 files, 1.9GiB total`. Working tree is clean. Head: 989656f982221debbf895082190abf24250b3814. BLOCK — pending epochs change live projections before activation; uniquely mapped queued Moves lose their accepted source hiding.

kayg commented 2026-10-08 19:25:52 +00:00

Author

Owner

Copy link

Started mail7e on job/mail7-1340, base/head 989656f982. Read the independent review and its five regressions. Pending windows mutate shared message, attachment and thread projections; activation must publish staged projections in the same transaction. Queued undispatched transfers need an unambiguous source handoff. Search must use accepted-intent visibility. No regression expectations will change.

Started mail7e on job/mail7-1340, base/head 989656f982221debbf895082190abf24250b3814. Read the independent review and its five regressions. Pending windows mutate shared message, attachment and thread projections; activation must publish staged projections in the same transaction. Queued undispatched transfers need an unambiguous source handoff. Search must use accepted-intent visibility. No regression expectations will change.

kayg commented 2026-10-08 19:33:46 +00:00

Author

Owner

Copy link

Findings and decisions for mail7e (#1340):

• The three pending content/FTS/thread regressions pass with staged normalized input and atomic publication. Bodies and attachments stay unchanged across restart; activation can publish earlier pages from an empty final window.
• A focused run exposed a staging replay error: the exhausted pending checkpoint test expected UIDs [1,2] but replay restored UID 3. Fixed publication to join only retained memberships and update content metadata without writing UID membership or older flags. The original expectation is unchanged and the test now passes.
• The five review regressions pass. Transfer handoff creates a fresh source virtual UID only for unique, undispatched receipt sources. It keeps the receipt ID, destination UID and state. Dispatched/confirmed receipts remain pinned to prevent another COPY.
• Merged origin/dev once (f7a253dc4). Resolved the server upgrade test conflict by retaining both additions. Mail migration 23 is free on origin/dev, whose current Mail frontier is 20. Updated live upgrade pins for Mail 22/23; deployed fixture pins are unchanged.
• Decision: persist normalized input as JSON in a generation/UID table and publish in 100-row pages using the existing writer. Memory remains bounded by one page. Only replacement epochs pay serialization/replay work; first sync uses the original live path. Publication preserves provider reconciliation and recent flags.
• Retaining the independent review's passing scratch probes as permanent tests for migration, User isolation, ambiguity, second reset and seven SIGKILL stages.

Findings and decisions for mail7e (#1340): - The three pending content/FTS/thread regressions pass with staged normalized input and atomic publication. Bodies and attachments stay unchanged across restart; activation can publish earlier pages from an empty final window. - A focused run exposed a staging replay error: the exhausted pending checkpoint test expected UIDs [1,2] but replay restored UID 3. Fixed publication to join only retained memberships and update content metadata without writing UID membership or older flags. The original expectation is unchanged and the test now passes. - The five review regressions pass. Transfer handoff creates a fresh source virtual UID only for unique, undispatched receipt sources. It keeps the receipt ID, destination UID and state. Dispatched/confirmed receipts remain pinned to prevent another COPY. - Merged origin/dev once (f7a253dc4). Resolved the server upgrade test conflict by retaining both additions. Mail migration 23 is free on origin/dev, whose current Mail frontier is 20. Updated live upgrade pins for Mail 22/23; deployed fixture pins are unchanged. - Decision: persist normalized input as JSON in a generation/UID table and publish in 100-row pages using the existing writer. Memory remains bounded by one page. Only replacement epochs pay serialization/replay work; first sync uses the original live path. Publication preserves provider reconciliation and recent flags. - Retaining the independent review's passing scratch probes as permanent tests for migration, User isolation, ambiguity, second reset and seven SIGKILL stages.

kayg commented 2026-10-08 19:49:52 +00:00

Author

Owner

Copy link

The full Mail run passed the five review regressions and all retained safety probes. It exposed an exact schema-inventory assertion that did not yet include the new mail_epoch_content table. Added that table name, with its issue comment, without removing any existing entry (fad128f80). Live server upgrade pins also include Mail receipts 22/23; deployed fixtures remain unchanged. The original epoch_tests.rs prefix at 989656f98 is byte-for-byte unchanged. Final full runs and focused server upgrade checks are in progress.

The full Mail run passed the five review regressions and all retained safety probes. It exposed an exact schema-inventory assertion that did not yet include the new mail_epoch_content table. Added that table name, with its issue comment, without removing any existing entry (fad128f80). Live server upgrade pins also include Mail receipts 22/23; deployed fixtures remain unchanged. The original epoch_tests.rs prefix at 989656f98 is byte-for-byte unchanged. Final full runs and focused server upgrade checks are in progress.

kayg commented 2026-10-08 20:19:12 +00:00

Author

Owner

Copy link

Built #1340 Mail epoch correctness fixes on job/mail7-1340.
Head: 3b1b0868cd. Base: 989656f982. Merged origin/dev once at cc1af3c12b (merge commit f7a253dc4). No push, deploy or merge into another branch was performed.

Built

• Persist replacement input until activation. Pending windows cannot edit live text, HTML, attachments, FTS text, thread keys or aliases. Activation publishes retained UID pages in the same transaction as the epoch switch. Reconciliation's removed UIDs and newer flags stay authoritative.
• Read message content and attachments in one WAL snapshot.
• Map unique undispatched transfer sources to fresh virtual UIDs while retaining one receipt, accepted destination UID and queued state. Dispatched/confirmed receipts remain pinned, so a reset cannot authorize another COPY.
• Search uses accepted-intent membership visibility, like list and count.
• Added restart/body/attachment publication, anonymous bridge and transfer dispatch-boundary regressions. Retained the independent review's migration, ambiguity, second-reset, User isolation and seven-stage SIGKILL probes as permanent tests.
• The five original review tests and all prior epoch tests are byte-for-byte unchanged. The existing expectations changed only to extend exact migration inventories/frontiers for the added schema; no entries or receipt checks were removed.

Files

• crates/plugins/mail/src/cache/store.rs
• crates/plugins/mail/src/search.rs
• crates/plugins/mail/migrations/0023_epoch_content.sql
• crates/plugins/mail/src/epoch_tests.rs
• crates/plugins/mail/src/epoch_safety_tests.rs
• crates/calternal-server/src/upgrade_tests.rs (merge resolution and live migration pins only; deployed fixture receipts stay unchanged)

Decisions

• Use one durable normalized JSON input per generation/UID, then reuse the normal projection writer in 100-row publication pages. This avoids a second content/thread implementation. Abandoned staged inputs are removed on another reset.
• Only uniquely matched queued transfers can change source coordinates. Keep receipt and destination identity stable, and do not invent a flag mutation for a transfer with none.
• Migration 23 was free on the fetched origin/dev Mail frontier (20); this branch already owns 21/22.

Expected cost
Replacement publication is O(messages × normalized content and projection work), with at most 100 messages loaded per page. Staging writes batch 200 rows. Initial live sync does not serialize/replay staged input. The identity lookup remains indexed; only messages without a provider identity use the ID fallback. No performance target is claimed from correctness test timing.

UX gaps closed
Pending replacement content and thread leakage; accepted Move source resurrection; accepted deletion returning in Search. No UI controls changed.

Known gaps / For the merge round
The isolated external IMAP and Apple-client acceptance runs were not requested in this fix round. Full server, full adversarial matrices, release builds and performance profiles remain merge-round work. Run cargo test -p calternal-server -- --test-threads=4 for combined server coverage and the existing Mail provider acceptance profiles for real upstream exactly-once delivery. On the perf VM, run perf-run calternal mail7e 15 -- python3 bench/mail-sync.py "$MAIL_TEST_BIN" --test profile_full_history --profile-prefix "MAIL_HISTORY_PROFILE " --mailbox-messages 10000 --replacement --single --label mail7e with a prebuilt release test binary. Compare with docs/perf/baseline.json and repeat without --replacement for first sync. This report proves journal handoff and dispatch invariants, not a new external-provider acceptance run.

Gates (verbatim)

Command: cargo fmt --check. Exit 0, no output.

Command: cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings

    Blocking waiting for file lock on build directory
    Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 13m 36s

Command: cargo test -p calternal-plugin-mail (run 1; RUST_TEST_THREADS=4)

   Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 58.71s
     Running unittests src/lib.rs (/home/kayg/build/targets/mail7e/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 161 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok
test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok
test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok
test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok
test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok
test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok
test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok
test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok
test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes has been running for over 60 seconds
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 152 passed; 0 failed; 9 ignored; 0 measured; 0 filtered out; finished in 162.00s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s

Command: cargo test -p calternal-plugin-mail (run 2; RUST_TEST_THREADS=4)

    Blocking waiting for file lock on build directory
    Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 03s
     Running unittests src/lib.rs (/home/kayg/build/targets/mail7e/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 161 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok
test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok
test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok
test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok
test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok
test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok
test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok
test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok
test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 152 passed; 0 failed; 9 ignored; 0 measured; 0 filtered out; finished in 89.19s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Command: cargo clippy -p calternal-server --all-targets -- -D warnings

    Blocking waiting for file lock on build directory
   Compiling calternal-server v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-server)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 12s

Command: cargo test -p calternal-server upgrade_tests (RUST_TEST_THREADS=4)

   Compiling calternal-server v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-server)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 2m 11s
     Running unittests src/main.rs (/home/kayg/build/targets/mail7e/debug/deps/calternal_server-dd832c4034023fb8)

running 4 tests
test wire::upgrade_tests::production_schema_copy_upgrades_once_without_reinterpreting_notes_0028 ... ok
test wire::upgrade_tests::production_7c_upgrades_content_types_once ... ok
test wire::upgrade_tests::production_round_9_schema_upgrades_once_with_original_receipts ... ok
test wire::upgrade_tests::production_7b_schema_copy_upgrades_combined_pending_migrations ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 283 filtered out; finished in 5.05s

     Running tests/perf_guards.rs (/home/kayg/build/targets/mail7e/debug/deps/perf_guards-ec32631100a89d1d)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

     Running tests/private_index_permissions.rs (/home/kayg/build/targets/mail7e/debug/deps/private_index_permissions-d2fb274c9222eb69)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

Cleanup (verbatim):

     Removed 18470 files, 12.5GiB total

Removed generated apps/web/build, apps/web/.svelte-kit and target/tmp. Git status and git diff --check are clean. The Mail source is unchanged after its two passing full runs; later commits update only server upgrade inventories and comments. Re-read touched file documentation.

Head 3b1b0868cd — PASS: mailrev2_pending_window_keeps_live_text, mailrev2_pending_window_keeps_live_fts, mailrev2_pending_bridge_keeps_live_threads, mailrev2_mapped_move_keeps_source_hidden, mailrev2_mapped_delete_stays_out_of_search

Built #1340 Mail epoch correctness fixes on job/mail7-1340. Head: 3b1b0868cd655f037364b9a2e4f1bfc316d569f7. Base: 989656f982221debbf895082190abf24250b3814. Merged origin/dev once at cc1af3c12b68ad26c8403adde1be39c096b9ddea (merge commit f7a253dc4). No push, deploy or merge into another branch was performed. Built - Persist replacement input until activation. Pending windows cannot edit live text, HTML, attachments, FTS text, thread keys or aliases. Activation publishes retained UID pages in the same transaction as the epoch switch. Reconciliation's removed UIDs and newer flags stay authoritative. - Read message content and attachments in one WAL snapshot. - Map unique undispatched transfer sources to fresh virtual UIDs while retaining one receipt, accepted destination UID and queued state. Dispatched/confirmed receipts remain pinned, so a reset cannot authorize another COPY. - Search uses accepted-intent membership visibility, like list and count. - Added restart/body/attachment publication, anonymous bridge and transfer dispatch-boundary regressions. Retained the independent review's migration, ambiguity, second-reset, User isolation and seven-stage SIGKILL probes as permanent tests. - The five original review tests and all prior epoch tests are byte-for-byte unchanged. The existing expectations changed only to extend exact migration inventories/frontiers for the added schema; no entries or receipt checks were removed. Files - crates/plugins/mail/src/cache/store.rs - crates/plugins/mail/src/search.rs - crates/plugins/mail/migrations/0023_epoch_content.sql - crates/plugins/mail/src/epoch_tests.rs - crates/plugins/mail/src/epoch_safety_tests.rs - crates/calternal-server/src/upgrade_tests.rs (merge resolution and live migration pins only; deployed fixture receipts stay unchanged) Decisions - Use one durable normalized JSON input per generation/UID, then reuse the normal projection writer in 100-row publication pages. This avoids a second content/thread implementation. Abandoned staged inputs are removed on another reset. - Only uniquely matched queued transfers can change source coordinates. Keep receipt and destination identity stable, and do not invent a flag mutation for a transfer with none. - Migration 23 was free on the fetched origin/dev Mail frontier (20); this branch already owns 21/22. Expected cost Replacement publication is O(messages × normalized content and projection work), with at most 100 messages loaded per page. Staging writes batch 200 rows. Initial live sync does not serialize/replay staged input. The identity lookup remains indexed; only messages without a provider identity use the ID fallback. No performance target is claimed from correctness test timing. UX gaps closed Pending replacement content and thread leakage; accepted Move source resurrection; accepted deletion returning in Search. No UI controls changed. Known gaps / For the merge round The isolated external IMAP and Apple-client acceptance runs were not requested in this fix round. Full server, full adversarial matrices, release builds and performance profiles remain merge-round work. Run `cargo test -p calternal-server -- --test-threads=4` for combined server coverage and the existing Mail provider acceptance profiles for real upstream exactly-once delivery. On the perf VM, run `perf-run calternal mail7e 15 -- python3 bench/mail-sync.py "$MAIL_TEST_BIN" --test profile_full_history --profile-prefix "MAIL_HISTORY_PROFILE " --mailbox-messages 10000 --replacement --single --label mail7e` with a prebuilt release test binary. Compare with docs/perf/baseline.json and repeat without --replacement for first sync. This report proves journal handoff and dispatch invariants, not a new external-provider acceptance run. Gates (verbatim) Command: cargo fmt --check. Exit 0, no output. Command: cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings ``` Blocking waiting for file lock on build directory Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `dev` profile [unoptimized + debuginfo] target(s) in 13m 36s ``` Command: cargo test -p calternal-plugin-mail (run 1; RUST_TEST_THREADS=4) ``` Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `test` profile [unoptimized + debuginfo] target(s) in 58.71s Running unittests src/lib.rs (/home/kayg/build/targets/mail7e/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 161 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test routes::tests::read_state_is_journaled_without_provider_access ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test search::epoch_tests::epoch_sigkill_stages_converge ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes has been running for over 60 seconds test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 152 passed; 0 failed; 9 ignored; 0 measured; 0 filtered out; finished in 162.00s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s ``` Command: cargo test -p calternal-plugin-mail (run 2; RUST_TEST_THREADS=4) ``` Blocking waiting for file lock on build directory Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 03s Running unittests src/lib.rs (/home/kayg/build/targets/mail7e/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 161 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test routes::tests::read_state_is_journaled_without_provider_access ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test search::epoch_tests::epoch_sigkill_stages_converge ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 152 passed; 0 failed; 9 ignored; 0 measured; 0 filtered out; finished in 89.19s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Command: cargo clippy -p calternal-server --all-targets -- -D warnings ``` Blocking waiting for file lock on build directory Compiling calternal-server v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-server) Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 12s ``` Command: cargo test -p calternal-server upgrade_tests (RUST_TEST_THREADS=4) ``` Compiling calternal-server v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/calternal-server) Finished `test` profile [unoptimized + debuginfo] target(s) in 2m 11s Running unittests src/main.rs (/home/kayg/build/targets/mail7e/debug/deps/calternal_server-dd832c4034023fb8) running 4 tests test wire::upgrade_tests::production_schema_copy_upgrades_once_without_reinterpreting_notes_0028 ... ok test wire::upgrade_tests::production_7c_upgrades_content_types_once ... ok test wire::upgrade_tests::production_round_9_schema_upgrades_once_with_original_receipts ... ok test wire::upgrade_tests::production_7b_schema_copy_upgrades_combined_pending_migrations ... ok test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 283 filtered out; finished in 5.05s Running tests/perf_guards.rs (/home/kayg/build/targets/mail7e/debug/deps/perf_guards-ec32631100a89d1d) running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s Running tests/private_index_permissions.rs (/home/kayg/build/targets/mail7e/debug/deps/private_index_permissions-d2fb274c9222eb69) running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s ``` Cleanup (verbatim): ``` Removed 18470 files, 12.5GiB total ``` Removed generated apps/web/build, apps/web/.svelte-kit and target/tmp. Git status and git diff --check are clean. The Mail source is unchanged after its two passing full runs; later commits update only server upgrade inventories and comments. Re-read touched file documentation. Head 3b1b0868cd655f037364b9a2e4f1bfc316d569f7 — PASS: mailrev2_pending_window_keeps_live_text, mailrev2_pending_window_keeps_live_fts, mailrev2_pending_bridge_keeps_live_threads, mailrev2_mapped_move_keeps_source_hidden, mailrev2_mapped_delete_stays_out_of_search

kayg commented 2026-10-08 20:42:34 +00:00

Author

Owner

Copy link

Independent mailrev3 review started on job/mail7-1340, base/head 3b1b0868cd. I will inspect the implementation and run independent epoch, restart, action, isolation, upgrade and 10k staging probes. Author reports and result files will not be read. Findings: artifacts/mailrev3-findings.md.

Independent mailrev3 review started on job/mail7-1340, base/head 3b1b0868cd655f037364b9a2e4f1bfc316d569f7. I will inspect the implementation and run independent epoch, restart, action, isolation, upgrade and 10k staging probes. Author reports and result files will not be read. Findings: artifacts/mailrev3-findings.md.

kayg commented 2026-10-08 20:51:16 +00:00

Author

Owner

Copy link

Independent mailrev3 findings — #1340

Reviewed starting head: 3b1b0868cd.
Branch: job/mail7-1340. Author reports and result files were not read.

Review scope

Reader list, Search, count and detail consistency; real SIGKILL recovery; queued and dispatched actions; User isolation; migrations 0022 and 0023; second UIDVALIDITY reset; staged-content growth at 10,000 messages.

Probes in progress

• Migration 0023 creates an empty staging table. Check old pending checkpoints with shared content already written under 0022.
• Dispatched transfer source coordinates stay pinned; check accepted source visibility and receipt recovery after activation.
• Staging is keyed by folder/generation/UID and deleted on activation/reset. Measure logical bytes, peak resident use and abandoned generations.

P1 / BLOCK — deletion maps to the wrong old duplicate

Evidence: handoff_epoch_intents accepts one new membership, but only rejects an old duplicate when that duplicate has its own mutation or transfer. Ordinary duplicate source memberships do not make the match ambiguous.

Repro: sync old epoch 77 with remote UIDs 9 and 10, both with one normalized Message-ID identity. Select both through IMAP. Queue Deleted and selective EXPUNGE for only the UID-9 copy. That copy vanishes upstream; the UID-10 copy survives in replacement epoch 88 at UID 110. Activation creates a redirect for the UID-9 deletion to UID 110. Its expunge intent now hides the surviving copy and authorizes the mutation worker to delete it. There is no proof that the selected old copy survived.

Regression: mailrev3_old_duplicate_delete_must_not_target_surviving_copy. Expected 0 redirects, actual 1. This is a wrong-target destructive intent, not a timing finding. The probe does not perform destructive IO against a real provider.

P1 / BLOCK — abandoned epochs retain unbounded full body sets

Evidence: ensure_generation deletes abandoned mail_epoch_content JSON only. Pending project_window also inserts full normalized content for new identities into shared mail_messages. Abandoned memberships remain and keep those bodies alive. Cleanup occurs only on successful activation. Each reset with distinct identities adds another full body set.

Repro: start epochs 88, 99 and 111 without completing them; stage one different identity with a 64 KiB body in each. The staging table correctly has 1 row, but shared Mail has 3 body rows and 196,608 body bytes. No reset cap or abandoned-membership/body cleanup exists. At 10k distinct messages this adds at least 625 MiB of plain body bytes per abandoned epoch, plus JSON, HTML, indexes and WAL. Provider epoch churn can exhaust disk. Do not remove live rows or transfer/mutation-owned content when fixing this.

Regression: mailrev3_abandoned_epochs_do_not_accumulate_body_sets. Expected 1 retained pending body, actual 3. Separate 10k activation/JSON cleanup probe is still running.

P2 — dispatched Move loses accepted source hiding after activation

Evidence: dispatched receipts correctly keep their old coordinates. The visibility view hides only those old coordinates, so the uniquely matched new source returns. Delivery requires the original source to remain live and cannot reconcile the unchanged destination receipt after the switch.

Repro: queue Move INBOX → Archive, model the durable dispatched state (COPY sent, receipt not hydrated), activate replacement UID 109 for the same unique identity, close and reopen the Index. The receipt remains pinned and Archive has its accepted destination, but INBOX shows the source again. This probe models dispatch state; it does not claim a real wire-dispatched transfer.

Regression: mailrev3_dispatched_move_keeps_mapped_source_hidden. Receipt pinning passes; source hiding fails. Retain the non-replay rule when fixing the accepted read projection/recovery.

Passing independent probes so far

• Real subprocess SIGKILL at generation creation, shared-message insert, membership write, cursor update, activation, pre-COMMIT and post-COMMIT. Corrected subject/body/attachments and a References bridge stay old before COMMIT; after COMMIT they are new. List, detail, count and drained FTS agree. Restart and duplicate replay converge to exact UIDs 109 and 110.
• Actual schema through 0022, old-format rows, a pending generation with committed membership and low-water cursor, and queued Seen intent. Apply 0023, resume the checkpoint, finalize with an empty page, restart and apply migrations again. Body, UID and accepted Seen survive.

# Independent mailrev3 findings — #1340 Reviewed starting head: 3b1b0868cd655f037364b9a2e4f1bfc316d569f7. Branch: job/mail7-1340. Author reports and result files were not read. ## Review scope Reader list, Search, count and detail consistency; real SIGKILL recovery; queued and dispatched actions; User isolation; migrations 0022 and 0023; second UIDVALIDITY reset; staged-content growth at 10,000 messages. ## Probes in progress - Migration 0023 creates an empty staging table. Check old pending checkpoints with shared content already written under 0022. - Dispatched transfer source coordinates stay pinned; check accepted source visibility and receipt recovery after activation. - Staging is keyed by folder/generation/UID and deleted on activation/reset. Measure logical bytes, peak resident use and abandoned generations. ## P1 / BLOCK — deletion maps to the wrong old duplicate Evidence: `handoff_epoch_intents` accepts one new membership, but only rejects an old duplicate when that duplicate has its own mutation or transfer. Ordinary duplicate source memberships do not make the match ambiguous. Repro: sync old epoch 77 with remote UIDs 9 and 10, both with one normalized Message-ID identity. Select both through IMAP. Queue Deleted and selective EXPUNGE for only the UID-9 copy. That copy vanishes upstream; the UID-10 copy survives in replacement epoch 88 at UID 110. Activation creates a redirect for the UID-9 deletion to UID 110. Its expunge intent now hides the surviving copy and authorizes the mutation worker to delete it. There is no proof that the selected old copy survived. Regression: `mailrev3_old_duplicate_delete_must_not_target_surviving_copy`. Expected 0 redirects, actual 1. This is a wrong-target destructive intent, not a timing finding. The probe does not perform destructive IO against a real provider. ## P1 / BLOCK — abandoned epochs retain unbounded full body sets Evidence: `ensure_generation` deletes abandoned `mail_epoch_content` JSON only. Pending `project_window` also inserts full normalized content for new identities into shared `mail_messages`. Abandoned memberships remain and keep those bodies alive. Cleanup occurs only on successful activation. Each reset with distinct identities adds another full body set. Repro: start epochs 88, 99 and 111 without completing them; stage one different identity with a 64 KiB body in each. The staging table correctly has 1 row, but shared Mail has 3 body rows and 196,608 body bytes. No reset cap or abandoned-membership/body cleanup exists. At 10k distinct messages this adds at least 625 MiB of plain body bytes per abandoned epoch, plus JSON, HTML, indexes and WAL. Provider epoch churn can exhaust disk. Do not remove live rows or transfer/mutation-owned content when fixing this. Regression: `mailrev3_abandoned_epochs_do_not_accumulate_body_sets`. Expected 1 retained pending body, actual 3. Separate 10k activation/JSON cleanup probe is still running. ## P2 — dispatched Move loses accepted source hiding after activation Evidence: dispatched receipts correctly keep their old coordinates. The visibility view hides only those old coordinates, so the uniquely matched new source returns. Delivery requires the original source to remain live and cannot reconcile the unchanged destination receipt after the switch. Repro: queue Move INBOX → Archive, model the durable dispatched state (COPY sent, receipt not hydrated), activate replacement UID 109 for the same unique identity, close and reopen the Index. The receipt remains pinned and Archive has its accepted destination, but INBOX shows the source again. This probe models dispatch state; it does not claim a real wire-dispatched transfer. Regression: `mailrev3_dispatched_move_keeps_mapped_source_hidden`. Receipt pinning passes; source hiding fails. Retain the non-replay rule when fixing the accepted read projection/recovery. ## Passing independent probes so far - Real subprocess SIGKILL at generation creation, shared-message insert, membership write, cursor update, activation, pre-COMMIT and post-COMMIT. Corrected subject/body/attachments and a References bridge stay old before COMMIT; after COMMIT they are new. List, detail, count and drained FTS agree. Restart and duplicate replay converge to exact UIDs 109 and 110. - Actual schema through 0022, old-format rows, a pending generation with committed membership and low-water cursor, and queued Seen intent. Apply 0023, resume the checkpoint, finalize with an empty page, restart and apply migrations again. Body, UID and accepted Seen survive.

kayg commented 2026-10-08 20:58:21 +00:00

Author

Owner

Copy link

Independent mailrev3 final report — #1340

Independent mailrev3 findings — #1340

Reviewed starting head: 3b1b0868cd.
Branch: job/mail7-1340. Author reports and result files were not read.

Review scope

Reader list, Search, count and detail consistency; real SIGKILL recovery; queued and dispatched actions; User isolation; migrations 0022 and 0023; second UIDVALIDITY reset; staged-content growth at 10,000 messages.

Probes in progress

• Migration 0023 creates an empty staging table. Check old pending checkpoints with shared content already written under 0022.
• Dispatched transfer source coordinates stay pinned; check accepted source visibility and receipt recovery after activation.
• Staging is keyed by folder/generation/UID and deleted on activation/reset. Measure logical bytes, peak resident use and abandoned generations.

P1 / BLOCK — deletion maps to the wrong old duplicate

Evidence: handoff_epoch_intents accepts one new membership, but only rejects an old duplicate when that duplicate has its own mutation or transfer. Ordinary duplicate source memberships do not make the match ambiguous.

Repro: sync old epoch 77 with remote UIDs 9 and 10, both with one normalized Message-ID identity. Select both through IMAP. Queue Deleted and selective EXPUNGE for only the UID-9 copy. That copy vanishes upstream; the UID-10 copy survives in replacement epoch 88 at UID 110. Activation creates a redirect for the UID-9 deletion to UID 110. Its expunge intent now hides the surviving copy and authorizes the mutation worker to delete it. There is no proof that the selected old copy survived.

Regression: mailrev3_old_duplicate_delete_must_not_target_surviving_copy. Expected 0 redirects, actual 1. This is a wrong-target destructive intent, not a timing finding. The probe does not perform destructive IO against a real provider.

P1 / BLOCK — abandoned epochs retain unbounded full body sets

Evidence: ensure_generation deletes abandoned mail_epoch_content JSON only. Pending project_window also inserts full normalized content for new identities into shared mail_messages. Abandoned memberships remain and keep those bodies alive. Cleanup occurs only on successful activation. Each reset with distinct identities adds another full body set.

Repro: start epochs 88, 99 and 111 without completing them; stage one different identity with a 64 KiB body in each. The staging table correctly has 1 row, but shared Mail has 3 body rows and 196,608 body bytes. No reset cap or abandoned-membership/body cleanup exists. At 10k distinct messages this adds at least 625 MiB of plain body bytes per abandoned epoch, plus JSON, HTML, indexes and WAL. Provider epoch churn can exhaust disk. Do not remove live rows or transfer/mutation-owned content when fixing this.

Regression: mailrev3_abandoned_epochs_do_not_accumulate_body_sets. Expected 1 retained pending body, actual 3. Separate 10k activation/JSON cleanup probe is still running.

P2 — dispatched Move loses accepted source hiding after activation

Evidence: dispatched receipts correctly keep their old coordinates. The visibility view hides only those old coordinates, so the uniquely matched new source returns. Delivery requires the original source to remain live and cannot reconcile the unchanged destination receipt after the switch.

Repro: queue Move INBOX → Archive, model the durable dispatched state (COPY sent, receipt not hydrated), activate replacement UID 109 for the same unique identity, close and reopen the Index. The receipt remains pinned and Archive has its accepted destination, but INBOX shows the source again. This probe models dispatch state; it does not claim a real wire-dispatched transfer.

Regression: mailrev3_dispatched_move_keeps_mapped_source_hidden. Receipt pinning passes; source hiding fails. Retain the non-replay rule when fixing the accepted read projection/recovery.

Passing independent probes so far

• Real subprocess SIGKILL at generation creation, shared-message insert, membership write, cursor update, activation, pre-COMMIT and post-COMMIT. Corrected subject/body/attachments and a References bridge stay old before COMMIT; after COMMIT they are new. List, detail, count and drained FTS agree. Restart and duplicate replay converge to exact UIDs 109 and 110.
• Actual schema through 0022, old-format rows, a pending generation with committed membership and low-water cursor, and queued Seen intent. Apply 0023, resume the checkpoint, finalize with an empty page, restart and apply migrations again. Body, UID and accepted Seen survive.

Completed probes and storage result

The 10k probe passed: each of epochs 88 and 99 held exactly 10,000 staged rows / 701,424,475 JSON bytes. Replaying a page did not add rows. Reset removed the prior JSON, but retained 10,000 abandoned memberships. Activation removed all staged rows and left exactly 10,000 current memberships. Local debug test runtime for two staged passes plus activation: 285.26 s. Test-process VmHWM: 85,340 kB; final VmRSS: 76,080 kB. This is a large-body storage/recovery probe on the shared build host, not a first-sync latency benchmark or a perf-VM budget sample. It does not remove the distinct-identity body-retention defect above.

The full Mail crate gate also passed the existing seven-stage reader/count/FTS SIGKILL test, unambiguous flag/read/delete handoff, missing and duplicate-new-identity retention, queued Move retargeting across resets, two-User overlapping identity/UID isolation, pre-0022 upgrade, second pending reset, deferred FTS durability and selective wire-delivery/receipt tests. The changed-content independent SIGKILL and 0022→0023 pending-upgrade probes passed separately and stay in artifacts/mailrev3-probes.rs.

Gate findings outside the new regressions

Two existing remote-content tests failed at Db::connect: remote.rs:577 and remote.rs:539, both Sqlx(PoolTimedOut). No assertion in their Mail behavior was reached. These are SLOW/fixture-startup findings on the shared host; the gate failure is preserved verbatim below. No test expectations were changed and the full gate was not repeated.

Final report

Built: independent review probes; only the three failing regressions were committed, in one commit per concern. No production changes.

Files: crates/plugins/mail/src/epoch_tests.rs (committed); artifacts/mailrev3-findings.md, artifacts/mailrev3-probes.rs, artifacts/mailrev3-probe-output.txt, artifacts/mailrev3-cargo-test.txt (local review evidence, not committed).

Commits: 1e17caa90 (old duplicate deletion), 656c982d8 (dispatched Move visibility), 2021de94c (abandoned body retention).
Head: 2021de94c029f63df10b2d4c79299576fe5e98f6.

git fetch origin and git merge origin/dev ran once before gates: Already up to date. The worktree is clean. Added doc comments were re-read. cargo fmt --check passed (exit 0, no output).

cargo test -p calternal-plugin-mail (verbatim tail; full output in artifacts):

failures:
    remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access
    remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch
    search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets
    search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden
    search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy

test result: FAILED. 150 passed; 5 failed; 9 ignored; 0 measured; 0 filtered out; finished in 142.13s

error: test failed, to rerun pass `-p calternal-plugin-mail --lib`

Cleanup (verbatim):

     Removed 3962 files, 1.9GiB total

Known gaps: the three recorded regressions remain unfixed because this job is an independent review. The dispatched Move test models the persisted post-COPY dispatch state; it does not drive an external provider. Search checks drain the durable FTS queue; Search remains asynchronous after activation. The migration probe preserves legacy shared content; it cannot restore content already overwritten by the pre-staging writer. Two existing tests did not get past Index pool startup. No UI changed; no screenshot or web gate applies.

Decisions: no new production design decisions. The review treats old-side duplicate membership identities as ambiguous, even if only one copy has queued work. It counts full normalized bodies outside the staging table when assessing abandoned-epoch storage bounds.

BLOCK — wrong-target deletion handoff for old duplicate UIDs; unbounded full-body retention across abandoned epochs. Dispatched Move source visibility is an additional P2 finding.

# Independent mailrev3 final report — #1340 # Independent mailrev3 findings — #1340 Reviewed starting head: 3b1b0868cd655f037364b9a2e4f1bfc316d569f7. Branch: job/mail7-1340. Author reports and result files were not read. ## Review scope Reader list, Search, count and detail consistency; real SIGKILL recovery; queued and dispatched actions; User isolation; migrations 0022 and 0023; second UIDVALIDITY reset; staged-content growth at 10,000 messages. ## Probes in progress - Migration 0023 creates an empty staging table. Check old pending checkpoints with shared content already written under 0022. - Dispatched transfer source coordinates stay pinned; check accepted source visibility and receipt recovery after activation. - Staging is keyed by folder/generation/UID and deleted on activation/reset. Measure logical bytes, peak resident use and abandoned generations. ## P1 / BLOCK — deletion maps to the wrong old duplicate Evidence: `handoff_epoch_intents` accepts one new membership, but only rejects an old duplicate when that duplicate has its own mutation or transfer. Ordinary duplicate source memberships do not make the match ambiguous. Repro: sync old epoch 77 with remote UIDs 9 and 10, both with one normalized Message-ID identity. Select both through IMAP. Queue Deleted and selective EXPUNGE for only the UID-9 copy. That copy vanishes upstream; the UID-10 copy survives in replacement epoch 88 at UID 110. Activation creates a redirect for the UID-9 deletion to UID 110. Its expunge intent now hides the surviving copy and authorizes the mutation worker to delete it. There is no proof that the selected old copy survived. Regression: `mailrev3_old_duplicate_delete_must_not_target_surviving_copy`. Expected 0 redirects, actual 1. This is a wrong-target destructive intent, not a timing finding. The probe does not perform destructive IO against a real provider. ## P1 / BLOCK — abandoned epochs retain unbounded full body sets Evidence: `ensure_generation` deletes abandoned `mail_epoch_content` JSON only. Pending `project_window` also inserts full normalized content for new identities into shared `mail_messages`. Abandoned memberships remain and keep those bodies alive. Cleanup occurs only on successful activation. Each reset with distinct identities adds another full body set. Repro: start epochs 88, 99 and 111 without completing them; stage one different identity with a 64 KiB body in each. The staging table correctly has 1 row, but shared Mail has 3 body rows and 196,608 body bytes. No reset cap or abandoned-membership/body cleanup exists. At 10k distinct messages this adds at least 625 MiB of plain body bytes per abandoned epoch, plus JSON, HTML, indexes and WAL. Provider epoch churn can exhaust disk. Do not remove live rows or transfer/mutation-owned content when fixing this. Regression: `mailrev3_abandoned_epochs_do_not_accumulate_body_sets`. Expected 1 retained pending body, actual 3. Separate 10k activation/JSON cleanup probe is still running. ## P2 — dispatched Move loses accepted source hiding after activation Evidence: dispatched receipts correctly keep their old coordinates. The visibility view hides only those old coordinates, so the uniquely matched new source returns. Delivery requires the original source to remain live and cannot reconcile the unchanged destination receipt after the switch. Repro: queue Move INBOX → Archive, model the durable dispatched state (COPY sent, receipt not hydrated), activate replacement UID 109 for the same unique identity, close and reopen the Index. The receipt remains pinned and Archive has its accepted destination, but INBOX shows the source again. This probe models dispatch state; it does not claim a real wire-dispatched transfer. Regression: `mailrev3_dispatched_move_keeps_mapped_source_hidden`. Receipt pinning passes; source hiding fails. Retain the non-replay rule when fixing the accepted read projection/recovery. ## Passing independent probes so far - Real subprocess SIGKILL at generation creation, shared-message insert, membership write, cursor update, activation, pre-COMMIT and post-COMMIT. Corrected subject/body/attachments and a References bridge stay old before COMMIT; after COMMIT they are new. List, detail, count and drained FTS agree. Restart and duplicate replay converge to exact UIDs 109 and 110. - Actual schema through 0022, old-format rows, a pending generation with committed membership and low-water cursor, and queued Seen intent. Apply 0023, resume the checkpoint, finalize with an empty page, restart and apply migrations again. Body, UID and accepted Seen survive. ## Completed probes and storage result The 10k probe passed: each of epochs 88 and 99 held exactly 10,000 staged rows / 701,424,475 JSON bytes. Replaying a page did not add rows. Reset removed the prior JSON, but retained 10,000 abandoned memberships. Activation removed all staged rows and left exactly 10,000 current memberships. Local debug test runtime for two staged passes plus activation: 285.26 s. Test-process VmHWM: 85,340 kB; final VmRSS: 76,080 kB. This is a large-body storage/recovery probe on the shared build host, not a first-sync latency benchmark or a perf-VM budget sample. It does not remove the distinct-identity body-retention defect above. The full Mail crate gate also passed the existing seven-stage reader/count/FTS SIGKILL test, unambiguous flag/read/delete handoff, missing and duplicate-new-identity retention, queued Move retargeting across resets, two-User overlapping identity/UID isolation, pre-0022 upgrade, second pending reset, deferred FTS durability and selective wire-delivery/receipt tests. The changed-content independent SIGKILL and 0022→0023 pending-upgrade probes passed separately and stay in `artifacts/mailrev3-probes.rs`. ## Gate findings outside the new regressions Two existing remote-content tests failed at `Db::connect`: `remote.rs:577` and `remote.rs:539`, both `Sqlx(PoolTimedOut)`. No assertion in their Mail behavior was reached. These are SLOW/fixture-startup findings on the shared host; the gate failure is preserved verbatim below. No test expectations were changed and the full gate was not repeated. ## Final report Built: independent review probes; only the three failing regressions were committed, in one commit per concern. No production changes. Files: `crates/plugins/mail/src/epoch_tests.rs` (committed); `artifacts/mailrev3-findings.md`, `artifacts/mailrev3-probes.rs`, `artifacts/mailrev3-probe-output.txt`, `artifacts/mailrev3-cargo-test.txt` (local review evidence, not committed). Commits: `1e17caa90` (old duplicate deletion), `656c982d8` (dispatched Move visibility), `2021de94c` (abandoned body retention). Head: `2021de94c029f63df10b2d4c79299576fe5e98f6`. `git fetch origin` and `git merge origin/dev` ran once before gates: `Already up to date.` The worktree is clean. Added doc comments were re-read. `cargo fmt --check` passed (exit 0, no output). `cargo test -p calternal-plugin-mail` (verbatim tail; full output in artifacts): ``` failures: remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy test result: FAILED. 150 passed; 5 failed; 9 ignored; 0 measured; 0 filtered out; finished in 142.13s error: test failed, to rerun pass `-p calternal-plugin-mail --lib` ``` Cleanup (verbatim): ``` Removed 3962 files, 1.9GiB total ``` Known gaps: the three recorded regressions remain unfixed because this job is an independent review. The dispatched Move test models the persisted post-COPY dispatch state; it does not drive an external provider. Search checks drain the durable FTS queue; Search remains asynchronous after activation. The migration probe preserves legacy shared content; it cannot restore content already overwritten by the pre-staging writer. Two existing tests did not get past Index pool startup. No UI changed; no screenshot or web gate applies. Decisions: no new production design decisions. The review treats old-side duplicate membership identities as ambiguous, even if only one copy has queued work. It counts full normalized bodies outside the staging table when assessing abandoned-epoch storage bounds. BLOCK — wrong-target deletion handoff for old duplicate UIDs; unbounded full-body retention across abandoned epochs. Dispatched Move source visibility is an additional P2 finding.

kayg commented 2026-10-08 21:38:50 +00:00

Author

Owner

Copy link

mail7f repair started on job/mail7-1340, base/head 2021de94c0. Read both review rounds and all three unchanged regressions. Fixing bidirectional identity mapping, abandoned full-body retention, and dispatched Move visibility. No UI or dependency changes.

mail7f repair started on job/mail7-1340, base/head 2021de94c029f63df10b2d4c79299576fe5e98f6. Read both review rounds and all three unchanged regressions. Fixing bidirectional identity mapping, abandoned full-body retention, and dispatched Move visibility. No UI or dependency changes.

kayg commented 2026-10-08 21:46:57 +00:00

Author

Owner

Copy link

mail7f findings and fixes in progress:

• handoff_epoch_intents rejected only old duplicate virtual UIDs with queued work. It now requires exactly one old membership and one new membership for the identity. All old copies count, including copies with no virtual UID.
• ensure_generation removed mail_epoch_content only. It now removes memberships from epochs other than live/current and uses the existing intent-aware orphan collector. Startup runs the same cleanup for stale folders in bounded batches. First sync does not scan account bodies when no abandoned rows exist.
• Both the web projection and IMAP snapshot hid Move sources only at old coordinates. Migration 0024 and IMAP reads now hide dispatched sources by same-User, same-folder identity; the journal coordinates stay pinned. The new predicate has a partial covering index.
• Receipt recovery can read the pinned destination after source replacement. Source deletion at replacement coordinates remains forbidden. A complete current/live snapshot with no source identity permits cleanup; a pending snapshot or a remaining source keeps the receipt and accepted hiding.
• Db::connect has not changed. The earlier remote test PoolTimedOut failures were before any Mail migrations/configuration, so this cleanup cannot explain those startup failures. Both full test runs are queued with RUST_TEST_THREADS=4.
• git fetch origin and git merge origin/dev ran once: Already up to date. Checked origin/dev migrations; 0024 is free.
• Kept all original regression assertions. Added startup/readers/absence checks and retained the independent pending-0022 upgrade and changed-content SIGKILL probes as permanent tests.
  No dependency or UI changes. Focused build and final gates are in progress.

mail7f findings and fixes in progress: - handoff_epoch_intents rejected only old duplicate virtual UIDs with queued work. It now requires exactly one old membership and one new membership for the identity. All old copies count, including copies with no virtual UID. - ensure_generation removed mail_epoch_content only. It now removes memberships from epochs other than live/current and uses the existing intent-aware orphan collector. Startup runs the same cleanup for stale folders in bounded batches. First sync does not scan account bodies when no abandoned rows exist. - Both the web projection and IMAP snapshot hid Move sources only at old coordinates. Migration 0024 and IMAP reads now hide dispatched sources by same-User, same-folder identity; the journal coordinates stay pinned. The new predicate has a partial covering index. - Receipt recovery can read the pinned destination after source replacement. Source deletion at replacement coordinates remains forbidden. A complete current/live snapshot with no source identity permits cleanup; a pending snapshot or a remaining source keeps the receipt and accepted hiding. - Db::connect has not changed. The earlier remote test PoolTimedOut failures were before any Mail migrations/configuration, so this cleanup cannot explain those startup failures. Both full test runs are queued with RUST_TEST_THREADS=4. - git fetch origin and git merge origin/dev ran once: Already up to date. Checked origin/dev migrations; 0024 is free. - Kept all original regression assertions. Added startup/readers/absence checks and retained the independent pending-0022 upgrade and changed-content SIGKILL probes as permanent tests. No dependency or UI changes. Focused build and final gates are in progress.

kayg commented 2026-10-08 21:58:41 +00:00

Author

Owner

Copy link

Additional mail7f finding: destination hydration in project_window updates mail_proxy_transfers.message_id when it observes the receipt keyword. That field is not an immutable source identity. Using it to hide a replacement source, prove source absence, or retain source content can lose the accepted Move after hydration.

Fix: migration 0024 stores source_message_id separately, backfills existing receipts from their pinned virtual UID, and captures it when a transfer is accepted. Hydration leaves this field unchanged. IMAP and web hiding, replacement absence proof, and orphan/expunge body retention use this immutable source identity. Added mail7f_receipt_hydration_keeps_original_move_source_hidden, which models changed destination identity and confirms source visibility and retention through another reset.

This is part of the dispatched Move repair. Provider coordinates and the no-replay rule remain unchanged.

Additional mail7f finding: destination hydration in project_window updates mail_proxy_transfers.message_id when it observes the receipt keyword. That field is not an immutable source identity. Using it to hide a replacement source, prove source absence, or retain source content can lose the accepted Move after hydration. Fix: migration 0024 stores source_message_id separately, backfills existing receipts from their pinned virtual UID, and captures it when a transfer is accepted. Hydration leaves this field unchanged. IMAP and web hiding, replacement absence proof, and orphan/expunge body retention use this immutable source identity. Added mail7f_receipt_hydration_keeps_original_move_source_hidden, which models changed destination identity and confirms source visibility and retention through another reset. This is part of the dispatched Move repair. Provider coordinates and the no-replay rule remain unchanged.

kayg commented 2026-10-08 22:13:22 +00:00

Author

Owner

Copy link

mail7f — #1340 repair complete

Built: one-to-one epoch intent handoff in both directions; abandoned membership/body cleanup on reset and startup; dispatched Move hiding and receipt recovery across epochs. Migration 0024 keeps the original source identity separate from destination hydration. Provider coordinates stay pinned. Missing or ambiguous source intent stays recorded and fails with Source unavailable. No original regression or expectation was changed.

Branch: job/mail7-1340. Starting head: 2021de94c0.
Head: e492303a63.
Commits: 4e3d980a2 (identity proof), 08b5eb7e8 (abandoned content), 782a0dcfd (dispatched Move), e492303a6 (upgrade/SIGKILL review probes).
No push, deploy, issue close or merge into dev/main. git fetch origin and git merge origin/dev ran once: Already up to date. Checked origin/dev migrations; 0024 was free.

Files:
crates/plugins/mail/migrations/0024_dispatched_move_visibility.sql
crates/plugins/mail/src/cache.rs
crates/plugins/mail/src/cache/store.rs
crates/plugins/mail/src/epoch_tests.rs
crates/plugins/mail/src/proxy.rs
crates/plugins/mail/src/proxy_transfers.rs
crates/plugins/mail/src/sync.rs

Verification: CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_INCREMENTAL=0, CARGO_BUILD_JOBS=4; TMPDIR in this worktree; full tests used RUST_TEST_THREADS=4. Only the Mail crate changed. Final formatting, clippy and both full test runs passed. Focused epoch suite: 25 passed, 0 failed, 3 ignored subprocess workers. Both remote-content tests that previously failed during Db::connect passed in both final runs. Db::connect is unchanged, and those failures occurred before Mail startup cleanup; evidence is consistent with shared-host load.

cargo fmt --check: exit 0, no output.

cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings (verbatim):

    Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 12.52s

cargo test -p calternal-plugin-mail, run 1 (verbatim tail):

test result: ok. 160 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 34.44s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-plugin-mail, run 2 (verbatim tail):

test result: ok. 160 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 34.08s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Earlier integration failures were fixed: the IMAP query assumed the new column before migration; it now reuses the shared visibility view. The new cleanup test queried both Users while asserting only Alice's bodies; its query now scopes Alice and separately checks Bob's body. Early failing logs stay in artifacts/mail7f-test--before-upgrade-fix.txt. All prior mailrev2_ probes and actual SIGKILL stages pass, including the retained changed-content probe and pending-0022 upgrade.

UX gaps closed: dispatched Move sources no longer return in IMAP, web pages, detail selection or folder counts; accepted destinations remain visible. Destination hydration cannot replace source identity or discard its body. UX gaps left: no new UI was authored.

Expected cost: intent handoff uses indexed probes that stop at two copies, then batch writes. Reset removes abandoned staging/memberships and scans account messages for orphans only when abandoned rows were removed. Startup reads stale folders in pages of 100 and rechecks them in a writer transaction. First sync skips the account orphan scan. The dispatched-source read predicate has a partial index.

Known gaps: 10 existing optional provider/profile tests remain ignored in normal gates. The new dispatched-state regressions model the durable receipt and provider snapshot in the real Index; they do not drive a new external-provider/Mac run. No fresh timing or 7 s performance claim is made in this safety repair.

For the merge round: use the isolated TLS fixture from tests/adversarial/mail-sync.md, then run
cargo test -p calternal-plugin-mail real_tls_copy_delete_preserves_receipt_and_replays_once -- --ignored --nocapture
to verify selective deletion and receipt non-replay against the provider. Run the existing replacement profile with a prebuilt release Mail test binary on the perf VM:
perf-run calternal mail7f-replacement 15 -- python3 bench/mail-sync.py <prebuilt-mail-test-binary> --test profile_full_history --profile-prefix 'MAIL_HISTORY_PROFILE ' --replacement --mailbox-messages 10000.
It must compare replacement latency, CPU, RSS and the burst under the same conditions; no comparable Mail sync entry exists in docs/perf/baseline.json. Release builds and full external matrices belong to the merge round under the verification policy.

Decisions: source identity is durable receipt state, independent of destination hydration. Unknown dispatched outcomes keep the source identity hidden and retain the receipt; only a complete current/live snapshot can prove absence. Startup cleanup runs once in the background, with writer admission, and preserves live/current epochs plus all mutation/transfer-owned content.

Re-read comments in every touched file. Working tree is clean. Review artifacts were not committed.
Cleanup (verbatim):

     Removed 7092 files, 2.5GiB total

e492303a63 — mailrev3_old_duplicate_delete_must_not_target_surviving_copy, mailrev3_abandoned_epochs_do_not_accumulate_body_sets, mailrev3_dispatched_move_keeps_mapped_source_hidden now pass in both full runs.

mail7f — #1340 repair complete Built: one-to-one epoch intent handoff in both directions; abandoned membership/body cleanup on reset and startup; dispatched Move hiding and receipt recovery across epochs. Migration 0024 keeps the original source identity separate from destination hydration. Provider coordinates stay pinned. Missing or ambiguous source intent stays recorded and fails with Source unavailable. No original regression or expectation was changed. Branch: job/mail7-1340. Starting head: 2021de94c029f63df10b2d4c79299576fe5e98f6. Head: e492303a63d619c83ef778e531451dd92f1fa3dc. Commits: 4e3d980a2 (identity proof), 08b5eb7e8 (abandoned content), 782a0dcfd (dispatched Move), e492303a6 (upgrade/SIGKILL review probes). No push, deploy, issue close or merge into dev/main. git fetch origin and git merge origin/dev ran once: Already up to date. Checked origin/dev migrations; 0024 was free. Files: crates/plugins/mail/migrations/0024_dispatched_move_visibility.sql crates/plugins/mail/src/cache.rs crates/plugins/mail/src/cache/store.rs crates/plugins/mail/src/epoch_tests.rs crates/plugins/mail/src/proxy.rs crates/plugins/mail/src/proxy_transfers.rs crates/plugins/mail/src/sync.rs Verification: CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_INCREMENTAL=0, CARGO_BUILD_JOBS=4; TMPDIR in this worktree; full tests used RUST_TEST_THREADS=4. Only the Mail crate changed. Final formatting, clippy and both full test runs passed. Focused epoch suite: 25 passed, 0 failed, 3 ignored subprocess workers. Both remote-content tests that previously failed during Db::connect passed in both final runs. Db::connect is unchanged, and those failures occurred before Mail startup cleanup; evidence is consistent with shared-host load. cargo fmt --check: exit 0, no output. cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings (verbatim): ``` Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `dev` profile [unoptimized + debuginfo] target(s) in 12.52s ``` cargo test -p calternal-plugin-mail, run 1 (verbatim tail): ``` test result: ok. 160 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 34.44s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` cargo test -p calternal-plugin-mail, run 2 (verbatim tail): ``` test result: ok. 160 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 34.08s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Earlier integration failures were fixed: the IMAP query assumed the new column before migration; it now reuses the shared visibility view. The new cleanup test queried both Users while asserting only Alice's bodies; its query now scopes Alice and separately checks Bob's body. Early failing logs stay in artifacts/mail7f-test-*-before-upgrade-fix.txt. All prior mailrev2_* probes and actual SIGKILL stages pass, including the retained changed-content probe and pending-0022 upgrade. UX gaps closed: dispatched Move sources no longer return in IMAP, web pages, detail selection or folder counts; accepted destinations remain visible. Destination hydration cannot replace source identity or discard its body. UX gaps left: no new UI was authored. Expected cost: intent handoff uses indexed probes that stop at two copies, then batch writes. Reset removes abandoned staging/memberships and scans account messages for orphans only when abandoned rows were removed. Startup reads stale folders in pages of 100 and rechecks them in a writer transaction. First sync skips the account orphan scan. The dispatched-source read predicate has a partial index. Known gaps: 10 existing optional provider/profile tests remain ignored in normal gates. The new dispatched-state regressions model the durable receipt and provider snapshot in the real Index; they do not drive a new external-provider/Mac run. No fresh timing or 7 s performance claim is made in this safety repair. For the merge round: use the isolated TLS fixture from tests/adversarial/mail-sync.md, then run `cargo test -p calternal-plugin-mail real_tls_copy_delete_preserves_receipt_and_replays_once -- --ignored --nocapture` to verify selective deletion and receipt non-replay against the provider. Run the existing replacement profile with a prebuilt release Mail test binary on the perf VM: `perf-run calternal mail7f-replacement 15 -- python3 bench/mail-sync.py <prebuilt-mail-test-binary> --test profile_full_history --profile-prefix 'MAIL_HISTORY_PROFILE ' --replacement --mailbox-messages 10000`. It must compare replacement latency, CPU, RSS and the burst under the same conditions; no comparable Mail sync entry exists in docs/perf/baseline.json. Release builds and full external matrices belong to the merge round under the verification policy. Decisions: source identity is durable receipt state, independent of destination hydration. Unknown dispatched outcomes keep the source identity hidden and retain the receipt; only a complete current/live snapshot can prove absence. Startup cleanup runs once in the background, with writer admission, and preserves live/current epochs plus all mutation/transfer-owned content. Re-read comments in every touched file. Working tree is clean. Review artifacts were not committed. Cleanup (verbatim): ``` Removed 7092 files, 2.5GiB total ``` e492303a63d619c83ef778e531451dd92f1fa3dc — mailrev3_old_duplicate_delete_must_not_target_surviving_copy, mailrev3_abandoned_epochs_do_not_accumulate_body_sets, mailrev3_dispatched_move_keeps_mapped_source_hidden now pass in both full runs.

kayg commented 2026-10-08 22:18:32 +00:00

Author

Owner

Copy link

Independent mailrev4 review started on job/mail7-1340 at e492303a63 (merge base with origin/dev to follow). I read the repository rules, vocabulary and Mail design sections and the issue body. I will inspect the diff and behavior without author reports. Focus: duplicate intent mappings in both epochs; abandoned body cleanup and SIGKILL; dispatched Move settlement/failure; pending upgrades through 0024; rerun mailrev2/mailrev3. Findings go to artifacts/mailrev4-findings.md. Only failing regression tests will be committed; no production edits.

Independent mailrev4 review started on job/mail7-1340 at e492303a63d619c83ef778e531451dd92f1fa3dc (merge base with origin/dev to follow). I read the repository rules, vocabulary and Mail design sections and the issue body. I will inspect the diff and behavior without author reports. Focus: duplicate intent mappings in both epochs; abandoned body cleanup and SIGKILL; dispatched Move settlement/failure; pending upgrades through 0024; rerun mailrev2/mailrev3. Findings go to artifacts/mailrev4-findings.md. Only failing regression tests will be committed; no production edits.

kayg commented 2026-10-08 22:27:12 +00:00

Author

Owner

Copy link

Independent mailrev4 findings — #1340

Starting head: e492303a63.
Branch: job/mail7-1340. Author reports and result files were not read.

Review in progress

Round 4 checks bidirectional ambiguity, abandoned-body cleanup and crash safety,
dispatched Move settlement/failure, 0022/0023 to 0024 pending upgrades, and all
mailrev2/mailrev3 regressions. No production changes are authorized by this review.

P1 / BLOCK — failed dispatched Move keeps a surviving source hidden indefinitely

Evidence: migration 0024 hides a source identity whenever a moving receipt is not
queued. It has no Jobs state or provider-settlement predicate. Transfer delivery
keeps a receipt and returns Source unavailable when a complete replacement still
contains that identity (proxy_transfers.rs, replacement_source_absent/deliver).
Normal sync never restores this source or wakes a dead transfer Job.

Repro: queue Move INBOX to Archive for old UID 9; persist dispatched state; sync
replacement epoch 88 with the unique same identity at UID 109; reach the real
Jobs retry limit via lease_next/fail and assert state dead; sync the surviving
UID 109 again. Receipt remains recorded (correct), but INBOX has 0 messages,
expected 1. Retry can only repeat the same failure while that identity survives.
No provider delete or second COPY is performed by the probe.

Regression: mailrev4_failed_dispatched_move_reappears_after_sync.
Assertion: left 0, right 1. This is persistent projection loss, not SLOW.

P2 / BLOCK for this reader-consistency round — dispatch hides an unselected duplicate

Evidence: migration 0024's identity-wide hiding applies to the original epoch,
not only a replacement. It does not check old-side/new-side unique membership
proof. Distinct UIDs can share one normalized message identity.

Repro: live epoch 77 has UIDs 9 and 10 with the same message identity. Select both
through IMAP; queue Move for one UID. INBOX correctly retains the unselected UID.
Change only the durable receipt to dispatched. INBOX now has 0 messages, expected

1. Dispatch has not deleted or moved the unselected provider UID. This is caused
   by the new view, before any UIDVALIDITY change.

Regression: mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate.
Assertion: left 0, right 1. The probe models the persisted dispatch boundary.

Passing round-4 probes so far

• Flag/read/delete and queued Move require unique identities on both sides:
  old/new copy counts 1/2, 2/1 and 2/2 retain old intent without redirects; 1/1
  retargets once and duplicate replay keeps one mutation/receipt.
• Real subprocess SIGKILL after staged JSON DELETE, membership DELETE and body
  DELETE, before COMMIT and after COMMIT. Restart rolls back the first four
  stages, keeps committed cleanup in the last, then reruns cleanup idempotently.
  Current generation-3 staging activates; live generation-1 and accepted old
  intent/body survive; another User's body survives.
• Actual schemas through 0022 and 0023 with a dispatched Move and pending epoch:
  0024 backfills the source identity, keeps low-water 109/current generation 2,
  publishes retained/staged bodies, and survives two opens/idempotent migrations.

Focused round-4 output (verbatim):

test result: FAILED. 4 passed; 2 failed; 1 ignored; 0 measured; 170 filtered out; finished in 5.77s

error: test failed, to rerun pass `-p calternal-plugin-mail --lib`

# Independent mailrev4 findings — #1340 Starting head: e492303a63d619c83ef778e531451dd92f1fa3dc. Branch: job/mail7-1340. Author reports and result files were not read. ## Review in progress Round 4 checks bidirectional ambiguity, abandoned-body cleanup and crash safety, dispatched Move settlement/failure, 0022/0023 to 0024 pending upgrades, and all mailrev2/mailrev3 regressions. No production changes are authorized by this review. ## P1 / BLOCK — failed dispatched Move keeps a surviving source hidden indefinitely Evidence: migration 0024 hides a source identity whenever a moving receipt is not queued. It has no Jobs state or provider-settlement predicate. Transfer delivery keeps a receipt and returns Source unavailable when a complete replacement still contains that identity (proxy_transfers.rs, replacement_source_absent/deliver). Normal sync never restores this source or wakes a dead transfer Job. Repro: queue Move INBOX to Archive for old UID 9; persist dispatched state; sync replacement epoch 88 with the unique same identity at UID 109; reach the real Jobs retry limit via lease_next/fail and assert state dead; sync the surviving UID 109 again. Receipt remains recorded (correct), but INBOX has 0 messages, expected 1. Retry can only repeat the same failure while that identity survives. No provider delete or second COPY is performed by the probe. Regression: mailrev4_failed_dispatched_move_reappears_after_sync. Assertion: left 0, right 1. This is persistent projection loss, not SLOW. ## P2 / BLOCK for this reader-consistency round — dispatch hides an unselected duplicate Evidence: migration 0024's identity-wide hiding applies to the original epoch, not only a replacement. It does not check old-side/new-side unique membership proof. Distinct UIDs can share one normalized message identity. Repro: live epoch 77 has UIDs 9 and 10 with the same message identity. Select both through IMAP; queue Move for one UID. INBOX correctly retains the unselected UID. Change only the durable receipt to dispatched. INBOX now has 0 messages, expected 1. Dispatch has not deleted or moved the unselected provider UID. This is caused by the new view, before any UIDVALIDITY change. Regression: mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate. Assertion: left 0, right 1. The probe models the persisted dispatch boundary. ## Passing round-4 probes so far - Flag/read/delete and queued Move require unique identities on both sides: old/new copy counts 1/2, 2/1 and 2/2 retain old intent without redirects; 1/1 retargets once and duplicate replay keeps one mutation/receipt. - Real subprocess SIGKILL after staged JSON DELETE, membership DELETE and body DELETE, before COMMIT and after COMMIT. Restart rolls back the first four stages, keeps committed cleanup in the last, then reruns cleanup idempotently. Current generation-3 staging activates; live generation-1 and accepted old intent/body survive; another User's body survives. - Actual schemas through 0022 and 0023 with a dispatched Move and pending epoch: 0024 backfills the source identity, keeps low-water 109/current generation 2, publishes retained/staged bodies, and survives two opens/idempotent migrations. Focused round-4 output (verbatim): ``` test result: FAILED. 4 passed; 2 failed; 1 ignored; 0 measured; 170 filtered out; finished in 5.77s error: test failed, to rerun pass `-p calternal-plugin-mail --lib` ```

kayg commented 2026-10-08 22:33:58 +00:00

Author

Owner

Copy link

Independent mailrev4 data-safety review — #1340

Verdict: BLOCK. Two new reader consistency defects remain.

Built: independent review probes; committed only the two failing regressions,
one concern per commit. No production change, push or deploy. Author reports
and result files were not read.

Files: crates/plugins/mail/src/epoch_tests.rs (committed). Local evidence:
artifacts/mailrev4-findings.md, mailrev4-probes.rs, mailrev4-scratch.rs,
mailrev4-probe-output.txt, mailrev4-tls-output.txt,
mailrev4-cleanup-strengthened.txt and mailrev4-cargo-test.txt.

Commits: 9aabc56e5 (failed Move source); 32625ada8 (unselected duplicate).
Head: 32625ada8a.
Base with origin/dev: cc1af3c12b.
Required fetch/merge ran once: Already up to date.

Blocking findings:

• P1: failed dispatched Move hides a provider-confirmed surviving source forever.
  Queue Move for UID 9, dispatch, replace with unique UID 109, reach Jobs' real
  retry limit (dead), then sync UID 109 again. The receipt correctly remains,
  but INBOX has 0 messages instead of 1. The 0024 view ignores permanent Job
  failure; sync does not restore the source or wake the dead receipt. Regression:
  mailrev4_failed_dispatched_move_reappears_after_sync.
• P2, blocking this reader-consistency review: dispatch hides an unselected old
  duplicate. Live UIDs 9 and 10 share one identity. Move one UID: the other stays
  visible while queued. Change only the receipt to dispatched: the other also
  vanishes (0 instead of 1), before any reset. The 0024 identity predicate applies
  to the original epoch without unique-membership proof. Regression:
  mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate.

Passing evidence:

• All 15 mailrev2/mailrev3 non-worker regressions passed in the full crate gate.
  Their parent tests ran real SIGKILL workers, including corrected text, bodies,
  attachments, thread bridges, list, Search, count and restart convergence.
• New scratch matrices checked old/new duplicates 1/2, 2/1, 2/2 and 1/1 for
  flag/read/delete intent and queued Move: ambiguous identities stay pinned;
  unique identities map once; duplicate replay does not add intent.
• Cleanup SIGKILL after staged JSON, membership and body deletes, before COMMIT
  and after COMMIT passed. Current staging still activates; live Seen state,
  unavailable accepted intent/body and the other User survive. Cleanup is
  idempotent after restart. Strengthened focused output:
  test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 176 filtered out; finished in 4.24s
• True 0022 and 0023 schemas with pending checkpoints and dispatched Moves
  upgraded to 0024, preserved bodies/source identity/low-water, activated, and
  survived restart and idempotent migration. No migration added by this review.
• Real Dovecot TLS COPY dispatch and two actual source UIDVALIDITY resets:
  surviving identity refused deletion; the next complete empty epoch permitted
  receipt settlement; repeat recovery left one Archive message, no receipt and
  no operation keyword. Output:
  test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 179 filtered out; finished in 2.86s

Gates (verbatim):

cargo fmt --check: exit 0, no output.

cargo test -p calternal-plugin-mail -- --test-threads=4:

failures:
    search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate
    search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync

test result: FAILED. 160 passed; 2 failed; 10 ignored; 0 measured; 0 filtered out; finished in 43.16s

error: test failed, to rerun pass `-p calternal-plugin-mail --lib`

Known gaps: both new regressions remain unfixed because this is an independent
review. Queued flags/read/delete were checked at the durable journal boundary
and by existing wire-delivery tests; the new real TLS probe exercises dispatched
Move recovery. Search checks drain the durable FTS queue; indexing is asynchronous
and reports pending status. No large-mailbox performance or native client run
was part of this safety review. The two visibility regressions model the durable
dispatch state; the separate TLS probe performed a real COPY. Initial TLS fixture
attempts stopped at unsupported INBOX rename and a Dovecot cached-index error;
reconnecting around resets made the completed run pass.

Decisions: no new production design decisions. A failed Move must expose its
surviving provider source while retaining its recovery receipt. Dispatch cannot
hide another UID that shares an identity. These follow this job's explicit
requirements and DESIGN §53; both are asserted without weakening existing tests.

For the merge round: fix the two recorded defects, then run
cargo test -p calternal-plugin-mail -- --test-threads=4 and verify both new
regressions pass. No requested review scenario was deferred.

Cleanup (verbatim):

     Removed 3968 files, 2.1GiB total

Working tree is clean. Passing scratch includes were removed. Doc comments were
re-read. The local provider is stopped and its generated keys/data are removed.
No web build output was created.

BLOCK — failed Move hides a surviving source indefinitely; dispatched Move hides an unselected duplicate.

Independent mailrev4 data-safety review — #1340 Verdict: BLOCK. Two new reader consistency defects remain. Built: independent review probes; committed only the two failing regressions, one concern per commit. No production change, push or deploy. Author reports and result files were not read. Files: crates/plugins/mail/src/epoch_tests.rs (committed). Local evidence: artifacts/mailrev4-findings.md, mailrev4-probes.rs, mailrev4-scratch.rs, mailrev4-probe-output.txt, mailrev4-tls-output.txt, mailrev4-cleanup-strengthened.txt and mailrev4-cargo-test.txt. Commits: 9aabc56e5 (failed Move source); 32625ada8 (unselected duplicate). Head: 32625ada8ae7b7b0c48e977e744ee4818c51e70c. Base with origin/dev: cc1af3c12b68ad26c8403adde1be39c096b9ddea. Required fetch/merge ran once: Already up to date. Blocking findings: - P1: failed dispatched Move hides a provider-confirmed surviving source forever. Queue Move for UID 9, dispatch, replace with unique UID 109, reach Jobs' real retry limit (dead), then sync UID 109 again. The receipt correctly remains, but INBOX has 0 messages instead of 1. The 0024 view ignores permanent Job failure; sync does not restore the source or wake the dead receipt. Regression: mailrev4_failed_dispatched_move_reappears_after_sync. - P2, blocking this reader-consistency review: dispatch hides an unselected old duplicate. Live UIDs 9 and 10 share one identity. Move one UID: the other stays visible while queued. Change only the receipt to dispatched: the other also vanishes (0 instead of 1), before any reset. The 0024 identity predicate applies to the original epoch without unique-membership proof. Regression: mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate. Passing evidence: - All 15 mailrev2/mailrev3 non-worker regressions passed in the full crate gate. Their parent tests ran real SIGKILL workers, including corrected text, bodies, attachments, thread bridges, list, Search, count and restart convergence. - New scratch matrices checked old/new duplicates 1/2, 2/1, 2/2 and 1/1 for flag/read/delete intent and queued Move: ambiguous identities stay pinned; unique identities map once; duplicate replay does not add intent. - Cleanup SIGKILL after staged JSON, membership and body deletes, before COMMIT and after COMMIT passed. Current staging still activates; live Seen state, unavailable accepted intent/body and the other User survive. Cleanup is idempotent after restart. Strengthened focused output: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 176 filtered out; finished in 4.24s - True 0022 and 0023 schemas with pending checkpoints and dispatched Moves upgraded to 0024, preserved bodies/source identity/low-water, activated, and survived restart and idempotent migration. No migration added by this review. - Real Dovecot TLS COPY dispatch and two actual source UIDVALIDITY resets: surviving identity refused deletion; the next complete empty epoch permitted receipt settlement; repeat recovery left one Archive message, no receipt and no operation keyword. Output: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 179 filtered out; finished in 2.86s Gates (verbatim): cargo fmt --check: exit 0, no output. cargo test -p calternal-plugin-mail -- --test-threads=4: ``` failures: search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync test result: FAILED. 160 passed; 2 failed; 10 ignored; 0 measured; 0 filtered out; finished in 43.16s error: test failed, to rerun pass `-p calternal-plugin-mail --lib` ``` Known gaps: both new regressions remain unfixed because this is an independent review. Queued flags/read/delete were checked at the durable journal boundary and by existing wire-delivery tests; the new real TLS probe exercises dispatched Move recovery. Search checks drain the durable FTS queue; indexing is asynchronous and reports pending status. No large-mailbox performance or native client run was part of this safety review. The two visibility regressions model the durable dispatch state; the separate TLS probe performed a real COPY. Initial TLS fixture attempts stopped at unsupported INBOX rename and a Dovecot cached-index error; reconnecting around resets made the completed run pass. Decisions: no new production design decisions. A failed Move must expose its surviving provider source while retaining its recovery receipt. Dispatch cannot hide another UID that shares an identity. These follow this job's explicit requirements and DESIGN §53; both are asserted without weakening existing tests. For the merge round: fix the two recorded defects, then run cargo test -p calternal-plugin-mail -- --test-threads=4 and verify both new regressions pass. No requested review scenario was deferred. Cleanup (verbatim): ``` Removed 3968 files, 2.1GiB total ``` Working tree is clean. Passing scratch includes were removed. Doc comments were re-read. The local provider is stopped and its generated keys/data are removed. No web build output was created. BLOCK — failed Move hides a surviving source indefinitely; dispatched Move hides an unselected duplicate.

kayg commented 2026-10-08 22:36:52 +00:00

Author

Owner

Copy link

Starting mail7g review-round-4 fixes on job/mail7-1340, base/head 32625ada8a. Read the contract and artifacts/mailrev4-findings.md. I will preserve all regression expectations, keep recovery receipts, and fix exact-source visibility and settlement. Required mail gates will run before the final report.

Starting mail7g review-round-4 fixes on job/mail7-1340, base/head 32625ada8ae7b7b0c48e977e744ee4818c51e70c. Read the contract and artifacts/mailrev4-findings.md. I will preserve all regression expectations, keep recovery receipts, and fix exact-source visibility and settlement. Required mail gates will run before the final report.

kayg commented 2026-10-08 22:40:24 +00:00

Author

Owner

Copy link

Finding: 0024's visibility predicate matches source_message_id without UID coordinates and has no settlement state. The IMAP SELECT path also hides source_uid independently of the shared view, so restoring the web view alone cannot restore original-epoch IMAP visibility. Fix uses separate temporary hidden coordinates, maps them only with one-to-one membership proof, and leaves dispatched receipt coordinates unchanged. A later live provider window clears the hide; Jobs dead transition and definite COPY NO/BAD clear it immediately. Migration 0025 adds the fields and upgrade probes cover 0022/0023/0024 pending checkpoints. Required origin/dev fetch/merge completed once: Already up to date. origin/dev Mail migrations end at 0020.

Finding: 0024's visibility predicate matches source_message_id without UID coordinates and has no settlement state. The IMAP SELECT path also hides source_uid independently of the shared view, so restoring the web view alone cannot restore original-epoch IMAP visibility. Fix uses separate temporary hidden coordinates, maps them only with one-to-one membership proof, and leaves dispatched receipt coordinates unchanged. A later live provider window clears the hide; Jobs dead transition and definite COPY NO/BAD clear it immediately. Migration 0025 adds the fields and upgrade probes cover 0022/0023/0024 pending checkpoints. Required origin/dev fetch/merge completed once: Already up to date. origin/dev Mail migrations end at 0020.

kayg commented 2026-10-08 22:51:00 +00:00

Author

Owner

Copy link

Recovery evidence: both committed mailrev4 regressions now pass. Focused mailrev2/mailrev3/mailrev4 run also passed real SIGKILL stages:

test result: ok. 19 passed; 0 failed; 3 ignored; 0 measured; 156 filtered out; finished in 9.11s

The round-4 cleanup and queued-duplicate probes passed with test-provider enabled:

test result: ok. 4 passed; 0 failed; 2 ignored; 0 measured; 175 filtered out; finished in 3.31s

A fresh native Dovecot TLS fixture then passed the existing real dispatched-COPY recovery probe:

real TLS dispatched Move: actual UIDVALIDITY reset, surviving identity refused, later complete absence settles, repeat recovery produces one destination
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 180 filtered out; finished in 1.82s

Initial fixture startup attempts failed on generated-key permissions and a missing login user in a different image; the first test invocation lacked the test-only CA hex setting and stopped with BlockedAddress before connecting. Corrected the fixture and invocation; no production network-policy exception was added. Temporary review includes will not be committed. Final mail gates are running.

Recovery evidence: both committed mailrev4 regressions now pass. Focused mailrev2/mailrev3/mailrev4 run also passed real SIGKILL stages: test result: ok. 19 passed; 0 failed; 3 ignored; 0 measured; 156 filtered out; finished in 9.11s The round-4 cleanup and queued-duplicate probes passed with test-provider enabled: test result: ok. 4 passed; 0 failed; 2 ignored; 0 measured; 175 filtered out; finished in 3.31s A fresh native Dovecot TLS fixture then passed the existing real dispatched-COPY recovery probe: real TLS dispatched Move: actual UIDVALIDITY reset, surviving identity refused, later complete absence settles, repeat recovery produces one destination test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 180 filtered out; finished in 1.82s Initial fixture startup attempts failed on generated-key permissions and a missing login user in a different image; the first test invocation lacked the test-only CA hex setting and stopped with BlockedAddress before connecting. Corrected the fixture and invocation; no production network-policy exception was added. Temporary review includes will not be committed. Final mail gates are running.

kayg commented 2026-10-08 23:00:13 +00:00

Author

Owner

Copy link

Full-suite finding fixed without changing any expectation: migration 0025 originally required core Jobs even when a caller opens only the Mail projection. Six existing migration/route/Search tests failed with no such table: main.jobs. The optional Mail failure trigger now registers once at queue/startup initialization, under the reserved writer, and backfills dead receipts there. Mail migrations remain independent of Jobs. Added a startup/repeat-startup regression for a receipt that failed before registration.

Corrected full suite:
test result: ok. 165 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 40.80s

New startup regression:
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 175 filtered out; finished in 0.43s

Atomic commits: 557945229 (exact visibility + upgrade/settlement tests), 1c4782253 (ambiguity matrix), 1d4a93a21 (independent schema and optional failure hook). Final full mail suite is now running twice on 1d4a93a21.

Full-suite finding fixed without changing any expectation: migration 0025 originally required core Jobs even when a caller opens only the Mail projection. Six existing migration/route/Search tests failed with `no such table: main.jobs`. The optional Mail failure trigger now registers once at queue/startup initialization, under the reserved writer, and backfills dead receipts there. Mail migrations remain independent of Jobs. Added a startup/repeat-startup regression for a receipt that failed before registration. Corrected full suite: test result: ok. 165 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 40.80s New startup regression: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 175 filtered out; finished in 0.43s Atomic commits: 557945229 (exact visibility + upgrade/settlement tests), 1c4782253 (ambiguity matrix), 1d4a93a21 (independent schema and optional failure hook). Final full mail suite is now running twice on 1d4a93a21.

kayg commented 2026-10-08 23:08:36 +00:00

Author

Owner

Copy link

READY FOR MERGE — mail7g review-round-4 repair, #1340

Built: temporary Move source visibility now uses exact membership coordinates, separate from immutable receipt coordinates. A later live provider window restores a surviving source. A terminal Job failure or definite COPY NO/BAD restores it immediately. Replacement visibility needs the selected old membership and unique identities on both sides. An unselected duplicate stays visible. Receipts stay durable and cannot authorize another COPY or deletion at replacement coordinates.

Files:

• crates/plugins/mail/migrations/0025_exact_move_visibility.sql
• crates/plugins/mail/src/cache/store.rs
• crates/plugins/mail/src/proxy.rs
• crates/plugins/mail/src/proxy_transfers.rs
• crates/plugins/mail/src/sync.rs
• crates/plugins/mail/src/epoch_tests.rs

Head: 7d847a42cd. Branch: job/mail7-1340. Start: 32625ada8a. Required fetch/merge of origin/dev ran once: Already up to date. Its Mail migrations ended at 0020. Migrations 0022–0024 and all existing test expectations are unchanged.

Atomic commits:

• 557945229: exact visibility, source settlement and 0022–0024 upgrade tests.
• 1c4782253: ambiguous and unselected replacement matrix.
• 1d4a93a21: keep standalone Mail migrations independent of core Jobs; register the optional failure hook at runtime.
• 7d847a42c: keep hook registration separate from epoch cleanup's COMMIT.

Findings fixed: the first full suite found six existing standalone-schema tests failing with no such table: main.jobs. Registration moved out of migration 0025. No expectation changed. Final review also found that registration inside epoch cleanup would add an early COMMIT before cleanup; runtime startup now registers first in its own transaction. The unchanged round-4 cleanup SIGKILL probe passes all five checkpoints.

Recovery evidence on final code:

• Both named mailrev4 regressions pass in both final full suites below.
• All mailrev2/mailrev3 regressions and their subprocess SIGKILL workers pass.
• New tests cover original-epoch restoration on sync and terminal failure, 0022/0023/0024 pending upgrades and repeated opens, ambiguous old/new identities, disappearance of the selected old UID while an unselected duplicate survives, and repeated startup restoration of a receipt that failed before hook installation.
• Round-4 cleanup SIGKILL probe:

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 181 filtered out; finished in 4.89s

• Fresh native Dovecot TLS probe performed real marked COPY, two real UIDVALIDITY resets, refused replacement-source deletion, settled only after complete source absence, and left one Archive message with no operation marker. No receipt replay:

test search::epoch_tests::mailrev4_real_tls_dispatched_move_settles_after_absent_sync ... real TLS dispatched Move: actual UIDVALIDITY reset, surviving identity refused, later complete absence settles, repeat recovery produces one destination
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 181 filtered out; finished in 2.62s

Gate output (verbatim). cargo fmt --check exited 0 with no output.

cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings:

    Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 23.95s

cargo test -p calternal-plugin-mail -- --test-threads=4, run 1:

   Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 44.84s
     Running unittests src/lib.rs (/home/kayg/build/targets/mail7g/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 176 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok
test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok
test search::epoch_tests::mail7f_dispatched_move_readers_and_absence_proof ... ok
test search::epoch_tests::mail7f_receipt_hydration_keeps_original_move_source_hidden ... ok
test search::epoch_tests::mail7f_startup_collects_only_abandoned_epoch_content ... ok
test search::epoch_tests::mail7g_startup_restores_dead_receipt_before_failure_hook ... ok
test search::epoch_tests::mail7g_original_source_visibility_settles_without_losing_receipt ... ok
test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok
test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test search::epoch_tests::mail7g_dispatched_visibility_requires_selected_unique_membership ... ok
test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok
test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok
test search::epoch_tests::mail7g_upgrade_0025_pending_from_0022_through_0024 ... ok
test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok
test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok
test search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets ... ok
test search::epoch_tests::mailrev3_changed_crash_worker ... ignored, subprocess for mailrev3 changed-content SIGKILL probe
test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok
test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok
test search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden ... ok
test search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy ... ok
test search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate ... ok
test search::epoch_tests::mailrev3_upgrade_0022_pending_resume ... ok
test search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test search::epoch_tests::mailrev3_sigkill_changed_content_reads ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 166 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 72.43s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-plugin-mail -- --test-threads=4, run 2:

    Blocking waiting for file lock on package cache
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1.53s
     Running unittests src/lib.rs (/home/kayg/build/targets/mail7g/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 176 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok
test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok
test search::epoch_tests::mail7f_dispatched_move_readers_and_absence_proof ... ok
test search::epoch_tests::mail7f_receipt_hydration_keeps_original_move_source_hidden ... ok
test search::epoch_tests::mail7f_startup_collects_only_abandoned_epoch_content ... ok
test search::epoch_tests::mail7g_startup_restores_dead_receipt_before_failure_hook ... ok
test search::epoch_tests::mail7g_original_source_visibility_settles_without_losing_receipt ... ok
test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok
test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test search::epoch_tests::mail7g_dispatched_visibility_requires_selected_unique_membership ... ok
test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok
test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok
test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok
test search::epoch_tests::mail7g_upgrade_0025_pending_from_0022_through_0024 ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok
test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok
test search::epoch_tests::mailrev3_changed_crash_worker ... ignored, subprocess for mailrev3 changed-content SIGKILL probe
test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok
test search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets ... ok
test search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden ... ok
test search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy ... ok
test search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate ... ok
test search::epoch_tests::mailrev3_upgrade_0022_pending_resume ... ok
test search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test search::epoch_tests::mailrev3_sigkill_changed_content_reads ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 166 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 66.88s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

UX gaps closed: a failed or provider-confirmed surviving Move source becomes readable again; dispatch cannot hide an unselected duplicate. IMAP uses the same visibility view as web reads.

Known gaps / UX gaps left: retained receipts keep the existing source-edit barrier until recovery; this repair preserves delivery ordering. A dedicated wire NO/BAD-rejection probe was not added; the real TLS probe covers receipt recovery. No new large-mailbox timing or native Apple Mail session was run. Initial fixture attempts failed on generated-key permissions, a missing login user in another image, and a missing test-only CA setting (BlockedAddress before connection); the corrected final fixture passes. No production network-policy exception was added.

Decisions: use separate nullable visibility coordinates instead of changing wire receipt coordinates. Keep the first replacement activation hidden when the outcome is unknown; a later live observation settles it. Install the optional core-Jobs hook at startup/enqueue, not in an independent Mail migration. Keep its writer transaction separate from epoch cleanup. These are implementation choices for the explicit #1340 rules and DESIGN §53; no new product behavior was invented.

Expected cost: one batched UID observation per live fetch window; receipt lookups use the owner/folder index and the existing 1,024-operation queue limit. Epoch identity probes stop at two matches on each side. No per-message IO, provider call under a writer transaction, or full-Home request scan was added. Existing bench/mail-sync.py profiles remain available; no new operation or route was added.

For the merge round: a fresh performance comparison can use the existing release test binary on the perf VM:

perf-run calternal mail7g-1340 10 -- python3 bench/mail-sync.py <prebuilt-release-mail-test-binary> --test profile_full_history --profile-prefix 'MAIL_HISTORY_PROFILE ' --mailbox-messages 10000 --replacement

It must preserve the exact history and FTS counts and compare replacement timings with the prior #1340 results. No requested recovery scenario was deferred.

Cleanup (verbatim):

     Removed 7098 files, 2.6GiB total

The generated provider and its keys/data are removed. No web build output was created. Review artifacts and temporary probe includes were not committed. Changed module/function comments were re-read. Working tree is clean. No push, deploy, merge into dev, or issue close was done.

7d847a42cd — both tests passing.

READY FOR MERGE — mail7g review-round-4 repair, #1340 Built: temporary Move source visibility now uses exact membership coordinates, separate from immutable receipt coordinates. A later live provider window restores a surviving source. A terminal Job failure or definite COPY NO/BAD restores it immediately. Replacement visibility needs the selected old membership and unique identities on both sides. An unselected duplicate stays visible. Receipts stay durable and cannot authorize another COPY or deletion at replacement coordinates. Files: - crates/plugins/mail/migrations/0025_exact_move_visibility.sql - crates/plugins/mail/src/cache/store.rs - crates/plugins/mail/src/proxy.rs - crates/plugins/mail/src/proxy_transfers.rs - crates/plugins/mail/src/sync.rs - crates/plugins/mail/src/epoch_tests.rs Head: 7d847a42cd8cc1d3f2dba22b57e5a36de393a24d. Branch: job/mail7-1340. Start: 32625ada8ae7b7b0c48e977e744ee4818c51e70c. Required fetch/merge of origin/dev ran once: Already up to date. Its Mail migrations ended at 0020. Migrations 0022–0024 and all existing test expectations are unchanged. Atomic commits: - 557945229: exact visibility, source settlement and 0022–0024 upgrade tests. - 1c4782253: ambiguous and unselected replacement matrix. - 1d4a93a21: keep standalone Mail migrations independent of core Jobs; register the optional failure hook at runtime. - 7d847a42c: keep hook registration separate from epoch cleanup's COMMIT. Findings fixed: the first full suite found six existing standalone-schema tests failing with `no such table: main.jobs`. Registration moved out of migration 0025. No expectation changed. Final review also found that registration inside epoch cleanup would add an early COMMIT before cleanup; runtime startup now registers first in its own transaction. The unchanged round-4 cleanup SIGKILL probe passes all five checkpoints. Recovery evidence on final code: - Both named mailrev4 regressions pass in both final full suites below. - All mailrev2/mailrev3 regressions and their subprocess SIGKILL workers pass. - New tests cover original-epoch restoration on sync and terminal failure, 0022/0023/0024 pending upgrades and repeated opens, ambiguous old/new identities, disappearance of the selected old UID while an unselected duplicate survives, and repeated startup restoration of a receipt that failed before hook installation. - Round-4 cleanup SIGKILL probe: ``` test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 181 filtered out; finished in 4.89s ``` - Fresh native Dovecot TLS probe performed real marked COPY, two real UIDVALIDITY resets, refused replacement-source deletion, settled only after complete source absence, and left one Archive message with no operation marker. No receipt replay: ``` test search::epoch_tests::mailrev4_real_tls_dispatched_move_settles_after_absent_sync ... real TLS dispatched Move: actual UIDVALIDITY reset, surviving identity refused, later complete absence settles, repeat recovery produces one destination test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 181 filtered out; finished in 2.62s ``` Gate output (verbatim). cargo fmt --check exited 0 with no output. cargo clippy -p calternal-plugin-mail --all-targets -- -D warnings: ``` Checking calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `dev` profile [unoptimized + debuginfo] target(s) in 23.95s ``` cargo test -p calternal-plugin-mail -- --test-threads=4, run 1: ``` Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `test` profile [unoptimized + debuginfo] target(s) in 44.84s Running unittests src/lib.rs (/home/kayg/build/targets/mail7g/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 176 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test routes::tests::read_state_is_journaled_without_provider_access ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok test search::epoch_tests::mail7f_dispatched_move_readers_and_absence_proof ... ok test search::epoch_tests::mail7f_receipt_hydration_keeps_original_move_source_hidden ... ok test search::epoch_tests::mail7f_startup_collects_only_abandoned_epoch_content ... ok test search::epoch_tests::mail7g_startup_restores_dead_receipt_before_failure_hook ... ok test search::epoch_tests::mail7g_original_source_visibility_settles_without_losing_receipt ... ok test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test search::epoch_tests::epoch_sigkill_stages_converge ... ok test search::epoch_tests::mail7g_dispatched_visibility_requires_selected_unique_membership ... ok test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok test search::epoch_tests::mail7g_upgrade_0025_pending_from_0022_through_0024 ... ok test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok test search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets ... ok test search::epoch_tests::mailrev3_changed_crash_worker ... ignored, subprocess for mailrev3 changed-content SIGKILL probe test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok test search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden ... ok test search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy ... ok test search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate ... ok test search::epoch_tests::mailrev3_upgrade_0022_pending_resume ... ok test search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test search::epoch_tests::mailrev3_sigkill_changed_content_reads ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 166 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 72.43s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` cargo test -p calternal-plugin-mail -- --test-threads=4, run 2: ``` Blocking waiting for file lock on package cache Finished `test` profile [unoptimized + debuginfo] target(s) in 1.53s Running unittests src/lib.rs (/home/kayg/build/targets/mail7g/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 176 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::read_state_is_journaled_without_provider_access ... ok test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok test search::epoch_tests::mail7f_dispatched_move_readers_and_absence_proof ... ok test search::epoch_tests::mail7f_receipt_hydration_keeps_original_move_source_hidden ... ok test search::epoch_tests::mail7f_startup_collects_only_abandoned_epoch_content ... ok test search::epoch_tests::mail7g_startup_restores_dead_receipt_before_failure_hook ... ok test search::epoch_tests::mail7g_original_source_visibility_settles_without_losing_receipt ... ok test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test search::epoch_tests::mail7g_dispatched_visibility_requires_selected_unique_membership ... ok test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok test search::epoch_tests::epoch_sigkill_stages_converge ... ok test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok test search::epoch_tests::mail7g_upgrade_0025_pending_from_0022_through_0024 ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok test search::epoch_tests::mailrev3_changed_crash_worker ... ignored, subprocess for mailrev3 changed-content SIGKILL probe test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok test search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets ... ok test search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden ... ok test search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy ... ok test search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate ... ok test search::epoch_tests::mailrev3_upgrade_0022_pending_resume ... ok test search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test search::epoch_tests::mailrev3_sigkill_changed_content_reads ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 166 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 66.88s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` UX gaps closed: a failed or provider-confirmed surviving Move source becomes readable again; dispatch cannot hide an unselected duplicate. IMAP uses the same visibility view as web reads. Known gaps / UX gaps left: retained receipts keep the existing source-edit barrier until recovery; this repair preserves delivery ordering. A dedicated wire NO/BAD-rejection probe was not added; the real TLS probe covers receipt recovery. No new large-mailbox timing or native Apple Mail session was run. Initial fixture attempts failed on generated-key permissions, a missing login user in another image, and a missing test-only CA setting (BlockedAddress before connection); the corrected final fixture passes. No production network-policy exception was added. Decisions: use separate nullable visibility coordinates instead of changing wire receipt coordinates. Keep the first replacement activation hidden when the outcome is unknown; a later live observation settles it. Install the optional core-Jobs hook at startup/enqueue, not in an independent Mail migration. Keep its writer transaction separate from epoch cleanup. These are implementation choices for the explicit #1340 rules and DESIGN §53; no new product behavior was invented. Expected cost: one batched UID observation per live fetch window; receipt lookups use the owner/folder index and the existing 1,024-operation queue limit. Epoch identity probes stop at two matches on each side. No per-message IO, provider call under a writer transaction, or full-Home request scan was added. Existing bench/mail-sync.py profiles remain available; no new operation or route was added. For the merge round: a fresh performance comparison can use the existing release test binary on the perf VM: ``` perf-run calternal mail7g-1340 10 -- python3 bench/mail-sync.py <prebuilt-release-mail-test-binary> --test profile_full_history --profile-prefix 'MAIL_HISTORY_PROFILE ' --mailbox-messages 10000 --replacement ``` It must preserve the exact history and FTS counts and compare replacement timings with the prior #1340 results. No requested recovery scenario was deferred. Cleanup (verbatim): ``` Removed 7098 files, 2.6GiB total ``` The generated provider and its keys/data are removed. No web build output was created. Review artifacts and temporary probe includes were not committed. Changed module/function comments were re-read. Working tree is clean. No push, deploy, merge into dev, or issue close was done. 7d847a42cd8cc1d3f2dba22b57e5a36de393a24d — both tests passing.

kayg commented 2026-10-08 23:10:06 +00:00

Author

Owner

Copy link

Independent mailrev5 review started on job/mail7-1340, base/head 7d847a42cd. Review scope: migration 0025 exact temporary Move hides and mailrev2/3/4 regressions. Author reports and result files will not be read. Findings: artifacts/mailrev5-findings.md. No production changes; commit only a regression that exposes a defect.

Independent mailrev5 review started on job/mail7-1340, base/head 7d847a42cd8cc1d3f2dba22b57e5a36de393a24d. Review scope: migration 0025 exact temporary Move hides and mailrev2/3/4 regressions. Author reports and result files will not be read. Findings: artifacts/mailrev5-findings.md. No production changes; commit only a regression that exposes a defect.

kayg commented 2026-10-08 23:20:27 +00:00

Author

Owner

Copy link

Independent mailrev5 evidence on head 7d847a42cd: both round-4 blockers now pass, together with all committed mailrev2/3/4 non-worker regressions so far. New local probes passed SIGKILL before/after exact-hide settlement COMMIT, restart with a pending hide, a second UIDVALIDITY change during replacement, dead-Move upgrades from actual 0022/0023/0024 schemas, and simultaneous owner-bound hides for Users with overlapping UIDs/account labels. Receipt coordinates remain immutable; failed/sync-observed sources become visible without hiding another User. Earlier round-4 scratch duplicate and cleanup-SIGKILL probes also pass unchanged. No blocker found so far. The 10,000-row staging regression and TLS NO/BAD/EOF probe remain pending. Evidence is appended to artifacts/mailrev5-findings.md. No production changes or commits.

Independent mailrev5 evidence on head 7d847a42cd8cc1d3f2dba22b57e5a36de393a24d: both round-4 blockers now pass, together with all committed mailrev2/3/4 non-worker regressions so far. New local probes passed SIGKILL before/after exact-hide settlement COMMIT, restart with a pending hide, a second UIDVALIDITY change during replacement, dead-Move upgrades from actual 0022/0023/0024 schemas, and simultaneous owner-bound hides for Users with overlapping UIDs/account labels. Receipt coordinates remain immutable; failed/sync-observed sources become visible without hiding another User. Earlier round-4 scratch duplicate and cleanup-SIGKILL probes also pass unchanged. No blocker found so far. The 10,000-row staging regression and TLS NO/BAD/EOF probe remain pending. Evidence is appended to artifacts/mailrev5-findings.md. No production changes or commits.

kayg commented 2026-10-08 23:25:42 +00:00

Author

Owner

Copy link

Independent mailrev5 data-safety review — #1340

Verdict: READY. No data-loss, wrong-message, cross-User or permanent-hide defect found in the migration 0025 change.

Built: local independent review probes only. No production edits or new commits. Passing tests remain uncommitted, as the review prompt requires. Author reports and result files were not read.

Head: 7d847a42cd
Branch: job/mail7-1340. Required fetch/merge ran once: Already up to date. Working tree is clean.

Evidence:

• All enabled mailrev2/mailrev3/mailrev4 regressions passed, including both prior round-4 blockers. Parent SIGKILL tests drove their ignored subprocess workers. Also reran earlier local round-4 bidirectional duplicates, queued Move, pending upgrades and cleanup SIGKILL tests, and the 10,000-row round-3 staging regression.
• New SIGKILL tests before and after live settlement COMMIT: rollback keeps the pending hide; committed settlement restores its source after restart. IMAP, web list, folder count and Search agree; receipt fields do not change.
• Restart during a pending hide, then a second UIDVALIDITY change: visibility maps to the unique current membership while receipt coordinates remain generation 1 / UID 9 / epoch 77. Later live sync clears the hide; a third replacement does not revive it.
• True 0022, 0023 and 0024 schemas with dead dispatched Moves upgrade to 0025 and restore their sources through runtime initialization, retaining receipts. Existing pending-checkpoint upgrade tests also pass.
• Two Users with the same account labels and overlapping remote/virtual UIDs each queue a Move. Failure of Alice's Job restores Alice only; Bob's hide and destination remain intact.
• Real TLS protocol peer drives production TransferJob through UID COPY NO, BAD and lost-response EOF. NO/BAD immediately clear only the exact source hide. EOF retains it until source sync. Restart preserves each result. Six connections record COPY counts 1,0,1,0,1,0: each retry searches and never copies again.

Probe output (verbatim):

test result: ok. 175 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 140.81s
test result: ok. 5 passed; 0 failed; 1 ignored; 0 measured; 185 filtered out; finished in 5.60s

Files: artifacts/mailrev5-findings.md, mailrev5-scratch.rs, mailrev5-provider.py, mailrev5-initial-gate.txt, mailrev5-tls-and-probes.txt, mailrev5-provider-output.txt, mailrev5-cargo-test.txt, mailrev5-fmt.txt, mailrev5-clean.txt. Temporary epoch_tests.rs includes were removed; no tracked file differs from HEAD.

Known gaps: the normal crate gate leaves 10 opt-in provider/profile tests ignored. This review does not establish native-client acceptance or the 7-second performance target. The isolated TLS peer checks rejection and transport-loss behavior; the older optional Dovecot acceptance probe was not rerun. The first TLS setup used a CA certificate as its leaf and failed before dispatch; separate CA-signed server leaf fixed the fixture. Its failed output is retained in mailrev5-tls-fixture-failure.txt and mailrev5-provider-fixture-failure.txt. No existing assertion was weakened.

Decisions: no production design decisions. Reused the existing two-User Index fixture and test-provider trust hook. Used local TLS fault responses to exercise the production worker without a live external account.

Gates: cargo fmt --check exited 0 with no output. Final cargo test -p calternal-plugin-mail -- --test-threads=4 output follows verbatim, after removal of local includes:

   Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 24.54s
     Running unittests src/lib.rs (/home/kayg/build/targets/mailrev5/debug/deps/calternal_plugin_mail-a3120b18c18525f8)

running 176 tests
test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok
test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok
test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok
test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok
test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok
test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok
test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok
test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok
test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok
test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator
test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok
test cache::store::tests::migration_creates_a_rebuildable_projection ... ok
test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok
test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok
test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok
test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok
test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok
test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok
test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok
test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok
test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok
test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok
test html::tests::body_root_attributes_survive_for_sender_selectors ... ok
test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok
test html::tests::css_selectors_keep_child_combinators ... ok
test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok
test html::tests::html_and_css_work_are_bounded ... ok
test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok
test html::tests::links_are_extracted_for_explicit_safe_controls ... ok
test html::tests::measurement_images_are_removed_before_prefetch ... ok
test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok
test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok
test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok
test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok
test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok
test html::tests::ten_megabyte_html_stops_before_parsing ... ok
test imap::tests::endpoint_normalization_pins_host_identity ... ok
test imap::tests::login_disabled_capability_prevents_sending_password ... ok
test imap::tests::login_rejections_map_to_fixed_errors ... ok
test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok
test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok
test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok
test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok
test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok
test imap::tests::seen_change_writes_all_duplicate_uids ... ok
test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok
test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok
test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok
test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok
test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok
test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok
test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok
test proxy::tests::list_identifies_unified_special_use_destinations ... ok
test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok
test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok
test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok
test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile
test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok
test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok
test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok
test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok
test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok
test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok
test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok
test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok
test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok
test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok
test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok
test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok
test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok
test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok
test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok
test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok
test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok
test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok
test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok
test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok
test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok
test proxy::tests::user_intents_wait_for_writer_before_reading ... ok
test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok
test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok
test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok
test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok
test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok
test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok
test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok
test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok
test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok
test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok
test remote::tests::reclamation_retains_capacity_for_new_mail ... ok
test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok
test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok
test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok
test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok
test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok
test routes::tests::account_routes_require_a_data_scoped_user ... ok
test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok
test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok
test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok
test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok
test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok
test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok
test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok
test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok
test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test routes::tests::read_state_is_journaled_without_provider_access ... ok
test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok
test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok
test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok
test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok
test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok
test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok
test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok
test search::epoch_tests::mail7f_dispatched_move_readers_and_absence_proof ... ok
test search::epoch_tests::mail7f_receipt_hydration_keeps_original_move_source_hidden ... ok
test search::epoch_tests::mail7f_startup_collects_only_abandoned_epoch_content ... ok
test search::epoch_tests::mail7g_startup_restores_dead_receipt_before_failure_hook ... ok
test search::epoch_tests::mail7g_original_source_visibility_settles_without_losing_receipt ... ok
test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok
test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression
test search::epoch_tests::mail7g_dispatched_visibility_requires_selected_unique_membership ... ok
test search::epoch_tests::epoch_sigkill_stages_converge ... ok
test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok
test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok
test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok
test search::epoch_tests::mail7g_upgrade_0025_pending_from_0022_through_0024 ... ok
test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok
test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok
test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok
test search::epoch_tests::mailrev3_changed_crash_worker ... ignored, subprocess for mailrev3 changed-content SIGKILL probe
test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok
test search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets ... ok
test search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden ... ok
test search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy ... ok
test search::epoch_tests::mailrev3_upgrade_0022_pending_resume ... ok
test search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate ... ok
test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok
test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test
test search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync ... ok
test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok
test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok
test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok
test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok
test search::epoch_tests::mailrev3_sigkill_changed_content_reads ... ok
test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok
test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok
test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok
test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok
test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok
test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok
test sync::tests::incomplete_backfill_cannot_commit_success ... ok
test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok
test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok
test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok
test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test
test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok
test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340
test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile
test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok
test sync::tests::manual_sync_keeps_one_listener_per_account ... ok
test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok
test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok
test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok
test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok
test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py
test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok
test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok
test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok
test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok
test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok
test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok
test sync::tests::truncation_keeps_utf8_boundaries ... ok
test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok
test sync::tests::provider_fetch_list_stores_all_messages ... ok
test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok

test result: ok. 166 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 28.86s

   Doc-tests calternal_plugin_mail

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Cleanup (verbatim):

     Removed 3968 files, 2.1GiB total

The generated TLS keys/data are removed; the provider exited; no web build output was created.

READY

Independent mailrev5 data-safety review — #1340 Verdict: READY. No data-loss, wrong-message, cross-User or permanent-hide defect found in the migration 0025 change. Built: local independent review probes only. No production edits or new commits. Passing tests remain uncommitted, as the review prompt requires. Author reports and result files were not read. Head: 7d847a42cd8cc1d3f2dba22b57e5a36de393a24d Branch: job/mail7-1340. Required fetch/merge ran once: Already up to date. Working tree is clean. Evidence: - All enabled mailrev2/mailrev3/mailrev4 regressions passed, including both prior round-4 blockers. Parent SIGKILL tests drove their ignored subprocess workers. Also reran earlier local round-4 bidirectional duplicates, queued Move, pending upgrades and cleanup SIGKILL tests, and the 10,000-row round-3 staging regression. - New SIGKILL tests before and after live settlement COMMIT: rollback keeps the pending hide; committed settlement restores its source after restart. IMAP, web list, folder count and Search agree; receipt fields do not change. - Restart during a pending hide, then a second UIDVALIDITY change: visibility maps to the unique current membership while receipt coordinates remain generation 1 / UID 9 / epoch 77. Later live sync clears the hide; a third replacement does not revive it. - True 0022, 0023 and 0024 schemas with dead dispatched Moves upgrade to 0025 and restore their sources through runtime initialization, retaining receipts. Existing pending-checkpoint upgrade tests also pass. - Two Users with the same account labels and overlapping remote/virtual UIDs each queue a Move. Failure of Alice's Job restores Alice only; Bob's hide and destination remain intact. - Real TLS protocol peer drives production TransferJob through UID COPY NO, BAD and lost-response EOF. NO/BAD immediately clear only the exact source hide. EOF retains it until source sync. Restart preserves each result. Six connections record COPY counts 1,0,1,0,1,0: each retry searches and never copies again. Probe output (verbatim): ``` test result: ok. 175 passed; 0 failed; 12 ignored; 0 measured; 0 filtered out; finished in 140.81s test result: ok. 5 passed; 0 failed; 1 ignored; 0 measured; 185 filtered out; finished in 5.60s ``` Files: artifacts/mailrev5-findings.md, mailrev5-scratch.rs, mailrev5-provider.py, mailrev5-initial-gate.txt, mailrev5-tls-and-probes.txt, mailrev5-provider-output.txt, mailrev5-cargo-test.txt, mailrev5-fmt.txt, mailrev5-clean.txt. Temporary epoch_tests.rs includes were removed; no tracked file differs from HEAD. Known gaps: the normal crate gate leaves 10 opt-in provider/profile tests ignored. This review does not establish native-client acceptance or the 7-second performance target. The isolated TLS peer checks rejection and transport-loss behavior; the older optional Dovecot acceptance probe was not rerun. The first TLS setup used a CA certificate as its leaf and failed before dispatch; separate CA-signed server leaf fixed the fixture. Its failed output is retained in mailrev5-tls-fixture-failure.txt and mailrev5-provider-fixture-failure.txt. No existing assertion was weakened. Decisions: no production design decisions. Reused the existing two-User Index fixture and test-provider trust hook. Used local TLS fault responses to exercise the production worker without a live external account. Gates: cargo fmt --check exited 0 with no output. Final cargo test -p calternal-plugin-mail -- --test-threads=4 output follows verbatim, after removal of local includes: ``` Compiling calternal-plugin-mail v0.0.1 (/home/kayg/Developer/calternal-wt/mail7-1340/crates/plugins/mail) Finished `test` profile [unoptimized + debuginfo] target(s) in 24.54s Running unittests src/lib.rs (/home/kayg/build/targets/mailrev5/debug/deps/calternal_plugin_mail-a3120b18c18525f8) running 176 tests test cache::store::tests::attachment_metadata_has_bounded_display_and_transfer_values ... ok test cache::store::tests::duplicate_uid_migration_preserves_existing_memberships ... ok test cache::store::tests::delta_arrivals_emit_once_without_body_or_backfill ... ok test cache::store::tests::duplicate_uids_share_one_message_and_flag_all_copies ... ok test cache::store::tests::expunge_cleanup_is_bounded_resumable_and_blocks_new_sweeps ... ok test cache::store::tests::expunge_staging_rejects_a_short_provider_page_before_pruning ... ok test cache::store::tests::expunge_validation_keeps_the_scan_when_only_new_uids_arrive ... ok test cache::store::tests::folder_page_keys_use_covering_indexes_without_a_sort ... ok test cache::store::tests::expunged_duplicate_uid_keeps_source_message_after_move ... ok test cache::store::tests::mail_folder_page_profile_200k ... ignored, run via bench/mail-folder-page.py on the HDD emulator test cache::store::tests::folder_page_metadata_tracks_message_refreshes ... ok test cache::store::tests::migration_creates_a_rebuildable_projection ... ok test cache::store::tests::page_cursor_does_not_skip_the_first_unreturned_message ... ok test cache::store::tests::incomplete_replacement_epoch_keeps_live_messages_until_verified ... ok test cache::store::tests::provider_sync_publishes_mail_folder_invalidation_after_commit ... ok test cache::store::tests::read_state_change_publishes_mail_folder_invalidation ... ok test cache::store::tests::folder_page_work_stays_within_the_indexed_window ... ok test cache::store::tests::references_merge_partial_threads_and_preserve_old_links ... ok test cache::store::tests::repeated_sync_preserves_only_the_same_prepared_body ... ok test crypto::tests::debug_output_never_contains_plaintext_or_ciphertext ... ok test crypto::tests::encrypts_and_binds_the_password_to_its_user_and_account ... ok test crypto::tests::rejects_zero_keys_and_invalid_nonces ... ok test html::tests::body_root_attributes_survive_for_sender_selectors ... ok test html::tests::css_safety_removes_active_properties_and_retains_safe_siblings ... ok test html::tests::css_selectors_keep_child_combinators ... ok test html::tests::hostile_mail_cannot_request_resources_or_run_code ... ok test html::tests::html_and_css_work_are_bounded ... ok test html::tests::image_sources_require_embedded_rasters_or_prefetched_references ... ok test html::tests::links_are_extracted_for_explicit_safe_controls ... ok test html::tests::measurement_images_are_removed_before_prefetch ... ok test html::tests::mime_html_part_is_decoded_before_sanitizing ... ok test html::tests::nested_charset_and_css_payload_does_not_escape_sanitizer ... ok test html::tests::newsletter_layout_survives_without_direct_network_sources ... ok test html::tests::newsletters_preserve_their_tables_images_and_conditional_styles ... ok test html::tests::sender_fonts_use_cached_sources_without_local_probes ... ok test html::tests::ten_megabyte_html_stops_before_parsing ... ok test imap::tests::endpoint_normalization_pins_host_identity ... ok test imap::tests::login_disabled_capability_prevents_sending_password ... ok test imap::tests::login_rejections_map_to_fixed_errors ... ok test imap::tests::mail_resolver_accepts_only_listed_private_networks ... ok test imap::tests::password_login_happens_after_a_secure_transport_is_supplied ... ok test imap::tests::review_1160_pinned_socket_refuses_starttls_downgrade ... ok test imap::tests::review_1160_pinning_preserves_tls_hostname_and_requires_tls ... ok test imap::tests::review_1160_plaintext_transport_is_not_configurable ... ok test imap::tests::seen_change_writes_all_duplicate_uids ... ok test imap::tls_provider_tests::tls_connector_builds_without_a_process_default_provider ... ok test cache::store::tests::windows_commit_identity_membership_and_cursor_together ... ok test proxy::tests::full_body_without_tagged_completion_is_rejected ... ok test proxy::tests::concurrent_select_and_flag_refresh_keep_stable_uid_revisions ... ok test cache::store::tests::window_waits_for_writer_before_reading_generation ... ok test proxy::tests::invalid_flag_intent_does_not_queue_a_job ... ok test proxy::tests::large_message_metadata_does_not_fetch_or_cache_full_mime ... ok test proxy::tests::list_identifies_unified_special_use_destinations ... ok test proxy::tests::metadata_refuses_an_inconsistent_advertised_size ... ok test proxy::tests::overlapping_views_keep_modseq_after_flags_return_to_the_original_value ... ok test proxy::tests::pending_epoch_rebuild_keeps_live_view_and_cached_body ... ok test proxy::tests::profile_mail_proxy ... ignored, manual Mail cache profile test proxy::tests::populated_local_listener_accepts_store_fetch_and_selective_expunge ... ok test proxy::tests::provider_epoch_reset_allocates_new_virtual_uid ... ok test proxy::tests::queued_expunge_requires_current_deleted_intent ... ok test proxy::tests::queued_flags_survive_restart_and_condstore_uses_committed_revision ... ok test proxy::tests::multibyte_flags_obey_the_metadata_byte_limit ... ok test proxy::tests::raw_cache_and_disabled_accounts_do_not_bypass_owner_checks ... ok test proxy::tests::restored_account_requires_a_new_mailbox_epoch ... ok test proxy::tests::sync_preserves_queued_flags_without_restoring_stale_state ... ok test proxy::tests::unchanged_select_does_not_wait_for_index_or_security_writer ... ok test proxy::tests::unified_inbox_is_owner_scoped_and_uid_stable_across_sessions ... ok test proxy_mutations::tests::observed_recent_does_not_relax_client_flag_validation ... ok test proxy::tests::unsupported_selective_deletion_keeps_the_message_visible ... ok test proxy_mutations::tests::wire_delivery_is_uid_scoped_and_checks_epoch_and_rejection ... ok test cache::store::tests::sync_stages_reserve_writer_before_reading ... ok test proxy_mutations::tests::unavailable_sources_keep_the_durable_intent_for_retry ... ok test proxy_transfers::tests::copy_retains_preceding_offline_flags ... ok test proxy_transfers::tests::move_upgrades_pending_copy_without_a_second_destination ... ok test proxy_transfers::tests::copy_then_delete_accepts_pipelined_commands_and_defers_delivery ... ok test proxy_transfers::tests::imported_recent_is_not_captured_in_transfer_intent ... ok test proxy_transfers::tests::offline_move_is_durable_readable_and_blocks_conflicting_writes ... ok test proxy_transfers::tests::pending_copy_of_existing_message_keeps_one_web_row ... ok test proxy::tests::user_intents_wait_for_writer_before_reading ... ok test proxy_transfers::tests::pending_destination_edits_project_immediately ... ok test proxy_transfers::tests::uncertain_dispatch_never_replays_non_idempotent_commands ... ok test proxy_transfers::tests::raw_draft_routes_from_and_survives_restart_before_delivery ... ok test proxy_transfers::wire_tests::interrupted_draft_resolution_checks_complete_exact_metadata ... ok test proxy_transfers::wire_tests::marker_receipt_requires_epoch_and_persistent_keyword_evidence ... ok test remote::tests::cached_sender_fonts_prepare_and_hydrate_under_account_access ... ok test remote::tests::deliveries_keep_a_successor_behind_an_already_running_batch ... ok test remote::tests::only_bounded_visible_rasters_become_inline_content ... ok test remote::tests::private_literals_and_non_web_ports_do_not_open_sockets ... ok test proxy_transfers::tests::source_pruning_keeps_the_pending_receipt_and_message ... ok test remote::tests::reclamation_retains_capacity_for_new_mail ... ok test proxy_transfers::tests::transfer_batch_rejects_foreign_revision_without_partial_work ... ok test routes::tests::attachment_ranges_reassemble_large_content_and_keep_small_shape ... ok test proxy_transfers::tests::unified_trash_routes_two_accounts_without_cross_account_copy ... ok test routes::tests::hostile_account_fields_are_rejected_before_network_access ... ok test routes::tests::disabled_account_message_detail_embeds_no_cached_images ... ok test routes::tests::account_routes_require_a_data_scoped_user ... ok test routes::tests::mail_events_are_authenticated_and_owner_filtered ... ok test routes::tests::another_user_cannot_list_or_open_a_mail_account ... ok test routes::tests::mail_read_marking_preference_is_owner_scoped_and_persistent ... ok test routes::tests::mail_events_are_user_scoped_and_contain_no_message_data ... ok test routes::tests::oldest_unread_hides_threads_the_caller_cannot_see ... ok test routes::tests::preference_receipts_replay_undo_after_restart_and_isolate_users ... ok test routes::tests::provider_read_capacity_rejects_overload_without_queuing ... ok test routes::tests::preference_undo_preserves_legacy_edit_and_rejects_invalid_inputs ... ok test search::epoch_tests::epoch_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test routes::tests::read_state_is_journaled_without_provider_access ... ok test routes::tests::remote_content_defaults_on_and_is_user_scoped ... ok test search::epoch_tests::epoch_missing_identity_retains_intent_for_reported_failure ... ok test search::epoch_tests::epoch_pending_destination_refuses_transfer_and_draft ... ok test search::epoch_tests::epoch_queued_move_handoff_preserves_receipt_and_dispatch_boundary ... ok test search::epoch_tests::epoch_staged_anonymous_bridge_publishes_only_on_activation ... ok test search::epoch_tests::epoch_actions_handoff_is_atomic_and_owner_scoped ... ok test search::epoch_tests::epoch_staged_content_publishes_after_restart ... ok test search::epoch_tests::mail7f_dispatched_move_readers_and_absence_proof ... ok test search::epoch_tests::mail7f_receipt_hydration_keeps_original_move_source_hidden ... ok test search::epoch_tests::mail7f_startup_collects_only_abandoned_epoch_content ... ok test search::epoch_tests::mail7g_startup_restores_dead_receipt_before_failure_hook ... ok test search::epoch_tests::mail7g_original_source_visibility_settles_without_losing_receipt ... ok test search::epoch_tests::mailrev2_ambiguous_identity_retains_intent ... ok test search::epoch_tests::mailrev2_crash_worker ... ignored, spawned by epoch replacement SIGKILL regression test search::epoch_tests::mail7g_dispatched_visibility_requires_selected_unique_membership ... ok test search::epoch_tests::epoch_sigkill_stages_converge ... ok test search::epoch_tests::mailrev2_mapped_delete_stays_out_of_search ... ok test search::epoch_tests::mailrev2_mapped_move_keeps_source_hidden ... ok test search::epoch_tests::mailrev2_pending_bridge_keeps_live_threads ... ok test search::epoch_tests::mail7g_upgrade_0025_pending_from_0022_through_0024 ... ok test search::epoch_tests::mailrev2_migration_0022_existing_intent_restart ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_fts ... ok test search::epoch_tests::mailrev2_pending_window_keeps_live_text ... ok test search::epoch_tests::mailrev2_second_reset_abandons_pending_epoch ... ok test search::epoch_tests::mailrev3_changed_crash_worker ... ignored, subprocess for mailrev3 changed-content SIGKILL probe test search::epoch_tests::mailrev2_two_users_overlapping_intents ... ok test search::epoch_tests::mailrev3_abandoned_epochs_do_not_accumulate_body_sets ... ok test search::epoch_tests::mailrev3_dispatched_move_keeps_mapped_source_hidden ... ok test search::epoch_tests::mailrev3_old_duplicate_delete_must_not_target_surviving_copy ... ok test search::epoch_tests::mailrev3_upgrade_0022_pending_resume ... ok test search::epoch_tests::mailrev4_dispatched_move_does_not_hide_unselected_old_duplicate ... ok test search::tests::deferred_full_text_queue_is_bounded_durable_and_owner_scoped ... ok test sync::tests::backfill_process_crash_worker ... ignored, spawned by the Mail process crash recovery test test search::epoch_tests::mailrev4_failed_dispatched_move_reappears_after_sync ... ok test sync::tests::backfill_resumes_after_process_and_provider_interrupts_without_duplicate_threads ... ok test sync::tests::exhausted_pending_epoch_resumes_with_fresh_count ... ok test search::epoch_tests::mailrev2_sigkill_reader_search_counts ... ok test sync::tests::expunge_reconciliation_is_bounded_and_resumes_for_sparse_uids ... ok test search::epoch_tests::mailrev3_sigkill_changed_content_reads ... ok test sync::tests::flowed_text_preserves_quotes_and_signature_boundary ... ok test sync::tests::folder_roles_follow_special_use_then_inbox_name ... ok test sync::tests::fake_imap_full_history_commits_messages_before_advancing_cursor ... ok test sync::tests::idle_keepalives_end_at_the_bounded_window_without_timeout ... ok test sync::tests::hostile_mime_from_real_imap_session_keeps_text_and_isolates_html ... ok test sync::tests::idle_waits_have_a_separate_pool_from_mail_sync ... ok test sync::tests::incomplete_backfill_cannot_commit_success ... ok test sync::tests::expunge_sweep_converges_while_new_mail_keeps_arriving ... ok test sync::tests::interleaved_flags_fetch_is_ignored_after_uid_set_validation ... ok test sync::tests::malformed_headers_and_deep_multipart_nesting_still_sync ... ok test sync::tests::migrated_production_shape_resumes_without_refetch ... ignored, requires CALTERNAL_INTEGRATIONS_REVIEW_DB from the Server production-shape test test sync::tests::listener_reports_mail_that_arrived_before_idle ... ok test sync::tests::profile_full_history ... ignored, large duplicate-UID Mail history profile for #626 and #1340 test sync::tests::profile_interleaved_flags_fetch ... ignored, local performance profile test sync::tests::provider_and_cache_failures_log_only_classified_fields ... ok test sync::tests::manual_sync_keeps_one_listener_per_account ... ok test sync::tests::provider_identity_uses_a_message_id_only_after_normalization ... ok test sync::tests::provider_retry_delay_grows_and_stays_bounded ... ok test sync::tests::oversized_mime_part_is_bounded_during_imap_sync ... ok test sync::tests::quiet_idle_cycles_keep_one_durable_reconciliation_poll ... ok test sync::tests::real_tls_provider_backfill ... ignored, requires the isolated TLS provider from tests/adversarial/mail_sync_provider.py test sync::tests::rejected_fetch_is_an_error_and_next_command_stays_readable ... ok test sync::tests::standard_mail_signals_choose_a_category_without_subject_threading ... ok test sync::tests::statement_arrivals_use_only_conservative_subject_signals ... ok test sync::tests::quiet_listener_starts_the_poll_for_older_accounts ... ok test sync::tests::sync_reconciles_an_expunged_duplicate_uid ... ok test sync::tests::thread_keys_accept_strict_message_ids_and_reject_hostile_headers ... ok test sync::tests::truncation_keeps_utf8_boundaries ... ok test sync::tests::sync_start_records_attempt_without_claiming_completion ... ok test sync::tests::provider_fetch_list_stores_all_messages ... ok test sync::tests::delta_catch_up_shares_the_bounded_window_budget_and_resumes ... ok test result: ok. 166 passed; 0 failed; 10 ignored; 0 measured; 0 filtered out; finished in 28.86s Doc-tests calternal_plugin_mail running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Cleanup (verbatim): ``` Removed 3968 files, 2.1GiB total ``` The generated TLS keys/data are removed; the provider exited; no web build output was created. READY

kayg referenced this issue 2026-10-09 05:37:24 +00:00

Merge round 35c #1500

kayg referenced this issue from a commit 2026-10-09 11:11:17 +00:00

Merge job/mail7-1340 (#1340)

kayg commented 2026-10-09 11:11:32 +00:00

Author

Owner

Copy link

Deployed to calternal.cloud in merge round 35c (build 5a2c02c13, live 2026-10-09 ~16:35 IST). Closing.

Deployed to calternal.cloud in merge round 35c (build 5a2c02c13, live 2026-10-09 ~16:35 IST). Closing.

kayg closed this issue 2026-10-09 11:11:33 +00:00

Commenting is not possible because the repository is archived.

No Branch/Tag specified

Branches Tags

dev

wip/cards2-1083

wip/cards-1083

wip/canvas-visual

wip/canvasvis2-976

wip/calhdr-1112

wip/blocks-1125

wip/allday-1107

wip/agenda-1086

wip/trayicons2-1095

wip/trayicons-1095

wip/sidebar3-1094

wip/previewcard-1098

wip/palette2-1123

wip/palette-1093

wip/nlpchip-1127

wip/mchrome-1084

wip/mailghost2-1094

wip/mailghost-1094

wip/kbpreview2-1118

wip/kbpreview-1118

wip/kanban-1092

wip/editreg-1132

wip/editorrail3-1113

wip/editorrail2-1113

wip/editorrail-1113

wip/draw4-1101

wip/draw3-1101

wip/draw2-1101

wip/draw-1101

wip/delete-1119

job/txentry-1198

job/rev1586

job/rev2-money-ident

job/rev2-webperf

job/morph-1104

job/previewcard-1098

job/importhang-1121

job/kanban-1092

job/merge-round-7b

job/merge-round-7c

job/calcards-1115

job/collabrev3-1197

job/e2e-b-1071

job/agenda-1086

job/agenda-decks

job/allday-1107

job/browserfix

job/calact9

job/adv7c-1105

wip/collabrev3-1197

wip/calcards-1115

wip/calact9

wip/calact10

wip/browserfix

wip/agenda-decks

wip/adv7c-1105

wip/txentry-1198

wip/rev2-webperf

wip/rev2-money-ident

wip/rev1586

wip/morph-1104

wip/merge-round-7c5

wip/merge-round-7c4

wip/merge-round-7c3

wip/merge-round-7c2

wip/merge-round-7c

wip/importhang-1121

wip/fix1586

wip/e2e-b2-1071

wip/e2e-b-1071

job/noncanon-1618

wip/noncanon-1618

job/edrevert-1616

job/partial-1605

wip/partial-1605

job/calact6

job/cold-1586

job/perfvm-1587

wip/cold-1586

job/cal404-1581

wip/calact6

wip/cal404-1581

job/dname-1584

wip/dname-1584

job/merge41

wip/merge41b

wip/merge41

job/rev1573c

job/hscroll-1579

wip/rev1573c

job/rev1573b

job/rev1573

wip/fix1573

wip/hscroll2

wip/hscroll-1579

job/rev1574

wip/fix1574

wip/rev1574

job/startup-1573c

job/merge40

wip/startup-1573d

wip/startup-1573c

job/calact5

wip/remtext2

wip/merge40c

wip/merge40b

wip/merge40

job/calact4

job/startup-1573

job/calprof

wip/calprof

wip/startup-1573

job/fs1575

job/fix1570

job/calact3

job/rec1540c

wip/fix1570

job/rev1570-review

wip/calact3

job/calact2

wip/calact2

job/calcold-1560

wip/calcold-1560

wip/rec1540c

job/fix1569

wip/fix1569

job/calact-1557

wip/calact-1557

job/remtext-1568

wip/remtext-1568

job/fix1566

wip/fix1566

job/rec1540b

job/reconcile-1540

job/money2rev

wip/money2rev

job/merge39

wip/merge39

wip/reconcile-1540

job/tabswitch2

job/merge38

wip/merge38

job/autherr-1553

wip/autherr-1553

job/money-688

wip/money688rev3

wip/money688rev2

wip/money688rev

wip/money688l

wip/money688k

wip/money688j

wip/money688i

wip/money688h

wip/money688g

wip/money688f

wip/money688e

wip/money688d

wip/money688c

wip/money688b

wip/money-688

wip/tabswitch2

wip/merge37

job/merge37

job/jobnames-1538

wip/jobnames-1538

job/tabcomp-perf

job/calscript-1492

wip/tabcomp-perf

wip/calscript-1492

job/mailbody-1533

wip/mailbody-1533

job/intlfmt-1461

job/merge36

wip/intlfmt2

wip/intlfmt-1461

wip/merge36

job/merge35c

job/tagperf-1186

wip/tagrev2

wip/tagrev

wip/tagperf4

wip/tagperf2-1186

wip/tagperf-1186

wip/tagfix5

wip/tagfix3

wip/tagfast2

wip/tagfast

wip/merge35c

job/notesel-1493

wip/notesel2

wip/notesel-1493

job/merge35b

wip/merge35b

wip/m35btriage

wip/m35bratchet

wip/calsel-1491

job/calsel-1491

job/notifread-1387

job/bulkwrite-1393

wip/notifread2

wip/notifread-1387

job/slowsel-1392

wip/bulkwrite-1393

wip/bulkrev4

wip/bulkrev3

wip/bulkrev2

wip/bulkrev

wip/bulkfix4

wip/bulkfix3

wip/bulkfix2

wip/bulkfix

job/mail7-1340

job/bigscroll-1481

job/searchperf-1425

wip/mailrev4

wip/mailrev3

wip/mailrev2

wip/mailrev

wip/mail7f

wip/mail7e

wip/mail7d

wip/searchperf2

wip/searchperf-1425

wip/mail7c

wip/mail7b

wip/mail7-1340

job/photoopen-1367

job/mailsel-1449

wip/bigscroll-1481

wip/slowsel3

wip/slowsel2

wip/slowsel-1392

wip/selroot

job/journaldup

wip/journaldup8

wip/journaldup7

wip/journaldup6

wip/journaldup5

wip/journaldup4

wip/journaldup3

wip/journaldup2

wip/journaldup

wip/jdrev5

wip/jdrev4

wip/jdrev3

wip/jdrev2

wip/jdrev

wip/jdfix9

job/bignote-1349

wip/mailsel-1449

wip/bignote3

wip/bignote2

wip/bignote-1349

job/statusmark-1457

wip/statusmark-1457

wip/photoopen4

wip/photoopen3

wip/photoopen2

wip/photoopen-1367

job/authflash-1473

job/merge35

wip/authflash-1473

job/phototags-1370

wip/phototags-1370

wip/merge35

wip/m35triage

job/tick-1446

wip/tick-1446

job/restartidx-1450

job/exmove-1333

job/coldnote-1451

wip/restartidx-1450

wip/exmove-1333

wip/coldnote-1451

job/undo412-1430

wip/undo412-1430

job/calwrite-1441

job/xcancel-1443

job/calscroll-1362

wip/xcancel-1443

wip/calwrite-1441

job/xsub-1445

wip/calscroll3

wip/calscroll2

wip/calscroll-1362

wip/xsub-1445

job/linkdup-1440

wip/linkdup2

wip/linkdup-1440

wip/tabswitch4

wip/tabswitch3

wip/tabswitch-1233

wip/navrev

job/tabswitch-1233

job/mdpaste-1431

job/taskundo-1433

job/directory-1199

wip/taskundo-1433

job/remind-1432

job/nfd-1411

wip/remind-1432

wip/dirfix4

wip/dirfix3

wip/dirfix2

wip/dirfix-1199

wip/directory-1199-r

wip/directory-1199

job/readerx-1427

wip/readerx-1427

job/scrollfade-1416

job/tipphone-1439

wip/nfd-1411

job/purge-1420

job/notifrow-1414

wip/purge-1420

wip/notifrow-1414

job/slop-1385

wip/slop2

wip/slop-1385

job/tbover-1409

job/qltitle-1408

job/grantlive-1405

job/taskclick-1434

wip/tbover-1409

wip/taskclick-1434

wip/qltitle-1408

job/scrub-1422

job/hotfix35

job/ticks-1361

wip/scrub-1422

job/taskprev-1401

wip/grantlive-1405

job/ghost-1323

wip/ticks3

wip/ticks2

wip/ticks-1361

wip/taskprev-1401

job/bash3-settings

wip/ghost-1323

job/mailmorph-1368

wip/bash3-settings

job/apiaudit-1374

wip/bash3-filesphotos

job/bash3-filesphotos

job/xuser39-1402

job/bash3-money

job/mailhtml-1372

wip/mailhtml2

wip/mailhtml-1372

job/bash3-interop

job/bash3-calendar

wip/bash3-money

wip/bash3-interop

wip/bash3-calendar

wip/mailmorph2

wip/mailmorph-1368

wip/bash3-tasks

job/outro-1391

job/linkpick-1379

job/linkdel-1384

wip/linkpick2

wip/linkpick-1379

wip/linkdel3

wip/linkdel2

wip/linkdel-1384

job/mailrows-1390

wip/mailrows-1390

wip/apiaudit2

wip/apiaudit-1374

job/bash-core

wip/outro-1391

wip/bash-core2

wip/bash-core

job/bash3-notes

job/bash3-tasks

job/merge34

job/notifdel-1363

wip/merge34

wip/notifdel2

wip/notifdel-1363

job/wklink-1371

wip/wklink-1371

job/searchbar-1360

wip/searchbar2

wip/searchbar-1360

job/merge33-1122

wip/merge33c

wip/merge33b

wip/merge33-1122

job/bash-security

job/mailglass-1373

wip/bash-security2

wip/bash-security

wip/mailglass-1373

job/instaudit-1369

wip/instaudit-1369

job/tabstorm-1338

wip/tabstorm2

wip/tabstorm-1338

wip/bash-files2

wip/bash-files

job/bash-files

job/bash-money

job/editfuzz-1335

wip/bash-money

wip/moneyrev

job/tasksperf-1286

wip/tasksperf3

wip/tasksperf2

wip/tasksperf-1286

wip/editfuzz2

wip/editfuzz-1335

review/dirrev-1199

job/merge32

wip/merge32b

wip/merge32

job/bash-chrome

job/tagdnd-1187

job/bash-notes

wip/bash-notes2

wip/bash-notes

wip/bash-chrome

wip/reconcile-c1

wip/adv-c1

wip/tagdnd2-1187

wip/tagdnd-1187

job/bash-calendar

job/deploy-c1

job/editbreak-1334

job/bash-perf

wip/bash-calendar2

wip/bash-calendar

wip/bash-perf

job/hunt-20261006

job/bash-interop

job/collabloss-1197

wip/collabrev-1197

wip/collabloss5-1197

wip/collabloss4-1197

wip/collabloss3-1197

wip/collabloss2-1197

wip/collabloss-1197

wip/bash-interop

job/bash-mail

wip/bash-mail

job/vis-1293

wip/vis-1293

wip/restyle-search-r

wip/restyle-search-fix2

wip/restyle-search-fix

wip/restyle-search

job/restyle-settings

job/restyle-search

wip/restyle-settings-fix

wip/restyle-settings

job/adv31

wip/advfix-a

wip/adv31

job/bash-tasks

wip/bash-tasks

job/restyle-files

wip/restyle-files-fix

wip/restyle-files

job/fmtbar-1210

wip/fmtbar2-1210

wip/fmtbar-1210

job/perf2-1124

wip/perf2-1124

job/proflog-1204

job/merge31

wip/restyle-notes

wip/merge31

wip/proflog-1204

job/restyle-notes

job/fix-1213

wip/fix-1213

job/hide-1153

job/restyle-mailmoney

job/notestatus-1209

wip/hide-1153

wip/hiderev2-1153

wip/hiderev-1153

wip/hide5-1153

wip/hide4-1153

wip/hide3-1153

wip/hide2-1153

job/sidebarblur-1208

wip/notestatus-1209

wip/sidebarblur-1208

wip/restyle-mailmoney

job/onboard-1141

wip/onboard2-1141

wip/onboard-1141.aborted-early

wip/onboard-1141

job/adv-1202

job/segmented-1200

wip/segmented-1200

job/merge30

job/notifloop-1194

job/toastmorph-1201

wip/notifloop-1194

job/perf-1124

job/tocrail-1191

job/cards-1179

wip/cards2-1179

wip/cards-1179

wip/tocrail-1191

wip/perf-1124

wip/merge30j

job/wizchoices-1140

wip/wizchoices-1140

wip/restyle-1190

job/moneyfmt-1180

wip/moneyfmt2-1180

wip/moneyfmt-1180-r

wip/moneyfmt-1180

job/pillglass-1189

job/flags-1181

wip/flags-1181

job/restyle-1190

job/settingsreg-1195

job/wizard-1140

site/website

wip/wizardrev2-1140

wip/wizardrev-1140

wip/wizard5-1140

wip/wizard4-1140

wip/wizard3-1140

wip/wizard2-1140

wip/wizard-1140

wip/pillglass-1189

wip/settingsreg-1195

job/merge29

job/fu-1171

wip/merge29j

wip/fu-1171

job/fu-1166

job/txresearch-1188

wip/fu-1166

job/merge28

job/search-1066

wip/search-1066

wip/merge28j

job/gateslot-1182

job/bulkimport-1157

job/mailnet-1160

wip/mailnetrev-1160

wip/mailnet-1160

wip/bulkrev-1157

wip/bulkimport-1157

job/startup-1161

wip/startup-1161

job/merge27

job/linkcards-1151

wip/linkcards3-1151

wip/linkcards2-1151

wip/linkcards-1151

job/traydate-1144

wip/traydate3-1144

wip/traydate2-1144

wip/traydate-1144

job/draw-1101

wip/merge27j

job/blockpill-1152

wip/blockpill3-1152

wip/blockpill2-1152

wip/blockpill-1152

job/minihover-1149

wip/minihover2-1149

wip/minihover-1149

job/merge25

wip/merge25-r

wip/merge25b

wip/merge25

job/inspector-1129

job/tags-1110

wip/inspector3-1129

wip/inspector2-1129

wip/inspector-1129

wip/tagsrev-1110

wip/tags2-1110

wip/tags-1110

job/dates-1148

wip/datesrev-1148

wip/dates2-1148

wip/dates-1148

job/licence-1145

wip/licence2-1145

wip/licence-1145

job/selfhost-1156

fixbatch-25

job/merge23

wip/merge23

job/tagfilter-1109

wip/tagfilter2-1109

wip/tagfilter-1109

job/kbd-1134

wip/kbd2-1134

wip/kbd-1134

job/palfoot-1137

wip/selfhost-1156

wip/palfoot2-1137

wip/palfoot-1137

job/toggle-1158

wip/toggle-1158

fixbatch-23

job/kbpreview-1118

job/docratchet-1155

job/perflint-1133

job/devtests-1159

wip/docratchet-1155

wip/devtests-1159

job/segv-1136

wip/toast-1142

wip/segv-1136

job/toast-1142

job/blockreload-1147

wip/blockreload-1147

job/font-1150

wip/font-1150

job/importui-1120

job/minimonth-1149

wip/importui-1120

wip/minimonth-1149

job/depcheck-1146

wip/perflint-1133

wip/depcheck-1146

job/blocks-1125

job/plus-1128

job/shift-1138

wip/plus2-1128

wip/plus-1128

wip/shift-1138

job/moneyfid-1130

job/editorrail-1113

wip/moneyrev-1130

wip/moneyfid-1130

job/noext-851

wip/noext-851

wip/noext3-851

wip/noext2-851

job/week-1135

wip/week-1135

job/editreg-1132

job/smoke-1122

wip/smoke-1122

job/docs-1143

job/palette2-1123

job/calhdr-1112

job/nlpchip-1127

job/mailghost-1094

job/reconnect-1131

wip/reconnect-1131

job/trayicons-1095

job/delete-1119

hotfix-sidebar2

job/cards-1083

job/palette-1093

job/mchrome-1084

job/e2e-a-1071

job/canvas-visual

wip/e2e-a2-1071

wip/e2e-a-1071

backup/canvas-visual-before-reconcile

wip/surfaces-p2

job/merge-round-9

wip/merge-round-9

job/7cfix-small

wip/7cfix-small

job/mailui-1078

job/merge-round-8

wip/merge-round-8

wip/mailui-1078

job/mailround-1038

job/applemail-accept

wip/settitle-1068

wip/mailround2-1038

wip/mailround-1038

wip/e2e-7b

job/crash-1069

wip/crash-1069

job/searchlost-1066

wip/searchlost-1066

job/7b-reconcile

job/flake-1065

wip/flake-1065

wip/merge-round-7b7

wip/merge-round-7b6

wip/merge-round-7b5

wip/merge-round-7b4

wip/7b-reconcile

job/appupdate-1059

job/nfd-1044

wip/appupdate-1059

job/e2e-7b

job/loop-1062

wip/loop-1062

job/pdfprev-1045

job/invtoggle-1053

wip/pdfprev-1045

wip/nfd-1044

wip/invtoggle-1053

job/7bfix-e2e

job/mailstress-b

wip/7bfix-e2e

wip/mailstress-b

job/7bfix-adv

wip/7bfix-adv

job/mailstress-a

job/stack-1054

wip/stack-1054

wip/mailstress-a

job/mailstress-1038

wip/mailstress-1038

job/upload500-1051

wip/upload500-1051

job/share-1034

wip/share-1034

job/syncerr-1037

job/7bfix-photos

wip/7bfix-photos

job/paste-1036

job/setside-1039

wip/setside-1039

wip/paste-1036

job/lease-1042

wip/syncerr-1037

wip/lease-1042

job/7bfix-data

job/passkeybind-1043

wip/apprevoke-1041

job/invite-1035

wip/invite-1035

job/merge-round-7b2

wip/merge-round-7b2

job/mailproxy-486

job/apprevoke-1041

job/rebuild-1033

job/pillborder-1029

wip/pillborder-1029

wip/mailproxy-486

wip/applemail-486

job/headless-998

wip/headless-998

job/groups-1028

wip/groups-1028

job/rebuildwarn-1016

wip/rebuildwarn-1016

job/startup-1011

wip/startup-1011

job/monthpill-1009

job/bgthumb-1025

job/sharetitle-1012

wip/monthpill-1009

wip/bgthumb-1025

wip/sharetitle-1012

job/canvas-cards-977

wip/canvas-cards-977

job/canvas-pencil-978

job/canvas-sketch-990

wip/canvas-sketch-990

wip/canvas-pencil-978

job/canvas-files-989

wip/canvas-files-989

job/canvas-collab-991

wip/canvas-collab-991

job/weekscroll-1018

wip/weekscroll-1018

wip/canvas-core-976

job/canvas-core-976

job/round-drag

wip/round-drag

job/round-settings

wip/oapi-974

job/oapi-974

job/hist2-integrate

job/mailhtml-726

wip/mailhtml-726

wip/hist2-integrate

job/moneyfu-984

job/drag-1015

wip/drag-1015

job/rename-1017

wip/rename-1017

job/hist2-api

wip/hist2-api

job/oneacct-1014

wip/oneacct-1014

wip/moneyfu-984

job/hist2-bench

job/hist2-restore

wip/hist2-bench

job/hist2-write

job/hotfix-724

wip/hotfix-724

wip/hist2-write

wip/hist2-restore

job/hist2-store

job/hist2-ui

wip/hist2-ui

wip/hist2-store

job/searchstarve-965

job/shutdown-963

wip/shutdown-963

wip/pubedit-981

job/pubedit-981

job/analytics-973

wip/searchstarve-965

job/authflash-850

job/weeklane-969

job/pvtitle-1004

job/hist-975

wip/authflash-850

job/voicepill-617

wip/pvtitle-1004

job/headring-1003

wip/weeklane-969

wip/voicepill-617

wip/headring-1003

wip/analytics-973

job/agentscope-980

wip/thumbsandbox-988

job/thumbsandbox-988

wip/hist-975

job/links-856

wip/links-856

job/davetag-966

wip/davetag-966

job/filesstorm-1000

job/hoverpad-725

wip/filesstorm-1000

job/ffmpegblas-993

job/merge-round-7a

wip/hoverpad-725

wip/ffmpegblas-993

job/nowdot-1002

wip/verify-7a

job/noteid-857

wip/nowdot-1002

wip/noteid-857

wip/merge-round-7a

wip/agentscope-980

job/imapedge

job/a11yfix2

wip/imapedge-941

wip/imapedge

wip/a11yfix2

job/notetask-986

job/logheading

wip/logheading-998

job/textthumb-652

job/photolive-987

wip/photolive-987

job/davactive-983

job/savefix-985

job/tabicons-607

wip/davactive-983

wip/tabicons-607

wip/notetask-986

wip/savefix-985

job/dirid-627

job/buildspeed-1007

wip/dirid-627

job/staging-imap

job/perfguards-impl

job/undo-a11y

wip/undo-a11y

job/mailperf

job/wal-824

wip/settings-50

job/settings-50

job/notesfilter-606

wip/notesfilter-606

job/surfaces-p2

wip/wal-824

job/maillayouts

wip/mailperf

wip/maillayouts

job/taskmeta-659

job/money-ident

wip/money-ident

wip/taskmeta-659

job/errstates

wip/perfguards-impl

job/headings-881

wip/headings-881

wip/errstates

job/voice-619

job/gaps-827

job/notesperf

wip/notesperf

wip/voice-619

job/hddsql-549

job/perf-stream-668

wip/perf-stream-668

wip/deeplinks-fix

job/deeplinks-fix

job/authfix

job/docsfix-rust

wip/docsfix-rust

job/webperf

job/docsfix-web

job/datafix2

job/webdav-lock-476

job/copyfix

wip/copyfix

wip/webperf

job/focus-658

wip/protofix

job/mediafix

job/protofix

wip/mediafix

job/agentfix

job/hhmm-724

wip/agentfix

job/undo-722

job/reuse

wip/webdav-lock-476

wip/reuse

job/scopefix

job/datafix

wip/hhmm-724

wip/undo-722

job/surfaces-p1

wip/hddsql-549

job/voicememos-618

wip/datafix2

wip/surfaces-p1

job/fix-940

wip/fix-940

job/blaze-surfaces

wip/datafix

wip/blaze-surfaces

job/taskday-655

job/linknav-639

wip/linknav-639

wip/gaps-827

job/isolation-707

job/audiophotos-720

wip/audiophotos-720

job/advfind-664

wip/voicememos-618

wip/taskday-655

wip/isolation-707

wip/advfind-664

wip/scopefix

wip/focus-658

job/testgaps

wip/testgaps

job/overscroll-718

wip/authfix

job/deps

wip/overscroll-718

job/rev2-agentfix

job/rev2-mailperf

wip/deps

job/hardening-728

wip/hardening-728

job/searchgen-832

wip/searchgen-832

job/photopw-849

job/mailsql-825

wip/photopw-849

job/sharefix

wip/sharefix

job/rev2-mailhtml-726

job/rev2-perfguards

job/copyval-723

job/lightglass-r2

wip/lightglass-r2

wip/docsfix-web

job/copy-audit

job/macinterop-staging-r2

job/design-sync

job/rev2-taskmeta-659

job/docs-audit

job/rev2-advfind-664

job/rev2-mailproxy-486

job/states-audit

job/rev2-datafix

job/design-drift

job/test-gaps

job/rev2-voicememos-618

job/rev2-mediafix

job/rev2-deps

job/rev2-datafix2

job/licence-audit

job/issue-hygiene

job/rev2-protofix

job/rev2-voice-619

job/rev2-isolation-707

job/rev2-surfaces-p1

job/deeplink-audit2

job/rev2-audiophotos-720

wip/test-gaps

job/rev2-overscroll-718

job/rev2-undo-722

wip/states-audit

job/rev2-dropmd-719

job/rev2-linknav-639

job/merge-7b-plan

wip/merge-7b-plan

job/rev2-taskday-655

wip/mailsql-825

job/rev2-webdav-lock-476

job/rev2-browserfix

wip/design-drift

job/rev2-hddsql-549

wip/deeplink-audit2

job/rev2-scopefix

job/rev2-authfix

job/rev2-hardening-728

job/rev2-wal-824

job/rev2-sharefix

job/calsidebar-638

job/chrome-audit

job/ioperf

wip/ioperf

wip/chrome-audit

wip/calsidebar-638

job/dropmd-719

wip/dropmd-719

job/ocr-build

wip/ocr-build

job/blaze-settings

wip/copyval-723

job/toastring-721

wip/toastring-721

job/deployfix-732

wip/deployfix-732

wip/blaze-settings

job/money-import-recheck

job/rev-a11y

job/perf-arch-db

job/rev-7b-data

wip/textthumb-652

wip/perf-arch-db

job/sec-protocols

job/sidehdr-660

job/rev-7b-security

job/research-surfaces

job/rev-design-gaps

job/rev-mcp-api

wip/sidehdr-660

job/perf-arch-memory

wip/sec-protocols

job/perf-arch-bundle

job/snapedge-714

wip/rev-mcp-api

job/sec-supplychain

wip/research-surfaces

job/perf-arch-sync

job/rev-consistency

job/perf-arch-server

wip/perf-arch-server

wip/perf-arch-memory

job/perf-arch-io

job/perf-arch-client

job/sec-fs

job/sec-mcp-scopes

job/sec-sharing

job/perf-guards

job/sec-browser

job/sec-admin-deploy

job/sec-auth

wip/snapedge-714

job/bgpicker-717

wip/perf-arch-bundle

wip/money-import-recheck

job/advsetup-654

wip/bgpicker-717

wip/advsetup-654

job/burst-709

job/kbdcaps-710

job/app-pw-chooser

wip/burst-709

wip/app-pw-chooser

job/imaptest-625

wip/kbdcaps-710

job/fix-499

wip/fix-499

job/perf-mut-667

job/calimg-589

job/perf-snap-666

wip/calimg-589

wip/perf-snap-666

wip/perf-mut-667

job/perf-cache-665

wip/perf-cache-665

job/voicefiles-620

wip/voicefiles-620

job/admin-burst-705

wip/admin-burst-705

job/voicememos-review

wip/voicememos-review

wip/ryw-653

job/ryw-653

job/writeonopen-661

job/instant-663

wip/writeonopen-661

job/money-import-review

wip/money-import-review

wip/importjs-610

review/integrations-407-round6

wip/integrations-review

job/dragghost-612

wip/dragghost-612

job/integrations

wip/integrations

job/decider-656

job/merge-round-6

job/perf-rerun

wip/merge-round-6

job/integrations-review-round5

job/selalign-576

wip/selalign-576

job/mcp-events-491

job/files-631

job/cal-e2e-569

wip/cal-e2e-569

job/reload-423

wip/reload-423

wip/mcp-events-491

wip/files-631

job/notesbridge-644

wip/notesbridge-644

job/editor-series

job/calcard-series

wip/calcard-series

job/mcp-events-review-491

wip/mcp-events-review

wip/editor-series

job/quirks-546

job/integrations-recheck

job/tocrail-636

wip/tocrail-636

wip/quirks-546

wip/reminders-643

job/reminders-643

wip/davscale-573

job/davscale-573

job/integrations-review

wip/ocr-eval-584

job/ocr-eval-584

job/esc-537

wip/esc-537

job/toastname-586

wip/toastname-586

job/submenu-579

wip/submenu-579

job/tasks-mode

wip/tasks-mode

job/agentdocs-630

job/dupwrite-634

wip/agentdocs-630

wip/dupwrite-634

job/lightglass-588

wip/lightglass-588

job/tabswitch-549

job/ghosttask-623

wip/ghosttask-623

job/toaststack-616

job/weekstate-609

job/mailsync-613

wip/mailsync-613

wip/weekstate-609

job/maildup-626

wip/tabswitch-549

wip/maildup-626

wip/toaststack-616

job/motion-611

wip/motion-611

job/tlstest-601

wip/tlstest-601

job/perf-495

job/floating-sheet

wip/floating-sheet

job/remdup-585

wip/remdup-585

job/fix-502

wip/fix-502

job/attachplay-622

job/perf-batch

wip/perf-batch-563

wip/perf-495

hotfix/mail-sync-diag

job/mail-m3

wip/mail-m3

job/attach-poof-603

job/calhover-608

job/editorbar-604

job/mentions-605

job/merge-round-4

job/allday-514

wip/merge-round-4

wip/allday-514

job/merge-round-4a

wip/merge-round-4a

job/sharestack-580

job/fix-501

wip/sharestack-580

wip/fix-501

job/perf-batch-563

job/theme-detail

job/apw-cache-review

wip/apw-cache-review

job/probe-520

wip/probe-520

job/mac-393

wip/mac-393

job/header-571

job/flake-513

wip/flake-513

job/docs-thumb-547

wip/header-571

job/webcal-572

wip/webcal-572

wip/shortcuts-542

job/shortcuts-542

wip/docs-thumb-547

job/caldav-stress

wip/caldav-stress

wip/sweep-478

job/apw-cache-512

wip/apw-cache-512

job/money-empty-540

wip/restart-505

wip/money-empty-540

wip/fix-510

job/restart-505

job/fix-503

job/perf-496

wip/perf-496

job/fix-498

wip/fix-498

job/info-inspector-465

wip/info-inspector-465

job/fix-510

job/fix-507

wip/fix-507

wip/fix-503

job/fix-493

job/money-kinds

wip/money-kinds

job/hygiene-548

job/merge-round-3

wip/fix-493

job/drag-snap-536

wip/merge-round-3

wip/merge-round-0930

wip/drag-snap-536

job/align-538

wip/align-538

job/bg-flash

wip/bg-flash

job/money-import

job/search-count-544

wip/search-count-544

wip/money-import

job/settings-key-541

wip/settings-key-541

job/toast-539

job/preview-421

wip/preview-421

wip/toast-539

job/tasks-500-531

job/title-plain-526

wip/title-plain-526

wip/tasks-500-531

job/notes-bridge

wip/parity-484

job/parity-484

job/files-slow

job/crash-525

wip/notes-bridge

wip/files-slow

wip/crash-525

job/kbd-motion-527

wip/bg-422

job/analytics-504

wip/analytics-504

wip/kbd-motion-527

job/upload-pill-523

wip/upload-pill-523

wip/tray-order

job/tray-order

wip/overflow-mid

wip/merge-round-2

job/perf-494

wip/perf-494

job/merge-round-3-pre-rebuild

wip/mcp-fast-492

wip/motion-477

wip/asr-ab-489

wip/theme-variants-506

wip/overflow-511

wip/week-header-508

wip/attach-427

job/dav-delete-471

job/iso-435

wip/iso-435

wip/files-sel-keys

wip/dav-delete-471

job/align-253

job/siwc-490

wip/siwc-490

job/money-kinds-review

wip/align-253

wip/money-kinds-review

job/small-bugs-3

wip/overlay-title-487

wip/multiget-500

wip/hidden-420

wip/webcal-ui

wip/webcal-431

job/perf-367

job/location

wip/small-bugs-3

wip/location

wip/perf-367

wip/admin-deny-483

job/tag-unicode-473

wip/tag-unicode-473

job/blur-436

wip/photos-470

wip/blur-436

wip/small-bugs-4

wip/hunt-20260930

wip/settings-hdr-482

wip/chips-416

job/dedup-375

wip/dedup-375

job/doc-stack

wip/doc-stack

job/tokens-literals

wip/tokens-literals

job/jobs-leftovers

wip/send-fast

wip/paste-467

wip/money-numbers

job/money-plugin

wip/money-plugin

backup/doc-stack-r3-reset-source

job/break-dav

wip/merge-batch

wip/crossday-469

wip/mac-verify

wip/mail-m2

wip/break-dav

wip/money-review2

job/money-md

job/modes-424

wip/money-md

wip/jobs-leftovers

job/agenda-413

wip/agenda-413

job/dedup-375-clean

job/dedup-375-before-rebase

wip/modes-424

job/recog-417

wip/recog-417

wip/bounce-425

wip/ab-384-luna

backup/doc-stack-round3-before-rebuild

job/webdav-perf

wip/webdav-perf

job/toast-ring

wip/toast-ring

job/money-review

wip/money-review

wip/micro-motion

wip/settings-card

wip/minical

job/notes-imap-428

job/least-priv

wip/ui-small-2

wip/flaky-426

wip/drag-end-418

job/jank

wip/jank

wip/least-priv

wip/docs-site

job/agenda

job/sec-batch

wip/sec-batch

wip/per-user-index

job/area-calendars

wip/area-calendars

job/parity

wip/parity

job/documents-research

wip/documents-research

job/test-infra

job/reminders-sync

wip/small-bugs-2

wip/reminders-sync

wip/gestures

job/google-oauth

wip/tags-merge

wip/tags

job/e2e-theme

wip/e2e-theme

job/icon-align

wip/test-infra

wip/select-align

wip/editor-385

job/voice

wip/webdav

job/webdav

job/app-pw-ui

job/editor-integrity

wip/editor-integrity

wip/voice

wip/quota

wip/cal-followups

wip/icon-align

job/composer-scale

wip/composer-scale

job/jobs-page

wip/jobs-page

job/hig-type

wip/hig-type

wip/app-pw-ui

job/motion-spring

job/mcp

wip/motion-spring

wip/mcp

job/small-bugs

wip/push-hosts

job/profile-sign

wip/touch-369

wip/profile-sign

job/mobile-focus

wip/mobile-focus

wip/ui-polish-354

wip/small-bugs

wip/dup-task

job/toast-polish

job/app-pw-scopes

wip/toast-polish

wip/app-pw-scopes

wip/cli-agent

wip/selection-pills

job/preview-attach

wip/preview-attach

job/dav-proppatch

wip/dav-proppatch

wip/cal-switcher

job/atomic-race

wip/atomic-race

job/photos-shared

wip/photos-shared

wip/cal-grid

wip/note-rewrite

wip/search-rebuild

job/mail-m1

job/paperless-import

wip/paperless-import

wip/mail-m1

wip/hidden-activity

wip/search-d

wip/pricing-research

wip/cursors

wip/auto-scheme

job/single-pills

wip/single-pills

wip/xuser-matrix

wip/money-format

wip/app-pw-setup

wip/purge-dos

wip/vault-health

wip/caldav-apple

wip/xuser-audit

wip/e2e-green

wip/tabbar

wip/adv-harness

wip/maple-mono

job/search-fix

wip/search-fix

wip/search-perf-c

job/adv-harness

wip/sidebar-headers

job/glass

wip/temp-index

job/polish

wip/polish

wip/file-protocols

wip/money-research

wip/glass

wip/voice-models

wip/collab-redo

job/voice-research

wip/hunt-20260928

wip/notes-actions-research

wip/search-pad

wip/search-perf

wip/search-sticky

wip/editor-undo

wip/chrome-rules

wip/motion

wip/appearance-research

wip/appearance

wip/audit-bugs

wip/cal-glass

wip/block-actions

wip/authz-order

wip/event-stripes

wip/chrome-sidebar

wip/auth-flaky

wip/robust-2

wip/gate-fix

wip/menu-blur

wip/import-calternaljs

wip/tray-fix

job/import-calternaljs

wip/index-order

wip/audit-fixes

wip/search-chevrons

research/mail

wip/phone-chrome

wip/dedup-break

wip/csp

wip/ui-audit

wip/select-toast

wip/perf

wip/flat-layout

wip/fonts

wip/event-tint

wip/sync-converge

wip/data-split

wip/glass-audit

wip/robustness

wip/sync-chaos

wip/search-thumbs

wip/fuzz

wip/menu-icons

wip/search-pill

wip/sync-changing

wip/heading-links

wip/date-formats

wip/a11y

wip/break-editor

wip/e2e-fix

wip/settings-sections

wip/sync-root-guard

wip/search-palette

wip/share-edit

job/toasts

wip/toasts

wip/cont-analytics

wip/authz-review

wip/popovers

wip/overlay-glass

wip/change-feed

wip/editor-modes

wip/composer-align

wip/cont-agenda

wip/agenda-merge

job/agent-conventions

wip/agent-conventions

wip/backend-misc

job/route-audit

wip/route-audit

wip/ui-batch

wip/heif-hardening

wip/grid-resize

wip/ask-page

wip/webmcp

job/deeplink-audit

wip/deeplinks

wip/shortcuts

wip/cont-tz-days

main

No results found.

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal-v0#1340

No description provided.