PERF: composer 'Send 5' is slow; batch write path, derived work off the request path, optimistic UI

kayg commented

2026-09-29 17:10:44 +00:00

Owner

Owner (2026-09-29): "In the composer, 'Send 5' items takes tooooo long! Please optimise our plugins to be extremely fast!"

Scope: the composer's batch send (Log entries, tasks, events, notes), and then every plugin write path users wait on. Performance-first rule (CLAUDE.md priority 1).

1. Measure (before changing anything)

Production build. Measure on the perf-test VM, holding flock /root/perf.lock with load below 1 inside the lock. Also measure on the build host for fast iteration, labelled as such.
Time "Send 1", "Send 5" and "Send 20" of mixed Log entries (with tags and times; one with an attachment) end to end: click → all entries visible, and click → the server has acknowledged every write. Take 10 runs and report the median and p95.
Add server-side tracing (a tracing span per stage, off by default, enabled by a named config flag) for the write path: request parse, NLP, daily-note read, Markdown rewrite, atomic_write_and_fsync, version snapshot, Index and FTS update, the Search reindex, the CalDAV/area projection and sync tokens, the change feed and SSE, notifications and reminders, analytics. Show where the time goes for a batch of 5. Are the 5 sent one by one, each doing a full daily-note rewrite, fsync, version snapshot and reindex?

2. Fix at the root (keep a fix only if it wins by at least 10%, median of 5 interleaved A/B runs; durability must not weaken: an acknowledged write stays durable)

One batch request, POST …/log/batch or equivalent: all N entries are applied to each affected daily note in one read-modify-write, one fsync (plus the directory fsync), and one version snapshot per note. The web composer, CLI, MCP and WebMCP use it (#395 parity).
Move derived work off the request path: Search/FTS, CalDAV projection, analytics and notifications become one coalesced background job per batch, while the response returns once the file is durable. Freshness targets stay (visible in Search within 1 s).
Optimistic UI: entries appear at once in the Calendar and Log with a pending state, reconcile on the server's acknowledgement, and roll back with an error toast on failure (#427 made batch send appear live; keep that).
Then profile the other plugin write paths users wait on (a task create or edit, a note save, an event create or edit, a file rename or move, a tag rename) with the same tracing. Fix the top offenders the same way, one issue per root cause if large.

Targets

Send 5: server acknowledgement ≤ 150 ms p95 on the VM, and entries visible in ≤ 50 ms after the click (optimistic). Send 20 ≤ 400 ms p95.

Proof

Before and after tables under docs/perf/runs/, trace summaries, e2e tests of optimistic send with rollback on a failure, and gates per crate plus web. The adversarial round on the batch endpoint: 1,000 entries, entries for 50 different days, concurrent batches to the same day, and a batch while the note is open in the editor, with no lost writes.

## Owner (2026-09-29): "In the composer, 'Send 5' items takes tooooo long! Please optimise our plugins to be extremely fast!" Scope: the composer's batch send (Log entries, tasks, events, notes), and then **every plugin write path** users wait on. Performance-first rule (CLAUDE.md priority 1). ## 1. Measure (before changing anything) - Production build. Measure on the perf-test VM, holding `flock /root/perf.lock` with load below 1 inside the lock. Also measure on the build host for fast iteration, labelled as such. - Time "Send 1", "Send 5" and "Send 20" of mixed Log entries (with tags and times; one with an attachment) end to end: click → all entries visible, and click → the server has acknowledged every write. Take 10 runs and report the median and p95. - **Add server-side tracing** (a `tracing` span per stage, off by default, enabled by a named config flag) for the write path: request parse, NLP, daily-note read, Markdown rewrite, `atomic_write_and_fsync`, version snapshot, Index and FTS update, the Search reindex, the CalDAV/area projection and sync tokens, the change feed and SSE, notifications and reminders, analytics. Show where the time goes for a batch of 5. **Are the 5 sent one by one, each doing a full daily-note rewrite, fsync, version snapshot and reindex?** ## 2. Fix at the root (keep a fix only if it wins by at least 10%, median of 5 interleaved A/B runs; durability must not weaken: an acknowledged write stays durable) - **One batch request**, `POST …/log/batch` or equivalent: all N entries are applied to each affected daily note in **one** read-modify-write, one fsync (plus the directory fsync), and one version snapshot per note. The web composer, CLI, MCP and WebMCP use it (#395 parity). - Move derived work **off the request path**: Search/FTS, CalDAV projection, analytics and notifications become one coalesced background job per batch, while the response returns once the file is durable. Freshness targets stay (visible in Search within 1 s). - **Optimistic UI:** entries appear at once in the Calendar and Log with a pending state, reconcile on the server's acknowledgement, and roll back with an error toast on failure (#427 made batch send appear live; keep that). - Then profile the other plugin write paths users wait on (a task create or edit, a note save, an event create or edit, a file rename or move, a tag rename) with the same tracing. Fix the top offenders the same way, one issue per root cause if large. ## Targets Send 5: server acknowledgement ≤ 150 ms p95 on the VM, and entries visible in ≤ 50 ms after the click (optimistic). Send 20 ≤ 400 ms p95. ## Proof Before and after tables under `docs/perf/runs/`, trace summaries, e2e tests of optimistic send with rollback on a failure, and gates per crate plus web. The adversarial round on the batch endpoint: 1,000 entries, entries for 50 different days, concurrent batches to the same day, and a batch while the note is open in the editor, with no lost writes.

kayg commented

2026-09-29 17:11:30 +00:00

Author

Owner

Starting #468 on job/send-fast, based on 918b4764aded5aa385ef784acb8887cf4361bf36 (dev). I am reading the write paths and collecting baseline timings before implementation.

Starting #468 on `job/send-fast`, based on `918b4764aded5aa385ef784acb8887cf4361bf36` (`dev`). I am reading the write paths and collecting baseline timings before implementation.

kayg commented

2026-09-29 17:16:02 +00:00

Author

Owner

Baseline finding: the existing Send all implementation awaits one commitFn per draft (apps/web/src/lib/composer/Composer.svelte), and each Log draft calls POST /api/v1/notes/journal/log (apps/web/src/lib/composer/commit.ts). The handler performs a Daily note update and Index update for every entry, then emits a change and syncs nearby-day navigation. The perf VM's /root/perf.lock was held; flock -n returned immediately without a sample. I am recording build-host production timings and will not substitute those for the VM requirement.

Baseline finding: the existing Send all implementation awaits one `commitFn` per draft (apps/web/src/lib/composer/Composer.svelte), and each Log draft calls `POST /api/v1/notes/journal/log` (apps/web/src/lib/composer/commit.ts). The handler performs a Daily note update and Index update for every entry, then emits a change and syncs nearby-day navigation. The perf VM's `/root/perf.lock` was held; `flock -n` returned immediately without a sample. I am recording build-host production timings and will not substitute those for the VM requirement.

kayg commented

2026-09-29 17:56:58 +00:00

Author

Owner

Baseline setup evidence: the build-host production server build was waiting behind the shared sccache daemon. At 19:54 UTC, its process tree showed one active rustc child while this job had four compile requests waiting. I restarted only this build with SCCACHE_CLIENT_SIDE=1 and a separate daemon port, keeping CARGO_BUILD_JOBS=4 and the preset target directory. The production SPA build is ready. The nonblocking /root/perf.lock attempt on perf-test was busy, so no unlocked VM sample was taken.

Baseline setup evidence: the build-host production server build was waiting behind the shared sccache daemon. At 19:54 UTC, its process tree showed one active rustc child while this job had four compile requests waiting. I restarted only this build with `SCCACHE_CLIENT_SIDE=1` and a separate daemon port, keeping `CARGO_BUILD_JOBS=4` and the preset target directory. The production SPA build is ready. The nonblocking `/root/perf.lock` attempt on perf-test was busy, so no unlocked VM sample was taken.

kayg commented

2026-09-29 18:19:07 +00:00

Author

Owner

The pre-change production browser baseline is running from release binary e3f688f6e00815304b3b05035873fdf0128dd696 on the build host. It covers 10 runs each of Send 1, Send 5 and Send 20; the 20-item sequential sends are the longest set. At 20:18 local, host load averages were 19.49, 24.01 and 23.23. The probe has not reported a failed HTTP response. The perf-test lock was busy at the earlier nonblocking check, so these results are labelled build-host only.

The pre-change production browser baseline is running from release binary `e3f688f6e00815304b3b05035873fdf0128dd696` on the build host. It covers 10 runs each of Send 1, Send 5 and Send 20; the 20-item sequential sends are the longest set. At 20:18 local, host load averages were 19.49, 24.01 and 23.23. The probe has not reported a failed HTTP response. The perf-test lock was busy at the earlier nonblocking check, so these results are labelled build-host only.

kayg commented

2026-09-29 18:21:22 +00:00

Author

Owner

Production browser baseline committed as 43560736 (docs/perf/runs/2026-09-29-issue-468-send-fast.md and its JSON samples). Source server revision: e3f688f6e00815304b3b05035873fdf0128dd696.

Send	Successful write requests	Ack median / p95	Calendar visible median / p95
1	1	559 ms / 765 ms	873 ms / 1,095 ms
5	5	3,303 ms / 12,244 ms	3,830 ms / 12,782 ms
20	20	21,562 ms / 27,862 ms	22,151 ms / 28,490 ms

All 10 runs per size had successful responses. The 1, 5 and 15 minute build-host load averages were 21.12, 28.23, 23.56 before and 18.50, 22.57, 22.80 after. The perf-test VM was not measured because its lock was busy at the nonblocking check. These results are build-host measurements only. The request counts confirm that Send 5 and Send 20 run as separate Log writes.

Production browser baseline committed as `43560736` (`docs/perf/runs/2026-09-29-issue-468-send-fast.md` and its JSON samples). Source server revision: `e3f688f6e00815304b3b05035873fdf0128dd696`. | Send | Successful write requests | Ack median / p95 | Calendar visible median / p95 | | --- | ---: | ---: | ---: | | 1 | 1 | 559 ms / 765 ms | 873 ms / 1,095 ms | | 5 | 5 | 3,303 ms / 12,244 ms | 3,830 ms / 12,782 ms | | 20 | 20 | 21,562 ms / 27,862 ms | 22,151 ms / 28,490 ms | All 10 runs per size had successful responses. The 1, 5 and 15 minute build-host load averages were `21.12, 28.23, 23.56` before and `18.50, 22.57, 22.80` after. The perf-test VM was not measured because its lock was busy at the nonblocking check. These results are build-host measurements only. The request counts confirm that Send 5 and Send 20 run as separate Log writes.

kayg referenced this issue from a commit

2026-09-29 18:29:04 +00:00

perf: record Composer send baseline (#468)

kayg commented

2026-09-29 18:34:05 +00:00

Author

Owner

Route trace finding (#468): POST /api/v1/notes/journal/log currently does one daily-note path lookup, one next_log_block_id SQL query, one store::update/durable write, one store::index (including Calendar Log projection and FTS), and one neighboring Daily note navigation sync per entry. The 10-run build-host baseline also showed Send 5 as five HTTP writes (median acknowledgement 3,303 ms; p95 12,244 ms). The batch route will group by Daily note and enqueue one projection job after the durable writes.

Route trace finding (#468): `POST /api/v1/notes/journal/log` currently does one daily-note path lookup, one `next_log_block_id` SQL query, one `store::update`/durable write, one `store::index` (including Calendar Log projection and FTS), and one neighboring Daily note navigation sync per entry. The 10-run build-host baseline also showed Send 5 as five HTTP writes (median acknowledgement 3,303 ms; p95 12,244 ms). The batch route will group by Daily note and enqueue one projection job after the durable writes.

kayg referenced this issue from a commit

2026-09-29 18:39:16 +00:00

perf(notes-core): append Log batches in one pass

kayg commented

2026-09-29 18:54:44 +00:00

Author

Owner

Gate finding: the full cargo test -p calternal-plugin-notes run has one unrelated existing failure, tests::vtodo_wire_upgrade_rotates_sync_epoch_once. It removes migrations().migrations.pop() as the simulated pre-upgrade state, but the Notes migration list now ends at 0019 (dav_area_deletes) after 0018 (reminder_wire_epoch.sql). The second apply therefore does not run the epoch rotation that the unchanged assertion expects. I have not changed the assertion or fixture; the batch tests pass independently.

Gate finding: the full `cargo test -p calternal-plugin-notes` run has one unrelated existing failure, `tests::vtodo_wire_upgrade_rotates_sync_epoch_once`. It removes `migrations().migrations.pop()` as the simulated pre-upgrade state, but the Notes migration list now ends at 0019 (`dav_area_deletes`) after 0018 (`reminder_wire_epoch.sql`). The second apply therefore does not run the epoch rotation that the unchanged assertion expects. I have not changed the assertion or fixture; the batch tests pass independently.

kayg referenced this issue from a commit

2026-09-29 19:00:29 +00:00

feat(notes): batch durable composer Log writes

kayg commented

2026-09-29 19:32:09 +00:00

Author

Owner

Finding from the production composer e2e: after a successful POST /api/v1/notes/journal/log/batch (201), GET /api/v1/notes/journal/ returned 409 Log entry has no block ID. The batch allocated IDs for its response but did not set them on each Markdown LogEntry before the durable write. I am fixing this and adding a regression assertion against the persisted day.

Finding from the production composer e2e: after a successful POST /api/v1/notes/journal/log/batch (201), GET /api/v1/notes/journal/<day> returned 409 `Log entry has no block ID`. The batch allocated IDs for its response but did not set them on each Markdown LogEntry before the durable write. I am fixing this and adding a regression assertion against the persisted day.

kayg commented

2026-09-29 19:43:04 +00:00

Author

Owner

The production Composer E2E reached its screenshot phase, then failed because the account's default appearance was system while Playwright's browser scheme was light. The harness compared the literal setting to the browser scheme. Updated the primary E2E page to persist Paper/light before the interaction screenshots, matching their stated theme and making the assertion concrete. Rerunning the production E2E.

The production Composer E2E reached its screenshot phase, then failed because the account's default appearance was `system` while Playwright's browser scheme was `light`. The harness compared the literal setting to the browser scheme. Updated the primary E2E page to persist Paper/light before the interaction screenshots, matching their stated theme and making the assertion concrete. Rerunning the production E2E.

kayg commented

2026-09-29 19:46:23 +00:00

Author

Owner

The E2E rerun passed the theme setup and reached the new optimistic rollback case. That test tried to click the screen-reader-only Stack button while the NLP chip row covered its hit target, so Playwright timed out before sending. Updated it to use the existing keyboard stacking gesture (two Enter presses); rerunning the full production Composer E2E.

kayg commented

2026-09-29 19:54:08 +00:00

Author

Owner

Adversarial UI run found a real optimistic-send race. When a Calendar range read started after the pending row was inserted but before the batch server acknowledgement, keepNewerCalendarLogs treated the read as authoritative and removed the pending row. The affected E2E exposed this only after earlier Calendar refreshes. I am changing range merges to retain pending rows until acknowledgement and adding regression coverage for both late reads and a server row that arrives before the acknowledgement callback.

Adversarial UI run found a real optimistic-send race. When a Calendar range read started after the pending row was inserted but before the batch server acknowledgement, `keepNewerCalendarLogs` treated the read as authoritative and removed the pending row. The affected E2E exposed this only after earlier Calendar refreshes. I am changing range merges to retain pending rows until acknowledgement and adding regression coverage for both late reads and a server row that arrives before the acknowledgement callback.

kayg commented

2026-09-29 20:01:29 +00:00

Author

Owner

Resuming #468 in job/send-fast. The branch was based on 918b4764aded5aa385ef784acb8887cf4361bf36; the current committed head is 64e2efbbb41c1873b7390ff38144cc9c3b3b9ffd. The worktree has uncommitted batch-send changes from the interrupted run. I am reviewing and validating those changes before committing them, then I will continue with the remaining measurement, tracing, and gates.

Resuming #468 in `job/send-fast`. The branch was based on `918b4764aded5aa385ef784acb8887cf4361bf36`; the current committed head is `64e2efbbb41c1873b7390ff38144cc9c3b3b9ffd`. The worktree has uncommitted batch-send changes from the interrupted run. I am reviewing and validating those changes before committing them, then I will continue with the remaining measurement, tracing, and gates.

kayg commented

2026-09-29 20:07:31 +00:00

Author

Owner

Review finding with evidence: the batch handler returns after durable Markdown writes while notes.log-batch-index is still queued. At acknowledgement, reconcilePendingCreates changes the Calendar row to pending: false, and composerSaved schedules a range refresh immediately. If that read finishes before the projection job indexes the Daily note, keepNewerCalendarLogs sees an equal version and no pending row, then replaces the optimistic row with the stale response. The file change feed can trigger the same race. I am adding an acked-but-unprojected state so the row remains visible until a range response contains its server ID, with a regression test.

Review finding with evidence: the batch handler returns after durable Markdown writes while `notes.log-batch-index` is still queued. At acknowledgement, `reconcilePendingCreates` changes the Calendar row to `pending: false`, and `composerSaved` schedules a range refresh immediately. If that read finishes before the projection job indexes the Daily note, `keepNewerCalendarLogs` sees an equal version and no pending row, then replaces the optimistic row with the stale response. The file change feed can trigger the same race. I am adding an acked-but-unprojected state so the row remains visible until a range response contains its server ID, with a regression test.

kayg commented

2026-09-30 01:15:17 +00:00

Author

Owner

Gate finding on job/send-fast: cargo test -p calternal-plugin-notes compiled the crate, then aborted in tests::task_create_attach_tick_detach_and_replay with thread ... has overflowed its stack and signal: 6, SIGABRT. The failure occurred after 65 tests had passed. I am isolating the test before attributing it to the change; no test expectation was changed.

Gate finding on `job/send-fast`: `cargo test -p calternal-plugin-notes` compiled the crate, then aborted in `tests::task_create_attach_tick_detach_and_replay` with `thread ... has overflowed its stack` and `signal: 6, SIGABRT`. The failure occurred after 65 tests had passed. I am isolating the test before attributing it to the change; no test expectation was changed.

kayg commented

2026-09-30 01:29:20 +00:00

Author

Owner

Isolation evidence for the Notes task-test crash: task_create_attach_tick_detach_and_replay passes with the filesystem trace commit at its parent revision, but overflows its test thread's stack with the trace commit present. The cause is the #[instrument] wrapper on calternal-fs::Root::write_inner, a large async future. I removed that wrapper and moved the atomic_write_and_fsync spans onto synchronous fsync and journal-publication boundaries; I am validating this change now.

Isolation evidence for the Notes task-test crash: `task_create_attach_tick_detach_and_replay` passes with the filesystem trace commit at its parent revision, but overflows its test thread's stack with the trace commit present. The cause is the `#[instrument]` wrapper on `calternal-fs::Root::write_inner`, a large async future. I removed that wrapper and moved the `atomic_write_and_fsync` spans onto synchronous fsync and journal-publication boundaries; I am validating this change now.

kayg referenced this issue from a commit

2026-09-30 01:50:59 +00:00

perf(server): opt-in write-stage tracing behind server.trace_writes

kayg referenced this issue from a commit

2026-09-30 01:51:00 +00:00

feat(cli,mcp): send Log entries through the durable batch endpoint

kayg referenced this issue from a commit

2026-09-30 01:51:00 +00:00

test(adversarial): Log batch probe (1,000 entries, 50 days, concurrency, editor)

kayg referenced this issue from a commit

2026-09-30 01:51:00 +00:00

test(notes): a crash before Log batch projection is repaired from the queue

kayg referenced this issue from a commit

2026-09-30 02:11:59 +00:00

fix(notes): initialise alarms in the Log batch etag after the dev merge

kayg referenced this issue from a commit

2026-09-30 02:22:06 +00:00

perf(e2e): in-page visible timing and the interleaved #468 A/B result

kayg referenced this issue from a commit

2026-09-30 02:43:09 +00:00

test(e2e): make the optimistic-send and rollback cases deterministic

kayg referenced this issue

2026-09-30 02:47:18 +00:00

Non-ASCII tags are truncated by the Log line reader ('#café' becomes tag 'caf' plus 'é' in the title) #473

kayg referenced this issue from a commit

2026-09-30 02:52:49 +00:00

test(adversarial): Log batch probe passes against a real server

kayg referenced this issue from a commit

2026-09-30 02:52:49 +00:00

docs(perf): describe the in-page visible time of the send probe

kayg commented

2026-09-30 03:12:09 +00:00

Author

Owner

Continued after the usage-limit stop (Claude). Head: 77db1f5774c66a3581d4ec3e1ef203461c422c3a on job/send-fast. origin/dev was merged twice, most recently at 99288dcfe. Not pushed or merged.

What landed since the stop

The uncommitted tracing work was reviewed and committed. server.trace_writes is off by default. It adds a coarse write span per request that never logs paths, and stage spans for Daily note read, Markdown rewrite, fsync, version snapshot, index/FTS, CalDAV sync tokens, Search reindex and notifications (0a4192e75).
The CLI log command and the MCP calternal_create_log tool now use /journal/log/batch (#395 parity) (63ba8e64a).
Crash repair is proven: composer_log_batch_projection_is_repaired_from_the_queued_job. The projection job row is committed before the Daily note write. The test checks that after the response (entries durable, no projection rows), a fresh handler leased from the durable queue rebuilds the Calendar rows (169a13e8c).
Merge fix: JournalEvent::alarms (#469) in the batch etag (959e02c30).
Composer e2e fixes (0c5cb1faf). The rollback wait compared an unawaited count() Promise with 0, so it could never pass. The 2 sent, 2 failed expectation was wrong (nothing is acknowledged, so it is 0 sent, 2 failed). 3e2 and 3e3 now run on the day view, because the week grid folded the pending blocks into "+N". The production composer e2e passes, including optimistic rows <= 50 ms with cold NLP and a visible rollback, with drafts kept, on failure.
Adversarial Log batch probe passes on a real release server: 1,000 entries over 50 days, 10 concurrent same-day batches, Unicode, and a live Note editor open during a batch, with no lost writes (1a67d3d1c). Three failures in the first runs were probe bugs, not lost writes. Entry counts matched every time. Titles ending in bare numbers are read as times, and the editor save is debounced.
Filed #473 (non-blocking): the line reader truncates non-ASCII tags (#café becomes tag caf plus é in the title). This behaviour already existed; the batch reuses the parser.
The cross-user classification gate (#472) classifies the new route: 308 operations classified, test_xuser_classification.py OK.

Before / after (interleaved A/B, build host)

The perf-test VM was busy with the perf round, so this ran on the shared build host (load 9.7–15.5). A = origin/dev 55a2f90fe and B = 959e02c30, both release builds with production SPA. 5 rounds, with the order alternated each round. Send 1 and Send 5 have 10 samples each; Send 20 has 5. In-page visible = capture-phase click event to the DOM commit that shows every title.

Entries	Requests A→B	Ack median A→B	Ack p95 A→B	In-page visible median A→B	In-page visible p95 A→B
1	1→1	357→365 ms	3,901→495 ms	225→23 ms	3,770→61 ms
5	5→1	978→315 ms	1,369→549 ms	886→17 ms	1,116→88 ms
20	20→1	6,252→521 ms	8,342→555 ms	6,096→51 ms	8,060→97 ms

Trace of a Send 5 batch: the handler takes 100–250 ms, most of it atomic_write_and_fsync (44–145 ms on this host) and the version snapshot (16–20 ms). The Search reindex (185–310 ms) and the projection job run after the response. The Send 1 acknowledgement does not change, because it is one durable write in both builds. The VM targets (Send 5 ack <= 150 ms p95, Send 20 <= 400 ms p95) are not yet verified. The build host is disk-bound and loaded, so they need a locked perf-VM run. Samples are in docs/perf/runs/2026-09-30T0419Z-issue-468-send-fast-ab.json, and the table is in docs/perf/runs/2026-09-29-issue-468-send-fast.md.

Gates (after the final merge, `77db1f577`)

fmt exit=0
clippy calternal-fs exit=0
clippy calternal-notes-core exit=0
clippy calternal-plugin-notes exit=0
clippy calternal-search exit=0
clippy calternal-plugin-notifications exit=0
clippy calternal-cli exit=0
clippy calternal-server exit=0
test calternal-fs exit=0          (39 + 42 passed)
test calternal-notes-core exit=0  (503 + 13 + 5 + 7 + 12 passed)
test calternal-plugin-notes exit=0 (126 + 1 passed)
test calternal-search exit=0
test calternal-plugin-notifications exit=0 (24 passed)
test calternal-cli exit=0 (26 + 15 passed)
test calternal-server exit=0 (85 passed, 2 ignored)
bun run check: COMPLETED 1949 FILES 0 ERRORS 0 WARNINGS 0 FILES_WITH_PROBLEMS
bun run test:  Test Files  136 passed (136) / Tests  877 passed (877)

The earlier vtodo_wire_upgrade_rotates_sync_epoch_once failure no longer occurs after the dev merge.

Open

A perf-VM run under flock /root/perf.lock for the absolute targets.
The second half of the issue (profiling task, note, event, file and tag write paths) has not started.

Continued after the usage-limit stop (Claude). Head: `77db1f5774c66a3581d4ec3e1ef203461c422c3a` on `job/send-fast`. `origin/dev` was merged twice, most recently at `99288dcfe`. Not pushed or merged. ## What landed since the stop - The uncommitted tracing work was reviewed and committed. `server.trace_writes` is off by default. It adds a coarse write span per request that never logs paths, and stage spans for Daily note read, Markdown rewrite, fsync, version snapshot, index/FTS, CalDAV sync tokens, Search reindex and notifications (`0a4192e75`). - The CLI `log` command and the MCP `calternal_create_log` tool now use `/journal/log/batch` (#395 parity) (`63ba8e64a`). - Crash repair is proven: `composer_log_batch_projection_is_repaired_from_the_queued_job`. The projection job row is committed before the Daily note write. The test checks that after the response (entries durable, no projection rows), a fresh handler leased from the durable queue rebuilds the Calendar rows (`169a13e8c`). - Merge fix: `JournalEvent::alarms` (#469) in the batch etag (`959e02c30`). - Composer e2e fixes (`0c5cb1faf`). The rollback wait compared an unawaited `count()` Promise with 0, so it could never pass. The `2 sent, 2 failed` expectation was wrong (nothing is acknowledged, so it is `0 sent, 2 failed`). 3e2 and 3e3 now run on the day view, because the week grid folded the pending blocks into "+N". The production composer e2e passes, including optimistic rows <= 50 ms with cold NLP and a visible rollback, with drafts kept, on failure. - Adversarial Log batch probe passes on a real release server: 1,000 entries over 50 days, 10 concurrent same-day batches, Unicode, and a live Note editor open during a batch, with no lost writes (`1a67d3d1c`). Three failures in the first runs were probe bugs, not lost writes. Entry counts matched every time. Titles ending in bare numbers are read as times, and the editor save is debounced. - Filed #473 (non-blocking): the line reader truncates non-ASCII tags (`#café` becomes tag `caf` plus `é` in the title). This behaviour already existed; the batch reuses the parser. - The cross-user classification gate (#472) classifies the new route: `308 operations classified`, `test_xuser_classification.py` OK. ## Before / after (interleaved A/B, build host) The perf-test VM was busy with the perf round, so this ran on the shared build host (load 9.7–15.5). A = `origin/dev` `55a2f90fe` and B = `959e02c30`, both release builds with production SPA. 5 rounds, with the order alternated each round. Send 1 and Send 5 have 10 samples each; Send 20 has 5. In-page visible = capture-phase click event to the DOM commit that shows every title. | Entries | Requests A→B | Ack median A→B | Ack p95 A→B | In-page visible median A→B | In-page visible p95 A→B | |---:|---:|---:|---:|---:|---:| | 1 | 1→1 | 357→365 ms | 3,901→495 ms | 225→23 ms | 3,770→61 ms | | 5 | 5→1 | 978→315 ms | 1,369→549 ms | 886→17 ms | 1,116→88 ms | | 20 | 20→1 | 6,252→521 ms | 8,342→555 ms | 6,096→51 ms | 8,060→97 ms | Trace of a Send 5 batch: the handler takes 100–250 ms, most of it `atomic_write_and_fsync` (44–145 ms on this host) and the version snapshot (16–20 ms). The Search reindex (185–310 ms) and the projection job run after the response. The Send 1 acknowledgement does not change, because it is one durable write in both builds. The VM targets (Send 5 ack <= 150 ms p95, Send 20 <= 400 ms p95) are **not yet verified**. The build host is disk-bound and loaded, so they need a locked perf-VM run. Samples are in `docs/perf/runs/2026-09-30T0419Z-issue-468-send-fast-ab.json`, and the table is in `docs/perf/runs/2026-09-29-issue-468-send-fast.md`. ## Gates (after the final merge, `77db1f577`) ``` fmt exit=0 clippy calternal-fs exit=0 clippy calternal-notes-core exit=0 clippy calternal-plugin-notes exit=0 clippy calternal-search exit=0 clippy calternal-plugin-notifications exit=0 clippy calternal-cli exit=0 clippy calternal-server exit=0 test calternal-fs exit=0 (39 + 42 passed) test calternal-notes-core exit=0 (503 + 13 + 5 + 7 + 12 passed) test calternal-plugin-notes exit=0 (126 + 1 passed) test calternal-search exit=0 test calternal-plugin-notifications exit=0 (24 passed) test calternal-cli exit=0 (26 + 15 passed) test calternal-server exit=0 (85 passed, 2 ignored) bun run check: COMPLETED 1949 FILES 0 ERRORS 0 WARNINGS 0 FILES_WITH_PROBLEMS bun run test: Test Files 136 passed (136) / Tests 877 passed (877) ``` The earlier `vtodo_wire_upgrade_rotates_sync_epoch_once` failure no longer occurs after the dev merge. ## Open - A perf-VM run under `flock /root/perf.lock` for the absolute targets. - The second half of the issue (profiling task, note, event, file and tag write paths) has not started.

kayg referenced this issue from a commit

2026-09-30 03:12:46 +00:00

Merge job/send-fast (#468: composer Send N is one batched request; derived work (search, calendar, reminders) queued durably before the write and run after the reply; optimistic UI with visible rollback; Send 5 on-screen median 886->17 ms)

kayg commented

2026-09-30 03:12:46 +00:00

Author

Owner

Log batch path merged into dev at dfb5964a2. Left open for: the perf-VM check of the p95 targets, and profiling the other write paths (task, note, event, file, tag).

Log batch path merged into dev at dfb5964a2. Left open for: the perf-VM check of the p95 targets, and profiling the other write paths (task, note, event, file, tag).

kayg referenced this issue

2026-09-30 03:43:05 +00:00

SECURITY: concurrent dedup can corrupt files across Users #375

kayg referenced this issue

2026-09-30 03:48:47 +00:00

LOCATION: one Location setting and its uses — location reminders via Apple Reminders geofencing (grill) #391

kayg referenced this issue

2026-09-30 05:03:48 +00:00

Composer: text selection is not drawn (double-click selects but shows no highlight) #419

kayg referenced this issue

2026-09-30 05:06:04 +00:00

SECURITY: concurrent dedup can corrupt files across Users #375

kayg referenced this issue

2026-09-30 06:45:54 +00:00

Log attachments: deleted file still shown, attachments not live, batch send not live, Log order, raw camera names #427

kayg referenced this issue

2026-09-30 09:30:41 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg referenced this issue

2026-10-02 13:25:21 +00:00

BLOCKER: retrying a Composer linked-body failure creates duplicate acknowledged Logs #844

kayg commented

2026-10-03 12:47:50 +00:00

Author

Owner

Audit against origin/dev: commit dfb5964a2 lands the Log batch path, but this issue also requires the perf-VM target check and profiling of Task, Note, Event, File, and Tag writes. The latest report and write-paths-468 queue entry confirm those parts remain open.

Audit against origin/dev: commit `dfb5964a2` lands the Log batch path, but this issue also requires the perf-VM target check and profiling of Task, Note, Event, File, and Tag writes. The latest report and `write-paths-468` queue entry confirm those parts remain open.

kayg referenced this issue

2026-10-03 12:51:31 +00:00

Issue hygiene sweep 2026-10-03 #1008

Rows
Columns

PERF: composer 'Send 5' is slow; batch write path, derived work off the request path, optimistic UI #468

Owner (2026-09-29): "In the composer, 'Send 5' items takes tooooo long! Please optimise our plugins to be extremely fast!"

1. Measure (before changing anything)

2. Fix at the root (keep a fix only if it wins by at least 10%, median of 5 interleaved A/B runs; durability must not weaken: an acknowledged write stays durable)

Targets

Proof

What landed since the stop

Before / after (interleaved A/B, build host)

Gates (after the final merge, 77db1f577)

Open

Gates (after the final merge, `77db1f577`)