kayg/calternal

Watch

Fork 0

Tags: rename/merge is slow — no full reconcile, batched parallel rewrite, optimistic UI #1186

New issue

Open

opened 2026-10-06 07:35:16 +00:00 by kayg · 7 comments

kayg commented

2026-10-06 07:35:16 +00:00

Owner

Owner report (2026-10-06): "updating tags takes too long (sequential? synchronous) — we are not writing performant code by default!"

Orchestrator reading of crates/calternal-tags/src/rewrite.rs: prepare runs index::reconcile_user (a full Home reconcile) before every rename/merge, then run_until processes targets one by one: take the namespace mutation lock, write the file, save the journal, refresh the Index, checkpoint (disk write) — per file.

Do (keep crash safety, resumability and exact Undo from #1110):

Measure first on the perf-test VM (root@10.69.69.63) with a Home of ~1,000 tagged items: wall time for rename and merge, p95 of other requests during it. Report before/after.
No full reconcile per operation: plan from the Index; verify each target's revision at write time instead (the checked writer already refuses stale bytes).
Batch: write targets in batches (e.g. 64) with bounded parallel file IO (the files are independent), one journal append and one checkpoint per batch, Index updates in one transaction per batch. Release the mutation lock between batches so ordinary edits are not starved.
UI: optimistic — the sidebar and views show the renamed tag immediately with a progress indicator for the background rewrite; Undo stays available.
Target: ≥10× faster on 1,000 items, no p95 regression for other requests.
Gates per crate + bun run check + bun run test quoted; numbers in the report and docs/perf/. A data-safety review follows.

## Owner report (2026-10-06): "updating tags takes too long (sequential? synchronous) — we are not writing performant code by default!" Orchestrator reading of crates/calternal-tags/src/rewrite.rs: `prepare` runs `index::reconcile_user` (a full Home reconcile) before every rename/merge, then `run_until` processes targets one by one: take the namespace mutation lock, write the file, save the journal, refresh the Index, checkpoint (disk write) — per file. Do (keep crash safety, resumability and exact Undo from #1110): 1. Measure first on the perf-test VM (root@10.69.69.63) with a Home of ~1,000 tagged items: wall time for rename and merge, p95 of other requests during it. Report before/after. 2. No full reconcile per operation: plan from the Index; verify each target's revision at write time instead (the checked writer already refuses stale bytes). 3. Batch: write targets in batches (e.g. 64) with bounded parallel file IO (the files are independent), one journal append and one checkpoint per batch, Index updates in one transaction per batch. Release the mutation lock between batches so ordinary edits are not starved. 4. UI: optimistic — the sidebar and views show the renamed tag immediately with a progress indicator for the background rewrite; Undo stays available. 5. Target: ≥10× faster on 1,000 items, no p95 regression for other requests. Gates per crate + `bun run check` + `bun run test` quoted; numbers in the report and docs/perf/. A data-safety review follows.

kayg commented

2026-10-06 09:31:59 +00:00

Author

Owner

Starting #1186 on job/tagperf-1186, based at f5fbced3c733370b5b37544d7816732628be2fd0 (worktree reports three commits behind origin/dev). I have read CLAUDE.md, CONTEXT.md, and the relevant Tag/index and instant interaction sections of docs/DESIGN.md. I am inspecting the existing benchmark and will record the ~1,000-item rename/merge baseline on the locked perf VM before changing the rewrite path.

Starting #1186 on `job/tagperf-1186`, based at `f5fbced3c733370b5b37544d7816732628be2fd0` (worktree reports three commits behind `origin/dev`). I have read `CLAUDE.md`, `CONTEXT.md`, and the relevant Tag/index and instant interaction sections of `docs/DESIGN.md`. I am inspecting the existing benchmark and will record the ~1,000-item rename/merge baseline on the locked perf VM before changing the rewrite path.

kayg referenced this issue from a commit

2026-10-06 10:14:55 +00:00

bench(tags): profile large rewrites under read load (#1186)

kayg commented

2026-10-06 10:31:41 +00:00

Author

Owner

Baseline setup is ready: the 1,000-item runner and concurrent request sampler are committed in 65a5a1f77 and 77629d50b, and a release server from base f5fbced3c is staged on the perf VM. Source inspection confirms rewrite::prepare calls index::reconcile_user before querying candidate Tag paths, then reads all indexed MIME values; run_until writes and checkpoints one source at a time, and refresh_index performs separate item lookups and writes. The required /root/perf.lock is held by another run (observed lock owners are flock and bash); this job is waiting for the exclusive measurement window. No baseline timings are available yet, and no implementation files have been changed.

Baseline setup is ready: the 1,000-item runner and concurrent request sampler are committed in `65a5a1f77` and `77629d50b`, and a release server from base `f5fbced3c` is staged on the perf VM. Source inspection confirms `rewrite::prepare` calls `index::reconcile_user` before querying candidate Tag paths, then reads all indexed MIME values; `run_until` writes and checkpoints one source at a time, and `refresh_index` performs separate item lookups and writes. The required `/root/perf.lock` is held by another run (observed lock owners are `flock` and `bash`); this job is waiting for the exclusive measurement window. No baseline timings are available yet, and no implementation files have been changed.

kayg referenced this issue from a commit

2026-10-06 10:35:04 +00:00

bench(tags): print rewrite sample progress (#1186)

kayg commented

2026-10-06 11:23:55 +00:00

Author

Owner

The first baseline attempt did not reach a rewrite sample. Seeding 1,000 Notes with 16 concurrent POSTs hit HTTP 500 at item 985 (Notes operation failed). The server log had repeated SQLite busy errors (sqlite_code=5) and a search-indexer TooManyOpenFiles error. The partial Home and profile were discarded. I am changing the fixture seeder to create Notes serially and report progress; this keeps setup outside the measured interval and avoids turning profile setup into a concurrency storm.

The first baseline attempt did not reach a rewrite sample. Seeding 1,000 Notes with 16 concurrent POSTs hit HTTP 500 at item 985 (`Notes operation failed`). The server log had repeated SQLite busy errors (`sqlite_code=5`) and a search-indexer `TooManyOpenFiles` error. The partial Home and profile were discarded. I am changing the fixture seeder to create Notes serially and report progress; this keeps setup outside the measured interval and avoids turning profile setup into a concurrency storm.

kayg commented

2026-10-06 13:28:02 +00:00

Author

Owner

The serial API seed reached 1,000 Notes, but the server then logged repeated Notes Index SQLite busy errors, and the final Tag-count read stalled beyond its 120-second timeout; /readyz also stopped responding (server RSS was about 811 MB). I discarded that Home and added crates/calternal-tags/examples/tag_rewrite_1186_seed.rs: it creates actual Markdown Notes through calternal-fs, runs one normal Tags reconciliation, and checks both Index counts. The profile now verifies that fixture before timing rewrites. cargo fmt --check, focused Clippy and all 44 calternal-tags tests pass; the baseline timing is still pending.

The serial API seed reached 1,000 Notes, but the server then logged repeated Notes Index SQLite busy errors, and the final Tag-count read stalled beyond its 120-second timeout; `/readyz` also stopped responding (server RSS was about 811 MB). I discarded that Home and added `crates/calternal-tags/examples/tag_rewrite_1186_seed.rs`: it creates actual Markdown Notes through `calternal-fs`, runs one normal Tags reconciliation, and checks both Index counts. The profile now verifies that fixture before timing rewrites. `cargo fmt --check`, focused Clippy and all 44 `calternal-tags` tests pass; the baseline timing is still pending.

kayg commented

2026-10-06 13:57:10 +00:00

Author

Owner

Performance baseline finding: the 1,000-item fixture is now seeded directly through calternal-fs and its Tag counts verify. Under the perf lock, the first baseline rename had not reached a terminal receipt after more than five minutes of observation, with no completed sample yet. The current path therefore includes substantial setup/serialized work; I am keeping the run active to capture end-to-end wall time and overlapping Tag-read p95 before implementation changes.

kayg commented

2026-10-06 14:14:23 +00:00

Author

Owner

Baseline finding: with the direct 1,000-Note fixture, the pre-change server accepted the first rename but did not reach a terminal receipt within the configured 1,800-second timeout. The locked VM reported load average 2.46875, 2.68115234375, 1.79248046875 at start and 5.14990234375, 7.09521484375, 6.75341796875 after stop. The initial profiler discarded concurrent-read samples on timeout, so it did not produce a p95; I am fixing that reporting path and will record the incomplete wall time as a lower bound rather than claim a completed sample.

kayg referenced this issue from a commit

2026-10-06 14:38:15 +00:00

perf(tags): batch durable rewrites and index updates (#1186)

kayg commented

2026-10-06 14:47:49 +00:00

Author

Owner

Finding for #1186: after the first batch implementation, receipt progress counted Sidecar files as source records. A folder Sidecar can represent many visible Files, so 1,000 tagged Files would show fewer than 1,000 completed items. Progress now counts the visible items in each source and validates journal cursors before indexing offsets. cargo clippy -p calternal-tags --all-targets -- -D warnings passed; cargo test -p calternal-tags passed (45 tests).

Finding for #1186: after the first batch implementation, receipt progress counted Sidecar files as source records. A folder Sidecar can represent many visible Files, so 1,000 tagged Files would show fewer than 1,000 completed items. Progress now counts the visible items in each source and validates journal cursors before indexing offsets. `cargo clippy -p calternal-tags --all-targets -- -D warnings` passed; `cargo test -p calternal-tags` passed (45 tests).

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/calternal#1186

No description provided.

Rows
Columns