kayg/calternal

Watch

Fork 0

Files: case-only folder rename leaves duplicates (Voice memos → Voice Memos); atomic case rename, case-insensitive routing, twin merge #621

New issue

Open

opened 2026-10-01 10:13:07 +00:00 by kayg · 6 comments

kayg commented

2026-10-01 10:13:07 +00:00

Owner

Owner report (2026-10-01): "Renaming the folder from Voice memos to Voice Memos seems to have created a lone voice memo there even though both are the same file?"

After a case-only rename of Documents/Voice memos → Documents/Voice Memos, a voice memo appears as a separate lone item in the renamed folder, as if the files were duplicated or the old folder was re-created (the composer still writes to the old lowercase path until #618 lands, so a new recording may have re-created Voice memos).
Investigate and fix (data-integrity class: never duplicate or lose files):

Reproduce: rename a folder by case only in Files (web) and over WebDAV (Finder), with files inside. Then record a voice memo.
A case-only rename must be a single atomic move in calternal-fs (via a temporary name), with one Files Index update. No duplicate rows, no stale index entries, and blob dedup keeps one copy.
Make the router write to the real existing folder case-insensitively (find an existing folder whose name matches ignoring case before creating one), so Voice memos is never re-created next to Voice Memos (this pairs with #618).
Repair: a one-time check finds case-twin folders (same parent, same name ignoring case), merges them safely (newest wins on true conflicts, with a conflict copy kept), and rewrites links. Report what it did.
Tests: Rust tests for the case-only rename (web + WebDAV MOVE), the case-insensitive folder lookup, and the twin merge; Files index consistency after each step. Per-crate gates.

## Owner report (2026-10-01): "Renaming the folder from Voice memos to Voice Memos seems to have created a lone voice memo there even though both are the same file?" After a **case-only rename** of `Documents/Voice memos` → `Documents/Voice Memos`, a voice memo appears as a separate lone item in the renamed folder, as if the files were duplicated or the old folder was re-created (the composer still writes to the old lowercase path until #618 lands, so a new recording may have re-created `Voice memos`). **Investigate and fix (data-integrity class: never duplicate or lose files):** 1. Reproduce: rename a folder by case only in Files (web) and over WebDAV (Finder), with files inside. Then record a voice memo. 2. A case-only rename must be a single atomic move in `calternal-fs` (via a temporary name), with one Files Index update. No duplicate rows, no stale index entries, and blob dedup keeps one copy. 3. Make the router write to the real existing folder case-insensitively (find an existing folder whose name matches ignoring case before creating one), so `Voice memos` is never re-created next to `Voice Memos` (this pairs with #618). 4. Repair: a one-time check finds case-twin folders (same parent, same name ignoring case), merges them safely (newest wins on true conflicts, with a conflict copy kept), and rewrites links. Report what it did. **Tests:** Rust tests for the case-only rename (web + WebDAV MOVE), the case-insensitive folder lookup, and the twin merge; Files index consistency after each step. Per-crate gates.

kayg commented

2026-10-02 01:22:27 +00:00

Author

Owner

Starting work on job/voicememos-618, based on 9a6c38c3b7c8bbbdad9919cd2419f773d9c34b6e (origin/dev, currently seven commits ahead of the branch). I am tracing the Files API, WebDAV provider, calternal-fs rename journal, and Notes link-rewrite paths before implementing the Title Case migration and case-twin repair.

Starting work on `job/voicememos-618`, based on `9a6c38c3b7c8bbbdad9919cd2419f773d9c34b6e` (`origin/dev`, currently seven commits ahead of the branch). I am tracing the Files API, WebDAV provider, `calternal-fs` rename journal, and Notes link-rewrite paths before implementing the Title Case migration and case-twin repair.

kayg commented

2026-10-02 01:27:40 +00:00

Author

Owner

Finding: crates/plugins/files/src/uploads.rs::create_impl takes the exact folder returned by attachment_folder() and calls mkdir_p on its parent. After a client renames Documents/Voice memos to Documents/Voice Memos, the old spelling is not resolved before the next recording, so Linux can create a second sibling folder. Both the Files API rename route and WebDAV MOVE call move_indexed, which performs one Index subtree update after the filesystem move. calternal-fs::Root::move_path currently journals one direct rename, with no intermediate name for case-only moves. I will fix these shared points and add a startup, retry-safe merge for voice memo case twins.

Finding: `crates/plugins/files/src/uploads.rs::create_impl` takes the exact folder returned by `attachment_folder()` and calls `mkdir_p` on its parent. After a client renames `Documents/Voice memos` to `Documents/Voice Memos`, the old spelling is not resolved before the next recording, so Linux can create a second sibling folder. Both the Files API rename route and WebDAV MOVE call `move_indexed`, which performs one Index subtree update after the filesystem move. `calternal-fs::Root::move_path` currently journals one direct rename, with no intermediate name for case-only moves. I will fix these shared points and add a startup, retry-safe merge for voice memo case twins.

kayg commented

2026-10-02 02:33:36 +00:00

Author

Owner

Journal recovery review found a case-only rename failure window: two separate Rename journal steps could finish the temporary-to-final rename, then fail before deleting the journal. Replay would retry the first step after both its source and temporary path were gone and report NotFound. I changed case-only moves to one replayable CaseRename journal step and added recovery coverage before, between and after the two underlying renames. The Markdown migration test also exposed that legacy \ destinations were not decoded; the rewrite now handles that spelling and emits a valid angle-bracket destination.

Journal recovery review found a case-only rename failure window: two separate Rename journal steps could finish the temporary-to-final rename, then fail before deleting the journal. Replay would retry the first step after both its source and temporary path were gone and report NotFound. I changed case-only moves to one replayable CaseRename journal step and added recovery coverage before, between and after the two underlying renames. The Markdown migration test also exposed that legacy `\ ` destinations were not decoded; the rewrite now handles that spelling and emits a valid angle-bracket destination.

kayg commented

2026-10-02 03:05:57 +00:00

Author

Owner

Finding during crash-safety review: the startup merge moved and indexed a Voice Memos folder before it rewrote ordinary Markdown attachment links. A crash after the durable Files move completed could therefore leave Notes links at the old spelling; the move intent is cleared before the later batch rewrite, so startup could not replay that rewrite. I am moving this rewrite inside the pending-move window and extending recovery for these paths. A regression test will simulate the interrupted move and verify the link and Index recover together.

kayg commented

2026-10-02 04:25:18 +00:00

Author

Owner

#621 complete on job/voicememos-618, HEAD df30c35a487a0a3eb9aedacbf102beec79e0050a.

Case-only Files rename and WebDAV MOVE now use one journaled temporary-name move and keep the folder/file Item IDs and one set of Files Index rows. The attachment router finds an existing case-folded Voice Memos folder, so recording after a rename does not recreate the old spelling. Startup repair merges case twins, keeps a conflict copy, and rewrites Notes and Log links. Recovery replays link rewrites for an interrupted move.

The one local real-server adversarial round passed Files rename, WebDAV MOVE, recording uploads, Index and byte checks, malformed metadata, and 16 concurrent recordings. Files crate tests passed: test result: ok. 153 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 138.12s. The full report, other gates, profile measurements, and the unrelated merged-origin/dev Notes test failure are on #618.

Decision for ties: the canonical target file wins when modification times match; the displaced file remains at a deterministic conflict-copy name derived from its stable Item ID.

#621 complete on `job/voicememos-618`, HEAD `df30c35a487a0a3eb9aedacbf102beec79e0050a`. Case-only Files rename and WebDAV MOVE now use one journaled temporary-name move and keep the folder/file Item IDs and one set of Files Index rows. The attachment router finds an existing case-folded Voice Memos folder, so recording after a rename does not recreate the old spelling. Startup repair merges case twins, keeps a conflict copy, and rewrites Notes and Log links. Recovery replays link rewrites for an interrupted move. The one local real-server adversarial round passed Files rename, WebDAV MOVE, recording uploads, Index and byte checks, malformed metadata, and 16 concurrent recordings. Files crate tests passed: `test result: ok. 153 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 138.12s`. The full report, other gates, profile measurements, and the unrelated merged-`origin/dev` Notes test failure are on #618. Decision for ties: the canonical target file wins when modification times match; the displaced file remains at a deterministic conflict-copy name derived from its stable Item ID.

kayg commented

2026-10-02 13:14:14 +00:00

Author

Owner

Filesystem audit evidence from sec-fs (#663), base c4a61e8cf0:

crates/calternal-fs/src/trash.rs:121–158 checks only exact destination absence during Root::restore. It does not call the shared ensure_name_available NFC/full-case-fold sibling check. The route in crates/plugins/files/src/lib.rs:3340–3412 also lacks it. These functions retain this behavior in round 7a at 2f4482ded0.

Please include restore in this issue's existing twin-prevention scope. Reuse the sibling check under the publication lock, reject a conflicting restore and keep the Trash entry intact. Do not silently normalize or retitle existing data. Add small tests for case and canonical Unicode conflicts for files, folders and paired Sidecars. This is source evidence of a policy inconsistency; no sync data loss was reproduced. No new duplicate issue was filed.

Filesystem audit evidence from sec-fs (#663), base c4a61e8cf090170f35b1bed3350d9de20c83ecd5: `crates/calternal-fs/src/trash.rs:121–158` checks only exact destination absence during `Root::restore`. It does not call the shared `ensure_name_available` NFC/full-case-fold sibling check. The route in `crates/plugins/files/src/lib.rs:3340–3412` also lacks it. These functions retain this behavior in round 7a at 2f4482ded066d9c5d9c59130377907f7fd2916c9. Please include restore in this issue's existing twin-prevention scope. Reuse the sibling check under the publication lock, reject a conflicting restore and keep the Trash entry intact. Do not silently normalize or retitle existing data. Add small tests for case and canonical Unicode conflicts for files, folders and paired Sidecars. This is source evidence of a policy inconsistency; no sync data loss was reproduced. No new duplicate issue was filed.

kayg referenced this issue

2026-10-02 13:14:15 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-02 13:17:21 +00:00

PERF: instant everywhere — app-wide speed architecture rules, audit and shared primitives #663

kayg referenced this issue

2026-10-03 12:51:32 +00:00

Issue hygiene sweep 2026-10-03 #1008