Editor rewrites an untouched Note's Markdown bytes on open

kayg commented

2026-09-28 15:58:02 +00:00

Owner

Evidence

A one-time real-server browser adversarial run on job/money-format at merged head 25e88ffccbab8e1cc41a67ddbc08ec9d5ccff050 reported that opening an untouched Note rewrote its Markdown bytes. The probe used seed 25608414. The sample included nested lists, an angle-bracket image link, and frontmatter quoting.

The run reported a byte mismatch without text input. It did not establish whether the transformed bytes changed rendered content or metadata semantically. The run was time-boxed and shared-host load was high, so reproduce before assigning a cause.

Follow-up

Add a deterministic regression that captures the source bytes, opens the Note without editing it, and checks persistence. Identify which serialization step writes the changed form. Do not weaken existing byte or content expectations to make the probe pass.

## Evidence A one-time real-server browser adversarial run on `job/money-format` at merged head `25e88ffccbab8e1cc41a67ddbc08ec9d5ccff050` reported that opening an untouched Note rewrote its Markdown bytes. The probe used seed `25608414`. The sample included nested lists, an angle-bracket image link, and frontmatter quoting. The run reported a byte mismatch without text input. It did not establish whether the transformed bytes changed rendered content or metadata semantically. The run was time-boxed and shared-host load was high, so reproduce before assigning a cause. ## Follow-up Add a deterministic regression that captures the source bytes, opens the Note without editing it, and checks persistence. Identify which serialization step writes the changed form. Do not weaken existing byte or content expectations to make the probe pass.

structured-dark-390.png

122 KiB

structured-dark-820.png

287 KiB

structured-light-1440.png

798 KiB

structured-light-390.png

149 KiB

structured-dark-1440.png

359 KiB

structured-light-820.png

320 KiB

kayg referenced this issue

2026-09-28 16:01:43 +00:00

SPIKE: Money plain-text format — can an hledger superset hold YNAB/Actual budgets losslessly? Else design a simple calternal format (NO product code) #335

kayg commented

2026-09-28 16:06:15 +00:00

Author

Owner

Started work on branch job/note-rewrite at base 887c96fd7e. I have read CLAUDE.md, CONTEXT.md, and the relevant Note and Markdown contract sections of docs/DESIGN.md. I will reproduce the byte rewrite against the local server and production web build before changing the save path or serializers.

Started work on branch job/note-rewrite at base 887c96fd7e818616695acb19e69e7c95771ef301. I have read CLAUDE.md, CONTEXT.md, and the relevant Note and Markdown contract sections of docs/DESIGN.md. I will reproduce the byte rewrite against the local server and production web build before changing the save path or serializers.

kayg commented

2026-09-28 16:19:01 +00:00

Author

Owner

Finding: the TypeScript Markdown converter is not a byte fixpoint for a body containing CRLF and tab-indented nested list items. A direct markdownToDoc → docToMarkdown check changed the body from CRLF and tab indentation to LF and expanded list indentation. The production browser probe is being prepared to confirm whether this reaches disk on open and close.

Finding: the TypeScript Markdown converter is not a byte fixpoint for a body containing CRLF and tab-indented nested list items. A direct `markdownToDoc` → `docToMarkdown` check changed the body from CRLF and tab indentation to LF and expanded list indentation. The production browser probe is being prepared to confirm whether this reaches disk on open and close.

kayg commented

2026-09-28 17:08:55 +00:00

Author

Owner

Finding: the new Notes unit regression fails when collab_write receives the body returned by collab_read unchanged. The etag changes from cdb4143c953e9e62064f8e6f11e89a1baddec0f38059483133c34e258ae75fd9 to de9cfba8aae30110cd35a142dad622c6a934ca04d0e3c593f1478c8bcb665c5b; the live write path updates frontmatter and replaces the file despite no body edit. The test will also assert unchanged bytes and file fingerprint once the boundary is fixed.

Finding: the new Notes unit regression fails when `collab_write` receives the body returned by `collab_read` unchanged. The etag changes from `cdb4143c953e9e62064f8e6f11e89a1baddec0f38059483133c34e258ae75fd9` to `de9cfba8aae30110cd35a142dad622c6a934ca04d0e3c593f1478c8bcb665c5b`; the live write path updates frontmatter and replaces the file despite no body edit. The test will also assert unchanged bytes and file fingerprint once the boundary is fixed.

kayg commented

2026-09-28 17:16:43 +00:00

Author

Owner

Finding: the focused production browser probe on the local server reproduced the write before any input. During the five-second idle window the file grew from 319 to 332 bytes and mtime changed (1790615654094820458 → 1790615658426768650). The Markdown body stayed byte-identical. The frontmatter changed from quoted title/date values to normalized plain values and gained slug. The fixture title differed from its first heading, pointing to the Note view heading-retitle callback during load; I am tracing and will gate retitling on a real user edit.

Finding: the focused production browser probe on the local server reproduced the write before any input. During the five-second idle window the file grew from 319 to 332 bytes and mtime changed (`1790615654094820458` → `1790615658426768650`). The Markdown body stayed byte-identical. The frontmatter changed from quoted `title`/`date` values to normalized plain values and gained `slug`. The fixture title differed from its first heading, pointing to the Note view heading-retitle callback during load; I am tracing and will gate retitling on a real user edit.

kayg referenced this issue

2026-09-28 17:35:11 +00:00

PERF ROUND: benchmark + optimise average and worst-case CPU/memory (after current merges) #367

kayg commented

2026-09-28 17:46:15 +00:00

Author

Owner

A second production-path issue is confirmed by the new Hub integration regression: ending a clean AgentTurn flushes a room with no Yjs document edit. The flush changes the last-edited field, converts CRLF to LF, and changes a tab-indented nested list to spaces. The byte and fingerprint assertions fail before the fix.

kayg commented

2026-09-28 18:03:16 +00:00

Author

Owner

A real edit in one paragraph also converted an unrelated unique wikilink elsewhere in the Note: Target Note became alias. The Markdown line patch preserved its bytes, but the Notes save hook still ran link resolution over the whole body. The live collab save now resolves only the changed byte range; I am verifying it with a matching target Note.

A real edit in one paragraph also converted an unrelated unique wikilink elsewhere in the Note: [[Target Note|alias]] became [alias](<./n1.md>). The Markdown line patch preserved its bytes, but the Notes save hook still ran link resolution over the whole body. The live collab save now resolves only the changed byte range; I am verifying it with a matching target Note.

kayg commented

2026-09-28 18:19:59 +00:00

Author

Owner

Resuming issue #361 on branch job/note-rewrite from base 887c96fd7e818616695acb19e69e7c95771ef301 (HEAD edce19bb). Preserving the two existing commits and all seven uncommitted files from the interrupted run. I will validate the existing patch, reproduce byte and mtime stability with the real server and production web build, then merge dev once before final gates.

Resuming issue #361 on branch `job/note-rewrite` from base `887c96fd7e818616695acb19e69e7c95771ef301` (HEAD `edce19bb`). Preserving the two existing commits and all seven uncommitted files from the interrupted run. I will validate the existing patch, reproduce byte and mtime stability with the real server and production web build, then merge `dev` once before final gates.

kayg commented

2026-09-28 18:52:02 +00:00

Author

Owner

Finding: a real live edit previously called the general set_note_frontmatter writer to change last edited. That writer re-emits all managed fields, so an untouched quoted date or other managed scalar could change spelling during a body edit. I added a regression with quoted frontmatter, CRLF, custom fields, trailing spaces, and no final newline; the edit must change only its body line and the timestamp line. It passes with a small public set_note_frontmatter_scalar helper in calternal-notes-core, which reuses the existing YAML-safe scalar encoder and raw-field updater. Focused core and Notes tests pass.

Finding: a real live edit previously called the general `set_note_frontmatter` writer to change `last edited`. That writer re-emits all managed fields, so an untouched quoted `date` or other managed scalar could change spelling during a body edit. I added a regression with quoted frontmatter, CRLF, custom fields, trailing spaces, and no final newline; the edit must change only its body line and the timestamp line. It passes with a small public `set_note_frontmatter_scalar` helper in `calternal-notes-core`, which reuses the existing YAML-safe scalar encoder and raw-field updater. Focused core and Notes tests pass.

kayg commented

2026-09-28 22:00:42 +00:00

Author

Owner

Completed Forgejo #361.

Branch: job/note-rewrite
Head: ec67bcd3dfca7a58a67dd512e56a87205a4db638 (pushed; origin matches).

Changes

A clean collaboration flush and a save of the exact loaded Note body now skip disk writes. The raw Markdown remains authoritative when serializer output only reflects normalization.
A real edit patches only its changed source-line range. Untouched CRLF/CR/LF endings, tabs, trailing spaces, frontmatter, and final-newline state remain in the source. If the source cannot be aligned to its canonical Markdown, the save fails closed rather than replacing the Note.
A body edit updates only the last edited scalar. Other frontmatter bytes and style stay intact. Wikilink resolution runs only on the changed region.
Opening a Note whose first heading differs from its title no longer retitles it during hydration. A later real heading change still updates the title.

Production reproduction

The production web build and real local server opened each fixture, waited 5 seconds past autosave/debounce, closed the editor, and compared bytes and nanosecond mtime:

structured: 319 bytes before, idle, and after close; mtime 1790625783945843394 at all three points; first byte difference -1.
byte-edges: 307 bytes before, idle, and after close; mtime 1790625801033655501 at all three points; first byte difference -1.

The fixtures cover nested lists, angle-bracket image links, quoted frontmatter, CRLF, trailing whitespace, no final newline, tabs, HTML blocks, wikilinks, and NFC/NFD text. The Rust untouched_bytes suite passed all 5 tests. The full workspace cargo test gate also passed.

Six production screenshots are attached:

Gates

Output excerpts below are verbatim. Each command exited 0.

$ cargo fmt --check
exit_code=0

$ cargo clippy --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 78m 43s
exit_code=0

$ cargo test
Finished `test` profile [unoptimized + debuginfo] target(s) in 16m 58s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 44.69s
exit_code=0

$ bun run check
svelte-check found 0 errors and 0 warnings
exit_code=0

$ bun run test
Test Files  113 passed (113)
      Tests  745 passed (745)
   Duration  115.29s (transform 57%, environment 16%, import 16%, tests 8%, setup 4%)
exit_code=0

Decisions and known gaps

The save boundary treats byte-identical loaded content as a strict no-op, even if a dirty Yjs room serializes to a normalized equivalent.
The source patch uses one contiguous changed line range and preserves source bytes outside it. An unaligned source is rejected so the editor cannot silently reformat it.
The existing title and first heading may differ on open; hydration does not choose one over the other. A subsequent heading edit follows the current title-update behavior.
A real body edit continues to update last edited, but changes only that scalar.
The one adversarial round was scoped to the new untouched-Note production probe; the general adversarial suite was not run.

Files

apps/web/src/lib/notes/NoteView.svelte; crates/calternal-collab/src/markdown.rs; crates/calternal-collab/src/session.rs; crates/calternal-collab/tests/untouched_bytes.rs; crates/calternal-notes-core/src/frontmatter.rs; crates/calternal-notes-core/src/lib.rs; crates/plugins/notes/src/lib.rs; tests/adversarial/editor.mjs; tests/adversarial/run.sh.

Completed Forgejo #361. Branch: `job/note-rewrite` Head: `ec67bcd3dfca7a58a67dd512e56a87205a4db638` (pushed; origin matches). ## Changes - A clean collaboration flush and a save of the exact loaded Note body now skip disk writes. The raw Markdown remains authoritative when serializer output only reflects normalization. - A real edit patches only its changed source-line range. Untouched CRLF/CR/LF endings, tabs, trailing spaces, frontmatter, and final-newline state remain in the source. If the source cannot be aligned to its canonical Markdown, the save fails closed rather than replacing the Note. - A body edit updates only the `last edited` scalar. Other frontmatter bytes and style stay intact. Wikilink resolution runs only on the changed region. - Opening a Note whose first heading differs from its title no longer retitles it during hydration. A later real heading change still updates the title. ## Production reproduction The production web build and real local server opened each fixture, waited 5 seconds past autosave/debounce, closed the editor, and compared bytes and nanosecond mtime: - `structured`: 319 bytes before, idle, and after close; mtime `1790625783945843394` at all three points; first byte difference `-1`. - `byte-edges`: 307 bytes before, idle, and after close; mtime `1790625801033655501` at all three points; first byte difference `-1`. The fixtures cover nested lists, angle-bracket image links, quoted frontmatter, CRLF, trailing whitespace, no final newline, tabs, HTML blocks, wikilinks, and NFC/NFD text. The Rust `untouched_bytes` suite passed all 5 tests. The full workspace `cargo test` gate also passed. Six production screenshots are attached: - [390 px, light](https://git.kayg.org/attachments/c0bcab49-7e62-4187-8cdf-60a56f9cf9d5), [390 px, dark](https://git.kayg.org/attachments/a4a91e88-ee57-463a-920f-84d93caf542b) - [820 px, light](https://git.kayg.org/attachments/0973d5a4-2c4d-4281-967e-e6ec9308e0e9), [820 px, dark](https://git.kayg.org/attachments/30d4ea72-e8a9-4f15-87f3-3fc77ead35b0) - [1440 px, light](https://git.kayg.org/attachments/4a6a89e5-c0c8-48fa-b053-1032d899d07c), [1440 px, dark](https://git.kayg.org/attachments/5e2b3975-567e-40f2-86be-7c8412a5bc75) ## Gates Output excerpts below are verbatim. Each command exited 0. ```text $ cargo fmt --check exit_code=0 $ cargo clippy --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 78m 43s exit_code=0 $ cargo test Finished `test` profile [unoptimized + debuginfo] target(s) in 16m 58s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 44.69s exit_code=0 $ bun run check svelte-check found 0 errors and 0 warnings exit_code=0 $ bun run test Test Files 113 passed (113) Tests 745 passed (745) Duration 115.29s (transform 57%, environment 16%, import 16%, tests 8%, setup 4%) exit_code=0 ``` ## Decisions and known gaps - The save boundary treats byte-identical loaded content as a strict no-op, even if a dirty Yjs room serializes to a normalized equivalent. - The source patch uses one contiguous changed line range and preserves source bytes outside it. An unaligned source is rejected so the editor cannot silently reformat it. - The existing title and first heading may differ on open; hydration does not choose one over the other. A subsequent heading edit follows the current title-update behavior. - A real body edit continues to update `last edited`, but changes only that scalar. - The one adversarial round was scoped to the new untouched-Note production probe; the general adversarial suite was not run. ## Files `apps/web/src/lib/notes/NoteView.svelte`; `crates/calternal-collab/src/markdown.rs`; `crates/calternal-collab/src/session.rs`; `crates/calternal-collab/tests/untouched_bytes.rs`; `crates/calternal-notes-core/src/frontmatter.rs`; `crates/calternal-notes-core/src/lib.rs`; `crates/plugins/notes/src/lib.rs`; `tests/adversarial/editor.mjs`; `tests/adversarial/run.sh`.

kayg referenced this issue from a commit

2026-09-28 22:16:12 +00:00

Merge job/note-rewrite (#361: opening an untouched Note never writes; real edits patch only the changed region)

kayg commented

2026-09-28 22:31:06 +00:00

Author

Owner

Merged into dev by Claude after visual review (a9a16e90) and deployed to calternal.cloud. Closing.

Merged into dev by Claude after visual review (a9a16e90) and deployed to calternal.cloud. Closing.

kayg closed this issue

2026-09-28 22:31:07 +00:00

Rows
Columns

Editor rewrites an untouched Note's Markdown bytes on open #361

Evidence

Follow-up

Changes

Production reproduction

Gates

Decisions and known gaps

Files