FOLLOW-UP from #235 adversarial round: DAV/Journal/bookmark timeouts, pasted-image undo duplication, 10k-block collab timeout

kayg commented

2026-09-27 19:55:50 +00:00

Owner

Non-SLOW findings from the #235 gate-fix adversarial round (2026-09-27, see #235 comments for evidence):

Editor: pasting an image then undo/redo duplicates the image (data-integrity class in the editor: must not duplicate content). Priority.
Collab: a 10,000-block note fails to sync within the timeout. Profile the Yjs/yrs path (initial sync size, update batching, persistence) — target: a 10k-block note opens and syncs in ≤ 2 s on the VM profile.
DAV sync and discovery requests time out under the round's load.
Calendar Event-from-Log, template creation, bookmark capture and the Journal PATCH storm time out (non-SLOW: requests never complete).
For each: reproduce on a quieter host, fix the root cause, add a regression. Items 1 and 2 first.

Non-SLOW findings from the #235 gate-fix adversarial round (2026-09-27, see #235 comments for evidence): 1. **Editor: pasting an image then undo/redo duplicates the image** (data-integrity class in the editor: must not duplicate content). Priority. 2. **Collab: a 10,000-block note fails to sync within the timeout.** Profile the Yjs/yrs path (initial sync size, update batching, persistence) — target: a 10k-block note opens and syncs in ≤ 2 s on the VM profile. 3. DAV sync and discovery requests time out under the round's load. 4. Calendar Event-from-Log, template creation, bookmark capture and the Journal PATCH storm time out (non-SLOW: requests never complete). For each: reproduce on a quieter host, fix the root cause, add a regression. Items 1 and 2 first.

kayg commented

2026-09-27 19:57:30 +00:00

Author

Owner

Starting Forgejo #265 on branch job/robust-2, based on origin/dev at 3ad937ec88. I fetched origin/dev; #191 is not present in its history, so item 1 remains gated. I am reproducing and fixing items 2, 3 and 4 in the requested order.

Starting Forgejo #265 on branch job/robust-2, based on origin/dev at 3ad937ec888c1fe45ce6e1d3052383e024dc5220. I fetched origin/dev; #191 is not present in its history, so item 1 remains gated. I am reproducing and fixing items 2, 3 and 4 in the requested order.

kayg commented

2026-09-27 20:31:40 +00:00

Author

Owner

Item 2 reproduction and root cause on job/robust-2: the real local editor probe timed out waiting for 10,000-block collaboration sync at 20 seconds. A focused server-side regression measured parse 707 ms, ProseMirror-to-Yrs construction 24.61 s, block index 80 ms, first sync encoding 87 ms for 616,204 bytes, and snapshot encoding 96 ms for 616,199 bytes. The delay is in crates/calternal-collab/src/lib.rs: each sibling is inserted at the current array length; Yrs 0.28 walks from the first child to resolve every indexed insertion, so building the 10,000-block XML fragment is quadratic. The new 2-second regression fails on this baseline as expected.

Item 2 reproduction and root cause on job/robust-2: the real local editor probe timed out waiting for 10,000-block collaboration sync at 20 seconds. A focused server-side regression measured parse 707 ms, ProseMirror-to-Yrs construction 24.61 s, block index 80 ms, first sync encoding 87 ms for 616,204 bytes, and snapshot encoding 96 ms for 616,199 bytes. The delay is in `crates/calternal-collab/src/lib.rs`: each sibling is inserted at the current array length; Yrs 0.28 walks from the first child to resolve every indexed insertion, so building the 10,000-block XML fragment is quadratic. The new 2-second regression fails on this baseline as expected.

kayg commented

2026-09-27 20:41:25 +00:00

Author

Owner

Item 2 fix committed as 075ba4fc. Reproduction on the baseline real server: the 10,000-block collaboration sync timed out at the probe's 20-second limit. The root cause was indexed Yrs insertion at the growing end of the XML fragment; Yrs 0.28 walks from the first child for each insertion, making construction quadratic. Building in reverse at index zero preserves the visible Markdown order and is linear.

The focused regression passed. Its 10,000-block phases were parse 568.5 ms, Yrs construction 516.7 ms, block index 45.6 ms, first sync 59.6 ms, snapshot 46.2 ms, total 1.236 s; it also checks the first and last blocks after round-trip. The real-server adversarial probe now has a 2,000 ms sync assertion. Full cargo test -p calternal-collab progressed through the Yjs vector test (231.67 s) but three existing hostile_clients timing/content tests failed under concurrent host load; I am recording those separately from this focused result.

Item 2 fix committed as 075ba4fc. Reproduction on the baseline real server: the 10,000-block collaboration sync timed out at the probe's 20-second limit. The root cause was indexed Yrs insertion at the growing end of the XML fragment; Yrs 0.28 walks from the first child for each insertion, making construction quadratic. Building in reverse at index zero preserves the visible Markdown order and is linear. The focused regression passed. Its 10,000-block phases were parse 568.5 ms, Yrs construction 516.7 ms, block index 45.6 ms, first sync 59.6 ms, snapshot 46.2 ms, total 1.236 s; it also checks the first and last blocks after round-trip. The real-server adversarial probe now has a 2,000 ms sync assertion. Full `cargo test -p calternal-collab` progressed through the Yjs vector test (231.67 s) but three existing `hostile_clients` timing/content tests failed under concurrent host load; I am recording those separately from this focused result.

kayg commented

2026-09-27 20:45:24 +00:00

Author

Owner

Item 3 reproduction/root-cause trace: the Notes performance fixture contains 2,000 real indexed Tasks. On baseline handlers, both PROPFIND /dav/calendars/{user}/ and PROPFIND /dav/calendars/{user}/reminders/ call RemindersProvider::list once and changes(None) once. The changes(None) implementation already loads and decodes the full initial snapshot. The focused regression failed with provider calls (list=1, changes=1) on all six discovery requests at 10, 200 and 2,000 Tasks. This duplicated DB query, JSON decode and iCalendar ETag work. The Calendar home also fetched and discarded the full Journal list. Initial Journal REPORT then fetched each changed Log entry with a separate get, causing one SQL lookup and filesystem parse per entry. I am removing these redundant scans and the initial REPORT N+1.

Item 3 reproduction/root-cause trace: the Notes performance fixture contains 2,000 real indexed Tasks. On baseline handlers, both `PROPFIND /dav/calendars/{user}/` and `PROPFIND /dav/calendars/{user}/reminders/` call `RemindersProvider::list` once and `changes(None)` once. The `changes(None)` implementation already loads and decodes the full initial snapshot. The focused regression failed with provider calls `(list=1, changes=1)` on all six discovery requests at 10, 200 and 2,000 Tasks. This duplicated DB query, JSON decode and iCalendar ETag work. The Calendar home also fetched and discarded the full Journal list. Initial Journal REPORT then fetched each changed Log entry with a separate `get`, causing one SQL lookup and filesystem parse per entry. I am removing these redundant scans and the initial REPORT N+1.

kayg commented

2026-09-27 20:52:56 +00:00

Author

Owner

Item 3 fix committed as 76cf1e4a. Calendar-home discovery now reads the Reminders sync token without enumerating Tasks. Reminders collection discovery uses the initial snapshot returned by changes(None) instead of also calling list. Read-only Reminders projection reads no longer wait on the Notes filesystem writer lock. Journal home discovery no longer scans and discards all Daily Notes, depth-0 Journal discovery skips the event scan, and Journal depth-1 discovery scans once. Initial Journal REPORT now gets all ETags from one event snapshot rather than issuing one provider get per Log entry.

Regression evidence: cargo test -p calternal-plugin-notes reminders_initial_report_timing_at_10_200_and_2000_tasks -- --nocapture passed. It ran discovery at 10, 200 and 2,000 Tasks and asserted home discovery made (list=0, changes=0, sync_token=1) provider calls, Reminders collection discovery made (0,1,0), and initial Journal sync made one list and zero per-entry gets. Initial Reminders REPORT at 2,000 Tasks measured 297.7 ms in this debug build. cargo test -p calternal-dav passed (10 tests, 0 failed; doc tests 0).

Item 3 fix committed as 76cf1e4a. Calendar-home discovery now reads the Reminders sync token without enumerating Tasks. Reminders collection discovery uses the initial snapshot returned by `changes(None)` instead of also calling `list`. Read-only Reminders projection reads no longer wait on the Notes filesystem writer lock. Journal home discovery no longer scans and discards all Daily Notes, depth-0 Journal discovery skips the event scan, and Journal depth-1 discovery scans once. Initial Journal REPORT now gets all ETags from one event snapshot rather than issuing one provider `get` per Log entry. Regression evidence: `cargo test -p calternal-plugin-notes reminders_initial_report_timing_at_10_200_and_2000_tasks -- --nocapture` passed. It ran discovery at 10, 200 and 2,000 Tasks and asserted home discovery made `(list=0, changes=0, sync_token=1)` provider calls, Reminders collection discovery made `(0,1,0)`, and initial Journal sync made one list and zero per-entry gets. Initial Reminders REPORT at 2,000 Tasks measured 297.7 ms in this debug build. `cargo test -p calternal-dav` passed (10 tests, 0 failed; doc tests 0).

kayg commented

2026-09-27 20:58:44 +00:00

Author

Owner

Item 4 template slice committed as 019c420f. The #235 adversarial report showed all 16 concurrent template creation requests timing out. The handler held the per-User writer lock while reading and rendering the template and resolving wikilinks, then queued every create before the actual write. I moved rendering and link resolution before the lock; directory creation, file write and Index update remain serialized. The new regression passed: 16 concurrent creates returned 201 with 16 unique IDs and paths (cargo test -p calternal-plugin-notes concurrent_template_creates_finish_with_distinct_note_ids -- --nocapture).

Item 4 template slice committed as 019c420f. The #235 adversarial report showed all 16 concurrent template creation requests timing out. The handler held the per-User writer lock while reading and rendering the template and resolving wikilinks, then queued every create before the actual write. I moved rendering and link resolution before the lock; directory creation, file write and Index update remain serialized. The new regression passed: 16 concurrent creates returned 201 with 16 unique IDs and paths (`cargo test -p calternal-plugin-notes concurrent_template_creates_finish_with_distinct_note_ids -- --nocapture`).

kayg commented

2026-09-27 21:03:04 +00:00

Author

Owner

Item 4 Journal PATCH fix committed as 8cadff07. The #235 reproduction sent 24 PATCH requests with the same If-Match; only 8 completed, all with 412, while 16 timed out. Each PATCH had acquired the per-User lock once for get, released it, then queued again for put. The PATCH path now keeps one lock from lookup through conditional validation and write, and passes the already-read Log entry to the shared Journal write path.

Regression evidence: cargo test -p calternal-plugin-notes journal_patch_storm_finishes_with_one_winner_and_stale_preconditions -- --nocapture passed in 2.08 seconds. It returned one 200 and 23 412 responses. cargo test -p calternal-plugin-notes journal_patch_ -- --nocapture passed all 6 matching tests, including cross-day move, CRLF preservation and checked-replace races.

Item 4 Journal PATCH fix committed as 8cadff07. The #235 reproduction sent 24 PATCH requests with the same `If-Match`; only 8 completed, all with `412`, while 16 timed out. Each PATCH had acquired the per-User lock once for `get`, released it, then queued again for `put`. The PATCH path now keeps one lock from lookup through conditional validation and write, and passes the already-read Log entry to the shared Journal write path. Regression evidence: `cargo test -p calternal-plugin-notes journal_patch_storm_finishes_with_one_winner_and_stale_preconditions -- --nocapture` passed in 2.08 seconds. It returned one `200` and 23 `412` responses. `cargo test -p calternal-plugin-notes journal_patch_ -- --nocapture` passed all 6 matching tests, including cross-day move, CRLF preservation and checked-replace races.

kayg referenced this issue

2026-09-27 21:06:30 +00:00

BUG: menus inside Settings/overlays have no blur (nested backdrop root); two rows highlighted #246

kayg referenced this issue

2026-09-27 21:06:48 +00:00

GATES: dev failures blocking the first calternal.cloud deploy #235

kayg commented

2026-09-27 21:23:21 +00:00

Author

Owner

Item 4 Event-from-Log and bookmark evidence: Event-from-Log previously read the same Log entry once in the route and again while building the Event deeplink; reverse linking then did two more provider reads. The route now uses the first read's stable Log ID and date, and Notes performs the checked reverse-link write under one lock without the final reread. cargo test -p calternal-plugin-notes linking_a_calendar_event -- --nocapture passed both link tests, including retry/idempotency and a concurrent child-line write. cargo test -p calternal-plugin-calendar --lib passed 46 tests.

Bookmark capture already has a per-User limit of four in-flight saves and returns 429 before excess requests queue behind the Notes writer. The long writer-lock holders were the DAV scans and PATCH path fixed above; the existing capture-storm regression passed (cargo test -p calternal-plugin-notes bookmark_capture_storm_rejects_excess_requests_before_the_user_lock_queue -- --nocapture). This slice keeps bookmark storage and that admission limit unchanged.

Item 4 Event-from-Log and bookmark evidence: Event-from-Log previously read the same Log entry once in the route and again while building the Event deeplink; reverse linking then did two more provider reads. The route now uses the first read's stable Log ID and date, and Notes performs the checked reverse-link write under one lock without the final reread. `cargo test -p calternal-plugin-notes linking_a_calendar_event -- --nocapture` passed both link tests, including retry/idempotency and a concurrent child-line write. `cargo test -p calternal-plugin-calendar --lib` passed 46 tests. Bookmark capture already has a per-User limit of four in-flight saves and returns `429` before excess requests queue behind the Notes writer. The long writer-lock holders were the DAV scans and PATCH path fixed above; the existing capture-storm regression passed (`cargo test -p calternal-plugin-notes bookmark_capture_storm_rejects_excess_requests_before_the_user_lock_queue -- --nocapture`). This slice keeps bookmark storage and that admission limit unchanged.

kayg commented

2026-09-27 21:54:57 +00:00

Author

Owner

Bookmark capture findings from the single final real-server adversarial round: a 16-request capture storm had four client timeouts and twelve 429 responses. The old per-User limit admitted four captures even though they serialized on the Notes writer lock. Commit 18f7578d lowers admission to one; the regression was RED with four saves before the fix and GREEN with exactly one save plus fifteen rejections.

The same round's Calendar projection probe did not find a newly captured bookmark within its existing 5-second poll. The Calendar range currently joins Notes rows through files_index, so this appears to be an index-arrival dependency. I am tracking this separately from the reported timeout fix. An unrelated authz matrix probe also received 422 rather than its expected 403 for malformed admin config; it was rejected without evidence of a state change.

Bookmark capture findings from the single final real-server adversarial round: a 16-request capture storm had four client timeouts and twelve `429` responses. The old per-User limit admitted four captures even though they serialized on the Notes writer lock. Commit `18f7578d` lowers admission to one; the regression was RED with four saves before the fix and GREEN with exactly one save plus fifteen rejections. The same round's Calendar projection probe did not find a newly captured bookmark within its existing 5-second poll. The Calendar range currently joins Notes rows through `files_index`, so this appears to be an index-arrival dependency. I am tracking this separately from the reported timeout fix. An unrelated authz matrix probe also received `422` rather than its expected `403` for malformed admin config; it was rejected without evidence of a state change.

kayg commented

2026-09-27 22:03:44 +00:00

Author

Owner

Follow-up to the bookmark projection finding: fixed in 193c6b64. Calendar range and grid queries now read timestamped bookmarks from the Notes Index, which owns their saved date, while the grid left-joins Files only to add an item ID when Files adopts the Note. Regressions cover the pre-adoption case, no duplicate after adoption, and the range/year projections. cargo test -p calternal-plugin-calendar passed: 47 unit tests, 1 cache integration test, 3 protocol integration tests, and doc tests (0 tests).

Follow-up to the bookmark projection finding: fixed in `193c6b64`. Calendar range and grid queries now read timestamped bookmarks from the Notes Index, which owns their saved date, while the grid left-joins Files only to add an item ID when Files adopts the Note. Regressions cover the pre-adoption case, no duplicate after adoption, and the range/year projections. `cargo test -p calternal-plugin-calendar` passed: 47 unit tests, 1 cache integration test, 3 protocol integration tests, and doc tests (0 tests).

kayg referenced this issue

2026-09-27 22:14:07 +00:00

BUG: menus inside Settings/overlays have no blur (nested backdrop root); two rows highlighted #246

kayg commented

2026-09-27 22:43:38 +00:00

Author

Owner

Finished Forgejo #265.

Branch: job/robust-2
Head: dd566bf593296465a555725f523d9eee0abd101d
Push: git push origin HEAD:job/robust-2 returned Everything up-to-date.

Built:

Collaboration sync now constructs Yrs top-level blocks in linear time. The 10,000-block regression passed in 1.236 s focused; the real-server probe measured 798 ms.
DAV discovery avoids discarded Journal scans and duplicate Reminder reads. Initial Journal sync uses one snapshot instead of per-entry reads.
Event-from-Log reuses the located Log identity and writes under one Notes lock. Template rendering happens before the per-User write lock. Journal PATCH holds one lock through ETag validation and write; a 24-request same-ETag storm produced one 200 and 23 412 responses.
Bookmark capture admits one request per User, so excess requests get 429 before building a writer-lock queue. Calendar range and grid read bookmarks from the Notes Index and add a Files item ID after adoption.

Files changed by this job: crates/calternal-collab/src/lib.rs, crates/calternal-collab/src/session.rs, crates/calternal-dav/src/protocol.rs, crates/plugins/notes/src/lib.rs, crates/plugins/notes/src/tasks_dav.rs, crates/plugins/notes/src/calendar_links.rs, crates/plugins/notes/src/bookmarks.rs, crates/plugins/calendar/src/routes.rs, crates/plugins/calendar/src/items.rs, crates/plugins/calendar/src/view.rs, and tests/adversarial/editor.mjs.

Gates, verbatim result lines:

cargo fmt --check — exit 0, no output.
cargo clippy --all-targets -- -D warnings — Finished [32mdev[0m profile [unoptimized + debuginfo] target(s) in 12m 18s; exit 0.
cargo test — exit 0. Relevant result lines: test result: 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.58s (collab unit tests), test result: 95 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 69.50s (Notes), and Calendar: 47 unit, 1 cache integration, 3 protocol integration tests passed; doc tests passed with 0 tests.
bun run check — svelte-check found 0 errors and 0 warnings.
bun run test — Test Files 89 passed (89) and Tests 619 passed (619).
cargo clean — Removed 19073 files, 17.5GiB total. Removed apps/web/build, apps/web/node_modules/.vite, and apps/web/.svelte-kit.

Adversarial round: the runner exited 1 with ROUND 2 FINDINGS 79; most listed entries were SLOW. The 10k sync passed. DAV, Event-from-Log, template, and Journal requests completed but several were marked SLOW under shared-host load. The bookmark storm and Calendar projection findings were found against the server binary already running before their fixes; regressions pass after the fixes. The run also recorded unrelated findings for follow-up: admin config returned 422 where the matrix expected 403; late Files dedup probes got 401 for seeded sessions, a dedup share probe got 400 where it expected 200, and the mid-run restart coordinator did not resume one dedup probe. The final restart probe reported 0 findings. These do not establish a Files data-integrity failure because the dedup setup requests were unauthorized. I posted the evidence on #265.

Item 1 remains: origin/dev at c70b196b8ce120ab325b51d7dd9b8fd090e67364 still has no #191 commit, so the pasted-image undo/redo editor change stayed gated.

Decisions not specified in DESIGN.md: reverse-prepend Yrs nodes to avoid indexed append cost; admit one bookmark capture per User because Notes has one writer; read bookmarks from the Notes Index as their source of truth. The DAV snapshot and lock-scope reductions follow the existing single-writer and rebuildable-index model.

Finished Forgejo #265. Branch: `job/robust-2` Head: `dd566bf593296465a555725f523d9eee0abd101d` Push: `git push origin HEAD:job/robust-2` returned `Everything up-to-date`. Built: - Collaboration sync now constructs Yrs top-level blocks in linear time. The 10,000-block regression passed in 1.236 s focused; the real-server probe measured 798 ms. - DAV discovery avoids discarded Journal scans and duplicate Reminder reads. Initial Journal sync uses one snapshot instead of per-entry reads. - Event-from-Log reuses the located Log identity and writes under one Notes lock. Template rendering happens before the per-User write lock. Journal PATCH holds one lock through ETag validation and write; a 24-request same-ETag storm produced one 200 and 23 412 responses. - Bookmark capture admits one request per User, so excess requests get 429 before building a writer-lock queue. Calendar range and grid read bookmarks from the Notes Index and add a Files item ID after adoption. Files changed by this job: `crates/calternal-collab/src/lib.rs`, `crates/calternal-collab/src/session.rs`, `crates/calternal-dav/src/protocol.rs`, `crates/plugins/notes/src/lib.rs`, `crates/plugins/notes/src/tasks_dav.rs`, `crates/plugins/notes/src/calendar_links.rs`, `crates/plugins/notes/src/bookmarks.rs`, `crates/plugins/calendar/src/routes.rs`, `crates/plugins/calendar/src/items.rs`, `crates/plugins/calendar/src/view.rs`, and `tests/adversarial/editor.mjs`. Gates, verbatim result lines: - `cargo fmt --check` — exit 0, no output. - `cargo clippy --all-targets -- -D warnings` — `Finished [32m`dev`[0m profile [unoptimized + debuginfo] target(s) in 12m 18s`; exit 0. - `cargo test` — exit 0. Relevant result lines: `test result: 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.58s` (collab unit tests), `test result: 95 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 69.50s` (Notes), and Calendar: 47 unit, 1 cache integration, 3 protocol integration tests passed; doc tests passed with 0 tests. - `bun run check` — `svelte-check found 0 errors and 0 warnings`. - `bun run test` — `Test Files 89 passed (89)` and `Tests 619 passed (619)`. - `cargo clean` — `Removed 19073 files, 17.5GiB total`. Removed `apps/web/build`, `apps/web/node_modules/.vite`, and `apps/web/.svelte-kit`. Adversarial round: the runner exited 1 with `ROUND 2 FINDINGS 79`; most listed entries were SLOW. The 10k sync passed. DAV, Event-from-Log, template, and Journal requests completed but several were marked SLOW under shared-host load. The bookmark storm and Calendar projection findings were found against the server binary already running before their fixes; regressions pass after the fixes. The run also recorded unrelated findings for follow-up: admin config returned 422 where the matrix expected 403; late Files dedup probes got 401 for seeded sessions, a dedup share probe got 400 where it expected 200, and the mid-run restart coordinator did not resume one dedup probe. The final restart probe reported 0 findings. These do not establish a Files data-integrity failure because the dedup setup requests were unauthorized. I posted the evidence on #265. Item 1 remains: `origin/dev` at `c70b196b8ce120ab325b51d7dd9b8fd090e67364` still has no `#191` commit, so the pasted-image undo/redo editor change stayed gated. Decisions not specified in DESIGN.md: reverse-prepend Yrs nodes to avoid indexed append cost; admit one bookmark capture per User because Notes has one writer; read bookmarks from the Notes Index as their source of truth. The DAV snapshot and lock-scope reductions follow the existing single-writer and rebuildable-index model.

kayg commented

2026-09-27 22:44:22 +00:00

Author

Owner

Clippy output without terminal color codes: Finished dev profile [unoptimized + debuginfo] target(s) in 12m 18s (exit 0).

Clippy output without terminal color codes: `Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 18s` (exit 0).

kayg commented

2026-09-27 22:53:49 +00:00

Author

Owner

Reproduced in the one adversarial round merged for #275. Seed 25608414; after paste, the editor reported 1 .cal-image-block; after 500 Ctrl+Z presses it still reported 1; after 500 Ctrl+Y presses it reported 3. editor.mjs failed with undo/redo duplicated the pasted image (3 !== 1). This matches finding 1 already tracked here. The run also had load-only SLOW notices; no additional editor probe run was made.

Reproduced in the one adversarial round merged for #275. Seed `25608414`; after paste, the editor reported 1 `.cal-image-block`; after 500 Ctrl+Z presses it still reported 1; after 500 Ctrl+Y presses it reported 3. `editor.mjs` failed with `undo/redo duplicated the pasted image` (`3 !== 1`). This matches finding 1 already tracked here. The run also had load-only SLOW notices; no additional editor probe run was made.

kayg referenced this issue

2026-09-27 23:40:36 +00:00

FLAKY: calternal-auth key_rotation_revokes_sessions_and_rejects_competing_recovery (left 0) #275

kayg referenced this issue from a commit

2026-09-28 00:24:59 +00:00

Merge job/robust-2 (#265)

kayg commented

2026-09-28 00:25:05 +00:00

Author

Owner

Merged in 976b804f; orchestrator ran full clippy (clean) and full workspace tests (1,329 passed, 0 failed) on the merge.

Merged in 976b804f; orchestrator ran full clippy (clean) and full workspace tests (1,329 passed, 0 failed) on the merge.

kayg commented

2026-09-28 00:25:12 +00:00

Author

Owner

Item 1 (pasted-image undo duplication) still open; unblocked once #191 merges.

kayg commented

2026-09-28 01:01:14 +00:00

Author

Owner

The #248 adversarial run reproduced the pasted-image undo/redo duplication already tracked here. With seed 25608414, the editor browser had one image after paste/undo and three after the history storm; the runner reported 3 !== 1. This run used the shared host while multiple Rust builds were active, so the surrounding latency findings were reported as SLOW load. No duplicate issue was opened.

The #248 adversarial run reproduced the pasted-image undo/redo duplication already tracked here. With seed `25608414`, the editor browser had one image after paste/undo and three after the history storm; the runner reported `3 !== 1`. This run used the shared host while multiple Rust builds were active, so the surrounding latency findings were reported as SLOW load. No duplicate issue was opened.

kayg commented

2026-09-28 01:11:42 +00:00

Author

Owner

Follow-up from the #248 adversarial run: the 3 MiB appearance PUT reached the local proxy, which returned 502 after an upstream EPIPE even though the real server correctly returns 413. The proxy now waits for 100 Continue on large Content-Length requests before streaming. A targeted real-server run through the updated proxy returned 413. The original 30-minute round also reproduced this issue's pasted-image undo/redo finding; the broad round stopped during later SLOW journalrace checks at its time limit.

Follow-up from the #248 adversarial run: the 3 MiB appearance PUT reached the local proxy, which returned 502 after an upstream EPIPE even though the real server correctly returns 413. The proxy now waits for `100 Continue` on large Content-Length requests before streaming. A targeted real-server run through the updated proxy returned 413. The original 30-minute round also reproduced this issue's pasted-image undo/redo finding; the broad round stopped during later SLOW journalrace checks at its time limit.

kayg referenced this issue

2026-09-28 01:22:16 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg commented

2026-09-28 01:24:48 +00:00

Author

Owner

Reproduced during the audit-bugs adversarial round (2026-09-28):

FINDING editor browser editor paste, Unicode, and history storm seed=25608414: undo/redo duplicated the pasted image

3 !== 1

The probe reported editor findings=1. This matches item 1 in this issue and remains a content integrity blocker. The UI work in #278 does not touch editor behavior; recording the reproduction here for the editor follow-up.

Reproduced during the audit-bugs adversarial round (2026-09-28): `FINDING editor browser editor paste, Unicode, and history storm seed=25608414: undo/redo duplicated the pasted image` `3 !== 1` The probe reported `editor findings=1`. This matches item 1 in this issue and remains a content integrity blocker. The UI work in #278 does not touch editor behavior; recording the reproduction here for the editor follow-up.

kayg referenced this issue

2026-09-28 01:40:53 +00:00

APPEARANCE: simplify to Theme & UI (scheme, theme families, background) + Text; custom gradients; Unsplash #248

kayg commented

2026-09-28 02:55:50 +00:00

Author

Owner

Add to item 1 (editor, after #191 — now merged): #278's adversarial editor probe found Ctrl+Z / Ctrl+Shift+Z round trips insert three spaces between adjacent blocks after a hostile paste + IME input. Same undo/redo area as the pasted-image duplication; fix both with regressions.

kayg commented

2026-09-28 02:57:06 +00:00

Author

Owner

Starting item 1 on branch job/editor-undo, based on dev at 6c2f3b41a4. #191 is present in history as bdcecc42. Reproducing image duplication and paste/IME spacing in the live Yjs editor before changing undo behavior.

Starting item 1 on branch job/editor-undo, based on dev at 6c2f3b41a448356a73797bd3a8bf9f93a8484a6e. #191 is present in history as bdcecc42. Reproducing image duplication and paste/IME spacing in the live Yjs editor before changing undo behavior.

kayg commented

2026-09-28 03:09:37 +00:00

Author

Owner

Root-cause finding: the editor's Yjs keymap returns false when its undo/redo stack is empty. A focused package test dispatching Ctrl+Z, Ctrl+Shift+Z and Ctrl+Y after clearing the real Yjs UndoManager confirms the events are not prevented, so the browser's native contenteditable history can run after collaborative history is exhausted. This matches the 500-keypress image duplication path; I am adding a higher-priority live-editor key handler that consumes those shortcuts even when Yjs has no stack item, then checking the real-server browser probe.

kayg commented

2026-09-28 03:24:26 +00:00

Author

Owner

Committed the first editor fix as b03c46f4. Live Notes now consumes Ctrl+Z, Ctrl+Shift+Z and Ctrl+Y even when the Yjs undo manager has no stack item; composition key events do not run history. Added an editor-package test with the real TipTap Collaboration extension and Yjs UndoManager. Focused result: , ; package check: . The browser probe now covers mixed image/rich text/code/list paste, 50 round trips with and without IME, then the 500-keypress history exhaustion case.

Committed the first editor fix as b03c46f4. Live Notes now consumes Ctrl+Z, Ctrl+Shift+Z and Ctrl+Y even when the Yjs undo manager has no stack item; composition key events do not run history. Added an editor-package test with the real TipTap Collaboration extension and Yjs UndoManager. Focused result: , ; package check: . The browser probe now covers mixed image/rich text/code/list paste, 50 round trips with and without IME, then the 500-keypress history exhaustion case.

kayg commented

2026-09-28 03:24:46 +00:00

Author

Owner

Correction to my previous comment: the focused editor-package run reported Test Files 1 passed (1), Tests 3 passed (3); package check reported svelte-check found 0 errors and 0 warnings. The browser probe covers mixed image, rich text, code and list paste, 50 round trips with and without IME, and the 500-keypress history exhaustion case.

kayg referenced this issue

2026-09-28 03:35:18 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg commented

2026-09-28 03:37:11 +00:00

Author

Owner

The first focused server run passed fixture setup and the 1,083-request authorization matrix, then timed out before paste assertions. The isolated history fixture had dropped the normal reading-to-editing Enter step, so Ctrl+V ran while the editor still selected blocks. I am restoring that transition and will rerun the editor-only probe with unrelated authz work skipped.

kayg commented

2026-09-28 03:48:53 +00:00

Author

Owner

The corrected real-browser regression reaches the mixed paste and saves it. After one Ctrl+Z / Ctrl+Shift+Z pair, Chromium reports changed rendered content and duplicate image/block content (the initial DOM had one image and one copy). The editor package test initially missed this because its initial paragraph existed only in ProseMirror state, not the Y.Doc; I corrected the fixture to seed via the collaborative editor and changed it to dispatch actual key events. That focused package case now passes for non-IME paste; I am tracing why the mounted Notes editor differs.

kayg commented

2026-09-28 03:58:27 +00:00

Author

Owner

Correction to my previous comment: the apparent duplicate after the first browser pair was a false positive from comparing full node-view HTML. Undo removed the one pasted image and blocks; redo restored one image and one copy of each block. The image node view changed from its failed-load placeholder to its loading state, so the raw HTML differed. I am changing the regression to compare stable editor content (while checking one image separately) and moving the first history pairs before the API save wait, which had delayed them by about 0.87 seconds and could hide capture-time grouping behavior.

kayg commented

2026-09-28 03:58:53 +00:00

Author

Owner

Another data point for item 1 (editor undo/redo): #236's adversarial editor probe recorded an undo/redo text mismatch at seed 25608414 (text delta posted on #236). Reproduce with that seed in the editor-undo job.

kayg commented

2026-09-28 04:05:57 +00:00

Author

Owner

The corrected semantic browser snapshot now reproduces the data-integrity failure when history starts immediately after paste: by round 3 of the Ctrl+Z / Ctrl+Shift+Z sweep, the image, rich paragraph, code block and list appear twice. The earlier save wait (about 0.87 s) hid it. I am recording each event's defaultPrevented state and the content after undo and redo to identify the responsible history path before changing behavior.

kayg commented

2026-09-28 04:37:45 +00:00

Author

Owner

The real-browser trace narrows the failure: each shortcut performs one Yjs operation (undo stack 1→0, redo stack 1→0), and Chromium emits no beforeinput historyUndo/historyRedo event. The ProseMirror document still gains duplicate image/code/list nodes on redo. The editor-package test's initial Y.Doc is generated by Yjs, while the live server seeds it with Yrs; I am capturing that server-origin state in a test fixture to isolate the difference.

kayg commented

2026-09-28 05:02:59 +00:00

Author

Owner

Real-browser finding for #265 item 1: with one editor client on the real Yrs server, the mixed image/rich-text/code/list paste is present once. Ctrl+Z restores the server-seeded document and changes the Yjs stacks from undo=1/redo=0 to undo=0/redo=1. Ctrl+Shift+Z restores one copy and returns the stacks to undo=1/redo=0. Before the next key, the editor receives an untracked Yrs update and the document gains duplicate copies while the Yjs stacks do not change. No native historyUndo/historyRedo events fire.

The likely source is the existing conflict-shadow flow in crates/calternal-collab/src/session.rs: an update that deletes an indexed original block opens a shadow, and later non-deleting updates are applied to that shadow and merged into the live room with apply_named_blocks. The redo of the paste can therefore be merged back as fresh Yrs structs while its original redo content is already live. The editor package's local UndoManager-only test cannot produce this server-origin update; the live e2e does.

This root cause changes another crate's live collaboration behavior. I am recording it before expanding this editor job's file scope. The current e2e trace and server code path are available in this worktree for review.

Real-browser finding for #265 item 1: with one editor client on the real Yrs server, the mixed image/rich-text/code/list paste is present once. Ctrl+Z restores the server-seeded document and changes the Yjs stacks from undo=1/redo=0 to undo=0/redo=1. Ctrl+Shift+Z restores one copy and returns the stacks to undo=1/redo=0. Before the next key, the editor receives an untracked Yrs update and the document gains duplicate copies while the Yjs stacks do not change. No native historyUndo/historyRedo events fire. The likely source is the existing conflict-shadow flow in `crates/calternal-collab/src/session.rs`: an update that deletes an indexed original block opens a shadow, and later non-deleting updates are applied to that shadow and merged into the live room with `apply_named_blocks`. The redo of the paste can therefore be merged back as fresh Yrs structs while its original redo content is already live. The editor package's local UndoManager-only test cannot produce this server-origin update; the live e2e does. This root cause changes another crate's live collaboration behavior. I am recording it before expanding this editor job's file scope. The current e2e trace and server code path are available in this worktree for review.

kayg commented

2026-09-28 05:15:19 +00:00

Author

Owner

Follow-up from the rebuilt production e2e: the strengthened probe now checks the state after each undo. It fails on undo 1 because the mixed paste is back in the ProseMirror document before redo starts. The Yjs UndoManager has already popped its item, and the browser emitted no native history event. This confirms an untracked collaboration update restores the deleted paste between keystrokes. The test uses a successful image response too, so the update is not caused by the image node view's load error.

The editor-owned paste/IME capture-boundary fix and its 50-pair package test pass. The remaining duplicate is the server-side conflict-shadow merge described in my previous comment; completing item 1 requires changing calternal-collab behavior.

Follow-up from the rebuilt production e2e: the strengthened probe now checks the state after each undo. It fails on undo 1 because the mixed paste is back in the ProseMirror document before redo starts. The Yjs UndoManager has already popped its item, and the browser emitted no native history event. This confirms an untracked collaboration update restores the deleted paste between keystrokes. The test uses a successful image response too, so the update is not caused by the image node view's load error. The editor-owned paste/IME capture-boundary fix and its 50-pair package test pass. The remaining duplicate is the server-side conflict-shadow merge described in my previous comment; completing item 1 requires changing `calternal-collab` behavior.

kayg commented

2026-09-28 05:43:25 +00:00

Author

Owner

Final report for Forgejo #265 item 1

Branch: job/editor-undo
Head: 06ad3c1d359a4bdee1c56c36e2a2b8ae3db97ac8

Built editor-side Yjs history key handling and explicit paste/IME capture boundaries. Added a deterministic editor unit test for mixed image, rich text, code and list paste, with and without IME, across 50 undo/redo rounds. Added an e2e assertion that checks the editor document after undo. Added the extension to the app's editor type boundary.

Files changed:

packages/editor/src/collaborationHistoryKeys.ts
packages/editor/src/collaborationUndo.test.ts
packages/editor/src/index.ts
packages/editor/package.json
packages/editor/README.md
apps/web/src/lib/notes/editorHost.ts
apps/web/src/lib/notes/editor-types/index.d.ts
bun.lock
tests/adversarial/editor.mjs

Gates:

cargo fmt --check: exit 0; no output.

cargo clippy --all-targets -- -D warnings: exit 0. Output:

Compiling calternal-server v0.0.1 (/home/kayg/Developer/calternal-wt/editor-undo/crates/calternal-server)
Checking calternal-collab v0.0.1 (/home/kayg/Developer/calternal-wt/editor-undo/crates/calternal-collab)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12.34s

cargo test: exit 0. Output included Finished test profile [unoptimized + debuginfo] target(s) in 3m 44s; collaboration tests reported test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.57s.
bun run check in apps/web: svelte-check found 0 errors and 0 warnings (exit 0).
bun run test in apps/web: Test Files 107 passed (107) and Tests 701 passed (701) (exit 0).
Focused editor test: Test Files 1 passed (1) and Tests 3 passed (3).

The single post-merge adversarial browser pass found a remaining failure:
FINDING editor isolated editor paste history storm seed=25608414: mixed paste undo 3 changed editor content
The issue also reproduces image duplication across redo. The evidence points to crates/calternal-collab/src/session.rs: merge_conflict_shadow applies later non-deleting updates to its shadow and merges them back into the live Yrs room, so a redo after deletion can be interpreted as a concurrent edit and duplicate content. Fixing this requires changing collaboration-server behavior, outside the editor job's owned behavior/files. The task therefore remains incomplete: the real server e2e does not prove 50 stable rounds or resolve the reported data-integrity bugs. No existing test expectation was weakened.

Decisions where the design docs were silent: close paste capture after the paste transaction; close IME capture 25 ms after composition end to follow ProseMirror's delayed composition flush; type the app's extension declaration as AnyExtension.

Final report for Forgejo #265 item 1 Branch: `job/editor-undo` Head: `06ad3c1d359a4bdee1c56c36e2a2b8ae3db97ac8` Built editor-side Yjs history key handling and explicit paste/IME capture boundaries. Added a deterministic editor unit test for mixed image, rich text, code and list paste, with and without IME, across 50 undo/redo rounds. Added an e2e assertion that checks the editor document after undo. Added the extension to the app's editor type boundary. Files changed: - `packages/editor/src/collaborationHistoryKeys.ts` - `packages/editor/src/collaborationUndo.test.ts` - `packages/editor/src/index.ts` - `packages/editor/package.json` - `packages/editor/README.md` - `apps/web/src/lib/notes/editorHost.ts` - `apps/web/src/lib/notes/editor-types/index.d.ts` - `bun.lock` - `tests/adversarial/editor.mjs` Gates: - `cargo fmt --check`: exit 0; no output. - `cargo clippy --all-targets -- -D warnings`: exit 0. Output: ``` Compiling calternal-server v0.0.1 (/home/kayg/Developer/calternal-wt/editor-undo/crates/calternal-server) Checking calternal-collab v0.0.1 (/home/kayg/Developer/calternal-wt/editor-undo/crates/calternal-collab) Finished `dev` profile [unoptimized + debuginfo] target(s) in 12.34s ``` - `cargo test`: exit 0. Output included `Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 44s`; collaboration tests reported `test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.57s`. - `bun run check` in `apps/web`: `svelte-check found 0 errors and 0 warnings` (exit 0). - `bun run test` in `apps/web`: `Test Files 107 passed (107)` and `Tests 701 passed (701)` (exit 0). - Focused editor test: `Test Files 1 passed (1)` and `Tests 3 passed (3)`. The single post-merge adversarial browser pass found a remaining failure: `FINDING editor isolated editor paste history storm seed=25608414: mixed paste undo 3 changed editor content` The issue also reproduces image duplication across redo. The evidence points to `crates/calternal-collab/src/session.rs`: `merge_conflict_shadow` applies later non-deleting updates to its shadow and merges them back into the live Yrs room, so a redo after deletion can be interpreted as a concurrent edit and duplicate content. Fixing this requires changing collaboration-server behavior, outside the editor job's owned behavior/files. The task therefore remains incomplete: the real server e2e does not prove 50 stable rounds or resolve the reported data-integrity bugs. No existing test expectation was weakened. Decisions where the design docs were silent: close paste capture after the paste transaction; close IME capture 25 ms after composition end to follow ProseMirror's delayed composition flush; type the app's extension declaration as `AnyExtension`.

kayg referenced this issue

2026-09-28 06:28:04 +00:00

COLLAB: redo after delete is merged as a concurrent edit and duplicates pasted content (server, session.rs) #292

kayg referenced this issue from a commit

2026-09-28 06:29:38 +00:00

Merge job/editor-undo (#265 item 1: editor-side paste/IME history boundaries)

kayg referenced this issue

2026-09-28 08:54:26 +00:00

POLISH: Calendar now pill drops AM/PM (formatToParts); one heavier sidebar row weight token #287

kayg commented

2026-09-28 10:37:24 +00:00

Author

Owner

Additional load-round evidence from #303 (2026-09-28). The server stayed alive at the end, but the full adversarial run produced non-SLOW timeouts during the cross-plugin load: DAV Basic app-password setup; Calendar Event-from-Log and two recurrence writes; Journal patch-target setup and a current-condition delete; one Reminder write; and several collaboration Note creates. It also returned explicit 503 service_unavailable responses with Authentication database is busy; retry shortly. Other requests succeeded with 5–30 second latency.

The full run took about 75 minutes on a host with concurrent build/perf jobs. This is evidence from one loaded run; the dedicated #303 attachment append cases completed without findings.

Additional load-round evidence from #303 (2026-09-28). The server stayed alive at the end, but the full adversarial run produced non-SLOW timeouts during the cross-plugin load: DAV Basic app-password setup; Calendar Event-from-Log and two recurrence writes; Journal patch-target setup and a current-condition delete; one Reminder write; and several collaboration Note creates. It also returned explicit `503 service_unavailable` responses with `Authentication database is busy; retry shortly`. Other requests succeeded with 5–30 second latency. The full run took about 75 minutes on a host with concurrent build/perf jobs. This is evidence from one loaded run; the dedicated #303 attachment append cases completed without findings.

kayg referenced this issue

2026-09-28 11:01:23 +00:00

Adversarial attack2-only run uses deleted users and skips restart coordination #285

kayg referenced this issue

2026-09-28 11:54:50 +00:00

CALENDAR: Attach and Record voice in the item preview actions (reuse the composer's tray and recorder) #303

kayg commented

2026-10-01 05:13:04 +00:00

Author

Owner

Hygiene review: later adversarial evidence still includes non-SLOW timeouts and explicit 503 responses; the editor report also leaves paste/undo duplication unresolved in the Collab crate. Keeping #265 open.

kayg referenced this issue

2026-10-02 13:10:12 +00:00

PERF: target external Note changes to one collaboration room (#663) #769

kayg referenced this issue

2026-10-02 13:10:33 +00:00

PERF: use one resettable flush task per collaboration room (#663) #778

kayg referenced this issue

2026-10-02 17:35:52 +00:00

Investigate Calendar Event from Log timeout under shared load #250

kayg commented

2026-10-02 17:35:57 +00:00

Author

Owner

Partial duplicate symptom with #250: both report Calendar Event-from-Log creation timing out at the 30-second client deadline during an adversarial run. This issue also contains distinct editor image undo and large-note collaboration findings. Recommend linking only the Event-from-Log observation to #250 and keeping the other findings separate.

kayg referenced this issue

2026-10-02 17:42:29 +00:00

Audit: open issue hygiene (duplicates, already fixed, stale) #866

kayg referenced this issue

2026-10-03 12:56:40 +00:00

Issue hygiene sweep 2026-10-03 #1008

kayg commented

2026-10-04 13:54:48 +00:00

Author

Owner

Additional #867 evidence on a1f3a0797: bookmark Calendar projection reported two request timeouts. tests/adversarial/attack.py:5656 uses a two-second per-request deadline. The retained server log contains Calendar projection SQL taking more than one second on the loaded host. This is insufficient to classify the timeout as a hang or as load alone. No idle-host live reproduction was performed in this job; the finding remains unresolved.

Additional #867 evidence on a1f3a0797: bookmark Calendar projection reported two request timeouts. tests/adversarial/attack.py:5656 uses a two-second per-request deadline. The retained server log contains Calendar projection SQL taking more than one second on the loaded host. This is insufficient to classify the timeout as a hang or as load alone. No idle-host live reproduction was performed in this job; the finding remains unresolved.

kayg referenced this issue

2026-10-04 14:01:23 +00:00

Plan: merge order and conflicts for merge round 7b #867

Rows
Columns

FOLLOW-UP from #235 adversarial round: DAV/Journal/bookmark timeouts, pasted-image undo duplication, 10k-block collab timeout #265