Fuzzing: cargo-fuzz targets for every parser that reads user bytes

kayg commented

2026-09-26 09:40:15 +00:00

Owner

Add cargo-fuzz (libFuzzer) targets and run each for a fixed budget on this host (short, e.g. 20 min each, CARGO_BUILD_JOBS=4): notes-core Markdown↔blocks converter (round-trip invariant: untouched bytes preserved, DESIGN #95), day-file/journal parser (Log section, time ranges across midnight), calternal-path normalisation (never escapes, Unicode tricks), ICS/VTODO parsing in calternal-dav, tus upload headers, frontmatter/YAML, tag parser, search query parser, share slug/password inputs. Every crash or invariant violation: minimal repro committed as a regression test, fix, commit atomically. Keep the corpora small and in-repo (fuzz/corpus), keep fuzz/ out of the default workspace build. Report: targets, execs/sec, crashes found and fixed.

kayg commented

2026-09-26 09:41:23 +00:00

Author

Owner

Starting #161 on branch job/fuzz at base SHA 82d4c258bb. I will add small libFuzzer targets for the requested Rust parsers, run fixed budgets, and commit each working slice with regressions for findings.

Starting #161 on branch job/fuzz at base SHA 82d4c258bbcc6f34e8805d0018004fa8e621b9a5. I will add small libFuzzer targets for the requested Rust parsers, run fixed budgets, and commit each working slice with regressions for findings.

kayg commented

2026-09-26 09:48:19 +00:00

Author

Owner

Finding: DESIGN §9/#95 places the byte-preserving Markdown bridge in calternal-collab, while notes-core has a separate coarse block parser. The Markdown fuzz target exercises both. The tus and public-link validators were private, so I added small public parsing/validation entry points that reuse their production code paths. Nine targets and seed corpora are registered; the first instrumented build is in progress under CARGO_BUILD_JOBS=4.

kayg commented

2026-09-26 09:56:09 +00:00

Author

Owner

Started #161 on branch job/fuzz at base 82d4c258bb. Continuing the existing uncommitted fuzz harness; I will build and commit it, run each target for 120 seconds, and record parser findings with reproductions.

Started #161 on branch job/fuzz at base 82d4c258bbcc6f34e8805d0018004fa8e621b9a5. Continuing the existing uncommitted fuzz harness; I will build and commit it, run each target for 120 seconds, and record parser findings with reproductions.

kayg commented

2026-09-26 10:03:59 +00:00

Author

Owner

Harness committed as 67d55b5993. cargo fmt --check passed. The isolated nightly path target built successfully; the combined target build remains in progress. The seed corpus includes CRLF and path edge cases.

Harness committed as 67d55b5993f2082a422d528a73b521075b5df34b. `cargo fmt --check` passed. The isolated nightly `path` target built successfully; the combined target build remains in progress. The seed corpus includes CRLF and path edge cases.

kayg commented

2026-09-26 10:06:48 +00:00

Author

Owner

Path target: 120-second campaign completed with 1,514,444 executions (12,516 exec/s), no crash or invariant failure. Source: target/tmp/fuzz-path.log.

kayg commented

2026-09-26 10:12:22 +00:00

Author

Owner

Build environment finding: the shared sccache daemon returned error writing dependencies ... tz-days/target/tmp/...: No such file or directory while compiling chrono. That temp directory belongs to another worktree and no longer exists. I am running this job with RUSTC_WRAPPER= to avoid the stale daemon temp path.

Build environment finding: the shared sccache daemon returned `error writing dependencies ... tz-days/target/tmp/...: No such file or directory` while compiling chrono. That temp directory belongs to another worktree and no longer exists. I am running this job with `RUSTC_WRAPPER=` to avoid the stale daemon temp path.

kayg commented

2026-09-26 10:19:26 +00:00

Author

Owner

Host load observation during sanitizer build: uptime reported load averages 88.55, 73.34, 68.46; the notes-core rustc process had 40 seconds of CPU time after 6:53 elapsed. This is SLOW-only shared-host contention, with no parser failure shown.

Host load observation during sanitizer build: `uptime` reported load averages 88.55, 73.34, 68.46; the notes-core rustc process had 40 seconds of CPU time after 6:53 elapsed. This is SLOW-only shared-host contention, with no parser failure shown.

kayg commented

2026-09-26 10:21:47 +00:00

Author

Owner

Harness build finding: tags target referenced private nlp::tags, causing E0603. I added a minimal public re-export of the existing span-aware extract_tags function in notes-core and changed the target to call it. No parser behavior changed; rebuilding now.

Harness build finding: `tags` target referenced private `nlp::tags`, causing E0603. I added a minimal public re-export of the existing span-aware `extract_tags` function in notes-core and changed the target to call it. No parser behavior changed; rebuilding now.

kayg commented

2026-09-26 10:25:25 +00:00

Author

Owner

Starting property-based continuation on branch job/fuzz at 67d55b5993. Saved crash input and two uncommitted edits are present. I will preserve a regression test, fix the panic, add bounded proptest coverage, merge dev, and run gates. No cargo-fuzz run.

Starting property-based continuation on branch job/fuzz at 67d55b5993f2082a422d528a73b521075b5df34b. Saved crash input and two uncommitted edits are present. I will preserve a regression test, fix the panic, add bounded proptest coverage, merge dev, and run gates. No cargo-fuzz run.

kayg commented

2026-09-26 10:30:52 +00:00

Author

Owner

Finding: the saved 106-byte input reproduces a Daily note fuzz invariant failure. A carriage return before a malformed [tz=...] marker lets parse_log_bullet accept it as title text. format_log_entry trims away that leading whitespace, exposing the reserved marker; the formatted line then fails to parse. The focused regression now checks this case. The standalone tag scanner did not panic on the same input.

kayg commented

2026-09-26 10:57:49 +00:00

Author

Owner

Build-host observation: the search property test has spent over 20 minutes compiling dependencies while several other Rust jobs run concurrently. There is no test failure or parser finding yet. This is host load (SLOW-only), not a product defect; I am keeping CARGO_BUILD_JOBS=4 and waiting for the focused test before commit.

kayg referenced this issue

2026-09-26 15:56:35 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg commented

2026-09-27 09:28:43 +00:00

Author

Owner

Resuming #161 on branch job/fuzz at checkpoint faf33e20d2. The branch was originally based at 82d4c258bb; current dev is 9cc5ab1bbc. I will review the committed parser properties, run the remaining fixed-budget targets, merge current dev before the final gates, and report measured results.

Resuming #161 on branch job/fuzz at checkpoint faf33e20d26fae1565f4c5141a7a89eeb374533b. The branch was originally based at 82d4c258bbcc6f34e8805d0018004fa8e621b9a5; current dev is 9cc5ab1bbcceb1b3a7eb71e936d927acc256ffba. I will review the committed parser properties, run the remaining fixed-budget targets, merge current dev before the final gates, and report measured results.

kayg commented

2026-09-27 09:39:46 +00:00

Author

Owner

Merge finding: current dev's media exports conflicted with this branch's share validator exports in crates/plugins/files/src/lib.rs. I retained both sets of exports. The focused gate passed: cargo test -p calternal-plugin-files — test result: ok. 101 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 50.99s. Doc-tests: 0 passed, 0 failed. Merge commit: a72a93f5.

Merge finding: current dev's media exports conflicted with this branch's share validator exports in crates/plugins/files/src/lib.rs. I retained both sets of exports. The focused gate passed: cargo test -p calternal-plugin-files — test result: ok. 101 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 50.99s. Doc-tests: 0 passed, 0 failed. Merge commit: a72a93f5.

kayg commented

2026-09-27 10:01:34 +00:00

Author

Owner

Markdown fuzz finding after merge a72a93f5: 5,285 executions triggered the target's stable-second-round-trip invariant at fuzz/fuzz_targets/markdown.rs:17. The input is 21 bytes (hex 23 23 23 21 21 21 21 21 21 21 20 48 0d 0d 0d 0a 0a 72 61 70 68); first output retains two CR bytes where the second output retains one. I am minimizing and tracing this into the production Markdown bridge before adding a regression and fix.

Markdown fuzz finding after merge a72a93f5: 5,285 executions triggered the target's stable-second-round-trip invariant at fuzz/fuzz_targets/markdown.rs:17. The input is 21 bytes (hex 23 23 23 21 21 21 21 21 21 21 20 48 0d 0d 0d 0a 0a 72 61 70 68); first output retains two CR bytes where the second output retains one. I am minimizing and tracing this into the production Markdown bridge before adding a regression and fix.

kayg commented

2026-09-27 10:04:57 +00:00

Author

Owner

Correction: the sanitizer panic is on fuzz/fuzz_targets/markdown.rs:17, after the full collab bridge's second round-trip check passes. The failing invariant is calternal-notes-core::canonicalize_markdown idempotence. The six-byte repro is 21 0d 0d 0a 0a 68 (). I am tracing the notes-core block serializer and will add the regression there.

kayg commented

2026-09-27 10:09:31 +00:00

Author

Owner

Correction: the sanitizer panic is on fuzz/fuzz_targets/markdown.rs:17, after the full collaboration bridge's second round-trip check passes. The failing invariant is calternal-notes-core::canonicalize_markdown idempotence. The six-byte repro is 21 0d 0d 0a 0a 68 (!\r\r\n\nh). The coarse parser splits only on LF, then flush strips a CRLF terminator from a segment that contains a preceding lone CR. The regression will cover stable output and the interior blank-line count.

Correction: the sanitizer panic is on fuzz/fuzz_targets/markdown.rs:17, after the full collaboration bridge's second round-trip check passes. The failing invariant is calternal-notes-core::canonicalize_markdown idempotence. The six-byte repro is 21 0d 0d 0a 0a 68 (`!\r\r\n\nh`). The coarse parser splits only on LF, then flush strips a CRLF terminator from a segment that contains a preceding lone CR. The regression will cover stable output and the interior blank-line count.

kayg commented

2026-09-27 10:14:00 +00:00

Author

Owner

Fixed in commit fb56ff02 (fix(notes): stabilize mixed newline Markdown blocks). The RED test reproduced the minimized six-byte case before the fix. The new line iterator treats CRLF, lone CR and LF as line endings and preserves the original spelling inside content spans. The minimized input is also a six-byte seed in fuzz/corpus/markdown.

Focused verification: cargo test -p calternal-notes-core passed. Output included test result: ok. 480 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.47s, test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.85s, test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.09s, test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.62s, and test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s. The focused rustfmt check exited 0.

Fixed in commit fb56ff02 (`fix(notes): stabilize mixed newline Markdown blocks`). The RED test reproduced the minimized six-byte case before the fix. The new line iterator treats CRLF, lone CR and LF as line endings and preserves the original spelling inside content spans. The minimized input is also a six-byte seed in fuzz/corpus/markdown. Focused verification: `cargo test -p calternal-notes-core` passed. Output included `test result: ok. 480 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.47s`, `test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.85s`, `test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.09s`, `test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.62s`, and `test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s`. The focused rustfmt check exited 0.

kayg referenced this issue

2026-09-27 10:23:51 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg referenced this issue

2026-09-27 11:13:19 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg commented

2026-09-27 13:45:26 +00:00

Author

Owner

Completed Forgejo #161 on job/fuzz.

Head: 806886a8678b2634777032e658507ff14740053b.
The branch includes current dev (6495aaf7) through merge commit 806886a8. The worktree is clean.

Built:

Added nine bounded libFuzzer targets and seed corpora for Markdown, dayfiles, frontmatter, paths, iCalendar, TUS headers, tags, search queries, and public-link inputs. Added a fixed-duration runner and setup notes.
Added parser/property regressions for notes, paths, and search.
Fixed Markdown block parsing for mixed newline input. The minimized failure was !\r\r\n\nh; added it as a regression and corpus seed.
Corrected the adversarial harness to wait for seeded editor state, refresh the passkey before key rotation, and send request-limit probes to the Rust backend when the Node proxy changes the result.

Fuzz runs:

path: 3,907,075 executions
dayfile: 232,688
frontmatter: 674,177
markdown: 110,171
icalendar: 2,010,486
search_query: 2,178,304
tus_headers: 452,387
share_inputs: 589,501
tags: 1,332,026

All nine fixed-duration runs completed without a finding after the Markdown fix.

Adversarial results:

The full first round reported 21 findings, all marked SLOW. The editor restart probe ended with 0 findings and the server alive.
The full second-round report counted 98 findings, mostly SLOW. Follow-up direct or isolated probes for oversized analytics URI (backend response 414), AI oversized input, public share options, and exhaustion each passed; the three focused Round 2 runs ended with 0 findings and the server alive.
One non-SLOW performance measurement remains recorded: an upload during a 20,000-file user purge took 2.24 s against a 2 s probe threshold. The upload succeeded. This was measured while the shared host was busy.
The first post-merge full Cargo run had one failure in concurrent_clients_and_external_writer_converge; its final-note assertion ran 900 ms after a 750 ms debounce. The isolated test, the complete two_clients binary, and the subsequent full Cargo retry all passed. No collaboration code changed in this job.

Final gate output (on the current code; the last dev merge changed only web/UI and audit files):

cargo fmt --check
(no stdout; exit status 0)

cargo clippy --all-targets -- -D warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.00s

cargo test
(exit status 0)
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

bun run check
svelte-check found 0 errors and 0 warnings

bun run test
 Test Files  80 passed (80)
      Tests  585 passed (585)
   Start at  15:38:45
   Duration  136.33s (transform 67%, import 13%, environment 11%, tests 6%, setup 2%)

Main files:

fuzz/Cargo.toml, fuzz/Cargo.lock, fuzz/README.md, fuzz/run-fixed.sh, fuzz/fuzz_targets/, and fuzz/corpus/
crates/calternal-notes-core/src/markdown.rs
crates/calternal-notes-core/tests/parser_properties.rs
crates/calternal-path/tests/path_properties.rs
crates/calternal-search/tests/query_properties.rs
tests/adversarial/attack.py, attack2.py, editor.mjs, and run.sh

Decisions not set by the design docs: I selected these nine existing parser/value boundaries for fuzzing, used their round-trip, idempotence, rejection, and no-panic invariants, and set each run to 120 seconds. I sent oversized URI probes directly to the Rust server because the Node proxy applies a different URI limit.

Completed Forgejo #161 on `job/fuzz`. Head: `806886a8678b2634777032e658507ff14740053b`. The branch includes current `dev` (`6495aaf7`) through merge commit `806886a8`. The worktree is clean. Built: - Added nine bounded libFuzzer targets and seed corpora for Markdown, dayfiles, frontmatter, paths, iCalendar, TUS headers, tags, search queries, and public-link inputs. Added a fixed-duration runner and setup notes. - Added parser/property regressions for notes, paths, and search. - Fixed Markdown block parsing for mixed newline input. The minimized failure was `!\r\r\n\nh`; added it as a regression and corpus seed. - Corrected the adversarial harness to wait for seeded editor state, refresh the passkey before key rotation, and send request-limit probes to the Rust backend when the Node proxy changes the result. Fuzz runs: - `path`: 3,907,075 executions - `dayfile`: 232,688 - `frontmatter`: 674,177 - `markdown`: 110,171 - `icalendar`: 2,010,486 - `search_query`: 2,178,304 - `tus_headers`: 452,387 - `share_inputs`: 589,501 - `tags`: 1,332,026 All nine fixed-duration runs completed without a finding after the Markdown fix. Adversarial results: - The full first round reported 21 findings, all marked `SLOW`. The editor restart probe ended with 0 findings and the server alive. - The full second-round report counted 98 findings, mostly `SLOW`. Follow-up direct or isolated probes for oversized analytics URI (backend response 414), AI oversized input, public share options, and exhaustion each passed; the three focused Round 2 runs ended with 0 findings and the server alive. - One non-SLOW performance measurement remains recorded: an upload during a 20,000-file user purge took 2.24 s against a 2 s probe threshold. The upload succeeded. This was measured while the shared host was busy. - The first post-merge full Cargo run had one failure in `concurrent_clients_and_external_writer_converge`; its final-note assertion ran 900 ms after a 750 ms debounce. The isolated test, the complete `two_clients` binary, and the subsequent full Cargo retry all passed. No collaboration code changed in this job. Final gate output (on the current code; the last dev merge changed only web/UI and audit files): ```text cargo fmt --check (no stdout; exit status 0) cargo clippy --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 27.00s cargo test (exit status 0) test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s bun run check svelte-check found 0 errors and 0 warnings bun run test Test Files 80 passed (80) Tests 585 passed (585) Start at 15:38:45 Duration 136.33s (transform 67%, import 13%, environment 11%, tests 6%, setup 2%) ``` Main files: - `fuzz/Cargo.toml`, `fuzz/Cargo.lock`, `fuzz/README.md`, `fuzz/run-fixed.sh`, `fuzz/fuzz_targets/`, and `fuzz/corpus/` - `crates/calternal-notes-core/src/markdown.rs` - `crates/calternal-notes-core/tests/parser_properties.rs` - `crates/calternal-path/tests/path_properties.rs` - `crates/calternal-search/tests/query_properties.rs` - `tests/adversarial/attack.py`, `attack2.py`, `editor.mjs`, and `run.sh` Decisions not set by the design docs: I selected these nine existing parser/value boundaries for fuzzing, used their round-trip, idempotence, rejection, and no-panic invariants, and set each run to 120 seconds. I sent oversized URI probes directly to the Rust server because the Node proxy applies a different URI limit.

kayg referenced this issue from a commit

2026-09-27 13:49:31 +00:00

Merge job/fuzz (#161)

kayg commented

2026-09-27 13:49:31 +00:00

Author

Owner

Merged in 74d60726.

Merged in 74d60726.

kayg closed this issue

2026-09-27 13:49:32 +00:00

kayg referenced this issue