Non-ASCII tags are truncated by the Log line reader ('#café' becomes tag 'caf' plus 'é' in the title) #473

Closed
opened 2026-09-30 02:47:18 +00:00 by kayg · 12 comments
Owner

Found in the #468 adversarial round (tests/adversarial/log_batch.mjs, job/send-fast).

Observed. POST /api/v1/notes/journal/log/batch (and the single /journal/log, which uses the same parse_composer_entry) with the text café 🧪 #café stores the title café 🧪 é and the tag area/caf. The server line reader stops a tag at the first non-ASCII character. The web composer sends canonical text from the client NLP, so a Hebrew tag typed in the composer (#חיים in the composer e2e) also stays in the title.

Expected. One rule for tag characters in the client NLP, the server line reader and the Index (notes_index.rs, markdown.rs use is_ascii_alphanumeric). Either accept Unicode letters in tags everywhere, or reject/keep the whole #word as text; never split a word into a truncated tag plus a stray letter.

Severity. Not a merge blocker: no data is lost (the bytes stay in the title), but the tag is wrong and the title is changed.

Not fixed in #468 (out of scope: the batch path only reuses the existing parser).

Found in the #468 adversarial round (`tests/adversarial/log_batch.mjs`, job/send-fast). **Observed.** `POST /api/v1/notes/journal/log/batch` (and the single `/journal/log`, which uses the same `parse_composer_entry`) with the text `café 🧪 #café` stores the title `café 🧪 é` and the tag `area/caf`. The server line reader stops a tag at the first non-ASCII character. The web composer sends canonical text from the client NLP, so a Hebrew tag typed in the composer (`#חיים` in the composer e2e) also stays in the title. **Expected.** One rule for tag characters in the client NLP, the server line reader and the Index (`notes_index.rs`, `markdown.rs` use `is_ascii_alphanumeric`). Either accept Unicode letters in tags everywhere, or reject/keep the whole `#word` as text; never split a word into a truncated tag plus a stray letter. **Severity.** Not a merge blocker: no data is lost (the bytes stay in the title), but the tag is wrong and the title is changed. Not fixed in #468 (out of scope: the batch path only reuses the existing parser).
Author
Owner

Starting #473 on branch job/tag-unicode-473, based on dfb5964a2f. I’m tracing the shared tag vectors and all Rust/TypeScript parser callers before making the Unicode/NFC change.

Starting #473 on branch job/tag-unicode-473, based on dfb5964a2fcf13dc8b9a50a319eee09bc386f322. I’m tracing the shared tag vectors and all Rust/TypeScript parser callers before making the Unicode/NFC change.
Author
Owner

Finding: the new decomposed-accent vector also exposed a Note-title splice bug: canonical dedup kept only the first tag span, so + combining acute remained in the title when it duplicated . I changed Note-title handling to scan every source span and dedupe only the returned NFC Tag values.

Finding: the new decomposed-accent vector also exposed a Note-title splice bug: canonical dedup kept only the first tag span, so + combining acute remained in the title when it duplicated . I changed Note-title handling to scan every source span and dedupe only the returned NFC Tag values.
Author
Owner

Search verification finding: cargo test -p calternal-search compiled successfully. Its indexer suite passed 18/20 tests; a_file_trashed_during_its_add_never_appears_in_search and search_drops_a_hit_whose_file_is_gone_and_queues_its_removal each timed out in the shared wait_for_hit helper while looking for ordinary ASCII file contents (zebracorn, back). The helper retries for 5 seconds. The shared-vector tag operator test passed. The failure is outside the changed tag fields; I will do one isolated check and leave existing expectations unchanged.

Search verification finding: `cargo test -p calternal-search` compiled successfully. Its indexer suite passed 18/20 tests; `a_file_trashed_during_its_add_never_appears_in_search` and `search_drops_a_hit_whose_file_is_gone_and_queues_its_removal` each timed out in the shared `wait_for_hit` helper while looking for ordinary ASCII file contents (`zebracorn`, `back`). The helper retries for 5 seconds. The shared-vector tag operator test passed. The failure is outside the changed tag fields; I will do one isolated check and leave existing expectations unchanged.
Author
Owner

Search follow-up: cargo clippy -p calternal-search --all-targets -- -D warnings passed. The parallel indexer suite had 2/20 timeout failures, but cargo test -p calternal-search --test indexer -- --test-threads=1 passed 20/20, including both cases. This confirms load-sensitive timing in the existing tests; expectations and behavior were not changed.

Search follow-up: `cargo clippy -p calternal-search --all-targets -- -D warnings` passed. The parallel indexer suite had 2/20 timeout failures, but `cargo test -p calternal-search --test indexer -- --test-threads=1` passed 20/20, including both cases. This confirms load-sensitive timing in the existing tests; expectations and behavior were not changed.
Author
Owner

Web check finding: after installing the locked Bun dependencies, bun run check found a TypeScript error in the new Composer pass-through test fixture. Its empty tags: [] arrays inferred as never[], so assigning shared vector values failed. I am fixing the fixture types; production code is not implicated.

Web check finding: after installing the locked Bun dependencies, `bun run check` found a TypeScript error in the new Composer pass-through test fixture. Its empty `tags: []` arrays inferred as `never[]`, so assigning shared vector values failed. I am fixing the fixture types; production code is not implicated.
Author
Owner

Web gate findings: bun run check now passes. The full bun run test run completed 842 tests and reported 15 existing tests timing out at Vitest's 5-second limit across unrelated UI suites while the shared host had competing test/build jobs. It also could not collect the modified Composer.svelte.test.ts: Vite gives that test a non-file import.meta.url, which readFileSync(URL) rejects. I am switching shared-vector reads in the three affected tests to the repository's existing process.cwd() path pattern, then I will run those changed tests directly. No existing expectations will change.

Web gate findings: `bun run check` now passes. The full `bun run test` run completed 842 tests and reported 15 existing tests timing out at Vitest's 5-second limit across unrelated UI suites while the shared host had competing test/build jobs. It also could not collect the modified `Composer.svelte.test.ts`: Vite gives that test a non-file `import.meta.url`, which `readFileSync(URL)` rejects. I am switching shared-vector reads in the three affected tests to the repository's existing `process.cwd()` path pattern, then I will run those changed tests directly. No existing expectations will change.
Author
Owner

Web follow-up: all three changed test files pass serially (44/44 tests). The shared-vector read issue was resolved by using resolve(process.cwd(), '../../contracts/vectors/tags.v1.json'), matching the existing SidebarSectionHeader test pattern. The remaining full-suite failures were 15 unrelated tests timing out at 5 seconds; the other 842 tests passed.

Web follow-up: all three changed test files pass serially (44/44 tests). The shared-vector read issue was resolved by using `resolve(process.cwd(), '../../contracts/vectors/tags.v1.json')`, matching the existing SidebarSectionHeader test pattern. The remaining full-suite failures were 15 unrelated tests timing out at 5 seconds; the other 842 tests passed.
Author
Owner

#473 implementation report

Head: c42843a48 (includes the required merge from origin/dev; no push).

Built: Shared tag parsing now accepts Unicode letters and digits, combining marks after the first character, namespaces, and NFC-equivalent spellings. Rust keeps original byte spans while returning NFC values. Notes, Composer, Tasks, Search, Tags, Calendar categories, and Money payee-tag boundaries use the shared grammar or canonical projections. Added the requested Unicode cases and caller vectors. Composer NLP still uses Rust as the source of truth; web tests pin transport and search-pill behavior.

Files: Cargo.lock; crates/calternal-notes-core/ parser, composer, dayfile, link, task, and vector parity files; crates/calternal-search/src/{index.rs,operators.rs}; crates/calternal-tags/src/lib.rs; crates/calternal-money/{Cargo.toml,src/edit.rs,src/ledger.rs}; crates/plugins/calendar/src/client/mod.rs; contracts/vectors/{README.md,tags.v1.json}; docs/tags.md; and the four Composer/Search web source and test files.

Decisions: A tag starts with a Unicode letter or _; later characters also allow Unicode numbers, combining marks, _, /, and -. Digit-first #42 remains plain text. Canonical values are NFC; source Note spelling stays intact. The Composer does not gain a TypeScript NLP parser.

Gate output:

cargo fmt --all --check
(exit 0, no output)

cargo clippy -p calternal-notes-core --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 19.59s

cargo test -p calternal-notes-core
504 passed; 0 failed
13 passed; 0 failed
5 passed; 0 failed
7 passed; 0 failed
12 passed; 0 failed

cargo clippy -p calternal-tags --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 31s
cargo test -p calternal-tags
11 passed; 0 failed

cargo clippy -p calternal-money --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in 9.70s
cargo test -p calternal-money
15 passed; 0 failed
11 passed; 0 failed
2 passed; 0 failed

cargo test -p calternal-plugin-calendar (pre-merge)
52 passed; 0 failed
1 passed; 0 failed
3 passed; 0 failed
cargo clippy -p calternal-plugin-calendar --all-targets -- -D warnings (pre-merge)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 29m 02s

cargo clippy -p calternal-search --all-targets -- -D warnings (pre-merge)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 55s
cargo test -p calternal-search
33 passed; 0 failed; 1 ignored
18 passed; 2 failed (parallel indexer wait timeouts)
cargo test -p calternal-search --test indexer -- --test-threads=1
20 passed; 0 failed

bun run check (after the test path fix, pre-merge)
svelte-check found 0 errors and 0 warnings
bun run test (pre-merge)
Tests  15 failed | 842 passed (857)
error: script "test" exited with code 1
bun run test --maxWorkers=1 src/lib/composer/Composer.svelte.test.ts src/lib/composer/nlp.test.ts src/lib/search/query.test.ts
Test Files  3 passed (3)
Tests  44 passed (44)

Known gaps: The full web suite was not rerun after fixing its shared-vector path; 15 unrelated tests had hit Vitest's 5-second timeout under concurrent host load. The Search parallel run had two indexer hit-wait timeouts; both passed in the serial indexer run, with expectations unchanged. The post-merge Search clippy retry was stopped while waiting on the shared Cargo package-cache lock. Calendar and web checks passed before the required merge; only formatting, Notes core, Tags, and Money were rerun after it. No live-server adversarial round or screenshot set was produced; no route schema or layout changed. Target cleanup is in progress.

#473 implementation report **Head:** `c42843a48` (includes the required merge from `origin/dev`; no push). **Built:** Shared tag parsing now accepts Unicode letters and digits, combining marks after the first character, namespaces, and NFC-equivalent spellings. Rust keeps original byte spans while returning NFC values. Notes, Composer, Tasks, Search, Tags, Calendar categories, and Money payee-tag boundaries use the shared grammar or canonical projections. Added the requested Unicode cases and caller vectors. Composer NLP still uses Rust as the source of truth; web tests pin transport and search-pill behavior. **Files:** `Cargo.lock`; `crates/calternal-notes-core/` parser, composer, dayfile, link, task, and vector parity files; `crates/calternal-search/src/{index.rs,operators.rs}`; `crates/calternal-tags/src/lib.rs`; `crates/calternal-money/{Cargo.toml,src/edit.rs,src/ledger.rs}`; `crates/plugins/calendar/src/client/mod.rs`; `contracts/vectors/{README.md,tags.v1.json}`; `docs/tags.md`; and the four Composer/Search web source and test files. **Decisions:** A tag starts with a Unicode letter or `_`; later characters also allow Unicode numbers, combining marks, `_`, `/`, and `-`. Digit-first `#42` remains plain text. Canonical values are NFC; source Note spelling stays intact. The Composer does not gain a TypeScript NLP parser. **Gate output:** ```text cargo fmt --all --check (exit 0, no output) cargo clippy -p calternal-notes-core --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 19.59s cargo test -p calternal-notes-core 504 passed; 0 failed 13 passed; 0 failed 5 passed; 0 failed 7 passed; 0 failed 12 passed; 0 failed cargo clippy -p calternal-tags --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 31s cargo test -p calternal-tags 11 passed; 0 failed cargo clippy -p calternal-money --all-targets -- -D warnings Finished `dev` profile [unoptimized + debuginfo] target(s) in 9.70s cargo test -p calternal-money 15 passed; 0 failed 11 passed; 0 failed 2 passed; 0 failed cargo test -p calternal-plugin-calendar (pre-merge) 52 passed; 0 failed 1 passed; 0 failed 3 passed; 0 failed cargo clippy -p calternal-plugin-calendar --all-targets -- -D warnings (pre-merge) Finished `dev` profile [unoptimized + debuginfo] target(s) in 29m 02s cargo clippy -p calternal-search --all-targets -- -D warnings (pre-merge) Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 55s cargo test -p calternal-search 33 passed; 0 failed; 1 ignored 18 passed; 2 failed (parallel indexer wait timeouts) cargo test -p calternal-search --test indexer -- --test-threads=1 20 passed; 0 failed bun run check (after the test path fix, pre-merge) svelte-check found 0 errors and 0 warnings bun run test (pre-merge) Tests 15 failed | 842 passed (857) error: script "test" exited with code 1 bun run test --maxWorkers=1 src/lib/composer/Composer.svelte.test.ts src/lib/composer/nlp.test.ts src/lib/search/query.test.ts Test Files 3 passed (3) Tests 44 passed (44) ``` **Known gaps:** The full web suite was not rerun after fixing its shared-vector path; 15 unrelated tests had hit Vitest's 5-second timeout under concurrent host load. The Search parallel run had two indexer hit-wait timeouts; both passed in the serial indexer run, with expectations unchanged. The post-merge Search clippy retry was stopped while waiting on the shared Cargo package-cache lock. Calendar and web checks passed before the required merge; only formatting, Notes core, Tags, and Money were rerun after it. No live-server adversarial round or screenshot set was produced; no route schema or layout changed. Target cleanup is in progress.
Author
Owner

Final cleanup complete at head c42843a48935f5e4352d09720fe4076ae93315bc. cargo clean removed 14,150 files (7.6 GiB); the generated web dependencies and .svelte-kit output were removed. git status --short is empty.

Final cleanup complete at head `c42843a48935f5e4352d09720fe4076ae93315bc`. `cargo clean` removed 14,150 files (7.6 GiB); the generated web dependencies and `.svelte-kit` output were removed. `git status --short` is empty.
Author
Owner

Merge round progress on job/merge-round-0930: both job branches and the one required origin/dev refresh are merged. Current committed head is 2ed9a9076; the worker and sandbox integration remain under test.

The real bubblewrap boundary test passes. It checks absent host paths, a read-only root, UID/GID 65534, stripped host environment, denied network sockets, inherited limits, denied native fork and permitted native threads. PDF stdin must have all four kernel seals and fit the 64 MiB cap. Explicit PDF and generated-SVG actions produce WebP; the PDF action rejects JPEG input. The source-inode concern in the older image/video modes is tracked separately in #501.

Contract regeneration has no diff. Rust per-crate gates continue on the shared host. Web checks pass. The full web suite had one 5-second font-menu timeout; that test passed alone. The real-server media round is running now. Production screenshots and the final gate report will follow.

Merge round progress on `job/merge-round-0930`: both job branches and the one required `origin/dev` refresh are merged. Current committed head is `2ed9a9076`; the worker and sandbox integration remain under test. The real bubblewrap boundary test passes. It checks absent host paths, a read-only root, UID/GID 65534, stripped host environment, denied network sockets, inherited limits, denied native fork and permitted native threads. PDF stdin must have all four kernel seals and fit the 64 MiB cap. Explicit PDF and generated-SVG actions produce WebP; the PDF action rejects JPEG input. The source-inode concern in the older image/video modes is tracked separately in #501. Contract regeneration has no diff. Rust per-crate gates continue on the shared host. Web checks pass. The full web suite had one 5-second font-menu timeout; that test passed alone. The real-server media round is running now. Production screenshots and the final gate report will follow.
Author
Owner

Head: 40ccef12818eb27592994a2666021ee34b2b3cf5 on job/merge-round-0930.

Built: merged job/tag-unicode-473, then job/doc-stack, and merged the one required origin/dev refresh. Resolved Calendar token conflicts. Regenerated contracts without a diff. Refreshed the parity inventory for the merged batch Log caller.

Security verdict: the document renderer boundary passes the defensive audit after hardening. The native decoder has private namespaces, no network, a read-only root, UID/GID 65534, an inherited seccomp policy and bounded resources. Host Homes, data, secret-bearing host directories and extra parent descriptors are absent. The outer environment is cleared before bubblewrap starts, including its PID 1 monitor; monitor host-root and working-directory access are denied by the tested namespace. PDF input is a bounded, fully sealed memory file, not a Home inode. The fixed private symlink names only fd 0. Explicit PDF and generated-SVG actions select loaders. Native tools can create threads only. Complete Jobs and document renders have explicit wall budgets.

Files:

  • deploy/media-sandbox and deploy/media-sandbox-dropcaps.c: isolation, fixed actions, seals and native thread restrictions.
  • crates/calternal-fs/src/{lib.rs,thumbnails.rs}: bounded sealed input helper and regression tests.
  • crates/plugins/files/src/thumbnails.rs: early size/read bounds, shared media slot, fixed loaders, Job/render budgets and failure tests.
  • tests/adversarial/{media-sandbox-policy.c,test-document-sandbox.sh,prepare-media-runtime.sh,media_uploads.py,run.sh}: namespace, descriptor, input and bounded parser checks.
  • packages/ui/src/components/{calendar/GridColumn.svelte,files/FileThumb.svelte}: merged shared radius/Caption tokens.
  • apps/web/e2e/calendar-doc-stack.mjs: production Calendar, Files and Search evidence at 390/820/1440 in light/dark.
  • docs/{DESIGN.md,parity-exceptions.json,parity-matrix.md}: reasoning and the merged parity inventory.

Gates, verbatim:

cargo fmt --check: exit=0
cargo clippy -p calternal-notes-core --all-targets -- -D warnings: exit=0
cargo test -p calternal-notes-core: exit=0
cargo clippy -p calternal-tags --all-targets -- -D warnings: exit=0
cargo test -p calternal-tags: exit=0
cargo clippy -p calternal-fs --all-targets -- -D warnings: exit=0
cargo clippy -p calternal-money --all-targets -- -D warnings: exit=0
cargo test -p calternal-money: exit=0
cargo test -p calternal-fs: exit=0
cargo clippy -p calternal-plugin-calendar --all-targets -- -D warnings: exit=0
cargo test -p calternal-plugin-calendar: exit=0
cargo clippy -p calternal-plugin-files --all-targets -- -D warnings: exit=0
cargo test -p calternal-plugin-files: exit=0
cargo clippy -p calternal-plugin-notes --all-targets -- -D warnings: exit=0
cargo test -p calternal-plugin-notes: exit=0
cargo clippy -p calternal-search --all-targets -- -D warnings: exit=0
cargo test -p calternal-search: exit=101
cargo clippy -p calternal-server --all-targets -- -D warnings: exit=0
cargo test -p calternal-server: exit=0
cargo test -p calternal-search follow-up: exit=0

calternal-notes-core:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 25m 36s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 2m 55s
test result: ok. 504 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.54s
test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.08s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.11s
test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-tags:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 22m 18s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 33m 31s
test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 10.41s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-money:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 23s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 57s
test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s
test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.16s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.49s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

calternal-plugin-calendar:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 19s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 29s
test result: ok. 52 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 38.15s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.22s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.46s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-plugin-files:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 29s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 46s
test result: ok. 134 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 385.38s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-plugin-notes:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 7m 53s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 46s
test result: ok. 126 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 142.82s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.61s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

calternal-search:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 58s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 9m 50s
test result: ok. 33 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 17.57s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.46s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.22s
test result: FAILED. 19 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 64.00s

calternal-server:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 44s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 13s
test result: ok. 85 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 31.36s

calternal-fs:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 43s
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 50s
test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 35.08s
test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 45.87s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The first Search suite failed while polling for the marker in the existing trash race test. The unchanged isolated test passed. The full follow-up passed (exit0) with RUST_TEST_THREADS=1, including every integration binary and doc tests. Both suite results are included above and in the attached raw logs.


running 1 test
test a_file_trashed_during_its_add_never_appears_in_search ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 19 filtered out; finished in 11.37s
test result: ok. 33 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 30.88s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 18.81s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.08s
test result: ok. 20 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 94.93s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s
test result: ok. 1 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 9.30s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

bun run check:

$ node scripts/check-type-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
Text sizes and UI shape values use shared role tokens.
Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/merge-check/apps/web
Getting Svelte diagnostics...

svelte-check found 0 errors and 0 warnings
exit=0

bun run test had one known 5-second font-menu timeout. The original suite and isolated rerun output are preserved. No expectation changed.

Error: Test timed out in 5000ms.
 Test Files  1 failed | 136 passed (137)
      Tests  1 failed | 891 passed (892)
 Test Files  1 passed (1)
      Tests  1 passed (1)

Generated-contract check exited 0; regenerated files have no diff. Parity, sandbox and production capture output:

Parity matrix: 187 web API actions, 113 shortcuts, 2 static commands, 131 menu actions, 30 settings groups, 169 actions with adapter gaps
document input must be a bounded sealed snapshot
document input must be a bounded sealed snapshot
document input must be a bounded sealed snapshot
width: 64
height: 64
PASS: document namespace, private input, inherited limits and network policy
PASS: PDF link/script metadata and bounded SVG references/entities stay isolated
PASS: explicit PDF and SVG loaders produce WebP; PDF action rejects JPEG
x86 policy build: exit=0
arm64 policy build: exit=0
CALENDAR DOC STACK SCREENSHOTS PASSED

The one real-server media round exited 1 for this SLOW-only finding. All other checks reported no finding. It was not repeated; the existing threshold is unchanged.

!! server health after crafted media uploads: status=200, elapsed=6.71s, body=b''
media upload findings: 1

Known gaps: #513 tracks the intermittent Search trash-race marker polling failure; the unchanged isolated test passed. #501 tracks the pre-existing image/video source-inode boundary; no native compromise was reproduced. #507 tracks a scheme preference inconsistency during multi-context review. #510 tracks rendering-kind ambiguity in shared thumbnail cache identity; it is a derived-preview finding. Blocking OS reads cannot be interrupted by async cancellation, although source reads/copies are byte-bounded. arm64 was cross-compiled; real sandbox execution was x86_64.

Decisions: fixed internal PDF/text actions; raw SVG uploads remain outside document mode; PDF snapshots are sealed and capped at 64 MiB; private /tmp is capped at 64 MiB. The complete thumbnail Job has 180 seconds, and document rendering has 60 seconds including snapshot/probe/both sizes. Minimal public addition in calternal-fs keeps all input filesystem work in that crate. Shared dev radius and Caption tokens win over literal values. These choices are recorded in DESIGN §39.

No push or deploy was issued. Issues remain open. Screenshots are attached to #410 for Claude; artifacts are not committed.

Cleanup output, verbatim:

     Removed 21798 files, 13.9GiB total
Job Rust, web and staged test outputs removed; artifacts retained.

Full raw gate output: #410 verbatim gate logs, #473 verbatim gate logs.

Head: `40ccef12818eb27592994a2666021ee34b2b3cf5` on `job/merge-round-0930`. **Built:** merged `job/tag-unicode-473`, then `job/doc-stack`, and merged the one required `origin/dev` refresh. Resolved Calendar token conflicts. Regenerated contracts without a diff. Refreshed the parity inventory for the merged batch Log caller. **Security verdict:** the document renderer boundary passes the defensive audit after hardening. The native decoder has private namespaces, no network, a read-only root, UID/GID 65534, an inherited seccomp policy and bounded resources. Host Homes, data, secret-bearing host directories and extra parent descriptors are absent. The outer environment is cleared before bubblewrap starts, including its PID 1 monitor; monitor host-root and working-directory access are denied by the tested namespace. PDF input is a bounded, fully sealed memory file, not a Home inode. The fixed private symlink names only fd 0. Explicit PDF and generated-SVG actions select loaders. Native tools can create threads only. Complete Jobs and document renders have explicit wall budgets. **Files:** - `deploy/media-sandbox` and `deploy/media-sandbox-dropcaps.c`: isolation, fixed actions, seals and native thread restrictions. - `crates/calternal-fs/src/{lib.rs,thumbnails.rs}`: bounded sealed input helper and regression tests. - `crates/plugins/files/src/thumbnails.rs`: early size/read bounds, shared media slot, fixed loaders, Job/render budgets and failure tests. - `tests/adversarial/{media-sandbox-policy.c,test-document-sandbox.sh,prepare-media-runtime.sh,media_uploads.py,run.sh}`: namespace, descriptor, input and bounded parser checks. - `packages/ui/src/components/{calendar/GridColumn.svelte,files/FileThumb.svelte}`: merged shared radius/Caption tokens. - `apps/web/e2e/calendar-doc-stack.mjs`: production Calendar, Files and Search evidence at 390/820/1440 in light/dark. - `docs/{DESIGN.md,parity-exceptions.json,parity-matrix.md}`: reasoning and the merged parity inventory. **Gates, verbatim:** ```text cargo fmt --check: exit=0 cargo clippy -p calternal-notes-core --all-targets -- -D warnings: exit=0 cargo test -p calternal-notes-core: exit=0 cargo clippy -p calternal-tags --all-targets -- -D warnings: exit=0 cargo test -p calternal-tags: exit=0 cargo clippy -p calternal-fs --all-targets -- -D warnings: exit=0 cargo clippy -p calternal-money --all-targets -- -D warnings: exit=0 cargo test -p calternal-money: exit=0 cargo test -p calternal-fs: exit=0 cargo clippy -p calternal-plugin-calendar --all-targets -- -D warnings: exit=0 cargo test -p calternal-plugin-calendar: exit=0 cargo clippy -p calternal-plugin-files --all-targets -- -D warnings: exit=0 cargo test -p calternal-plugin-files: exit=0 cargo clippy -p calternal-plugin-notes --all-targets -- -D warnings: exit=0 cargo test -p calternal-plugin-notes: exit=0 cargo clippy -p calternal-search --all-targets -- -D warnings: exit=0 cargo test -p calternal-search: exit=101 cargo clippy -p calternal-server --all-targets -- -D warnings: exit=0 cargo test -p calternal-server: exit=0 cargo test -p calternal-search follow-up: exit=0 ``` `calternal-notes-core`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 25m 36s Finished `test` profile [unoptimized + debuginfo] target(s) in 2m 55s test result: ok. 504 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.54s test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.08s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.11s test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `calternal-tags`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 22m 18s Finished `test` profile [unoptimized + debuginfo] target(s) in 33m 31s test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 10.41s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `calternal-money`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 23s Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 57s test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.16s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.49s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s ``` `calternal-plugin-calendar`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 19s Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 29s test result: ok. 52 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 38.15s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.22s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.46s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `calternal-plugin-files`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 5m 29s Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 46s test result: ok. 134 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 385.38s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `calternal-plugin-notes`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 7m 53s Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 46s test result: ok. 126 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 142.82s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.61s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `calternal-search`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 12m 58s Finished `test` profile [unoptimized + debuginfo] target(s) in 9m 50s test result: ok. 33 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 17.57s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.46s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.22s test result: FAILED. 19 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 64.00s ``` `calternal-server`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 44s Finished `test` profile [unoptimized + debuginfo] target(s) in 12m 13s test result: ok. 85 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 31.36s ``` `calternal-fs`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 11m 43s Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 50s test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 35.08s test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 45.87s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` The first Search suite failed while polling for the marker in the existing trash race test. The unchanged isolated test passed. The full follow-up passed (exit0) with RUST_TEST_THREADS=1, including every integration binary and doc tests. Both suite results are included above and in the attached raw logs. ```text running 1 test test a_file_trashed_during_its_add_never_appears_in_search ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 19 filtered out; finished in 11.37s ``` ```text test result: ok. 33 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 30.88s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 18.81s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.08s test result: ok. 20 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 94.93s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s test result: ok. 1 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 9.30s test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.03s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `bun run check`: ```text $ node scripts/check-type-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json Text sizes and UI shape values use shared role tokens. Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/merge-check/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings exit=0 ``` `bun run test` had one known 5-second font-menu timeout. The original suite and isolated rerun output are preserved. No expectation changed. ```text Error: Test timed out in 5000ms. Test Files 1 failed | 136 passed (137) Tests 1 failed | 891 passed (892) Test Files 1 passed (1) Tests 1 passed (1) ``` Generated-contract check exited 0; regenerated files have no diff. Parity, sandbox and production capture output: ```text Parity matrix: 187 web API actions, 113 shortcuts, 2 static commands, 131 menu actions, 30 settings groups, 169 actions with adapter gaps document input must be a bounded sealed snapshot document input must be a bounded sealed snapshot document input must be a bounded sealed snapshot width: 64 height: 64 PASS: document namespace, private input, inherited limits and network policy PASS: PDF link/script metadata and bounded SVG references/entities stay isolated PASS: explicit PDF and SVG loaders produce WebP; PDF action rejects JPEG x86 policy build: exit=0 arm64 policy build: exit=0 CALENDAR DOC STACK SCREENSHOTS PASSED ``` The one real-server media round exited 1 for this SLOW-only finding. All other checks reported no finding. It was not repeated; the existing threshold is unchanged. ```text !! server health after crafted media uploads: status=200, elapsed=6.71s, body=b'' media upload findings: 1 ``` **Known gaps:** #513 tracks the intermittent Search trash-race marker polling failure; the unchanged isolated test passed. #501 tracks the pre-existing image/video source-inode boundary; no native compromise was reproduced. #507 tracks a scheme preference inconsistency during multi-context review. #510 tracks rendering-kind ambiguity in shared thumbnail cache identity; it is a derived-preview finding. Blocking OS reads cannot be interrupted by async cancellation, although source reads/copies are byte-bounded. arm64 was cross-compiled; real sandbox execution was x86_64. **Decisions:** fixed internal PDF/text actions; raw SVG uploads remain outside document mode; PDF snapshots are sealed and capped at 64 MiB; private `/tmp` is capped at 64 MiB. The complete thumbnail Job has 180 seconds, and document rendering has 60 seconds including snapshot/probe/both sizes. Minimal public addition in calternal-fs keeps all input filesystem work in that crate. Shared dev radius and Caption tokens win over literal values. These choices are recorded in DESIGN §39. No push or deploy was issued. Issues remain open. Screenshots are attached to #410 for Claude; artifacts are not committed. **Cleanup output, verbatim:** ```text Removed 21798 files, 13.9GiB total Job Rust, web and staged test outputs removed; artifacts retained. ``` Full raw gate output: [#410 verbatim gate logs](https://git.kayg.org/attachments/2399d8f5-d4b4-4f6c-a27f-b24a11c01e71), [#473 verbatim gate logs](https://git.kayg.org/attachments/80189bd0-1244-45a4-9094-954f887d8484).
Author
Owner

Merged into dev at 8ac92b400 via merge round 1 (all per-crate gates; merged-tree web gates 0 errors, 892/892).

Merged into dev at 8ac92b400 via merge round 1 (all per-crate gates; merged-tree web gates 0 errors, 892/892).
kayg closed this issue 2026-09-30 12:04:47 +00:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#473
No description provided.