kayg commented

Owner

The owner wants to copy their data from calternal.js. calternal.js keeps a local plaintext Markdown vault (calternal.js CLI/daemon, default ~/calternal-vault; packages/cli/src/vault.ts): day files in events/ (post-#38) and journal/ (pre-#38, read as a union), notes/, attachments/, tasks (check), plus whatever else vault.ts defines. DESIGN §17: calternal Home folders are Notes/, Tasks/, Attachments/, Photos/, Calendar/; Daily notes live in Notes/Journal/ with the same YYYYMMDD-dailynote.md format; 'Import from a calternal.js vault translates its lowercase paths.' No importer exists yet.

Build 'calternal import calternal-js ' in the calternal Rust CLI (crates/calternal-cli or wherever the CLI lives), uploading through the normal authenticated API (server stays the single writer):

Path map: events/ + journal/ day files → Notes/Journal/ (union; on the same day in both, merge Log sections without dropping or duplicating lines, by block id then byte equality); notes/ → Notes/; attachments/ → Attachments/; tasks → Tasks/; anything unknown → Imported/calternal.js/ (never dropped).
Rewrite relative links and embeds whose targets moved, byte-preserving everything else; keep ^block-ids, frontmatter (calternal-id where present), tags.
Idempotent and resumable: re-running skips files already identical; a differing existing file becomes a conflict copy, never an overwrite. --dry-run prints the plan (counts per folder, conflicts, rewritten links). A final report lists every file with its action.
Preserve mtimes where the API allows; handle huge vaults (streamed uploads through tus), Unicode/odd names, symlinks refused, .obsidian/.git and other dot folders skipped (reported).
Tests: a fixture vault covering every folder, both journal/ and events/ for the same day, links across moved folders, attachments with spaces, non-ASCII, a big file; import twice → second run is a no-op; import into a Home with an existing Daily note → merge without loss; the imported day files parse identically in calternal (Journal API shows the same entries).
Document the exact steps for the owner in docs (how to get the vault from calternal.js: calternal sync / the vault folder location, then run the import).

The owner wants to copy their data from calternal.js. calternal.js keeps a local plaintext Markdown vault (calternal.js CLI/daemon, default ~/calternal-vault; packages/cli/src/vault.ts): day files in events/ (post-#38) and journal/ (pre-#38, read as a union), notes/, attachments/, tasks (check), plus whatever else vault.ts defines. DESIGN §17: calternal Home folders are Notes/, Tasks/, Attachments/, Photos/, Calendar/; Daily notes live in Notes/Journal/ with the same YYYYMMDD-dailynote.md format; 'Import from a calternal.js vault translates its lowercase paths.' No importer exists yet. Build 'calternal import calternal-js <vault-dir>' in the calternal Rust CLI (crates/calternal-cli or wherever the CLI lives), uploading through the normal authenticated API (server stays the single writer): - Path map: events/ + journal/ day files → Notes/Journal/ (union; on the same day in both, merge Log sections without dropping or duplicating lines, by block id then byte equality); notes/ → Notes/; attachments/ → Attachments/; tasks → Tasks/; anything unknown → Imported/calternal.js/<same path> (never dropped). - Rewrite relative links and embeds whose targets moved, byte-preserving everything else; keep ^block-ids, frontmatter (calternal-id where present), tags. - Idempotent and resumable: re-running skips files already identical; a differing existing file becomes a conflict copy, never an overwrite. --dry-run prints the plan (counts per folder, conflicts, rewritten links). A final report lists every file with its action. - Preserve mtimes where the API allows; handle huge vaults (streamed uploads through tus), Unicode/odd names, symlinks refused, .obsidian/.git and other dot folders skipped (reported). - Tests: a fixture vault covering every folder, both journal/ and events/ for the same day, links across moved folders, attachments with spaces, non-ASCII, a big file; import twice → second run is a no-op; import into a Home with an existing Daily note → merge without loss; the imported day files parse identically in calternal (Journal API shows the same entries). - Document the exact steps for the owner in docs (how to get the vault from calternal.js: calternal sync / the vault folder location, then run the import).

kayg commented

2026-09-26 07:30:22 +00:00

Author

Owner

Not needed: the owner copies the vault files by hand (same Markdown format; only the folder names change).

kayg closed this issue

2026-09-26 07:30:22 +00:00

kayg commented

2026-09-26 09:46:31 +00:00

Author

Owner

Reopened 2026-09-26: the owner wants to dogfood with their real calternal.js vault. A plain copy is NOT enough: calternal.js links are vault-root-relative (![photo](<attachments/2026/p.jpg>), [Note](<notes/a.md>) inside Log bullets), calternal resolves links relative to the file, so every embed and note link in day files would break after the move to Notes/Journal/. Reuse the existing helpers in crates/calternal-notes-core/src/dayfile.rs (migrate_path, rewrite_body_links) and extend them to all link forms and all moved folders (notes/, tasks/, attachments/, events/, journal/), for day files AND notes. Everything in the original issue body still applies (dry-run, idempotent, conflict copies, report, byte preservation of everything not rewritten). Deliver the CLI subcommand: calternal import calternal-js <vault-dir> [--dry-run], uploading through the authenticated API (server stays the single writer). Also check the calternal.js tasks/ format against calternal Tasks/ and convert only if the formats differ (document the result). Add the owner steps to docs (ASD-STE100): sync calternal.js, stop calternald, dry-run, import, check.

Reopened 2026-09-26: the owner wants to dogfood with their real calternal.js vault. A plain copy is NOT enough: calternal.js links are vault-root-relative (`![photo](<attachments/2026/p.jpg>)`, `[Note](<notes/a.md>)` inside Log bullets), calternal resolves links relative to the file, so every embed and note link in day files would break after the move to Notes/Journal/. Reuse the existing helpers in crates/calternal-notes-core/src/dayfile.rs (migrate_path, rewrite_body_links) and extend them to all link forms and all moved folders (notes/, tasks/, attachments/, events/, journal/), for day files AND notes. Everything in the original issue body still applies (dry-run, idempotent, conflict copies, report, byte preservation of everything not rewritten). Deliver the CLI subcommand: `calternal import calternal-js <vault-dir> [--dry-run]`, uploading through the authenticated API (server stays the single writer). Also check the calternal.js tasks/ format against calternal Tasks/ and convert only if the formats differ (document the result). Add the owner steps to docs (ASD-STE100): sync calternal.js, stop calternald, dry-run, import, check.

kayg commented

2026-09-26 09:48:20 +00:00

Author

Owner

Started #152 on job/import-calternaljs, based on dev at 6440a5f8382794f423847c07b98adfa4852a398d.

I read the reopening comment. I will cover root-relative Markdown links in day files and Notes, map every moved folder, preserve untouched bytes, compare the calternal.js Tasks format, and keep conflicts non-destructive.

Started #152 on `job/import-calternaljs`, based on `dev` at `6440a5f8382794f423847c07b98adfa4852a398d`. I read the reopening comment. I will cover root-relative Markdown links in day files and Notes, map every moved folder, preserve untouched bytes, compare the calternal.js Tasks format, and keep conflicts non-destructive.

kayg referenced this issue

2026-09-26 10:23:44 +00:00

Flat Home layout: Notes/ flat (tasks and Daily notes included), attachments by type, no folder-derived meaning (DESIGN §40) #165

kayg commented

2026-09-26 10:24:05 +00:00

Author

Owner

Target layout changed (DESIGN §40, owner 2026-09-26): calternal.js events/ + journal/ day files → Notes/YYYYMMDD-dailynote.md (flat, no Journal/); notes/ → Notes/ (flat; a calternal.js subfolder under notes/ flattens into Notes/ with a unique name); tasks/ → Notes/ with task frontmatter; attachments/ images+videos → Photos/YYYY/YYYY-MM-DD/ (EXIF capture date, else file mtime), audio → Documents/Voice memos/, known documents → Documents/, unknown → Documents/Other/. Link rewriting targets these new paths.

kayg commented

2026-09-26 10:26:00 +00:00

Author

Owner

Continuation of #152 on job/import-calternaljs. Original branch base: 6440a5f8382794f423847c07b98adfa4852a398d; current job HEAD: 225492ed92a81bcd8d5f5fd799b95c96188eb8f4. I read DESIGN §40 from current dev (44b15be) and the latest issue comment. I will retarget the import mapping and link rewriting to the flat Home layout while preserving dry-run, idempotency, conflict copies, reporting, byte preservation, and property tests. I will keep the destination routing in one small table until #165's routing table lands on dev.

Continuation of #152 on `job/import-calternaljs`. Original branch base: `6440a5f8382794f423847c07b98adfa4852a398d`; current job HEAD: `225492ed92a81bcd8d5f5fd799b95c96188eb8f4`. I read DESIGN §40 from current `dev` (`44b15be`) and the latest issue comment. I will retarget the import mapping and link rewriting to the flat Home layout while preserving dry-run, idempotency, conflict copies, reporting, byte preservation, and property tests. I will keep the destination routing in one small table until #165's routing table lands on `dev`.

kayg commented

2026-09-26 10:44:11 +00:00

Author

Owner

Task-format check: /home/kayg/Developer/calternal.js/packages/core/src/tasks/build.rs emits title, priority, start, scheduled, due, recurrence, area, created, and checkbox lines. crates/calternal-notes-core/src/tasks/extract.rs reads those fields. No task-content conversion is needed. The task path maps to Notes/; #165 supplies task recognition by frontmatter.

Task-format check: `/home/kayg/Developer/calternal.js/packages/core/src/tasks/build.rs` emits `title`, `priority`, `start`, `scheduled`, `due`, `recurrence`, `area`, `created`, and checkbox lines. `crates/calternal-notes-core/src/tasks/extract.rs` reads those fields. No task-content conversion is needed. The task path maps to `Notes/`; #165 supplies task recognition by frontmatter.

kayg commented

2026-09-26 12:12:50 +00:00

Author

Owner

Evidence: dev documents Daily notes at flat Notes/YYYYMMDD-dailynote.md in DESIGN §40, but the current Journal scanner in crates/plugins/notes/src/lib.rs still lists Notes/Journal, and calternal-notes-core::daily_note_path still returns Notes/Journal/.... A flat import therefore cannot satisfy the issue's Journal API check on this base. I will keep the importer paths aligned with §40 and include the minimal server-side compatibility change needed for the Journal API to read those flat Daily notes.

Evidence: `dev` documents Daily notes at flat `Notes/YYYYMMDD-dailynote.md` in DESIGN §40, but the current Journal scanner in `crates/plugins/notes/src/lib.rs` still lists `Notes/Journal`, and `calternal-notes-core::daily_note_path` still returns `Notes/Journal/...`. A flat import therefore cannot satisfy the issue's Journal API check on this base. I will keep the importer paths aligned with §40 and include the minimal server-side compatibility change needed for the Journal API to read those flat Daily notes.

kayg commented

2026-09-26 12:53:46 +00:00

Author

Owner

The importer fixture found that the second run moved a linked Note into a new conflict copy because conflict planning compared it before applying another file's destination mapping. I changed planning to rebuild the complete link map after each conflict decision. cargo test -p calternal-cli now passes, and the fixture confirms that a second import adds no writes.

The importer fixture found that the second run moved a linked Note into a new conflict copy because conflict planning compared it before applying another file's destination mapping. I changed planning to rebuild the complete link map after each conflict decision. `cargo test -p calternal-cli` now passes, and the fixture confirms that a second import adds no writes.

kayg commented

2026-09-26 13:21:58 +00:00

Author

Owner

Found and fixed an importer retry inconsistency. NotesJournalProvider::ensure_ids assigns IDs to legacy Log entries when the Journal reads an imported Daily note. A rerun then saw changed bytes and could append the same event again. The merge helper now recognizes the Journal-stamped line by its parsed content, and the importer fixture simulates the server ID before its second run.

Evidence: cargo test -p calternal-notes-core server_assigned_block_id_does_not_make_an_imported_line_new -- --nocapture passed (1 test); cargo test -p calternal-cli import_calternaljs::tests -- --nocapture passed (2 tests). Flat Daily note compatibility also passed its Journal provider and GET /journal/{date} test.

Found and fixed an importer retry inconsistency. `NotesJournalProvider::ensure_ids` assigns IDs to legacy Log entries when the Journal reads an imported Daily note. A rerun then saw changed bytes and could append the same event again. The merge helper now recognizes the Journal-stamped line by its parsed content, and the importer fixture simulates the server ID before its second run. Evidence: `cargo test -p calternal-notes-core server_assigned_block_id_does_not_make_an_imported_line_new -- --nocapture` passed (1 test); `cargo test -p calternal-cli import_calternaljs::tests -- --nocapture` passed (2 tests). Flat Daily note compatibility also passed its Journal provider and `GET /journal/{date}` test.

kayg commented

2026-09-26 15:14:01 +00:00

Author

Owner

The real-server adversarial round found one non-load cross-plugin issue and filed it as #187: #187. The probe installed/indexed Photos/2024/2024-06-02/adversarial-photo.jpg, shared the Photos folder from user A to user B, and opted B into Shared/<A>/Photos. Repeated timeline reads returned HTTP 200 with an empty days array for the full 12-second visibility window. This is outside the import change and is recorded with the reproduction in #187.

The real-server adversarial round found one non-load cross-plugin issue and filed it as #187: https://git.kayg.org/kayg/calternal/issues/187. The probe installed/indexed `Photos/2024/2024-06-02/adversarial-photo.jpg`, shared the `Photos` folder from user A to user B, and opted B into `Shared/<A>/Photos`. Repeated timeline reads returned HTTP 200 with an empty `days` array for the full 12-second visibility window. This is outside the import change and is recorded with the reproduction in #187.

kayg commented

2026-09-26 15:31:00 +00:00

Author

Owner

Adversarial probe update: the round-2 logrewrite and journalrace probes now use the owner-decided flat Daily note path (Notes/YYYYMMDD-dailynote.md). A fresh-server focused run passed with 0 findings; it retained 6 seed entries, 40 accepted creates, 2 conditional sync uploads, and 46 total Journal entries, and the server stayed alive.

The full round also found a separate Photos issue: a recipient's active shared Photos root is omitted from the Photos timeline. I filed it as #187 with reproduction details. The first full run's remaining Notes probe failures were stale test paths and have now been corrected and rechecked. The attack.py findings were SLOW-only under shared build load; hostile bytes reported 0 findings.

I am merging the latest dev into this job branch before running the final gates.

Adversarial probe update: the round-2 logrewrite and journalrace probes now use the owner-decided flat Daily note path (Notes/YYYYMMDD-dailynote.md). A fresh-server focused run passed with 0 findings; it retained 6 seed entries, 40 accepted creates, 2 conditional sync uploads, and 46 total Journal entries, and the server stayed alive. The full round also found a separate Photos issue: a recipient's active shared Photos root is omitted from the Photos timeline. I filed it as #187 with reproduction details. The first full run's remaining Notes probe failures were stale test paths and have now been corrected and rechecked. The attack.py findings were SLOW-only under shared build load; hostile bytes reported 0 findings. I am merging the latest dev into this job branch before running the final gates.

kayg referenced this issue

2026-09-26 15:56:35 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg commented

2026-09-26 19:06:50 +00:00

Author

Owner

Final-gate findings after merging the flat Home layout:

tasks_store::latest_or_stamp_legacy scanned only Notes/Journal/. A flat Daily note with an unindexed legacy Log entry therefore did not attach a newly created Task. It now scans both locations and prefers the flat Daily note for a date present in both.
tasks_api::by_day extracted attachment dates at the legacy path offset (Notes/Journal/), so Tasks attached to flat Daily notes were omitted from the requested day. It now recognizes both layouts through the existing Journal path parser.
Three Notes tests still used Notes/Journal/ as the canonical write path. They now use Notes/YYYYMMDD-dailynote.md from DESIGN §40.

Evidence: the first workspace cargo test run reported 5 failures in calternal-plugin-notes; after these changes, cargo test -p calternal-plugin-notes reported test result: ok. 90 passed; 0 failed; 0 ignored; 0 measured; finished in 91.16s.

Final-gate findings after merging the flat Home layout: - `tasks_store::latest_or_stamp_legacy` scanned only `Notes/Journal/`. A flat Daily note with an unindexed legacy Log entry therefore did not attach a newly created Task. It now scans both locations and prefers the flat Daily note for a date present in both. - `tasks_api::by_day` extracted attachment dates at the legacy path offset (`Notes/Journal/`), so Tasks attached to flat Daily notes were omitted from the requested day. It now recognizes both layouts through the existing Journal path parser. - Three Notes tests still used `Notes/Journal/` as the canonical write path. They now use `Notes/YYYYMMDD-dailynote.md` from DESIGN §40. Evidence: the first workspace `cargo test` run reported 5 failures in `calternal-plugin-notes`; after these changes, `cargo test -p calternal-plugin-notes` reported `test result: ok. 90 passed; 0 failed; 0 ignored; 0 measured; finished in 91.16s`.

kayg commented

2026-09-26 19:36:50 +00:00

Author

Owner

Real-server finding from the post-merge adversarial round:

The tz-days probe created mwgrvxzlvm in a flat Daily note for 2026-08-15. The entry was present in users/<user>/Notes/20260815-dailynote.md; Journal and Calendar reads passed, but search did not return it as a Log entry after the probe's 30-second wait. Root cause: calternal-search::indexer::daily_note_date and query::daily_note_date accept only Notes/Journal/ paths, so the flat Daily note's Log entries are omitted from search indexing and cannot get Journal result links/date filtering. I will add flat-layout support and regression coverage in calternal-search.

Real-server finding from the post-merge adversarial round: The `tz-days` probe created `mwgrvxzlvm` in a flat Daily note for 2026-08-15. The entry was present in `users/<user>/Notes/20260815-dailynote.md`; Journal and Calendar reads passed, but search did not return it as a Log entry after the probe's 30-second wait. Root cause: `calternal-search::indexer::daily_note_date` and `query::daily_note_date` accept only `Notes/Journal/` paths, so the flat Daily note's Log entries are omitted from search indexing and cannot get Journal result links/date filtering. I will add flat-layout support and regression coverage in `calternal-search`.

kayg referenced this issue

2026-09-26 19:53:08 +00:00

Investigate request timeouts in concurrent Journal and capture probes #205

kayg referenced this issue

2026-09-26 20:14:58 +00:00

Local adversarial run lacks the media sandbox for thumbnail checks #209

kayg referenced this issue

2026-09-26 20:32:31 +00:00

Block actions: hover controls, make-note (→), block reminders with quiet hours, phone capsule bar (DESIGN §42) #191

kayg commented

2026-09-26 20:36:12 +00:00

Author

Owner

#152 continuation report

Built:

Added calternal import calternal-js <vault-dir> [--dry-run] with authenticated, resumable uploads, source-to-destination mapping, byte-preserving moved-link rewrites, idempotency, conflict copies, symlink and dot-folder handling, and a per-file report.
Added the owner import guide and fixture coverage.
Fixed flat Daily note Task attachment lookup after the §40 layout merge.
Fixed Search indexing, date filtering and deep links for flat Daily note Log entries while retaining legacy Journal paths. The flat-path regression passes.

Files changed against dev:

Cargo.lock
crates/calternal-cli/Cargo.toml
crates/calternal-cli/src/import_calternaljs.rs
crates/calternal-cli/src/main.rs
crates/calternal-cli/src/remote_commands.rs
crates/calternal-cli/tests/import_help.rs
crates/calternal-collab/tests/journal_race.rs
crates/calternal-notes-core/Cargo.toml
crates/calternal-notes-core/PARITY.md
crates/calternal-notes-core/src/dayfile.rs
crates/calternal-notes-core/src/lib.rs
crates/calternal-notes-core/src/links.rs
crates/calternal-notes-core/tests/vectors/dayfile.json
crates/calternal-search/src/indexer.rs
crates/calternal-search/src/lib.rs
crates/calternal-search/src/query.rs
crates/calternal-search/tests/indexer.rs
crates/plugins/calendar/src/items.rs
crates/plugins/files/src/lib.rs
crates/plugins/notes/src/lib.rs
crates/plugins/notes/src/store.rs
crates/plugins/notes/src/tasks_api.rs
crates/plugins/notes/src/tasks_store.rs
docs/import-calternaljs.md
tests/adversarial/attack2.py

Head SHA: cf5874759c8d055c64b36e6f61db91c6b93ef919

Gate output:

cargo fmt --check: exit 0, no output.
cargo clippy --all-targets -- -D warnings:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 01s

cargo test stopped at the existing Notifications path mismatch:

test result: FAILED. 11 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.61s
error: test failed, to rerun pass `-p calternal-plugin-notifications --lib`

The changed Notes crate passed in this run: test result: ok. 90 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 101.28s.

apps/web: bun run check:

svelte-check found 0 errors and 0 warnings

apps/web: bun run test:

 Test Files  55 passed (55)
      Tests  432 passed (432)

Additional verification:

cargo test -p calternal-search: passed (24 unit, 15 indexer, 1 ask-evaluation, 1 citations, 4 operators, 1 relevance, and 4 retrieval tests; 3 ignored).
The post-fix real-server tz-days probe returned ==== ROUND 2 FINDINGS 0; the latest full post-merge round also found no Daily note Search failure.
Latest full adversarial round: Round 1 had 36 SLOW-only findings; hostile bytes had 0 findings. Round 2 had 28 entries: successful SLOW responses plus 15 missing shared-photo thumbnail responses. The restart probe reported 0 findings and the server stayed alive.
The thumbnail limitation is recorded on #209: this worktree has no calternal-media-sandbox, no thumbnail was generated, and local thumbnail results do not establish packaged-runtime behavior.
An earlier bookmark capture timeout under measured shared-host load was recorded on #205.

Known gap:

The workspace test gate remains red in calternal-plugin-notifications::reminders::tests::reminder_paths_and_block_ids_are_constrained. The reminder scanner and helper still require Notes/Journal/, while DESIGN §40 and calternal_notes_core::daily_note_path now use flat Notes/. This also prevents reminders on flat Daily notes. I recorded the finding on #191 and left the cross-crate behavior change out of #152.

Decisions:

Search accepts both canonical flat Daily note paths and exact legacy Notes/Journal/ paths; DESIGN §40 makes the flat path canonical.
I kept the Notifications behavior change on #191 because it is outside this job's owned behavior. No other DESIGN decision was needed.

#152 continuation report Built: - Added `calternal import calternal-js <vault-dir> [--dry-run]` with authenticated, resumable uploads, source-to-destination mapping, byte-preserving moved-link rewrites, idempotency, conflict copies, symlink and dot-folder handling, and a per-file report. - Added the owner import guide and fixture coverage. - Fixed flat Daily note Task attachment lookup after the §40 layout merge. - Fixed Search indexing, date filtering and deep links for flat Daily note Log entries while retaining legacy Journal paths. The flat-path regression passes. Files changed against `dev`: - `Cargo.lock` - `crates/calternal-cli/Cargo.toml` - `crates/calternal-cli/src/import_calternaljs.rs` - `crates/calternal-cli/src/main.rs` - `crates/calternal-cli/src/remote_commands.rs` - `crates/calternal-cli/tests/import_help.rs` - `crates/calternal-collab/tests/journal_race.rs` - `crates/calternal-notes-core/Cargo.toml` - `crates/calternal-notes-core/PARITY.md` - `crates/calternal-notes-core/src/dayfile.rs` - `crates/calternal-notes-core/src/lib.rs` - `crates/calternal-notes-core/src/links.rs` - `crates/calternal-notes-core/tests/vectors/dayfile.json` - `crates/calternal-search/src/indexer.rs` - `crates/calternal-search/src/lib.rs` - `crates/calternal-search/src/query.rs` - `crates/calternal-search/tests/indexer.rs` - `crates/plugins/calendar/src/items.rs` - `crates/plugins/files/src/lib.rs` - `crates/plugins/notes/src/lib.rs` - `crates/plugins/notes/src/store.rs` - `crates/plugins/notes/src/tasks_api.rs` - `crates/plugins/notes/src/tasks_store.rs` - `docs/import-calternaljs.md` - `tests/adversarial/attack2.py` Head SHA: `cf5874759c8d055c64b36e6f61db91c6b93ef919` Gate output: - `cargo fmt --check`: exit 0, no output. - `cargo clippy --all-targets -- -D warnings`: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 01s ``` - `cargo test` stopped at the existing Notifications path mismatch: ``` test result: FAILED. 11 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.61s error: test failed, to rerun pass `-p calternal-plugin-notifications --lib` ``` The changed Notes crate passed in this run: `test result: ok. 90 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 101.28s`. - `apps/web: bun run check`: ``` svelte-check found 0 errors and 0 warnings ``` - `apps/web: bun run test`: ``` Test Files 55 passed (55) Tests 432 passed (432) ``` Additional verification: - `cargo test -p calternal-search`: passed (24 unit, 15 indexer, 1 ask-evaluation, 1 citations, 4 operators, 1 relevance, and 4 retrieval tests; 3 ignored). - The post-fix real-server `tz-days` probe returned `==== ROUND 2 FINDINGS 0`; the latest full post-merge round also found no Daily note Search failure. - Latest full adversarial round: Round 1 had 36 SLOW-only findings; hostile bytes had 0 findings. Round 2 had 28 entries: successful SLOW responses plus 15 missing shared-photo thumbnail responses. The restart probe reported 0 findings and the server stayed alive. - The thumbnail limitation is recorded on #209: this worktree has no `calternal-media-sandbox`, no thumbnail was generated, and local thumbnail results do not establish packaged-runtime behavior. - An earlier bookmark capture timeout under measured shared-host load was recorded on #205. Known gap: - The workspace test gate remains red in `calternal-plugin-notifications::reminders::tests::reminder_paths_and_block_ids_are_constrained`. The reminder scanner and helper still require `Notes/Journal/`, while DESIGN §40 and `calternal_notes_core::daily_note_path` now use flat `Notes/`. This also prevents reminders on flat Daily notes. I recorded the finding on #191 and left the cross-crate behavior change out of #152. Decisions: - Search accepts both canonical flat Daily note paths and exact legacy `Notes/Journal/` paths; DESIGN §40 makes the flat path canonical. - I kept the Notifications behavior change on #191 because it is outside this job's owned behavior. No other DESIGN decision was needed.

kayg commented

2026-09-26 22:28:06 +00:00

Author

Owner

Done on job/import-calternaljs (22 commits), held by Claude until #165 (flat layout) lands: the branch fails calternal-plugin-notifications reminders tests that still expect Notes/Journal/. Merge after #165, then rerun gates.

kayg referenced this issue

2026-09-27 10:23:51 +00:00

Owner test tracker: every request and when it is testable on o2 #203

kayg commented

2026-09-27 17:34:28 +00:00

Author

Owner

Starting work on job/import-calternaljs at cf5874759c8d055c64b36e6f61db91c6b93ef919; current dev is 060094c352206389038d00ebb10c1f9ca050191d. I will merge dev once, update the importer for DESIGN §40, repair reminder expectations, then run the requested fixture import and final gates.

Starting work on `job/import-calternaljs` at `cf5874759c8d055c64b36e6f61db91c6b93ef919`; current `dev` is `060094c352206389038d00ebb10c1f9ca050191d`. I will merge `dev` once, update the importer for DESIGN §40, repair reminder expectations, then run the requested fixture import and final gates.

kayg commented

2026-09-27 18:33:42 +00:00

Author

Owner

Merge and test findings for #152:

The one required merge with dev exposed redact_note_references as a public export without its implementation in the conflicted links module. Restored the implementation and its test from dev; notes-core tests pass.
calternal-plugin-calendar did not handle the merged UserSettingsError::Changed case. It now maps this concurrent update to HTTP 409 and has a regression test; calendar tests pass.
Search had duplicate daily-note date parsers after the merge. Consolidated the parser so Clippy does not report dead code; search tests pass.
The notes-core full run found a stale parity vector for the now-unknown uppercase Notes/... source root. Updated its expected path to Imported/calternal.js/Notes/...; the full notes-core crate now passes (495 unit tests, 13 log rewrite, 5 parser property, 7 Unicode title, 11 vector parity, 0 doc tests).

Other changed crate checks completed successfully: calternal-cli (13 unit + 1 help), calendar (46 unit + 1 cache + 3 protocol), notifications (12), notes (93), search (25 unit with one ignored plus all integration suites), and files (108). Final workspace gates and the real-server import are still pending.

Merge and test findings for #152: - The one required merge with `dev` exposed `redact_note_references` as a public export without its implementation in the conflicted links module. Restored the implementation and its test from `dev`; notes-core tests pass. - `calternal-plugin-calendar` did not handle the merged `UserSettingsError::Changed` case. It now maps this concurrent update to HTTP 409 and has a regression test; calendar tests pass. - Search had duplicate daily-note date parsers after the merge. Consolidated the parser so Clippy does not report dead code; search tests pass. - The notes-core full run found a stale parity vector for the now-unknown uppercase `Notes/...` source root. Updated its expected path to `Imported/calternal.js/Notes/...`; the full notes-core crate now passes (495 unit tests, 13 log rewrite, 5 parser property, 7 Unicode title, 11 vector parity, 0 doc tests). Other changed crate checks completed successfully: calternal-cli (13 unit + 1 help), calendar (46 unit + 1 cache + 3 protocol), notifications (12), notes (93), search (25 unit with one ignored plus all integration suites), and files (108). Final workspace gates and the real-server import are still pending.

kayg commented

2026-09-27 19:20:07 +00:00

Author

Owner

#152 real-server progress:

A fresh-Home CLI/API import passed with 10 fixture files, 9 imported target files (same-date event and Journal Daily notes combined), 2 rewritten links, and verified Notes, dated Photos, Documents, Voice memos, Other, and Imported paths. Home root listing and stat checks found no Journal, Tasks, Attachments, Calendar, or Files folders.
A separate fixture variant seeded an existing Home Daily note. One import completed; a later run returned CLI exit 4, remote file changed before upload publication, at the optimistic replace guard. The server refused the replace, so this attempt did not overwrite that revision. The fixture now tests source Daily-note grouping on a fresh Home; package tests cover import behavior against an existing remote Daily note.
The adversarial authz matrix has one response mismatch so far: PUT /api/v1/admin/config as the standard User returned HTTP 422 where the matrix expected 403. The response rejected the request. Review pending after the round.
The calendar-event-tag subprobe did not launch because calendar_event_tags.mjs reads CALTERNAL_SERVER_BIN, while the outer harness only set ADVERSARIAL_SERVER_BIN. I will run that focused probe with the expected variable after the current round. Other probes are still running.

The real server binary is from the current dev worktree. Building the server again in this worktree spent 12 minutes in serial vendored OpenSSL compilation, so the existing local production build was used to keep the time-boxed round moving. The current branch CLI is rebuilt from this worktree.

#152 real-server progress: - A fresh-Home CLI/API import passed with 10 fixture files, 9 imported target files (same-date event and Journal Daily notes combined), 2 rewritten links, and verified Notes, dated Photos, Documents, Voice memos, Other, and Imported paths. Home root listing and `stat` checks found no Journal, Tasks, Attachments, Calendar, or Files folders. - A separate fixture variant seeded an existing Home Daily note. One import completed; a later run returned CLI exit 4, `remote file changed before upload publication`, at the optimistic replace guard. The server refused the replace, so this attempt did not overwrite that revision. The fixture now tests source Daily-note grouping on a fresh Home; package tests cover import behavior against an existing remote Daily note. - The adversarial authz matrix has one response mismatch so far: `PUT /api/v1/admin/config` as the standard User returned HTTP 422 where the matrix expected 403. The response rejected the request. Review pending after the round. - The calendar-event-tag subprobe did not launch because `calendar_event_tags.mjs` reads `CALTERNAL_SERVER_BIN`, while the outer harness only set `ADVERSARIAL_SERVER_BIN`. I will run that focused probe with the expected variable after the current round. Other probes are still running. The real server binary is from the current `dev` worktree. Building the server again in this worktree spent 12 minutes in serial vendored OpenSSL compilation, so the existing local production build was used to keep the time-boxed round moving. The current branch CLI is rebuilt from this worktree.

kayg commented

2026-09-27 19:41:14 +00:00

Author

Owner

The realistic fresh-Home import passed before the attack probes: 10 fixture files in dry-run and real API import, 9 imported, 2 links rewritten, same-day source Daily note entries merged, and no legacy folders created.

Adversarial round is still running against the existing local dev server binary (not a server built from this job branch). The shared host currently has several other calternal-server builds and another attack.py process using CPU. Findings so far: template storm had 2 timed-out requests and 14/16 unique Note IDs; template path-traversal request timed out instead of returning 400; two Reminders discovery requests timed out, while later sync and VTODO create/read/delete succeeded; bookmark capture storm had 4 request timeouts and 12 HTTP 429 responses without the expected Retry-After. HEIF/AVIF upload probe passed; media thumbnails were not ready in 10 seconds, so its header check was skipped. These are load-sensitive and do not implicate the importer; I will record final run status after the one time-boxed round completes.

The realistic fresh-Home import passed before the attack probes: 10 fixture files in dry-run and real API import, 9 imported, 2 links rewritten, same-day source Daily note entries merged, and no legacy folders created. Adversarial round is still running against the existing local dev server binary (not a server built from this job branch). The shared host currently has several other calternal-server builds and another attack.py process using CPU. Findings so far: template storm had 2 timed-out requests and 14/16 unique Note IDs; template path-traversal request timed out instead of returning 400; two Reminders discovery requests timed out, while later sync and VTODO create/read/delete succeeded; bookmark capture storm had 4 request timeouts and 12 HTTP 429 responses without the expected Retry-After. HEIF/AVIF upload probe passed; media thumbnails were not ready in 10 seconds, so its header check was skipped. These are load-sensitive and do not implicate the importer; I will record final run status after the one time-boxed round completes.

kayg commented

2026-09-27 19:51:21 +00:00

Author

Owner

Correction to my previous comment: the bookmark capture test accepts HTTP 429 and does not require Retry-After on 429. Its aggregate finding text is generic; the unexpected results were four timed-out requests (status -1). I should not have described the 429 responses as missing Retry-After.

kayg referenced this issue

2026-09-27 19:51:58 +00:00

[adversarial] API probes timed out during shared-host load #264

kayg referenced this issue

2026-09-27 20:23:49 +00:00

[auth] Concurrent recovery-key rotation can reject both requests #266

kayg commented

2026-09-27 20:35:01 +00:00

Author

Owner

Final report

Finished job/import-calternaljs at 11816527dc0c1cd2eecaffe8f31c1a800665bad8. git push origin job/import-calternaljs succeeded (Everything up-to-date; the branch had already been pushed by the worktree timer). The required one-time git merge dev is commit dd4dbc0467ffcd8b2d7e307ec9729abaf5e02be6, with dev at 060094c352206389038d00ebb10c1f9ca050191d.

Built

The importer writes the §40 flat Home layout. Notes, Daily notes and task Markdown go directly in Notes/; photos and videos use dated Photos/ folders; audio goes in Documents/Voice memos/; known documents go in Documents/; unknown attachments go in Documents/Other/. Same-date source Daily notes merge Log entries, moved-note links are rewritten, and legacy app folders are not created. The calendar merge maps changed user settings to HTTP 409. The reminder test fixture now uses a flat Notes/ Daily note.

Files changed across the branch: Cargo.lock; crates/calternal-cli/Cargo.toml, src/import_calternaljs.rs, src/main.rs, src/remote_commands.rs, tests/import_help.rs; crates/calternal-notes-core/Cargo.toml, PARITY.md, src/dayfile.rs, src/lib.rs, src/links.rs, tests/vectors/dayfile.json; crates/calternal-search/src/indexer.rs, src/lib.rs, src/query.rs, tests/indexer.rs; crates/plugins/calendar/src/items.rs; crates/plugins/notifications/src/reminders.rs; and docs/import-calternaljs.md.

Real-server fixture import

IMPORT_E2E PASS: dry-run and one real API import covered 10 fixture files; same-day source Daily note entries merged; links rewrote; Notes, dated Photos, Documents, Voice memos, Other, and Imported targets verified; no legacy folders in Home.
IMPORT_E2E NOTE_TARGET: Notes/Research notes-962364ddbf64e583.md
IMPORT_E2E PHOTO_TARGET: Photos/2026/2026-09-24/field visit-38eb88d50498268f.jpg
IMPORT_E2E SUMMARY: {"already_identical": 0, "conflicts": 0, "files_reported": 10, "imported": 9, "rewritten_links": 2}

Gates

cargo fmt --check passed after a formatting-only correction committed as 11816527:

EXIT_CODE=0

cargo clippy --all-targets -- -D warnings:

Finished `dev` profile [unoptimized + debuginfo] target(s) in 8m 22s
EXIT_CODE=0

The one full cargo test command stopped at calternal-auth:

---- store::tests::key_rotation_revokes_sessions_and_rejects_competing_recovery stdout ----
thread 'store::tests::key_rotation_revokes_sessions_and_rejects_competing_recovery' (1133462) panicked at crates/calternal-auth/src/store.rs:2676:9:
assertion `left == right` failed
  left: 0
 right: 1

test result: FAILED. 49 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 87.63s
error: test failed, to rerun pass `-p calternal-auth --lib`
EXIT_CODE=101

That test expected one of two simultaneous recovery-key rotations to succeed; both returned errors. It is outside the importer changes and is filed as #266. Remaining workspace Rust tests did not run. The test was not rerun.

bun run check:

$ svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/import-calternaljs/apps/web
Getting Svelte diagnostics...

svelte-check found 0 errors and 0 warnings
EXIT_CODE=0

bun run test:

 Test Files  81 passed (81)
      Tests  589 passed (589)
   Start at  22:26:42
   Duration  152.15s (transform 66%, environment 13%, import 12%, tests 7%, setup 2%)
EXIT_CODE=0

Adversarial round and gaps

The single time-boxed round ran against the existing local gates dev server binary, not a server built from this branch. It printed 133 findings, mostly SLOW responses during concurrent builds and other adversarial runs on the shared host. Non-SLOW timeouts and the skipped local sync subprobe are documented in #264. No crash or HTTP 5xx was observed. Hostile-byte checks reported zero findings; HEIF/AVIF upload passed; 64 SSE streams were accepted. The calendar Event tag probe passed separately with its required server-bin environment variable. The local calternald sync subprobe did not run in the round because target/debug/calternald was absent at that time.

The authz matrix got HTTP 422 rather than its expected 403 for a standard-user config request. The route’s JSON extractor rejected the incompatible body before the authorization handler; no write occurred. This is recorded in #264. The production web build passed earlier, with existing large-chunk and dropped "use client" bundler warnings.

After gates, cargo clean removed 20,278 files (18.1 GiB); apps/web/build, apps/web/.svelte-kit, and the temporary adversarial runner were deleted. The worktree is clean.

Decision not explicit in §40

An unrecognized source folder keeps its relative hierarchy under Imported/calternal.js/. This preserves unfamiliar vault content without creating unsupported roots in Home.

# Final report Finished `job/import-calternaljs` at `11816527dc0c1cd2eecaffe8f31c1a800665bad8`. `git push origin job/import-calternaljs` succeeded (`Everything up-to-date`; the branch had already been pushed by the worktree timer). The required one-time `git merge dev` is commit `dd4dbc0467ffcd8b2d7e307ec9729abaf5e02be6`, with `dev` at `060094c352206389038d00ebb10c1f9ca050191d`. ## Built The importer writes the §40 flat Home layout. Notes, Daily notes and task Markdown go directly in `Notes/`; photos and videos use dated `Photos/` folders; audio goes in `Documents/Voice memos/`; known documents go in `Documents/`; unknown attachments go in `Documents/Other/`. Same-date source Daily notes merge Log entries, moved-note links are rewritten, and legacy app folders are not created. The calendar merge maps changed user settings to HTTP 409. The reminder test fixture now uses a flat `Notes/` Daily note. Files changed across the branch: `Cargo.lock`; `crates/calternal-cli/Cargo.toml`, `src/import_calternaljs.rs`, `src/main.rs`, `src/remote_commands.rs`, `tests/import_help.rs`; `crates/calternal-notes-core/Cargo.toml`, `PARITY.md`, `src/dayfile.rs`, `src/lib.rs`, `src/links.rs`, `tests/vectors/dayfile.json`; `crates/calternal-search/src/indexer.rs`, `src/lib.rs`, `src/query.rs`, `tests/indexer.rs`; `crates/plugins/calendar/src/items.rs`; `crates/plugins/notifications/src/reminders.rs`; and `docs/import-calternaljs.md`. ## Real-server fixture import ```text IMPORT_E2E PASS: dry-run and one real API import covered 10 fixture files; same-day source Daily note entries merged; links rewrote; Notes, dated Photos, Documents, Voice memos, Other, and Imported targets verified; no legacy folders in Home. IMPORT_E2E NOTE_TARGET: Notes/Research notes-962364ddbf64e583.md IMPORT_E2E PHOTO_TARGET: Photos/2026/2026-09-24/field visit-38eb88d50498268f.jpg IMPORT_E2E SUMMARY: {"already_identical": 0, "conflicts": 0, "files_reported": 10, "imported": 9, "rewritten_links": 2} ``` ## Gates `cargo fmt --check` passed after a formatting-only correction committed as `11816527`: ```text EXIT_CODE=0 ``` `cargo clippy --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 8m 22s EXIT_CODE=0 ``` The one full `cargo test` command stopped at `calternal-auth`: ```text ---- store::tests::key_rotation_revokes_sessions_and_rejects_competing_recovery stdout ---- thread 'store::tests::key_rotation_revokes_sessions_and_rejects_competing_recovery' (1133462) panicked at crates/calternal-auth/src/store.rs:2676:9: assertion `left == right` failed left: 0 right: 1 test result: FAILED. 49 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 87.63s error: test failed, to rerun pass `-p calternal-auth --lib` EXIT_CODE=101 ``` That test expected one of two simultaneous recovery-key rotations to succeed; both returned errors. It is outside the importer changes and is filed as #266. Remaining workspace Rust tests did not run. The test was not rerun. `bun run check`: ```text $ svelte-kit sync && svelte-check --tsconfig ./tsconfig.json Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/import-calternaljs/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings EXIT_CODE=0 ``` `bun run test`: ```text Test Files 81 passed (81) Tests 589 passed (589) Start at 22:26:42 Duration 152.15s (transform 66%, environment 13%, import 12%, tests 7%, setup 2%) EXIT_CODE=0 ``` ## Adversarial round and gaps The single time-boxed round ran against the existing local `gates` dev server binary, not a server built from this branch. It printed 133 findings, mostly SLOW responses during concurrent builds and other adversarial runs on the shared host. Non-SLOW timeouts and the skipped local sync subprobe are documented in #264. No crash or HTTP 5xx was observed. Hostile-byte checks reported zero findings; HEIF/AVIF upload passed; 64 SSE streams were accepted. The calendar Event tag probe passed separately with its required server-bin environment variable. The local `calternald` sync subprobe did not run in the round because `target/debug/calternald` was absent at that time. The authz matrix got HTTP 422 rather than its expected 403 for a standard-user config request. The route’s JSON extractor rejected the incompatible body before the authorization handler; no write occurred. This is recorded in #264. The production web build passed earlier, with existing large-chunk and dropped `"use client"` bundler warnings. After gates, `cargo clean` removed 20,278 files (18.1 GiB); `apps/web/build`, `apps/web/.svelte-kit`, and the temporary adversarial runner were deleted. The worktree is clean. ## Decision not explicit in §40 An unrecognized source folder keeps its relative hierarchy under `Imported/calternal.js/`. This preserves unfamiliar vault content without creating unsupported roots in Home.

kayg referenced this issue

2026-10-01 05:26:06 +00:00

HYGIENE: delete merged branches/worktrees, close done issues, gc (no history rewrite) #548

Rows
Columns

Import: calternal.js vault → calternal (CLI 'calternal import calternal-js <vault>') #152

Final report

Built

Real-server fixture import

Gates

Adversarial round and gaps

Decision not explicit in §40