kayg commented

Owner

Owner question (2026-10-01): "How do we, again, import data from calternal.js?"

State: the importer exists on branch job/import-calternaljs (head 11816527d, 2026-09-27), not merged, and 4 days behind dev. Usage (from its docs/import-calternaljs.md): run calternal sync in calternal.js, stop calternald, then calternal --server "$SERVER_URL" import calternal-js "$SOURCE_DIR" --dry-run, then the real run. It follows the §40 Home layout: notes, daily notes and tasks go to Notes/; dated photos/videos to Photos/; audio/documents to Documents/; unknown attachments to Documents/Other/. Same-date daily notes merge Log entries, and links are rewritten.
Job:

Merge origin/dev into the branch and resolve conflicts (it touches calternal-cli, notes-core dayfile/links, search, calendar, notifications).
Rerun its tests and a full dry run + real run against a copy of the owner's calternal.js vault on a local server (aggregates only; never print note contents). Verify: counts per kind, every link resolves, Log entries merged, idempotent re-run (a second run changes nothing), attachments deduplicated through the blob store.
Add a web entry point in Settings → Data → Import ("Import from calternal.js") that uses the same API (parity, #484), with preview → confirm → progress → report.
Update the doc for the current CLI.
Per-crate gates + web gates. Report the dry-run summary table on this issue.

## Owner question (2026-10-01): "How do we, again, import data from calternal.js?" **State:** the importer exists on branch `job/import-calternaljs` (head 11816527d, 2026-09-27), **not merged**, and 4 days behind dev. Usage (from its `docs/import-calternaljs.md`): run `calternal sync` in calternal.js, stop `calternald`, then `calternal --server "$SERVER_URL" import calternal-js "$SOURCE_DIR" --dry-run`, then the real run. It follows the §40 Home layout: notes, daily notes and tasks go to `Notes/`; dated photos/videos to `Photos/`; audio/documents to `Documents/`; unknown attachments to `Documents/Other/`. Same-date daily notes merge Log entries, and links are rewritten. **Job:** - Merge origin/dev into the branch and resolve conflicts (it touches calternal-cli, notes-core dayfile/links, search, calendar, notifications). - Rerun its tests and a full dry run + real run against a **copy** of the owner's calternal.js vault on a local server (aggregates only; never print note contents). Verify: counts per kind, every link resolves, Log entries merged, idempotent re-run (a second run changes nothing), attachments deduplicated through the blob store. - Add a **web entry point** in Settings → Data → Import ("Import from calternal.js") that uses the same API (parity, #484), with preview → confirm → progress → report. - Update the doc for the current CLI. Per-crate gates + web gates. Report the dry-run summary table on this issue.

preview-phone-390-paper-light.png

61 KiB

preview-desktop-1440-paper-light.png

174 KiB

preview-desktop-1440-tokyo-night-dark.png

183 KiB

preview-phone-390-tokyo-night-dark.png

71 KiB

preview-tablet-820-paper-light.png

100 KiB

preview-tablet-820-tokyo-night-dark.png

109 KiB

report-desktop-1440-paper-light.png

169 KiB

report-desktop-1440-tokyo-night-dark.png

178 KiB

report-phone-390-paper-light.png

56 KiB

report-phone-390-tokyo-night-dark.png

63 KiB

report-tablet-820-paper-light.png

94 KiB

report-tablet-820-tokyo-night-dark.png

103 KiB

kayg commented

2026-10-02 00:11:40 +00:00

Author

Owner

Starting #610 on branch job/importjs-610, base SHA 1af8ead262. Worktree is clean. I am locating the unmerged importer changes and the current Settings → Data and API surfaces before integrating.

Starting #610 on branch job/importjs-610, base SHA 1af8ead262daf0a4ad22a60564dc75bab0005cf4. Worktree is clean. I am locating the unmerged importer changes and the current Settings → Data and API surfaces before integrating.

kayg commented

2026-10-02 01:01:51 +00:00

Author

Owner

Integrated origin/dev into the importer branch. Resolved the CLI conflicts by keeping the current conditional tus upload/reporting behavior and adding the importer calls on that retry-aware RemoteClient; retained the current Notes exports and Calendar date/time, snapping, quota and share behavior. The importer crate gates pass: calternal-cli (36 unit + 1 help + 15 output-contract tests), calternal-notes-core (528 + 19 + 5 + 7 + 12 tests), and calternal-plugin-calendar (83 + 1 + 3 tests); all three clippy gates pass.

kayg commented

2026-10-02 02:01:44 +00:00

Author

Owner

The web importer is in progress. I searched /home/kayg and /mnt for a mounted owner vault copy and checked for a running calternald; I found neither. I am waiting for the local vault-copy and server/data paths needed for the requested full dry run and real run. No note contents were read or printed.

The web importer is in progress. I searched `/home/kayg` and `/mnt` for a mounted owner vault copy and checked for a running `calternald`; I found neither. I am waiting for the local vault-copy and server/data paths needed for the requested full dry run and real run. No note contents were read or printed.

kayg commented

2026-10-02 02:25:37 +00:00

Author

Owner

The first full web test run found one importer-section style violation: ImportSection.svelte used numeric font-weight: 550, and font-weight-token.test.ts reported line 375. I replaced it with var(--weight-semibold). The focused guard then passed: 1 file, 3 tests. I will rerun the full web suite after the remaining changes.

The first full web test run found one importer-section style violation: `ImportSection.svelte` used numeric `font-weight: 550`, and `font-weight-token.test.ts` reported line 375. I replaced it with `var(--weight-semibold)`. The focused guard then passed: 1 file, 3 tests. I will rerun the full web suite after the remaining changes.

kayg commented

2026-10-02 02:35:13 +00:00

Author

Owner

The first production-browser scenario stopped during its dark preview capture. The shared E2E helper stored dark mode in local User settings and the server Appearance API, but calling it after rendering the import route left the route rendered light (storedMode=dark, serverMode=dark, root.dataset.mode=light). The existing theme capture suite calls the helper with navigateTo so preferences load before the route. I changed this scenario to follow that documented order and give each report screenshot a distinct test vault.

The first production-browser scenario stopped during its dark preview capture. The shared E2E helper stored dark mode in local User settings and the server Appearance API, but calling it after rendering the import route left the route rendered light (`storedMode=dark`, `serverMode=dark`, `root.dataset.mode=light`). The existing theme capture suite calls the helper with `navigateTo` so preferences load before the route. I changed this scenario to follow that documented order and give each report screenshot a distinct test vault.

kayg commented

2026-10-02 02:36:59 +00:00

Author

Owner

The second production-browser pass reached the first import report, then timed out because verifyImport still clicked the preview-only “Import data” button after the loop had already confirmed the import. I removed that duplicate action; verification now reads the report state produced by the preceding upload.

The second production-browser pass reached the first import report, then timed out because `verifyImport` still clicked the preview-only “Import data” button after the loop had already confirmed the import. I removed that duplicate action; verification now reads the report state produced by the preceding upload.

kayg commented

2026-10-02 02:38:19 +00:00

Author

Owner

The third production-browser pass verified that both linked photo paths return HTTP 200, then failed the dedup assertion because the E2E check read hash from the stat response. The Files API Entry contract names this field content_hash (as used by the importer and Files UI). I corrected the fixture assertion to compare that field.

The third production-browser pass verified that both linked photo paths return HTTP 200, then failed the dedup assertion because the E2E check read `hash` from the stat response. The Files API `Entry` contract names this field `content_hash` (as used by the importer and Files UI). I corrected the fixture assertion to compare that field.

kayg commented

2026-10-02 02:57:26 +00:00

Author

Owner

A full production-browser pass reached all 12 captures and the idempotent preview, then timed out locating “Import another vault” after the final screenshot loop. I moved the repeat-import check to immediately after the first report, where the report action is freshly rendered, and added an explicit visibility wait for that action before each report capture.

kayg commented

2026-10-02 03:51:41 +00:00

Author

Owner

The first perf profile exposed a fixture defect: its Note links targeted Tasks that were not in the generated corpus, so the 10,000-file run reported 0 rewritten links. I corrected the paths to target existing Tasks and reran once under /root/perf.lock (host load 0.00, 0.02, 0.00). The corrected run reported 2,500 rewrites in 10,000 files and 500 rewrites across four 500-file burst jobs.

kayg commented

2026-10-02 04:01:19 +00:00

Author

Owner

#610 implementation report

Branch: job/importjs-610
Base: merged origin/dev at 80bb909ab817c0058535d1bc4e12747c12705bd3 (merge parents 11816527dc0c1cd2eecaffe8f31c1a800665bad8 and 687ff703136e71e89f8dfba139e93cd0788b25c1).
Head: 8bd7dd2a6074b84c149b81b60abfe6e610b4a459.

Built

Revived the CLI importer and rebased its behavior through the origin/dev merge. It supports calternal --server "$SERVER_URL" import calternal-js "$SOURCE_DIR" --dry-run, then the real import. It maps items to the §40 Home layout, merges same-date Daily note Log entries, rewrites local Markdown links, and reports import actions.
Added Settings → Data → Import → “Import from calternal.js” with local preview, confirmation, progress, and report. It uses the existing authenticated Files API.
Updated the CLI import guide and added a benchmark profile and production-build browser E2E.

Requested owner-vault run

I could not locate a supplied owner-vault copy or a local server data root in the job environment. I asked for those paths and have no reply at report time. The full CLI dry run and real import against the owner’s copy therefore remain outstanding; owner-vault counts are unavailable. No note contents were printed.

Kind	Owner-vault CLI dry run	Synthetic browser E2E fixture
Daily note	unavailable	2
Note	unavailable	1
Task	unavailable	0
Photo	unavailable	2
Video	unavailable	0
Audio	unavailable	0
Document	unavailable	0
Other	unavailable	0
Total source files	unavailable	5

The production-build browser E2E used a test-only vault on a real local server. It verified the merged same-day Log entries, two rewritten links resolving to uploaded Photos, equal attachment content hashes, and an idempotent second preview with no writes. It passed with no unexpected browser errors. This is synthetic fixture evidence, not a substitute for the requested owner-vault run.

Gates

cargo fmt --check: no output; exit 0.
Clippy passed with no warnings for calternal-cli, calternal-notes-core, calternal-search, and calternal-plugin-notifications using --all-targets -- -D warnings. Search output: Finished dev profile [unoptimized + debuginfo] target(s) in 1m 39s. Notifications output: Finished dev profile [unoptimized + debuginfo] target(s) in 1m 03s.
Rust tests: CLI passed (36 unit, 1 help, 15 output-contract); Notes core passed (528 unit, 19 log_rewrite, 5 parser_properties, 7 unicode_titles, 12 vector_parity); Search passed (71 passed, 3 ignored); Notifications passed (24, 0 failures, doc tests 0).
bun run check: svelte-check found 0 errors and 0 warnings.
bun run test: 149 files and 1,019 tests passed. Existing jsdom scrollTo and CSS parser warnings were printed; exit status was 0.
Production browser E2E: calternal.js import e2e passed; production screenshots are in /home/kayg/Developer/calternal-wt/importjs-610/artifacts/importjs-610.
No calternal-server gate was needed because this change adds no server route or contract.

Performance

Measured once on perf-test while holding /root/perf.lock; load average inside the lock was 0.00, 0.02, 0.00.

Profile	Elapsed	CPU	Peak RSS	Writes	Rewritten links
Average, 250 files, 5 runs	p50 20.431 ms; p95 50.751 ms	43.725 ms average	62,476,288 B	—	63 per run
Worst, 10,000 files	670.036 ms	1,060.481 ms	158,957,568 B	10,000	2,500
Burst, 4 × 500 files	121.392 ms	158.018 ms	167,428,096 B	2,000 total	500 total

The closest existing baseline is notes_10k_import at p50 104.31 ms, p95 187.04 ms, and 311,087,104 B RSS. Its notes-only workload differs from this mixed-file browser preview, so the values are not a direct comparison.

The initial benchmark fixture reported zero rewritten links because its generated Notes pointed to missing Tasks. I corrected the source-relative links and reran the profile; the corrected counts above exercise the link rewrite path.

Files

CLI: crates/calternal-cli/Cargo.toml, src/import_calternaljs.rs, src/main.rs, src/remote_commands.rs, tests/import_help.rs.
Notes core: crates/calternal-notes-core/Cargo.toml, PARITY.md, src/dayfile.rs, src/lib.rs, src/links.rs, tests/vectors/dayfile.json.
Search and notifications: crates/calternal-search/src/{indexer.rs,lib.rs,query.rs}, crates/calternal-search/tests/indexer.rs, crates/plugins/notifications/src/reminders.rs.
Web: apps/web/e2e/importjs-610.mjs, apps/web/package.json, apps/web/src/lib/files/uploads.svelte{,.test}.ts, apps/web/src/lib/imports/calternalJs{,.test}.ts, apps/web/src/routes/settings/[...path]/+page.svelte, apps/web/src/routes/settings/data/ImportSection.svelte, apps/web/src/routes/settings/sections{,.test}.ts.
Docs and locks: docs/import-calternaljs.md, bench/calternaljs-import-610.mjs, Cargo.lock, bun.lock.

Decisions where the design is silent

The web preview scans the selected folder in the browser, then sends planned writes one at a time through the existing Files API and reports per-file progress. This keeps the server as the single writer without adding an import route.
Flattened-name collisions use the CLI’s stable source-path-derived conflict-copy suffix so the web flow stays in parity.

Screenshots

Production screenshots cover preview and report at phone (390 px), tablet (820 px), and desktop (1440 px), in light and dark themes.

# #610 implementation report Branch: `job/importjs-610` Base: merged `origin/dev` at `80bb909ab817c0058535d1bc4e12747c12705bd3` (merge parents `11816527dc0c1cd2eecaffe8f31c1a800665bad8` and `687ff703136e71e89f8dfba139e93cd0788b25c1`). Head: `8bd7dd2a6074b84c149b81b60abfe6e610b4a459`. ## Built - Revived the CLI importer and rebased its behavior through the `origin/dev` merge. It supports `calternal --server "$SERVER_URL" import calternal-js "$SOURCE_DIR" --dry-run`, then the real import. It maps items to the §40 Home layout, merges same-date Daily note Log entries, rewrites local Markdown links, and reports import actions. - Added Settings → Data → Import → “Import from calternal.js” with local preview, confirmation, progress, and report. It uses the existing authenticated Files API. - Updated the CLI import guide and added a benchmark profile and production-build browser E2E. ## Requested owner-vault run I could not locate a supplied owner-vault copy or a local server data root in the job environment. I asked for those paths and have no reply at report time. The full CLI dry run and real import against the owner’s copy therefore remain outstanding; owner-vault counts are unavailable. No note contents were printed. | Kind | Owner-vault CLI dry run | Synthetic browser E2E fixture | | --- | ---: | ---: | | Daily note | unavailable | 2 | | Note | unavailable | 1 | | Task | unavailable | 0 | | Photo | unavailable | 2 | | Video | unavailable | 0 | | Audio | unavailable | 0 | | Document | unavailable | 0 | | Other | unavailable | 0 | | Total source files | unavailable | 5 | The production-build browser E2E used a test-only vault on a real local server. It verified the merged same-day Log entries, two rewritten links resolving to uploaded Photos, equal attachment content hashes, and an idempotent second preview with no writes. It passed with no unexpected browser errors. This is synthetic fixture evidence, not a substitute for the requested owner-vault run. ## Gates - `cargo fmt --check`: no output; exit 0. - Clippy passed with no warnings for `calternal-cli`, `calternal-notes-core`, `calternal-search`, and `calternal-plugin-notifications` using `--all-targets -- -D warnings`. Search output: Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 39s. Notifications output: Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 03s. - Rust tests: CLI passed (36 unit, 1 help, 15 output-contract); Notes core passed (528 unit, 19 log_rewrite, 5 parser_properties, 7 unicode_titles, 12 vector_parity); Search passed (71 passed, 3 ignored); Notifications passed (24, 0 failures, doc tests 0). - `bun run check`: `svelte-check found 0 errors and 0 warnings`. - `bun run test`: 149 files and 1,019 tests passed. Existing jsdom `scrollTo` and CSS parser warnings were printed; exit status was 0. - Production browser E2E: `calternal.js import e2e passed; production screenshots are in /home/kayg/Developer/calternal-wt/importjs-610/artifacts/importjs-610`. - No `calternal-server` gate was needed because this change adds no server route or contract. ## Performance Measured once on `perf-test` while holding `/root/perf.lock`; load average inside the lock was `0.00, 0.02, 0.00`. | Profile | Elapsed | CPU | Peak RSS | Writes | Rewritten links | | --- | ---: | ---: | ---: | ---: | ---: | | Average, 250 files, 5 runs | p50 20.431 ms; p95 50.751 ms | 43.725 ms average | 62,476,288 B | — | 63 per run | | Worst, 10,000 files | 670.036 ms | 1,060.481 ms | 158,957,568 B | 10,000 | 2,500 | | Burst, 4 × 500 files | 121.392 ms | 158.018 ms | 167,428,096 B | 2,000 total | 500 total | The closest existing baseline is `notes_10k_import` at p50 104.31 ms, p95 187.04 ms, and 311,087,104 B RSS. Its notes-only workload differs from this mixed-file browser preview, so the values are not a direct comparison. The initial benchmark fixture reported zero rewritten links because its generated Notes pointed to missing Tasks. I corrected the source-relative links and reran the profile; the corrected counts above exercise the link rewrite path. ## Files - CLI: `crates/calternal-cli/Cargo.toml`, `src/import_calternaljs.rs`, `src/main.rs`, `src/remote_commands.rs`, `tests/import_help.rs`. - Notes core: `crates/calternal-notes-core/Cargo.toml`, `PARITY.md`, `src/dayfile.rs`, `src/lib.rs`, `src/links.rs`, `tests/vectors/dayfile.json`. - Search and notifications: `crates/calternal-search/src/{indexer.rs,lib.rs,query.rs}`, `crates/calternal-search/tests/indexer.rs`, `crates/plugins/notifications/src/reminders.rs`. - Web: `apps/web/e2e/importjs-610.mjs`, `apps/web/package.json`, `apps/web/src/lib/files/uploads.svelte{,.test}.ts`, `apps/web/src/lib/imports/calternalJs{,.test}.ts`, `apps/web/src/routes/settings/[...path]/+page.svelte`, `apps/web/src/routes/settings/data/ImportSection.svelte`, `apps/web/src/routes/settings/sections{,.test}.ts`. - Docs and locks: `docs/import-calternaljs.md`, `bench/calternaljs-import-610.mjs`, `Cargo.lock`, `bun.lock`. ## Decisions where the design is silent - The web preview scans the selected folder in the browser, then sends planned writes one at a time through the existing Files API and reports per-file progress. This keeps the server as the single writer without adding an import route. - Flattened-name collisions use the CLI’s stable source-path-derived conflict-copy suffix so the web flow stays in parity. ## Screenshots Production screenshots cover preview and report at phone (390 px), tablet (820 px), and desktop (1440 px), in light and dark themes. - [preview-desktop-1440-paper-light.png](https://git.kayg.org/attachments/3dee6f91-9267-471b-a09b-f379803ae439) - [preview-desktop-1440-tokyo-night-dark.png](https://git.kayg.org/attachments/aeb30238-7b2a-437e-9451-5093153100bf) - [preview-phone-390-paper-light.png](https://git.kayg.org/attachments/84a621bc-a87d-407d-a577-c0c3f6001df2) - [preview-phone-390-tokyo-night-dark.png](https://git.kayg.org/attachments/50adcaf1-0e57-42b1-a75d-5b1835aae57b) - [preview-tablet-820-paper-light.png](https://git.kayg.org/attachments/9c8b0fda-cc4b-42c0-a969-01d34140c710) - [preview-tablet-820-tokyo-night-dark.png](https://git.kayg.org/attachments/0be0f79f-1a19-4434-ba45-b14491cdc9c9) - [report-desktop-1440-paper-light.png](https://git.kayg.org/attachments/fbe18fb8-fdf4-4fb3-a844-e55b090f7bbc) - [report-desktop-1440-tokyo-night-dark.png](https://git.kayg.org/attachments/de8d387b-7613-4546-9466-76ad7efbfdb3) - [report-phone-390-paper-light.png](https://git.kayg.org/attachments/f0a68992-68fb-4efc-8149-383839c9a014) - [report-phone-390-tokyo-night-dark.png](https://git.kayg.org/attachments/d8cdce19-ac85-4923-a8e7-e210d2dc119a) - [report-tablet-820-paper-light.png](https://git.kayg.org/attachments/d28dbc4c-6aac-4d96-af44-5d33e49219da) - [report-tablet-820-tokyo-night-dark.png](https://git.kayg.org/attachments/b86f57c7-3040-4217-a18b-a9f73b084e92)

kayg commented

2026-10-02 11:21:26 +00:00

Author

Owner

Started round 2 on branch job/importjs-610. Base SHA: 687ff70313 (merge-base with origin/dev). Initial head: 8bd7dd2a60.

Started round 2 on branch job/importjs-610. Base SHA: 687ff703136e71e89f8dfba139e93cd0788b25c1 (merge-base with origin/dev). Initial head: 8bd7dd2a6074b84c149b81b60abfe6e610b4a459.

kayg commented

2026-10-02 11:23:20 +00:00

Author

Owner

Cancelled by the owner (2026-10-02): no dedicated calternal.js importer. Moving in is done by dropping Markdown folders into calternal; tracked in the new drop-in issue. Branch job/importjs-610 is removed unmerged.

kayg closed this issue

2026-10-02 11:23:21 +00:00

kayg referenced this issue

2026-10-02 11:23:54 +00:00

Moving in by dropping Markdown folders: find and fix what breaks (replaces the dedicated importer) #719

kayg commented

2026-10-02 11:23:55 +00:00

Author

Owner

Cancelled by the owner (2026-10-02): no dedicated calternal.js importer. Moving in = dropping Markdown folders into calternal, tracked in #719. Branch job/importjs-610 removed unmerged.

kayg referenced this issue

2026-10-02 17:37:19 +00:00

Moving in by dropping Markdown folders: find and fix what breaks (replaces the dedicated importer) #719

kayg referenced this issue

2026-10-02 17:42:29 +00:00

Audit: open issue hygiene (duplicates, already fixed, stale) #866

kayg referenced this issue

2026-10-03 08:12:08 +00:00

Moving in by dropping Markdown folders: find and fix what breaks (replaces the dedicated importer) #719