Moving in by dropping Markdown folders: find and fix what breaks (replaces the dedicated importer)

kayg commented

2026-10-02 11:23:54 +00:00

Owner

Owner decision (2026-10-02)

"Most people will be bringing their Markdown files from other apps, so I would love to see what breaks when I just drop a bunch of files." The dedicated calternal.js importer (#610) is cancelled and removed. Moving in = dropping a folder of Markdown and attachments into calternal (browser Files upload with a folder drag, WebDAV/Finder, or CLI put). calternal must make sense of it without a special importer.

Goal

Drop real-world-shaped Markdown collections into a fresh Home and find everything that breaks. Fix what calternal should handle; document what it deliberately leaves alone.

Corpora (tests only: openly licensed or generated, never the owner's data)

Describe each by its conventions; do not name products in code or issues.

Wiki-link vaults: [[Note]], [[Note|alias]], [[Note#Heading]], [[Note#^block]], ![[image.png]] embeds, nested folders, an attachments folder, YAML frontmatter with tags: / aliases:, inline #tags and #nested/tags, callouts (> [!note]), tasks - [ ], inline key:: value properties.
Outliner exports: every line a bullet, block references ((id)), id:: properties, journals named YYYY_MM_DD.md.
Inline-tag apps: #multi word tag#, tags at the end, .textbundle folders.
Database exports: long hash suffixes in names (Title 1a2b3c….md), URL-encoded relative links (My%20Page%20abc.md), CSV next to Markdown.
Daily-journal exports: one big file or one file per day, with different date formats in names and frontmatter (2026-10-02.md, 20261002.md, Oct 2, 2026.md, date:).
Edge cases: BOM, CRLF, non-UTF-8 (#631), NFC/NFD twins, emoji and RTL names, 10 MB notes, 20k files, symlinks, .DS_Store/._ files (#648), hidden folders, the same title in different folders.

For each corpus, report what the User sees

Files appear in Files and in Notes; plugin-owned paths stay correct (#606).
Links: wiki links, Markdown links (relative, URL-encoded, with headings), embeds and block references resolve; backlinks and "Used in" work; what a broken link looks like.
Daily notes: journal files recognised as Daily notes (Calendar Journal, Log entries) or left as plain notes; a safe, explicit way to adopt them.
Tags, tasks, properties: indexed and shown in Calendar, Tags and the Inspector.
Attachments: images and PDFs show inline and in Photos/Documents; duplicates handled.
Speed and safety: time to index 20k files on the perf VM (HDD emulation), server RSS; nothing is rewritten without a User action (#661: opening never writes).
Identity: a stable calternal-id is assigned without changing visible content beyond the documented frontmatter key.

Output

A findings table on this issue (corpus × check → pass/fail/evidence). One sub-issue per real gap, grouped by shared fix (link resolver, Daily-note recognition, tag parser, attachment resolver). Fix the small, clear gaps in this job with regression tests that use the corpora as fixtures.

## Owner decision (2026-10-02) "Most people will be bringing their Markdown files from other apps, so I would love to see what breaks when I just drop a bunch of files." The dedicated calternal.js importer (#610) is cancelled and removed. **Moving in = dropping a folder of Markdown and attachments into calternal** (browser Files upload with a folder drag, WebDAV/Finder, or CLI `put`). calternal must make sense of it without a special importer. ## Goal Drop real-world-shaped Markdown collections into a fresh Home and find everything that breaks. Fix what calternal should handle; document what it deliberately leaves alone. ## Corpora (tests only: openly licensed or generated, never the owner's data) Describe each by its conventions; do not name products in code or issues. - **Wiki-link vaults:** `[[Note]]`, `[[Note|alias]]`, `[[Note#Heading]]`, `[[Note#^block]]`, `![[image.png]]` embeds, nested folders, an attachments folder, YAML frontmatter with `tags:` / `aliases:`, inline `#tags` and `#nested/tags`, callouts (`> [!note]`), tasks `- [ ]`, inline `key:: value` properties. - **Outliner exports:** every line a bullet, block references `((id))`, `id::` properties, journals named `YYYY_MM_DD.md`. - **Inline-tag apps:** `#multi word tag#`, tags at the end, `.textbundle` folders. - **Database exports:** long hash suffixes in names (`Title 1a2b3c….md`), URL-encoded relative links (`My%20Page%20abc.md`), CSV next to Markdown. - **Daily-journal exports:** one big file or one file per day, with different date formats in names and frontmatter (`2026-10-02.md`, `20261002.md`, `Oct 2, 2026.md`, `date:`). - **Edge cases:** BOM, CRLF, non-UTF-8 (#631), NFC/NFD twins, emoji and RTL names, 10 MB notes, 20k files, symlinks, `.DS_Store`/`._` files (#648), hidden folders, the same title in different folders. ## For each corpus, report what the User sees 1. Files appear in Files and in Notes; plugin-owned paths stay correct (#606). 2. **Links:** wiki links, Markdown links (relative, URL-encoded, with headings), embeds and block references resolve; backlinks and "Used in" work; what a broken link looks like. 3. **Daily notes:** journal files recognised as Daily notes (Calendar Journal, Log entries) or left as plain notes; a safe, explicit way to adopt them. 4. **Tags, tasks, properties:** indexed and shown in Calendar, Tags and the Inspector. 5. **Attachments:** images and PDFs show inline and in Photos/Documents; duplicates handled. 6. **Speed and safety:** time to index 20k files on the perf VM (HDD emulation), server RSS; nothing is rewritten without a User action (#661: opening never writes). 7. **Identity:** a stable calternal-id is assigned without changing visible content beyond the documented frontmatter key. ## Output A findings table on this issue (corpus × check → pass/fail/evidence). One sub-issue per real gap, grouped by shared fix (link resolver, Daily-note recognition, tag parser, attachment resolver). Fix the small, clear gaps in this job with regression tests that use the corpora as fixtures.

wiki-links-browser-files.zip

5 MiB

wiki-links-browser-notes-photos-preview.zip

7 MiB

wiki-links-webdav-files.zip

5 MiB

wiki-links-webdav-notes-photos-preview.zip

4.9 MiB

kayg referenced this issue

2026-10-02 11:23:55 +00:00

Import from calternal.js: revive the unmerged importer branch, verify on a copy of the owner's vault, add Settings → Data → Import #610

kayg commented

2026-10-02 11:25:47 +00:00

Author

Owner

Starting #719 on branch job/dropmd-719 at c4a61e8cf090170f35b1bed3350d9de20c83ecd5. This is also the merge-base with origin/dev at start. I am reading the existing Files, Notes and upload paths, then I will build licensed/generated fixture corpora, exercise browser folder upload, WebDAV PUT and CLI put, and capture the requested measurements and evidence.

Starting #719 on branch `job/dropmd-719` at `c4a61e8cf090170f35b1bed3350d9de20c83ecd5`. This is also the merge-base with `origin/dev` at start. I am reading the existing Files, Notes and upload paths, then I will build licensed/generated fixture corpora, exercise browser folder upload, WebDAV PUT and CLI put, and capture the requested measurements and evidence.

kayg commented

2026-10-02 13:00:05 +00:00

Author

Owner

Browser folder upload finding (synthetic Wiki-link corpus, fresh Home): the 6-file upload repeatedly left files missing. Across two fresh browser Homes, Guide.md and/or Notebook.md failed during tus PATCH with HTTP 412. The Files toast said “The destination changed during the upload. Retry to upload again.” One run completed 4/6 files; the instrumented run completed 4/6 and recorded 412 for both failed files. Files showed only 4 corpus items, with 1 Note and no Tasks. The failed requests had no Retry-After header.

I am comparing this with serial WebDAV and CLI intake and will test the visible Retry action. The current evidence points to a browser folder upload precondition race; I am tracing the parent identity and finalization checks before changing app behavior.

Browser folder upload finding (synthetic Wiki-link corpus, fresh Home): the 6-file upload repeatedly left files missing. Across two fresh browser Homes, `Guide.md` and/or `Notebook.md` failed during tus PATCH with HTTP 412. The Files toast said “The destination changed during the upload. Retry to upload again.” One run completed 4/6 files; the instrumented run completed 4/6 and recorded 412 for both failed files. Files showed only 4 corpus items, with 1 Note and no Tasks. The failed requests had no `Retry-After` header. I am comparing this with serial WebDAV and CLI intake and will test the visible Retry action. The current evidence points to a browser folder upload precondition race; I am tracing the parent identity and finalization checks before changing app behavior.

kayg commented

2026-10-02 13:06:23 +00:00

Author

Owner

Channel control: the serial WebDAV pilot imported all 6 Wiki-link files with no failed MKCOL/PUT requests; the fresh Home showed 6 Files, 3 Notes and 1 Task.

CLI pilot is currently blocked before server start by the local media-sandbox guard: prepare-media-runtime.sh measured 4,790 local threads and refused because its bound is 4,090. The CLI path remains unmeasured. I will continue with the locked perf-VM run and non-server work; I will not raise or bypass this guard.

Channel control: the serial WebDAV pilot imported all 6 Wiki-link files with no failed MKCOL/PUT requests; the fresh Home showed 6 Files, 3 Notes and 1 Task. CLI pilot is currently blocked before server start by the local media-sandbox guard: `prepare-media-runtime.sh` measured 4,790 local threads and refused because its bound is 4,090. The CLI path remains unmeasured. I will continue with the locked perf-VM run and non-server work; I will not raise or bypass this guard.

kayg commented

2026-10-02 15:08:57 +00:00

Author

Owner

Additional intake constraint: the latest pilot could not start a fresh local production server. The shared media sandbox refused startup at 6,913 local threads against its 4,090 limit. I did not change or bypass the guard. This currently blocks more local corpus cases and a rerun of the browser screenshot matrix; the previously captured Wiki-link screenshots remain in the ignored artifacts directory.

kayg commented

2026-10-02 15:32:52 +00:00

Author

Owner

Full Files crate test result: 146 passed, 1 failed, 1 ignored (finished in 941.29s). The existing internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm exceeded its five-minute completion deadline; its concurrent writer also reported atomic write 752 failed: entry not found. The same host refused a fresh server at 6,913 threads against the 4,090 media-sandbox limit. I left the test expectation unchanged and am treating this as host-load-sensitive pending the exact failure review; no API response or data-loss evidence was produced by this test.

Full Files crate test result: 146 passed, 1 failed, 1 ignored (finished in 941.29s). The existing `internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm` exceeded its five-minute completion deadline; its concurrent writer also reported `atomic write 752 failed: entry not found`. The same host refused a fresh server at 6,913 threads against the 4,090 media-sandbox limit. I left the test expectation unchanged and am treating this as host-load-sensitive pending the exact failure review; no API response or data-loss evidence was produced by this test.

kayg referenced this issue

2026-10-02 16:08:14 +00:00

Resolve wiki-style links and embeds in imported Notes #856

kayg referenced this issue

2026-10-02 16:08:34 +00:00

Assign stable identity to imported Markdown Notes #857

kayg referenced this issue

2026-10-02 17:20:27 +00:00

Investigate atomic write failures during Files reconcile storm #870

kayg commented

2026-10-02 17:22:18 +00:00

Author

Owner

Findings

Synthetic fixtures: wiki-links, outliner, inline-tags, database-export, daily-journal, and edge-cases. The complete three-channel intake matrix could not be run because the local media sandbox refused server starts at 6,913 threads against its 4,090 limit.

Corpus	Browser folder upload	WebDAV PUT	CLI `put`
`wiki-links`	Pilot was partial: 4/6 files; two upload PATCH requests returned HTTP 412 while concurrent uploads changed the parent Folder fingerprint. This was before the Files fix; no post-fix replay was possible.	6/6 files imported.	Not run; local server start was blocked.
`outliner`	Not measured	Not measured	Not measured
`inline-tags`	Not measured	Not measured	Not measured
`database-export`	Not measured	Not measured	Not measured
`daily-journal`	Not measured	Not measured	Not measured
`edge-cases`	Not measured	Not measured	Not measured

WebDAV observations for wiki-links: Files showed all 6 files, Notes showed 3 Notes, Tasks showed 1 task, and Photos showed 2 items. Frontmatter and inline tags appeared. All 3 imported Notes had path identities and the tested backlink queries returned 0. The embedded image did not load; the PDF link appeared. Opening a Note left its body and ETag unchanged.

The browser pilot exposed a false HTTP 412 when another upload changed a parent directory's size and mtime. The Files Index now preserves Folder identity when device and inode still match, while file identity continues to use its full fingerprint. A regression test covers child upload and reindex. This fix is committed, but the browser run could not be repeated after the local server guard blocked further starts.

20k-file performance

The requested run used bench/hdd-emu.sh on the perf VM under /root/perf.lock, with a pre-fix release binary. It emitted no completed checkpoint before the six-hour job cutoff, so there is no 20k result and the fix has not been performance-measured. The existing docs/perf/baseline.json files_50k baseline is p50 111.05 ms, p95 195.09 ms, rate 8.26 uploads/s, mean RSS 295,948,947 bytes, peak RSS 414,351,360 bytes, and server CPU 1,883.9 s. The incomplete run was interrupted at the job limit and released the VM lock.

UX gaps closed

Child uploads no longer invalidate the parent Folder's identity when its inode remains the same; the regression test passes.
Read-only Note open was checked for body and ETag stability.

UX gaps left

Wiki links and backlinks did not resolve in the pilot; the embedded image did not load. Tracked in #856.
Imported Notes without calternal-id used path identities. Tracked in #857.
Five corpora and all CLI runs remain unmeasured; the post-fix browser pilot remains unverified.
The WebDAV Note-preview screenshot set has only the 390 px light capture. Browser captures cover 390/820/1440 px in light/dark. All screenshots use Playwright macOS emulation; the Mac VM is offline.
The Files reconcile-storm test failed twice with a timeout and concurrent entry not found writer errors. I filed #870; no API response or data-loss evidence was captured.

Screenshot archives attached: browser Files, browser Notes/Photos/Note preview, WebDAV Files, WebDAV Notes/Photos/Note preview. There are 55 screenshots total.

Gates (verbatim output)

cargo fmt --check — exit 0, no output.

cargo clippy -p calternal-plugin-files --all-targets -- -D warnings:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 20s

cargo test -p calternal-plugin-files -- --test-threads=2:

thread 'tests::internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm' panicked at crates/plugins/files/src/lib.rs:5148:14:
writes, reconcile scans, and watcher adoption complete within five minutes: Elapsed(())

thread 'tokio-rt-worker' (2256759) panicked at crates/plugins/files/src/lib.rs:5122:45:
atomic write 176 failed: entry not found

test result: FAILED. 146 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 1527.45s
error: test failed, to rerun pass `-p calternal-plugin-files --lib`

bun run check:

User browser caches use userStorage; only documented device/public-link exceptions remain.
Text sizes and UI shape values use shared role tokens.
UI transitions and animation options use shared motion tokens or documented exceptions.
Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/dropmd-719/apps/web
Getting Svelte diagnostics...
svelte-check found 0 errors and 0 warnings

bun run test:

Test Files  7 failed | 146 passed (153)
      Tests  10 failed | 1043 passed (1053)
   Duration  550.02s (transform 61%, import 17%, environment 12%, tests 6%, setup 4%)
error: script "test" exited with code 1

All ten reported failures were 5-second test timeouts. I left existing expectations unchanged.

Decisions

The Files Index treats a directory's device+inode as its stable identity across child additions because directory size and mtime change during uploads. Regular-file fingerprints remain strict.
Corpus content is generated and synthetic; it contains no copied User data or external product text.
A real-Mac check remains pending because the Mac VM is offline. Exact follow-up: open the attached screenshot archives and review all covered views at 390/820/1440 px in light/dark; repeat the full intake matrix, including CLI, once a fresh local server can start.

Revision

Head: ee959b533f84c2c38c3c46b74856dc0ca105b0d4 (includes the one required merge from origin/dev). No push, deploy, or merge to dev was done. Other gaps are tracked in #856, #857, and #870.

## Findings Synthetic fixtures: `wiki-links`, `outliner`, `inline-tags`, `database-export`, `daily-journal`, and `edge-cases`. The complete three-channel intake matrix could not be run because the local media sandbox refused server starts at 6,913 threads against its 4,090 limit. | Corpus | Browser folder upload | WebDAV PUT | CLI `put` | |---|---|---|---| | `wiki-links` | Pilot was partial: 4/6 files; two upload PATCH requests returned HTTP 412 while concurrent uploads changed the parent Folder fingerprint. This was before the Files fix; no post-fix replay was possible. | 6/6 files imported. | Not run; local server start was blocked. | | `outliner` | Not measured | Not measured | Not measured | | `inline-tags` | Not measured | Not measured | Not measured | | `database-export` | Not measured | Not measured | Not measured | | `daily-journal` | Not measured | Not measured | Not measured | | `edge-cases` | Not measured | Not measured | Not measured | WebDAV observations for `wiki-links`: Files showed all 6 files, Notes showed 3 Notes, Tasks showed 1 task, and Photos showed 2 items. Frontmatter and inline tags appeared. All 3 imported Notes had path identities and the tested backlink queries returned 0. The embedded image did not load; the PDF link appeared. Opening a Note left its body and ETag unchanged. The browser pilot exposed a false HTTP 412 when another upload changed a parent directory's size and mtime. The Files Index now preserves Folder identity when device and inode still match, while file identity continues to use its full fingerprint. A regression test covers child upload and reindex. This fix is committed, but the browser run could not be repeated after the local server guard blocked further starts. ### 20k-file performance The requested run used `bench/hdd-emu.sh` on the perf VM under `/root/perf.lock`, with a pre-fix release binary. It emitted no completed checkpoint before the six-hour job cutoff, so there is no 20k result and the fix has not been performance-measured. The existing `docs/perf/baseline.json` `files_50k` baseline is p50 111.05 ms, p95 195.09 ms, rate 8.26 uploads/s, mean RSS 295,948,947 bytes, peak RSS 414,351,360 bytes, and server CPU 1,883.9 s. The incomplete run was interrupted at the job limit and released the VM lock. ### UX gaps closed - Child uploads no longer invalidate the parent Folder's identity when its inode remains the same; the regression test passes. - Read-only Note open was checked for body and ETag stability. ### UX gaps left - Wiki links and backlinks did not resolve in the pilot; the embedded image did not load. Tracked in #856. - Imported Notes without `calternal-id` used path identities. Tracked in #857. - Five corpora and all CLI runs remain unmeasured; the post-fix browser pilot remains unverified. - The WebDAV Note-preview screenshot set has only the 390 px light capture. Browser captures cover 390/820/1440 px in light/dark. All screenshots use Playwright macOS emulation; the Mac VM is offline. - The Files reconcile-storm test failed twice with a timeout and concurrent `entry not found` writer errors. I filed #870; no API response or data-loss evidence was captured. Screenshot archives attached: [browser Files](https://git.kayg.org/attachments/8a815118-306f-4bd3-880e-ef8865506aee), [browser Notes/Photos/Note preview](https://git.kayg.org/attachments/3d5ddd9c-a2d6-4f25-b30f-f32275ff02f9), [WebDAV Files](https://git.kayg.org/attachments/83eb2ce2-9c22-4f22-83da-fe175aab91ba), [WebDAV Notes/Photos/Note preview](https://git.kayg.org/attachments/ef1f4a0f-6003-4f2d-b6f6-a1eb5909378d). There are 55 screenshots total. ## Gates (verbatim output) `cargo fmt --check` — exit 0, no output. `cargo clippy -p calternal-plugin-files --all-targets -- -D warnings`: ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 6m 20s ``` `cargo test -p calternal-plugin-files -- --test-threads=2`: ```text thread 'tests::internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm' panicked at crates/plugins/files/src/lib.rs:5148:14: writes, reconcile scans, and watcher adoption complete within five minutes: Elapsed(()) thread 'tokio-rt-worker' (2256759) panicked at crates/plugins/files/src/lib.rs:5122:45: atomic write 176 failed: entry not found test result: FAILED. 146 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 1527.45s error: test failed, to rerun pass `-p calternal-plugin-files --lib` ``` `bun run check`: ```text User browser caches use userStorage; only documented device/public-link exceptions remain. Text sizes and UI shape values use shared role tokens. UI transitions and animation options use shared motion tokens or documented exceptions. Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/dropmd-719/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings ``` `bun run test`: ```text Test Files 7 failed | 146 passed (153) Tests 10 failed | 1043 passed (1053) Duration 550.02s (transform 61%, import 17%, environment 12%, tests 6%, setup 4%) error: script "test" exited with code 1 ``` All ten reported failures were 5-second test timeouts. I left existing expectations unchanged. ## Decisions - The Files Index treats a directory's device+inode as its stable identity across child additions because directory size and mtime change during uploads. Regular-file fingerprints remain strict. - Corpus content is generated and synthetic; it contains no copied User data or external product text. - A real-Mac check remains pending because the Mac VM is offline. Exact follow-up: open the attached screenshot archives and review all covered views at 390/820/1440 px in light/dark; repeat the full intake matrix, including CLI, once a fresh local server can start. ## Revision Head: `ee959b533f84c2c38c3c46b74856dc0ca105b0d4` (includes the one required merge from `origin/dev`). No push, deploy, or merge to `dev` was done. Other gaps are tracked in #856, #857, and #870.

kayg commented

2026-10-02 17:34:07 +00:00

Author

Owner

Independent read-only review started on job/rev2-dropmd-719, base 440e19dce2. Target: ee959b533. No builds, tests, servers or browsers will run. Findings will be committed in review-dropmd-719.md and audit-findings.md.

Independent read-only review started on job/rev2-dropmd-719, base 440e19dce23040ac8ebaae88f0469b6535b1afcb. Target: ee959b533. No builds, tests, servers or browsers will run. Findings will be committed in review-dropmd-719.md and audit-findings.md.

kayg referenced this issue

2026-10-02 17:36:03 +00:00

P2: Complete Markdown folder intake byte and checklist evidence #893

kayg referenced this issue

2026-10-02 17:37:14 +00:00

IMPORT: calternal.js vault → calternal Home (Settings → Import + CLI, dry-run first) #300

kayg commented

2026-10-02 17:37:19 +00:00

Author

Owner

This owner decision supersedes the dedicated importer proposal in #300 and the later dedicated importer #610. DESIGN §40 confirms the current flat Notes layout, and this issue defines folder drop as the general import path. Recommend keeping #719 as the active tracker and linking #300 as the retired importer proposal. Do not close either issue in this audit.

kayg commented

2026-10-02 17:37:37 +00:00

Author

Owner

Independent read-only review complete. Target: ee959b533. Review head: 11d44b734e. Two atomic documentation commits; no product changes, pushes, merges, builds, servers or browsers.

No confirmed P1 production defect found. Findings share one fix and are tracked in #893:

P2 — tests/dropmd_719/run_intake.mjs:383, :446, :524: no complete source/upload byte comparison; read-only opens cover only Guide. Same-size corruption or changes to BOM, CRLF and frontmatter in other files are not detected. Fix: compare complete bytes after intake and read-only opens; record permitted identity insertion separately; report content changes as failures.
P3 — tests/dropmd_719/run_intake.mjs:376, :404, :428: backlink counts and filename matching do not prove link navigation or Calendar Journal recognition. Fix: observe destination Notes, headings, blocks, Journal and Log entries; give every corpus/check an explicit pass, fail or not checked result.

Production inspection: folder checks retain device/inode identity; file checks retain size and modification-time comparisons. User-scoped SQL, pending namespace checks, expected grant IDs and calternal-fs access remain. No existing test expectation was weakened. The added test covers internal identity retention, not HTTP completion or cross-User grant behaviour.

Built/files: audit-findings.md and review-dropmd-719.md. The report includes evidence, fixes, test ideas and exact merge-round commands.

Gate output: none. LIGHT instructions forbid builds and tests. git diff --check returned exit 0 with no output.

Known gaps / For the merge round: run Files crate gates and focused sibling-upload/grant regressions; run the intake matrix with production binaries after #893 fixes its evidence. Run the combined authorization and robustness matrices. No latency/RSS or 20k-file result is claimed. DESIGN §58 is absent from both inspected revisions; §18 has the available performance budgets.

UX gaps closed: none (review only). UX gaps left: navigation and intake evidence in #893.

Decisions: group both evidence findings in #893 because the runner owns their fix. Do not claim untested behaviour as a product defect. No merge was needed before gates because this LIGHT review runs no gates. No build output was created.

Independent read-only review complete. Target: ee959b533. Review head: 11d44b734ea110479a383299962df22e8a835930. Two atomic documentation commits; no product changes, pushes, merges, builds, servers or browsers. No confirmed P1 production defect found. Findings share one fix and are tracked in #893: - **P2** — `tests/dropmd_719/run_intake.mjs:383`, `:446`, `:524`: no complete source/upload byte comparison; read-only opens cover only Guide. Same-size corruption or changes to BOM, CRLF and frontmatter in other files are not detected. Fix: compare complete bytes after intake and read-only opens; record permitted identity insertion separately; report content changes as failures. - **P3** — `tests/dropmd_719/run_intake.mjs:376`, `:404`, `:428`: backlink counts and filename matching do not prove link navigation or Calendar Journal recognition. Fix: observe destination Notes, headings, blocks, Journal and Log entries; give every corpus/check an explicit pass, fail or not checked result. Production inspection: folder checks retain device/inode identity; file checks retain size and modification-time comparisons. User-scoped SQL, pending namespace checks, expected grant IDs and calternal-fs access remain. No existing test expectation was weakened. The added test covers internal identity retention, not HTTP completion or cross-User grant behaviour. Built/files: `audit-findings.md` and `review-dropmd-719.md`. The report includes evidence, fixes, test ideas and exact merge-round commands. Gate output: none. LIGHT instructions forbid builds and tests. `git diff --check` returned exit 0 with no output. Known gaps / For the merge round: run Files crate gates and focused sibling-upload/grant regressions; run the intake matrix with production binaries after #893 fixes its evidence. Run the combined authorization and robustness matrices. No latency/RSS or 20k-file result is claimed. DESIGN §58 is absent from both inspected revisions; §18 has the available performance budgets. UX gaps closed: none (review only). UX gaps left: navigation and intake evidence in #893. Decisions: group both evidence findings in #893 because the runner owns their fix. Do not claim untested behaviour as a product defect. No merge was needed before gates because this LIGHT review runs no gates. No build output was created.

kayg referenced this issue

2026-10-02 17:42:29 +00:00

Audit: open issue hygiene (duplicates, already fixed, stale) #866

kayg commented

2026-10-03 07:34:15 +00:00

Author

Owner

Starting the read-only vault audit on branch dev at base SHA 4a871b3838. I will report aggregate counts only and will not read through or output note contents, titles, filenames, or personal text.

Starting the read-only vault audit on branch dev at base SHA 4a871b383864dad0d6c87bcd22b7f0d589e3506e. I will report aggregate counts only and will not read through or output note contents, titles, filenames, or personal text.

kayg commented

2026-10-03 08:12:08 +00:00

Author

Owner

Read-only production Home audit — 2026-10-03

Method and inventory

I queried the owner User's Index SQLite database with a read-only URI and query-only mode. The Instance has one owner User. I did not create a session, read credential material, select note titles or bodies, or output user-authored names or tag values. I compared the aggregates with the calternal.js conventions in docs/DESIGN.md, docs/contracts.md, packages/core/src/dayfile.rs, packages/core/src/tasks/extract.rs and packages/cli/src/core/notes-index.ts.

The supplied batch is 656 Daily notes, 15 Notes, 2 Task files and 2 photos: 675 files. The current Index has 684 Markdown files in Notes/ (662 Daily-note-pattern files and 22 other files, including 3 projected Task files) and 2 media items under Photos/2026/. This is 686 files in those buckets. The 656 Daily-note-pattern files in the supplied date range are all present; six more Daily-note-pattern files are dated 2026-09-28 through 2026-10-03. There are five more non-Daily Markdown files than the supplied 15 Notes + 2 Tasks. The Index has no import-batch provenance, so it cannot show which of those five were already in Home before the upload.

Findings

calternal.js convention	Supplied or current Home count	What the Index recognizes	Gap and tracking
Daily note path: events/YYYYMMDD-dailynote.md	656 in 2022-11-20 through 2026-09-27; current Notes/ has 662 matching names	All 656 in the supplied date range are indexed as Notes and match the current Daily-note filename rule. The current rule checks the basename in any Home folder, so the old events/ folder is not required. No YYYY-MM-DD.md names are indexed.	No naming gap for these 656 files. Current code: crates/calternal-notes-core/src/dayfile.rs::is_daily_note_path and crates/plugins/notes/src/lib.rs::journal_path_date. General drop-in coverage is #719; #610 is closed and its importer was cancelled.
Log entries: bullets under ## 📝 Log, with - HH:MM[ - HH:MM]	Source-file syntax use cannot be counted from the Index.	2,487 Calendar Log rows are indexed across 228 of the 656 date-range Daily notes. The indexed rows in that range start on 2025-11-17 and end on 2026-09-27. The other 428 files have no indexed Log row.	The Index cannot distinguish an empty Log section from a Log format it did not parse. I did not read source bytes, so this is an observation, not a confirmed parser failure. The parser is crates/calternal-notes-core/src/dayfile.rs::parse_day_file; #719 covers the corpus check.
Tasks: Task Markdown files and - [ ] / - [x] checkboxes; frontmatter metadata	Supplied manifest says 2 Task files.	Current Home has 3 Task-file projections, all with calternal-id, plus 49 inline Task rows. 48 inline rows are on 18 date-range Daily notes. For the 3 Task files, one has scheduled metadata; none has due or priority in its projection.	The current Home has one more Task file than the supplied manifest; the Index cannot attribute it to this upload. Task syntax and metadata are indexed by crates/calternal-notes-core/src/tasks/extract.rs and crates/plugins/notes/src/tasks_store.rs. #719 covers drop-in parity.
Tags: inline #tag and frontmatter tags:	Raw source usage cannot be split by syntax in the Index.	In the date-range Daily notes, the tag union has 4,239 assignments on 625 files and 89 distinct tag names. This is the merged projection of frontmatter and inline tags.	Tags are recognized. The Index does not retain whether each assignment came from frontmatter or inline text. Code: crates/calternal-notes-core/src/links.rs::note_tags_union and crates/plugins/notes/src/store.rs.
Standard Markdown links and backlinks	Raw source reference count is not available separately from the Index projection.	Across Home, the Index has 2,544 link references: 813 local references resolved, 1,613 local references unresolved and 118 external references. The backlink graph has 809 edges between indexed Notes.	Unresolved local references may be broken Note links or links to other files; the Index-only audit cannot classify their targets without showing paths. #719 explicitly covers Markdown links and attachments. Code: crates/plugins/notes/src/store.rs and crates/calternal-notes-core/src/notes_index.rs.
Wiki links, aliases, headings and blocks: [[Note	alias]], Note#Heading, Note#^block	Source syntax prevalence cannot be fully read from the Index.	The Index records 203 wikilink references; all 203 are unresolved. It records 3,758 target anchors and 2,557 block IDs overall, but no wikilink reference has a resolved target or indexed anchor.
Wiki embeds: !	Source syntax prevalence cannot be fully read from the Index.	The Index records 20 embed references; all 20 are unresolved.	This is covered by open #856. Code: crates/calternal-notes-core/src/links.rs and crates/plugins/notes/src/store.rs.
Note frontmatter and calternal-id	Supplied manifest says 15 Notes; current Notes/ has 19 non-Daily, non-Task Note records.	Of those 19 records, 6 have UUID identities and 13 use path identities. The 3 Task files all have Task IDs. The Index cannot isolate which 15 of the 19 Notes came from this upload. The current Note metadata parser reads calternal-id, title, date, last edited, slug and tags; it does not index aliases:.	Stable Note identity remains a gap for 13 current Note records; open #857 covers assigning stable identity to imported Notes. Alias-key matching is not covered by #856's inline-link scope. Code: crates/calternal-notes-core/src/note_meta.rs::extract_note_meta, crates/calternal-notes-core/src/links.rs::wikilink_match_keys, crates/plugins/notes/src/lib.rs::view and crates/calternal-notes-core/src/notes_index.rs::path_keyed_note_id.
Photos and attachments	Supplied manifest says 2 photos.	Photos/2026 has 2 indexed media items: one HEIC image and one JPEG image. Calendar Log projections also contain 37 attachment references across Home.	Both expected Photos items are in the Photos projection. Wiki embeds remain unresolved under #856. The Index does not distinguish the uploaded 15 Notes from other Home Notes. Photo indexing: crates/plugins/photos/src/index.rs; Files indexing: crates/plugins/files/src/index.rs.

Issue coverage and proposed child

#856 covers the observed unresolved Wiki links and embeds.
#857 covers the stable Note identity gap.
#719 remains the open parent for folder-drop Markdown recognition, including standard links, tags, tasks, Daily notes and attachments.
#610 is closed; its dedicated importer was cancelled.

Proposed child of #719: Resolve frontmatter aliases in dropped Markdown Notes

Read aliases: as a scalar, flow list or block list without rewriting source bytes.
Add alias values to the unique Note target-key index.
Resolve a unique alias target and keep missing or ambiguous references intact.
Refresh Used in and backlinks when an alias target changes.
Test rebuilds, collisions, renames and read-only Note opening.

I did not propose an issue for the 428 Daily notes without Log rows: the Index cannot establish whether they contain unparsed Log syntax or no entries. A source-byte review is needed before calling that a code gap.

Decisions, limits and completion

I used the Index because this audit did not create an authenticated session. The Index has no upload provenance and does not retain arbitrary frontmatter-key prevalence, so I report those counts as unavailable instead of inferring them. No product or code decision was needed.

No code changed and no commit or merge was made for this audit. The shared local dev HEAD advanced independently from the starting SHA 4a871b3838 to c7cf2bf8b0 through two docs/design commits; origin/dev remains at 4a871b3838. The worktree is clean.

Gates: Not run (read-only audit; no code changes).
Cleanup output: cargo clean — "Removed 1 file, 356B total"; removed web build output apps/web/build and apps/web/.svelte-kit.

## Read-only production Home audit — 2026-10-03 ### Method and inventory I queried the owner User's Index SQLite database with a read-only URI and query-only mode. The Instance has one owner User. I did not create a session, read credential material, select note titles or bodies, or output user-authored names or tag values. I compared the aggregates with the calternal.js conventions in docs/DESIGN.md, docs/contracts.md, packages/core/src/dayfile.rs, packages/core/src/tasks/extract.rs and packages/cli/src/core/notes-index.ts. The supplied batch is 656 Daily notes, 15 Notes, 2 Task files and 2 photos: 675 files. The current Index has 684 Markdown files in Notes/ (662 Daily-note-pattern files and 22 other files, including 3 projected Task files) and 2 media items under Photos/2026/. This is 686 files in those buckets. The 656 Daily-note-pattern files in the supplied date range are all present; six more Daily-note-pattern files are dated 2026-09-28 through 2026-10-03. There are five more non-Daily Markdown files than the supplied 15 Notes + 2 Tasks. The Index has no import-batch provenance, so it cannot show which of those five were already in Home before the upload. ### Findings | calternal.js convention | Supplied or current Home count | What the Index recognizes | Gap and tracking | |---|---:|---|---| | Daily note path: events/YYYYMMDD-dailynote.md | 656 in 2022-11-20 through 2026-09-27; current Notes/ has 662 matching names | All 656 in the supplied date range are indexed as Notes and match the current Daily-note filename rule. The current rule checks the basename in any Home folder, so the old events/ folder is not required. No YYYY-MM-DD.md names are indexed. | No naming gap for these 656 files. Current code: crates/calternal-notes-core/src/dayfile.rs::is_daily_note_path and crates/plugins/notes/src/lib.rs::journal_path_date. General drop-in coverage is #719; #610 is closed and its importer was cancelled. | | Log entries: bullets under ## 📝 Log, with - HH:MM[ - HH:MM] | Source-file syntax use cannot be counted from the Index. | 2,487 Calendar Log rows are indexed across 228 of the 656 date-range Daily notes. The indexed rows in that range start on 2025-11-17 and end on 2026-09-27. The other 428 files have no indexed Log row. | The Index cannot distinguish an empty Log section from a Log format it did not parse. I did not read source bytes, so this is an observation, not a confirmed parser failure. The parser is crates/calternal-notes-core/src/dayfile.rs::parse_day_file; #719 covers the corpus check. | | Tasks: Task Markdown files and - [ ] / - [x] checkboxes; frontmatter metadata | Supplied manifest says 2 Task files. | Current Home has 3 Task-file projections, all with calternal-id, plus 49 inline Task rows. 48 inline rows are on 18 date-range Daily notes. For the 3 Task files, one has scheduled metadata; none has due or priority in its projection. | The current Home has one more Task file than the supplied manifest; the Index cannot attribute it to this upload. Task syntax and metadata are indexed by crates/calternal-notes-core/src/tasks/extract.rs and crates/plugins/notes/src/tasks_store.rs. #719 covers drop-in parity. | | Tags: inline #tag and frontmatter tags: | Raw source usage cannot be split by syntax in the Index. | In the date-range Daily notes, the tag union has 4,239 assignments on 625 files and 89 distinct tag names. This is the merged projection of frontmatter and inline tags. | Tags are recognized. The Index does not retain whether each assignment came from frontmatter or inline text. Code: crates/calternal-notes-core/src/links.rs::note_tags_union and crates/plugins/notes/src/store.rs. | | Standard Markdown links and backlinks | Raw source reference count is not available separately from the Index projection. | Across Home, the Index has 2,544 link references: 813 local references resolved, 1,613 local references unresolved and 118 external references. The backlink graph has 809 edges between indexed Notes. | Unresolved local references may be broken Note links or links to other files; the Index-only audit cannot classify their targets without showing paths. #719 explicitly covers Markdown links and attachments. Code: crates/plugins/notes/src/store.rs and crates/calternal-notes-core/src/notes_index.rs. | | Wiki links, aliases, headings and blocks: [[Note|alias]], [[Note#Heading]], [[Note#^block]] | Source syntax prevalence cannot be fully read from the Index. | The Index records 203 wikilink references; all 203 are unresolved. It records 3,758 target anchors and 2,557 block IDs overall, but no wikilink reference has a resolved target or indexed anchor. | This is a confirmed projection gap. #856 is open and covers Wiki links, aliases, headings, blocks and embeds. Frontmatter aliases: is not exposed by the current Note metadata or target-key index, and its source usage count is unavailable here. | | Wiki embeds: ![[image.png]] | Source syntax prevalence cannot be fully read from the Index. | The Index records 20 embed references; all 20 are unresolved. | This is covered by open #856. Code: crates/calternal-notes-core/src/links.rs and crates/plugins/notes/src/store.rs. | | Note frontmatter and calternal-id | Supplied manifest says 15 Notes; current Notes/ has 19 non-Daily, non-Task Note records. | Of those 19 records, 6 have UUID identities and 13 use path identities. The 3 Task files all have Task IDs. The Index cannot isolate which 15 of the 19 Notes came from this upload. The current Note metadata parser reads calternal-id, title, date, last edited, slug and tags; it does not index aliases:. | Stable Note identity remains a gap for 13 current Note records; open #857 covers assigning stable identity to imported Notes. Alias-key matching is not covered by #856's inline-link scope. Code: crates/calternal-notes-core/src/note_meta.rs::extract_note_meta, crates/calternal-notes-core/src/links.rs::wikilink_match_keys, crates/plugins/notes/src/lib.rs::view and crates/calternal-notes-core/src/notes_index.rs::path_keyed_note_id. | | Photos and attachments | Supplied manifest says 2 photos. | Photos/2026 has 2 indexed media items: one HEIC image and one JPEG image. Calendar Log projections also contain 37 attachment references across Home. | Both expected Photos items are in the Photos projection. Wiki embeds remain unresolved under #856. The Index does not distinguish the uploaded 15 Notes from other Home Notes. Photo indexing: crates/plugins/photos/src/index.rs; Files indexing: crates/plugins/files/src/index.rs. | ### Issue coverage and proposed child - #856 covers the observed unresolved Wiki links and embeds. - #857 covers the stable Note identity gap. - #719 remains the open parent for folder-drop Markdown recognition, including standard links, tags, tasks, Daily notes and attachments. - #610 is closed; its dedicated importer was cancelled. Proposed child of #719: **Resolve frontmatter aliases in dropped Markdown Notes** - Read aliases: as a scalar, flow list or block list without rewriting source bytes. - Add alias values to the unique Note target-key index. - Resolve a unique [[alias]] target and keep missing or ambiguous references intact. - Refresh Used in and backlinks when an alias target changes. - Test rebuilds, collisions, renames and read-only Note opening. I did not propose an issue for the 428 Daily notes without Log rows: the Index cannot establish whether they contain unparsed Log syntax or no entries. A source-byte review is needed before calling that a code gap. ### Decisions, limits and completion I used the Index because this audit did not create an authenticated session. The Index has no upload provenance and does not retain arbitrary frontmatter-key prevalence, so I report those counts as unavailable instead of inferring them. No product or code decision was needed. No code changed and no commit or merge was made for this audit. The shared local dev HEAD advanced independently from the starting SHA 4a871b383864dad0d6c87bcd22b7f0d589e3506e to c7cf2bf8b0079e96d8a60faead276731a2db9eca through two docs/design commits; origin/dev remains at 4a871b383864dad0d6c87bcd22b7f0d589e3506e. The worktree is clean. Gates: Not run (read-only audit; no code changes). Cleanup output: cargo clean — "Removed 1 file, 356B total"; removed web build output apps/web/build and apps/web/.svelte-kit.

kayg commented

2026-10-03 08:12:50 +00:00

Author

Owner

Correction to the completion SHA in my audit report: the shared local dev branch advanced again after that report was posted. Current local HEAD is f06679b11c; origin/dev is 3b733f9dc2. The additional commits observed in the reflog are docs/design changes from the concurrent shared-worktree updates. I made no commit or merge for this audit; the worktree is clean.

Correction to the completion SHA in my audit report: the shared local dev branch advanced again after that report was posted. Current local HEAD is f06679b11cde29cc0b7120fdab5f721389caf695; origin/dev is 3b733f9dc2f9c18a72d461e3644a1b3b5e2fbff9. The additional commits observed in the reflog are docs/design changes from the concurrent shared-worktree updates. I made no commit or merge for this audit; the worktree is clean.

kayg commented

2026-10-03 08:21:38 +00:00

Author

Owner

Starting the #719 read-only follow-up on branch dev at base SHA f06679b11cde29cc0b7120fdab5f721389caf695. I will query the production Index read-only and run a server-side classifier over only the 428 Daily notes without indexed Calendar Log rows. It will emit aggregate counts only. I will not print, copy off-server, or log note text, titles, names, filenames or other personal content. No code or production data will be changed.

Starting the #719 read-only follow-up on branch `dev` at base SHA `f06679b11cde29cc0b7120fdab5f721389caf695`. I will query the production Index read-only and run a server-side classifier over only the 428 Daily notes without indexed Calendar Log rows. It will emit aggregate counts only. I will not print, copy off-server, or log note text, titles, names, filenames or other personal content. No code or production data will be changed.

kayg referenced this issue

2026-10-03 08:42:28 +00:00

Expose Daily note Log parse state in the Index #998

kayg referenced this issue

2026-10-03 08:53:47 +00:00

Journal: Log lines written as HHMM (no colon) are not recognised; 4,654 imported entries missing from the Calendar #724

kayg referenced this issue

2026-10-03 08:56:44 +00:00

Journal: Log lines written as HHMM (no colon) are not recognised; 4,654 imported entries missing from the Calendar #724

kayg commented

2026-10-03 08:57:33 +00:00

Author

Owner

Read-only follow-up complete

The production scan ran against the owner's 428 Daily notes with no indexed Calendar Log rows. It read all 428 successfully and emitted aggregate counts only. No note text, titles, names, paths, or personal content were printed, copied, or logged. The production database and files were not changed.

Index reconciliation

Daily notes	Count
In the audit range	656
With indexed Calendar Log rows	228
Indexed Calendar Log rows	2,487
With no indexed Calendar Log rows	428

Daily note syntax shapes in the 428 no-row notes

The calternal.js stored Log grammar is a ## 📝 Log section (plain ## Log is also accepted) with - HH:MM[ - HH:MM] <title-or-link> [#tag ...] [^block-id] entries. A ## HH:MM heading is not a Log heading. Duration phrases are composer input, not stored Log syntax. Shape categories can overlap; each row counts only matching files and lines, without revealing values.

Shape	Files	Lines / sections	Current parser result
Empty or whitespace-only note	5	—	No body to parse
Frontmatter-only note	0	—	—
Note with body text	423	—	Contains one or more body shapes below
Accepted H2 Log heading	279	279 headings	`## Log` / `## 📝 Log` form
Empty accepted Log section	4	4 sections	No Calendar Log entry
Time-shaped H2 heading (`## HH:MM…`)	1	1	Rejected: the heading must identify Log
Compact `HHMM` time-stamped bullet inside an accepted Log section	269	4,655	Rejected: current `read_time_prefix` requires `HH:MM`; see #724
`HH:MM` bullet, no accepted Log heading	39	594	Rejected: no selected Log section
`HH:MM` bullet before an accepted Log heading	1	1	Rejected: outside the selected section
`HH:MM` bullet after the next H2	2	4	Rejected: the Log section ends at the next H2
`HH:MM` bullet inside a Log section with no title	1	1	Rejected: `parse_log_bullet` requires a non-empty title
Bare `HH:MM` prefix outside an accepted Log section	56	130	Rejected: no bullet marker and no selected section
Bare `HH:MM` prefix inside a Log section	1	1	Rejected: `parse_log_bullet` requires a bullet marker
Bare `H:MM` prefix	1	1	Rejected: the parser requires two hour digits; it also lacks a bullet marker
`H:MM` range shape	1	1	Rejected: the parser requires two hour digits
Bare `HHMM` prefix outside an accepted Log section	1	1	Rejected: no colon, no bullet marker, and no selected section
Bullet-journal lines without a time prefix	351	5,702	Not Calendar Log entries; an entry needs a time prefix
Other non-task bullets inside accepted Log sections without `HH:MM`	272	5,146	Not Calendar Log entries; unparsed top-level bullets can be returned by the per-day API
Task checkbox lines	18	37	Task syntax, not timed Log entries; 9 lines in 6 files are inside Log sections
Duration phrases	19	22	Composer input shape, not a stored Log entry
Frontmatter tag syntax	357	441	Metadata, not a Log entry
Inline tag syntax	340	5,373	Metadata, not a Log entry by itself
Plain paragraph lines	208	749	Not Log entries
Wiki-link shape in this no-row subset	1	1	Link syntax, not a Log entry
Wiki-embed shape in this no-row subset	0	0	None observed

The colon-form HH:MM bullet rows above total 600 lines. File counts can overlap across placements. The 4,655 compact HHMM bullet candidates are one more than #724's existing 4,654 count of imported - HHMM - HHMM text ranges. The classifiers use different shape definitions; the difference is noted on #724 for reconciliation. No source line was selected or exposed.

Unresolved links and embeds by shape

Kind	References	Source notes	Missing target
Wiki link	203	82	203
Embed	20	14	20
Wiki embed	0	0	0

Of the 203 wiki links, 201 have no resolved target path and 2 have a target path without a Files/Notes Index record. The shape scan found no ambiguous target. The embed scan found no alias/alt-label or heading/block anchor shapes. These link and identity gaps belong to #856 and #857; no duplicate issue was filed.

Findings and issue tracking

Created #998, Expose Daily note Log parse state in the Index, for the remaining Index-level gap. The per-day Notes API already returns heading diagnostics and malformed top-level list items under #53/#70; the Index still has no aggregate parse state for a Daily note with zero Calendar Log rows.
The compact time syntax and rebuild work are already tracked by #724. I added the 4,655-candidate evidence and the one-count definition difference there.
Unheaded, pre-heading, and post-section HH:MM bullets are outside the owner-defined Log section. Missing heading and in-section line diagnostics are covered by #53/#70; no new issue was filed for those shapes.
Link and identity findings refer to #856 and #857.

Decisions

The design does not specify an Index-level Log parse state. #998 proposes a derived state refreshed when a Daily note is indexed, while keeping file bytes as the source of truth, storing no source text in that state, and keeping repair or warning behavior out of the normal User flow.

Worktree and gates

No code or repository files changed. Branch dev remains at f06679b11cde29cc0b7120fdab5f721389caf695; the worktree is clean. No build or test gates were run because this was a read-only audit with no code changes.

## Read-only follow-up complete The production scan ran against the owner's 428 Daily notes with no indexed Calendar Log rows. It read all 428 successfully and emitted aggregate counts only. No note text, titles, names, paths, or personal content were printed, copied, or logged. The production database and files were not changed. ### Index reconciliation | Daily notes | Count | |---|---:| | In the audit range | 656 | | With indexed Calendar Log rows | 228 | | Indexed Calendar Log rows | 2,487 | | With no indexed Calendar Log rows | 428 | ### Daily note syntax shapes in the 428 no-row notes The calternal.js stored Log grammar is a `## 📝 Log` section (plain `## Log` is also accepted) with `- HH:MM[ - HH:MM] <title-or-link> [#tag ...] [^block-id]` entries. A `## HH:MM` heading is not a Log heading. Duration phrases are composer input, not stored Log syntax. Shape categories can overlap; each row counts only matching files and lines, without revealing values. | Shape | Files | Lines / sections | Current parser result | |---|---:|---:|---| | Empty or whitespace-only note | 5 | — | No body to parse | | Frontmatter-only note | 0 | — | — | | Note with body text | 423 | — | Contains one or more body shapes below | | Accepted H2 Log heading | 279 | 279 headings | `## Log` / `## 📝 Log` form | | Empty accepted Log section | 4 | 4 sections | No Calendar Log entry | | Time-shaped H2 heading (`## HH:MM…`) | 1 | 1 | Rejected: the heading must identify Log | | Compact `HHMM` time-stamped bullet inside an accepted Log section | 269 | 4,655 | Rejected: current `read_time_prefix` requires `HH:MM`; see #724 | | `HH:MM` bullet, no accepted Log heading | 39 | 594 | Rejected: no selected Log section | | `HH:MM` bullet before an accepted Log heading | 1 | 1 | Rejected: outside the selected section | | `HH:MM` bullet after the next H2 | 2 | 4 | Rejected: the Log section ends at the next H2 | | `HH:MM` bullet inside a Log section with no title | 1 | 1 | Rejected: `parse_log_bullet` requires a non-empty title | | Bare `HH:MM` prefix outside an accepted Log section | 56 | 130 | Rejected: no bullet marker and no selected section | | Bare `HH:MM` prefix inside a Log section | 1 | 1 | Rejected: `parse_log_bullet` requires a bullet marker | | Bare `H:MM` prefix | 1 | 1 | Rejected: the parser requires two hour digits; it also lacks a bullet marker | | `H:MM` range shape | 1 | 1 | Rejected: the parser requires two hour digits | | Bare `HHMM` prefix outside an accepted Log section | 1 | 1 | Rejected: no colon, no bullet marker, and no selected section | | Bullet-journal lines without a time prefix | 351 | 5,702 | Not Calendar Log entries; an entry needs a time prefix | | Other non-task bullets inside accepted Log sections without `HH:MM` | 272 | 5,146 | Not Calendar Log entries; unparsed top-level bullets can be returned by the per-day API | | Task checkbox lines | 18 | 37 | Task syntax, not timed Log entries; 9 lines in 6 files are inside Log sections | | Duration phrases | 19 | 22 | Composer input shape, not a stored Log entry | | Frontmatter tag syntax | 357 | 441 | Metadata, not a Log entry | | Inline tag syntax | 340 | 5,373 | Metadata, not a Log entry by itself | | Plain paragraph lines | 208 | 749 | Not Log entries | | Wiki-link shape in this no-row subset | 1 | 1 | Link syntax, not a Log entry | | Wiki-embed shape in this no-row subset | 0 | 0 | None observed | The colon-form `HH:MM` bullet rows above total 600 lines. File counts can overlap across placements. The 4,655 compact `HHMM` bullet candidates are one more than #724's existing 4,654 count of imported `- HHMM - HHMM text` ranges. The classifiers use different shape definitions; the difference is noted on #724 for reconciliation. No source line was selected or exposed. ### Unresolved links and embeds by shape | Kind | References | Source notes | Alias / alt label | Heading anchor | Block anchor | Missing target | Attachment-folder shape | |---|---:|---:|---:|---:|---:|---:|---:| | Wiki link | 203 | 82 | 0 | 0 | 0 | 203 | 0 | | Embed | 20 | 14 | 0 | 0 | 0 | 20 | 0 | | Wiki embed | 0 | 0 | 0 | 0 | 0 | 0 | 0 | Of the 203 wiki links, 201 have no resolved target path and 2 have a target path without a Files/Notes Index record. The shape scan found no ambiguous target. The embed scan found no alias/alt-label or heading/block anchor shapes. These link and identity gaps belong to #856 and #857; no duplicate issue was filed. ### Findings and issue tracking - Created #998, **Expose Daily note Log parse state in the Index**, for the remaining Index-level gap. The per-day Notes API already returns heading diagnostics and malformed top-level list items under #53/#70; the Index still has no aggregate parse state for a Daily note with zero Calendar Log rows. - The compact time syntax and rebuild work are already tracked by #724. I added the 4,655-candidate evidence and the one-count definition difference there. - Unheaded, pre-heading, and post-section `HH:MM` bullets are outside the owner-defined Log section. Missing heading and in-section line diagnostics are covered by #53/#70; no new issue was filed for those shapes. - Link and identity findings refer to #856 and #857. ### Decisions The design does not specify an Index-level Log parse state. #998 proposes a derived state refreshed when a Daily note is indexed, while keeping file bytes as the source of truth, storing no source text in that state, and keeping repair or warning behavior out of the normal User flow. ### Worktree and gates No code or repository files changed. Branch `dev` remains at `f06679b11cde29cc0b7120fdab5f721389caf695`; the worktree is clean. No build or test gates were run because this was a read-only audit with no code changes.

kayg referenced this issue from a commit

2026-10-03 09:01:11 +00:00

test(notes): import generated #719 corpora

kayg commented

2026-10-03 11:29:36 +00:00

Author

Owner

Starting the issue hygiene sweep on branch dev, based on d4e7188810a89fb0e8e6b162279917f7e23989f (also current origin/dev). I will inventory open issues, check merged fixes and owner decisions against the repository and paused merge queue, and leave uncertain or queued work open. I will publish the summary as a new issue as requested.

Starting the issue hygiene sweep on branch `dev`, based on `d4e7188810a89fb0e8e6b162279917f7e23989f` (also current `origin/dev`). I will inventory open issues, check merged fixes and owner decisions against the repository and paused merge queue, and leave uncertain or queued work open. I will publish the summary as a new issue as requested.

kayg referenced this issue

2026-10-03 11:58:57 +00:00

IMPORT: calternal.js vault → calternal Home (Settings → Import + CLI, dry-run first) #300

kayg commented

2026-10-03 11:58:58 +00:00

Author

Owner

The folder-identity sub-change is superseded by #627, whose approved branch is still queued in merge-round-7b and is not on origin/dev. #719 remains open: the audit still has unresolved wiki links/embeds and imported Note identity (#856/#857), five unmeasured corpora/channels, and no completed 20k-file run.

kayg referenced this issue

2026-10-03 12:04:41 +00:00

Issue hygiene sweep 2026-10-03 #1008

kayg commented

2026-10-03 12:05:35 +00:00

Author

Owner

Issue hygiene sweep finished. The disposition table and evidence references are in #1008.

Closed 46 issues with verified completion, supersession, or duplicate evidence.
Marked #26 “STALE?” and left it open for owner review.
Commented and left open #41, #55, #71, #93, #223, #249, #256, #300, #330, #719, and #866. #223 and #256 were reopened because they carry explicit OPEN markers; #249 was reopened because it is marked LATER.
Left 639 other original open issues open. The full current issue records and comment histories were fetched; the remaining 639 descriptions and their evidence were not individually validated. No uncertain issue was closed.
#643 and #647 are superseded by #940, which remains queued and is not reported as landed. Two other initial closures, #223 and #256, were reopened after the explicit-marker audit.

No repository code or tracked files changed. Branch dev remains at d4e7188810a89fb0e8e6b162279917f7e23989f9; the worktree is clean. No commit, push, merge, or deploy was made.

Gates: Rust and web gates were not run because this issue-hygiene sweep changed no code. No test or adversarial command was run.

Cleanup command output:

     Removed 1 file, 356B total

The web build output directories were removed.

Known gaps: #719 still has unresolved imported-link and identity behavior plus incomplete corpus/channel coverage. The remaining 639 original open issues need individual evidence review before any further disposition. The historical Git object for the neutralized #349 text remains in history; no history rewrite was performed.

Decisions beyond DESIGN: mark #26 stale based on its recovered state and unknown root cause; treat #643/#647 as superseded by queued #940; neutralize #349's issue text and comment and close it under DESIGN §43. No design or code decisions were made.

Issue hygiene sweep finished. The disposition table and evidence references are in [#1008](https://git.kayg.org/kayg/calternal/issues/1008). - Closed 46 issues with verified completion, supersession, or duplicate evidence. - Marked #26 “STALE?” and left it open for owner review. - Commented and left open #41, #55, #71, #93, #223, #249, #256, #300, #330, #719, and #866. #223 and #256 were reopened because they carry explicit OPEN markers; #249 was reopened because it is marked LATER. - Left 639 other original open issues open. The full current issue records and comment histories were fetched; the remaining 639 descriptions and their evidence were not individually validated. No uncertain issue was closed. - #643 and #647 are superseded by #940, which remains queued and is not reported as landed. Two other initial closures, #223 and #256, were reopened after the explicit-marker audit. No repository code or tracked files changed. Branch `dev` remains at `d4e7188810a89fb0e8e6b162279917f7e23989f9`; the worktree is clean. No commit, push, merge, or deploy was made. Gates: Rust and web gates were not run because this issue-hygiene sweep changed no code. No test or adversarial command was run. Cleanup command output: ``` Removed 1 file, 356B total ``` The web build output directories were removed. Known gaps: #719 still has unresolved imported-link and identity behavior plus incomplete corpus/channel coverage. The remaining 639 original open issues need individual evidence review before any further disposition. The historical Git object for the neutralized #349 text remains in history; no history rewrite was performed. Decisions beyond DESIGN: mark #26 stale based on its recovered state and unknown root cause; treat #643/#647 as superseded by queued #940; neutralize #349's issue text and comment and close it under DESIGN §43. No design or code decisions were made.

kayg commented

2026-10-03 12:44:36 +00:00

Author

Owner

Rechecked against origin/dev d4e7188810 and the merge queue. #627 supersedes only the folder-identity change, and that branch is still queued in merge-round-7b. The intake audit still lists unresolved wiki links and embeds (#856), stable Note identity (#857), and incomplete corpus and channel coverage. Keep #719 open for the remaining intake work.

Rechecked against origin/dev d4e7188810a89fb0e8e6b162279917f7e23989f9 and the merge queue. #627 supersedes only the folder-identity change, and that branch is still queued in merge-round-7b. The intake audit still lists unresolved wiki links and embeds (#856), stable Note identity (#857), and incomplete corpus and channel coverage. Keep #719 open for the remaining intake work.

kayg referenced this issue

2026-10-03 13:13:42 +00:00

Issue hygiene sweep 2026-10-03 #1008

kayg referenced this issue

2026-10-03 14:07:57 +00:00

Resolve wiki-style links and embeds in imported Notes #856

Rows
Columns

Moving in by dropping Markdown folders: find and fix what breaks (replaces the dedicated importer) #719

Owner decision (2026-10-02)

Goal

Corpora (tests only: openly licensed or generated, never the owner's data)

For each corpus, report what the User sees

Output

Findings

20k-file performance

UX gaps closed

UX gaps left

Gates (verbatim output)

Decisions

Revision

Read-only production Home audit — 2026-10-03

Method and inventory

Findings

Issue coverage and proposed child

Decisions, limits and completion

Read-only follow-up complete

Index reconciliation

Daily note syntax shapes in the 428 no-row notes

Unresolved links and embeds by shape

Findings and issue tracking

Decisions

Worktree and gates