kayg/calternal

Watch

Fork 0

Docs in code: Starlight site from docs/ plus reference generated from code comments #971

New issue

Open

opened 2026-10-03 05:36:03 +00:00 by kayg · 1 comment

kayg commented

2026-10-03 05:36:03 +00:00

Owner

Owner request (2026-10-03)

Keep documentation in the code repository under a docs folder, with a good modern docs framework, and build as much as possible from the comments in the code.

Current state

apps/docs already exists: Astro + Starlight (MIT) with Pagefind search, guides in apps/docs/src/content/docs/. It builds to apps/docs/dist/ (see deploy/docs/README.md).
Hand-written design and reference docs live in docs/ (DESIGN.md, action registry, MCP, CLI walkthrough, deep links, tags, perf...).
docs.calternal.com was served from the retired o2 host, so it is down since o2 was phased out (2026-10-02).

Plan

Keep Starlight as the framework (modern, fast, MIT, static, built-in search, i18n, good a11y). Make docs/ the single source: Starlight reads the repo docs/ Markdown through a content loader, so DESIGN and reference docs are not copied.
Generated reference from code comments:
- TypeScript packages (packages/ui, packages/api-client, packages/editor, web lib): TypeDoc (Apache-2.0) + starlight-typedoc (MIT) renders TSDoc comments into Starlight pages.
- Rust crates: rustdoc. Either publish cargo doc HTML under /reference/rust/ or convert rustdoc JSON to Starlight pages; pick the one that keeps search and navigation unified.
- HTTP API: contracts/openapi.json → starlight-openapi (MIT) pages; the action registry and MCP tool list from contracts/actions.json.
- CLI: generated from the clap command tree (the CLI already builds completions from it).
Freshness gate: the docs build runs in CI (bun run --cwd apps/docs build), fails on broken links and on missing doc comments for public APIs (ratchet like the perf guards), and regenerates reference pages from the current code.
Hosting: serve the static build at docs.calternal.com from the production edge (Traefik route + a small static container, per deploy/docs/README.md), deployed with each release.
Docs use ASD-STE100 Simplified Technical English; never name competing products.

Tests

Docs build succeeds from a clean checkout; link checker passes; generated reference pages exist for each public package/crate/route; a removed doc comment on a public item fails the gate.

## Owner request (2026-10-03) Keep documentation in the code repository under a docs folder, with a good modern docs framework, and build as much as possible from the comments in the code. ## Current state - `apps/docs` already exists: Astro + Starlight (MIT) with Pagefind search, guides in `apps/docs/src/content/docs/`. It builds to `apps/docs/dist/` (see `deploy/docs/README.md`). - Hand-written design and reference docs live in `docs/` (DESIGN.md, action registry, MCP, CLI walkthrough, deep links, tags, perf...). - docs.calternal.com was served from the retired o2 host, so it is **down** since o2 was phased out (2026-10-02). ## Plan 1. **Keep Starlight** as the framework (modern, fast, MIT, static, built-in search, i18n, good a11y). Make `docs/` the single source: Starlight reads the repo `docs/` Markdown through a content loader, so DESIGN and reference docs are not copied. 2. **Generated reference from code comments:** - **TypeScript packages** (`packages/ui`, `packages/api-client`, `packages/editor`, web lib): TypeDoc (Apache-2.0) + `starlight-typedoc` (MIT) renders TSDoc comments into Starlight pages. - **Rust crates:** rustdoc. Either publish `cargo doc` HTML under `/reference/rust/` or convert rustdoc JSON to Starlight pages; pick the one that keeps search and navigation unified. - **HTTP API:** `contracts/openapi.json` → `starlight-openapi` (MIT) pages; the action registry and MCP tool list from `contracts/actions.json`. - **CLI:** generated from the clap command tree (the CLI already builds completions from it). 3. **Freshness gate:** the docs build runs in CI (`bun run --cwd apps/docs build`), fails on broken links and on missing doc comments for public APIs (ratchet like the perf guards), and regenerates reference pages from the current code. 4. **Hosting:** serve the static build at docs.calternal.com from the production edge (Traefik route + a small static container, per `deploy/docs/README.md`), deployed with each release. 5. Docs use ASD-STE100 Simplified Technical English; never name competing products. ## Tests - Docs build succeeds from a clean checkout; link checker passes; generated reference pages exist for each public package/crate/route; a removed doc comment on a public item fails the gate.

docs-971-home-light.png

99 KiB

docs-971-home-dark.png

99 KiB

docs-971-reference_typescript_calternal_ui-light.png

133 KiB

docs-971-reference_typescript_calternal_ui-dark.png

134 KiB

docs-971-reference_api_operations_get_note-light.png

82 KiB

docs-971-reference_api_operations_get_note-dark.png

82 KiB

docs-971-reference_rust_calternal_path-light.png

116 KiB

docs-971-reference_rust_calternal_path-dark.png

117 KiB

kayg commented

2026-10-03 06:34:31 +00:00

Author

Owner

Implementation on `job/docs-971` (head `5faf41f56`, not merged, not deployed)

What the site has now

One Starlight navigation and one Pagefind search for everything:

Section	Source	Pages
Guides	`apps/docs/src/content/docs/`	7
Design	`docs/*.md` read in place by a content loader (no copies); DESIGN.md is one page with anchors	10
Reference → CLI	new `contracts/cli.json` from the clap tree (test `cli_contract` fails when stale)	27
Reference → Actions and MCP tools	`contracts/actions.json`, one page per API area, each action links to its HTTP API page	25
Reference → HTTP API	`contracts/openapi.json` via starlight-openapi (area-tagged copy in `.cache/`, so untagged operations get pages)	336
Reference → TypeScript packages	TSDoc in ui / editor / api-client (svelte2tsx declarations → TypeDoc), one page per package	5
Reference → Rust crates	`///` / `//!` doc comments of the 26 library crates, one page per public module	86

Total 498 pages, dist 43 MB.

Tools (all AGPL-compatible)

astro 7.3.5 (MIT), @astrojs/starlight 0.42.4 (MIT), starlight-typedoc 0.23.1 (MIT), typedoc 0.28.20 (Apache-2.0), typedoc-plugin-markdown 4.13.1 (MIT), starlight-openapi 0.26.3 (MIT), starlight-links-validator 0.26.0 (MIT), svelte2tsx 0.7.61 (MIT), satteri 0.10.5 (MIT), github-slugger 2.0.0 (ISC).

Rust: source scanner instead of rustdoc. cargo doc HTML would split navigation and search. rustdoc JSON needs nightly flags (format changes per release) and type-checks every dependency: minutes and GBs on a clean checkout, and a Rust toolchain in the docs build. The scanner (apps/docs/src/reference/rust-scan.ts) lexes the crates in about 1 s, follows the module tree, applies visibility (pub reachable from the root, pub use re-exports inlined, no pub(crate), #[cfg(test)], #[doc(hidden)]), and resolves intra-doc links to anchors. Limits: no macro expansion and no cross-crate type resolution. Reasoning is in apps/docs/README.md.

Gates

bun run --cwd apps/docs build from a fresh clone: bun install 4 s, build 34 s (498 page(s) built in 30.87s, All internal links are valid.). A warm rebuild takes about 28 s.
Link checker is on (starlight-links-validator). It found real breakage on the way (relative .md links, prose examples such as Accounts.md#^amex, the anchors in cli-walkthrough). All of it is fixed by the loader.
Doc comment ratchet apps/docs/doc-coverage.json: rust 411, typescript 211 undocumented public items today. The build fails when a count grows and names the new items. I checked it on a clean clone by removing the doc comment of RelPath:
Rust: 412 public items without a doc comment, baseline 411. Add doc comments. New undocumented items: calternal_path::RelPath (crates/calternal-path/src/lib.rs:14)
bun run --cwd apps/docs check: 0 errors, 0 warnings, 0 hints. bun run --cwd apps/docs test: 24 pass, 0 fail.
cargo fmt --check -p calternal-cli clean. cargo clippy -p calternal-cli --all-targets -- -D warnings clean. cargo test -p calternal-cli: 37 passed and 15 passed.

Bug found on the way

calternal mail remote-content <id> <allow> declared a positional bool without a value action, which clap rejects (it panics with a debug assertion, and a release build cannot parse the value). Fixed with ArgAction::Set. A Cli::command().debug_assert() test now covers the whole tree.

Size

Starlight puts the full sidebar into every page. With ~500 reference links the first build was 182 MB. src/sidebar-fold.ts folds each reference group that does not hold the current page into one link (home page 230 KB → 24 KB). TypeDoc writes one page per package. The generated paths/components OpenAPI types are left out of the TypeScript reference because the HTTP API section already covers them (2.6 MB page).

Screenshots (production build)

	Light	Dark
Home
TypeScript: @calternal/ui
HTTP API: get_note
Rust: calternal-path

Hosting at docs.calternal.com (prepared, not done)

The files are deploy/docs/{Containerfile,Caddyfile,calternal-docs.container} and deploy/deploy-docs.sh. They give a Caddy 2.11.6 image (pinned digest), an unprivileged user, a read-only root, no capabilities, a real 404 page, immutable cache on /_astro/, and a CSP. I tested the image locally under --read-only --cap-drop=all --security-opt=no-new-privileges: 200/404 are correct and Pagefind search works with 0 CSP errors.

Cloudflare zone calternal.com: A docs → 168.119.145.115, DNS only.
calternal-cloud VM: in /etc/calternal-firewall.nft, allow TCP 8081 from 10.70.3.101-103, then restart calternal-firewall.service.
homelab-private: add k8s/hatsuna/calternal/docs.yaml. The full manifest is in deploy/docs/README.md: Certificate, IngressRoute plus redirect that reuse calternal-ratelimit/calternal-headers, and a selector-less Service + EndpointSlice to 10.70.3.162:8081. Argo app calternal syncs it.
Run deploy/deploy-docs.sh on the build VM. Run it again with each release.

Open points for the owner

The Design section publishes docs/DESIGN.md as it is. It names other products (quality bars, references). Point 5 of this issue says docs never name competing products. Do you want to rewrite those lines, or keep DESIGN out of the public site?
OpenAPI operations have no summaries (handlers carry no utoipa doc comments), so HTTP API pages are titled by operation ID. Server doc comments would improve the pages and the action help text. This is a separate issue if wanted.

## Implementation on `job/docs-971` (head `5faf41f56`, not merged, not deployed) ### What the site has now One Starlight navigation and one Pagefind search for everything: | Section | Source | Pages | |---|---|---| | Guides | `apps/docs/src/content/docs/` | 7 | | Design | `docs/*.md` read in place by a content loader (no copies); DESIGN.md is one page with anchors | 10 | | Reference → CLI | new `contracts/cli.json` from the clap tree (test `cli_contract` fails when stale) | 27 | | Reference → Actions and MCP tools | `contracts/actions.json`, one page per API area, each action links to its HTTP API page | 25 | | Reference → HTTP API | `contracts/openapi.json` via starlight-openapi (area-tagged copy in `.cache/`, so untagged operations get pages) | 336 | | Reference → TypeScript packages | TSDoc in ui / editor / api-client (svelte2tsx declarations → TypeDoc), one page per package | 5 | | Reference → Rust crates | `///` / `//!` doc comments of the 26 library crates, one page per public module | 86 | Total 498 pages, dist 43 MB. ### Tools (all AGPL-compatible) astro 7.3.5 (MIT), @astrojs/starlight 0.42.4 (MIT), starlight-typedoc 0.23.1 (MIT), typedoc 0.28.20 (Apache-2.0), typedoc-plugin-markdown 4.13.1 (MIT), starlight-openapi 0.26.3 (MIT), starlight-links-validator 0.26.0 (MIT), svelte2tsx 0.7.61 (MIT), satteri 0.10.5 (MIT), github-slugger 2.0.0 (ISC). **Rust: source scanner instead of rustdoc.** `cargo doc` HTML would split navigation and search. rustdoc JSON needs nightly flags (format changes per release) and type-checks every dependency: minutes and GBs on a clean checkout, and a Rust toolchain in the docs build. The scanner (`apps/docs/src/reference/rust-scan.ts`) lexes the crates in about 1 s, follows the module tree, applies visibility (`pub` reachable from the root, `pub use` re-exports inlined, no `pub(crate)`, `#[cfg(test)]`, `#[doc(hidden)]`), and resolves intra-doc links to anchors. Limits: no macro expansion and no cross-crate type resolution. Reasoning is in `apps/docs/README.md`. ### Gates - `bun run --cwd apps/docs build` from a fresh clone: `bun install` 4 s, build 34 s (`498 page(s) built in 30.87s`, `All internal links are valid.`). A warm rebuild takes about 28 s. - Link checker is on (starlight-links-validator). It found real breakage on the way (relative `.md` links, prose examples such as `Accounts.md#^amex`, the anchors in cli-walkthrough). All of it is fixed by the loader. - Doc comment ratchet `apps/docs/doc-coverage.json`: **rust 411, typescript 211** undocumented public items today. The build fails when a count grows and names the new items. I checked it on a clean clone by removing the doc comment of `RelPath`: `Rust: 412 public items without a doc comment, baseline 411. Add doc comments. New undocumented items: calternal_path::RelPath (crates/calternal-path/src/lib.rs:14)` - `bun run --cwd apps/docs check`: `0 errors, 0 warnings, 0 hints`. `bun run --cwd apps/docs test`: `24 pass, 0 fail`. - `cargo fmt --check -p calternal-cli` clean. `cargo clippy -p calternal-cli --all-targets -- -D warnings` clean. `cargo test -p calternal-cli`: `37 passed` and `15 passed`. ### Bug found on the way `calternal mail remote-content <id> <allow>` declared a positional `bool` without a value action, which clap rejects (it panics with a debug assertion, and a release build cannot parse the value). Fixed with `ArgAction::Set`. A `Cli::command().debug_assert()` test now covers the whole tree. ### Size Starlight puts the full sidebar into every page. With ~500 reference links the first build was 182 MB. `src/sidebar-fold.ts` folds each reference group that does not hold the current page into one link (home page 230 KB → 24 KB). TypeDoc writes one page per package. The generated `paths`/`components` OpenAPI types are left out of the TypeScript reference because the HTTP API section already covers them (2.6 MB page). ### Screenshots (production build) | | Light | Dark | |---|---|---| | Home | ![](https://git.kayg.org/attachments/85b7c535-ba13-4682-8257-86bd65ae0787) | ![](https://git.kayg.org/attachments/a8929c5d-5c1c-4c79-9f2e-e64d1dfe50bf) | | TypeScript: @calternal/ui | ![](https://git.kayg.org/attachments/d784bef6-1a2b-4ef8-992b-c7ef65bfd248) | ![](https://git.kayg.org/attachments/f27f2889-3c08-44e0-a665-0c029be37194) | | HTTP API: get_note | ![](https://git.kayg.org/attachments/aa32e1ca-c6ea-46be-86f3-7aa8905c2d62) | ![](https://git.kayg.org/attachments/a42547d8-ac04-4794-a5f2-3f3115b7263b) | | Rust: calternal-path | ![](https://git.kayg.org/attachments/5e6833b7-769d-4bd2-aa12-10e810ba8980) | ![](https://git.kayg.org/attachments/ff2b77f2-389e-47ec-ba95-1ac94dbf260b) | ### Hosting at docs.calternal.com (prepared, not done) The files are `deploy/docs/{Containerfile,Caddyfile,calternal-docs.container}` and `deploy/deploy-docs.sh`. They give a Caddy 2.11.6 image (pinned digest), an unprivileged user, a read-only root, no capabilities, a real 404 page, immutable cache on `/_astro/`, and a CSP. I tested the image locally under `--read-only --cap-drop=all --security-opt=no-new-privileges`: 200/404 are correct and Pagefind search works with 0 CSP errors. 1. Cloudflare zone `calternal.com`: A `docs → 168.119.145.115`, DNS only. 2. calternal-cloud VM: in `/etc/calternal-firewall.nft`, allow TCP 8081 from 10.70.3.101-103, then restart `calternal-firewall.service`. 3. homelab-private: add `k8s/hatsuna/calternal/docs.yaml`. The full manifest is in `deploy/docs/README.md`: Certificate, IngressRoute plus redirect that reuse `calternal-ratelimit`/`calternal-headers`, and a selector-less Service + EndpointSlice to 10.70.3.162:8081. Argo app `calternal` syncs it. 4. Run `deploy/deploy-docs.sh` on the build VM. Run it again with each release. ### Open points for the owner - The Design section publishes `docs/DESIGN.md` as it is. It names other products (quality bars, references). Point 5 of this issue says docs never name competing products. Do you want to rewrite those lines, or keep DESIGN out of the public site? - OpenAPI operations have no summaries (handlers carry no utoipa doc comments), so HTTP API pages are titled by operation ID. Server doc comments would improve the pages and the action help text. This is a separate issue if wanted.

kayg referenced this issue

2026-10-03 06:35:31 +00:00

DOCS: summaries for every HTTP API operation (doc comments on handlers) #974

kayg referenced this issue from a commit

2026-10-03 13:14:50 +00:00

feat(docs): Starlight site from docs/ plus reference generated from code (#971)

kayg referenced this issue from a commit

2026-10-03 13:14:50 +00:00

fix(docs): list the TypeScript package pages in the sidebar (#971)

kayg referenced this issue

2026-10-05 12:38:33 +00:00

Host docs.calternal.com on Cloudflare Pages; add export guides #1143