Audit: test coverage gaps (routes without e2e or adversarial coverage; tests that assert internals)

kayg commented

2026-10-02 16:47:37 +00:00

Owner

Owner rules: e2e does the flow like a User with real inputs and asserts what they see; every API route is in tests/adversarial and classified in the cross-User matrix. Map contracts/openapi.json routes and apps/web/src/routes pages against apps/web/e2e/.mjs and tests/adversarial/. List: routes with no adversarial probe; user flows with no e2e; e2e that assert implementation details instead of visible effects; tests that were weakened (git log -S for changed expectations in the last 2 weeks). File issues grouped per area.

Read-only job (LIGHT class)

This job runs at the lowest CPU priority on a shared, overloaded build host. Do not build or test: no cargo build/test/clippy/check, no bun install/build/test/run check, no servers, no browsers. Use rg, sed, git, scripts/fj, and reading. Write findings to audit-findings.md at the worktree root as you go (append per finding) and commit it on your branch. For each real defect: search existing issues first (scripts/fj --host https://git.kayg.org issue search --repo kayg/calternal "<terms>"), then either add evidence to the existing issue or create one self-contained issue (title, evidence with file:line, owner rule or DESIGN section, expected behaviour, test idea) that a later build job can own. Group findings that share one fix into one issue (one owner per shared fix). No product code changes. Post a summary with the list of issues on your own issue. Docs use ASD-STE100 Simplified Technical English.

Owner rules: e2e does the flow like a User with real inputs and asserts what they see; every API route is in tests/adversarial and classified in the cross-User matrix. Map contracts/openapi.json routes and apps/web/src/routes pages against apps/web/e2e/*.mjs and tests/adversarial/*. List: routes with no adversarial probe; user flows with no e2e; e2e that assert implementation details instead of visible effects; tests that were weakened (git log -S for changed expectations in the last 2 weeks). File issues grouped per area. ## Read-only job (LIGHT class) This job runs at the lowest CPU priority on a shared, overloaded build host. **Do not build or test**: no `cargo build/test/clippy/check`, no `bun install/build/test/run check`, no servers, no browsers. Use `rg`, `sed`, `git`, `scripts/fj`, and reading. Write findings to `audit-findings.md` at the worktree root as you go (append per finding) and commit it on your branch. For each real defect: search existing issues first (`scripts/fj --host https://git.kayg.org issue search --repo kayg/calternal "<terms>"`), then either add evidence to the existing issue or create one self-contained issue (title, evidence with file:line, owner rule or DESIGN section, expected behaviour, test idea) that a later build job can own. Group findings that share one fix into one issue (one owner per shared fix). No product code changes. Post a summary with the list of issues on your own issue. Docs use ASD-STE100 Simplified Technical English.

kayg commented

2026-10-02 16:57:15 +00:00

Author

Owner

Starting read-only coverage audit on branch job/test-gaps, based on origin/dev at 440e19dce23040ac8ebaae88f0469b6535b1afcb. I will map OpenAPI routes and web pages against adversarial probes and e2e flows, review visible-effect assertions and recent expectation changes, and record findings in audit-findings.md. No builds, tests, servers, or browsers will run.

Starting read-only coverage audit on branch `job/test-gaps`, based on `origin/dev` at `440e19dce23040ac8ebaae88f0469b6535b1afcb`. I will map OpenAPI routes and web pages against adversarial probes and e2e flows, review visible-effect assertions and recent expectation changes, and record findings in `audit-findings.md`. No builds, tests, servers, or browsers will run.

kayg referenced this issue

2026-10-02 17:34:07 +00:00

TESTS: cover Calendar subscription mutations across Users and in Settings #888

kayg referenced this issue

2026-10-02 17:34:23 +00:00

E2E: exercise Mail threads, attachments and sender categories as a User #889

kayg referenced this issue

2026-10-02 17:34:39 +00:00

E2E: assert visible Photos zoom effects instead of storage and class names #890

kayg referenced this issue

2026-10-02 17:34:58 +00:00

TESTS: require the documented Money toggle success status #891

kayg referenced this issue

2026-10-02 17:35:20 +00:00

Audit: deep links and Copy link coverage for every item type #861

kayg commented

2026-10-02 17:36:35 +00:00

Author

Owner

Finding: all 335 OpenAPI operations appear in the static authz and identity-classification loops. The seeded cross-User replay explicitly skips Calendar subscription update, remove and refresh because it has no reachable feed fixture (tests/adversarial/xuser_matrix.py:166-170). Tracked in #888.

kayg commented

2026-10-02 17:36:43 +00:00

Author

Owner

Finding: current Mail E2E proves connect, sync, folder listing and reload, but it does not open a message or use thread attachments and sender-category actions (apps/web/e2e/mail-sync-613.mjs:185-208). Tracked in #889; existing reader issues #726 and #771 cover related HTML/privacy and unread-count behavior.

kayg commented

2026-10-02 17:36:51 +00:00

Author

Owner

Finding: the Photos motion E2E checks the local-storage zoom key after Ctrl-scroll, and the Settings E2E checks a parent CSS class (apps/web/e2e/motion-spring-evidence.mjs:445-468; apps/web/e2e/auth.mjs:138-147). Tracked in #890.

kayg commented

2026-10-02 17:36:55 +00:00

Author

Owner

Finding: git log -S found a6fd7f704 (2026-10-01), which widened the Money fixture check to accept HTTP 200 or 204. The current OpenAPI contract and handler declare 204 only (contracts/openapi.json:12140-12173; crates/calternal-server/src/main.rs:620-655). Tracked in #891.

Finding: git log -S found a6fd7f704 (2026-10-01), which widened the Money fixture check to accept HTTP 200 or 204. The current OpenAPI contract and handler declare 204 only (contracts/openapi.json:12140-12173; crates/calternal-server/src/main.rs:620-655). Tracked in #891.

kayg commented

2026-10-02 17:37:03 +00:00

Author

Owner

Finding: Calendar subscription Settings actions are not exercised by the browser test, which creates the subscription through the API (apps/web/e2e/calendar-feeds.mjs:185-191). This shares the live feed fixture gap and is tracked in #888. The /tags index is also not traversed as a User; I added its evidence to existing deep-link issue #861.

kayg commented

2026-10-02 17:39:46 +00:00

Author

Owner

Finding: the /tags index has no E2E that selects a tag row or copies its link. I added evidence to existing issue #861 instead of opening a duplicate.

kayg commented

2026-10-02 17:41:32 +00:00

Author

Owner

Audit complete

Commit: ccca413c9d1d5d74fdedf6ef955b134ba23d2d89

The audit record is audit-findings.md. It maps all 335 OpenAPI operations to the static adversarial inventories and maps the web route families to their E2E files. It found three Calendar subscription mutations with no seeded cross-User replay, missing Calendar Settings and Mail reader user flows, a missing /tags index flow, two E2E assertions tied to storage or CSS details, and one widened Money status check.

Filed or updated issues:

#888 Calendar subscription cross-User and Settings E2E coverage.
#889 Mail thread, attachment and sender-category E2E coverage.
#890 visible Photos zoom assertions and the Settings class assertion.
#891 exact Money toggle status.
Added /tags index evidence to #861.

Known gaps: the live adversarial matrix and E2E suite were not run. No browser screenshots were captured. The build/test restriction prevented runtime confirmation.

Decisions: none. This source-only audit required no product or design choice.

Gate output: not run by instruction. The read-only job rule says: “Do not build or test: no cargo build/test/clippy/check, no bun install/build/test/run check, no servers, no browsers.” cargo clean output was: Removed 1 file, 356B total.

## Audit complete Commit: `ccca413c9d1d5d74fdedf6ef955b134ba23d2d89` The audit record is `audit-findings.md`. It maps all 335 OpenAPI operations to the static adversarial inventories and maps the web route families to their E2E files. It found three Calendar subscription mutations with no seeded cross-User replay, missing Calendar Settings and Mail reader user flows, a missing `/tags` index flow, two E2E assertions tied to storage or CSS details, and one widened Money status check. Filed or updated issues: - #888 Calendar subscription cross-User and Settings E2E coverage. - #889 Mail thread, attachment and sender-category E2E coverage. - #890 visible Photos zoom assertions and the Settings class assertion. - #891 exact Money toggle status. - Added `/tags` index evidence to #861. Known gaps: the live adversarial matrix and E2E suite were not run. No browser screenshots were captured. The build/test restriction prevented runtime confirmation. Decisions: none. This source-only audit required no product or design choice. Gate output: not run by instruction. The read-only job rule says: “Do not build or test: no `cargo build/test/clippy/check`, no `bun install/build/test/run check`, no servers, no browsers.” `cargo clean` output was: ` Removed 1 file, 356B total`.

kayg referenced this issue from a commit

2026-10-02 17:52:14 +00:00

docs(test): record coverage audit findings (#865)

kayg commented

2026-10-02 19:56:19 +00:00

Author

Owner

Starting #865 test-gap work on job/testgaps, based on 440e19dce23040ac8ebaae88f0469b6535b1afcb (one commit behind origin/dev). I am covering #888–#891 with test-only changes unless a test exposes a real product bug.

Starting #865 test-gap work on `job/testgaps`, based on `440e19dce23040ac8ebaae88f0469b6535b1afcb` (one commit behind `origin/dev`). I am covering #888–#891 with test-only changes unless a test exposes a real product bug.

kayg referenced this issue

2026-10-02 20:04:51 +00:00

TESTS: require the documented Money toggle success status #891

kayg commented

2026-10-02 20:04:51 +00:00

Author

Owner

Closed the #891 test gap in commit 0a2a79f79. tests/adversarial/xuser_matrix.py now requires exactly HTTP 204 when enabling Money for Users A and B, matching the OpenAPI contract and route handler.

Closed the #891 test gap in commit `0a2a79f79`. `tests/adversarial/xuser_matrix.py` now requires exactly HTTP 204 when enabling Money for Users A and B, matching the OpenAPI contract and route handler.

kayg commented

2026-10-02 20:20:58 +00:00

Author

Owner

Closed the #888 gaps in commit ab7077b40. Added live cross-User Calendar subscription mutation rows with a reachable iCalendar fixture and a Settings user flow for add, colour, visibility, refresh and remove. Browser and adversarial execution is deferred because the shared host load was high; exact commands are on #888 and in the final report.

Closed the #888 gaps in commit `ab7077b40`. Added live cross-User Calendar subscription mutation rows with a reachable iCalendar fixture and a Settings user flow for add, colour, visibility, refresh and remove. Browser and adversarial execution is deferred because the shared host load was high; exact commands are on #888 and in the final report.

kayg commented

2026-10-02 21:07:51 +00:00

Author

Owner

#865 test gaps are implemented on job/testgaps. I merged origin/dev once before final checks. Head: d3f2f8bdfaa3d5adac9bde6f32e030b97035d1ee.

Commits:

0a2a79f79 — require Money enable to return the documented 204.
ab7077b40 — add Calendar subscription ownership matrix rows and Settings E2E.
a1c0b7ce0 — add Mail thread, attachment and sender-category E2E.
3b96113a7 — assert visible Photos zoom effects.
202579e13 — merge origin/dev (the merge updates docs/DESIGN.md).
514f853d5, d3f2f8bdf — document the macOS screenshot contexts and Calendar fixture counter.

Files changed for #865:

tests/adversarial/xuser_matrix.py
tests/adversarial/test_xuser_classification.py
tests/adversarial/mail_sync_provider.py
apps/web/e2e/calendar-feeds.mjs
apps/web/e2e/mail-sync-613.mjs
apps/web/e2e/motion-spring-evidence.mjs
apps/web/e2e/photos.mjs

What changed:

Calendar's live cross-User matrix now creates a real A-owned URL subscription, checks update/remove/refresh against B, C, D, Admin and anonymous using both A's ID and a missing ID, and verifies no upstream fetch or change to A's saved subscription. The Settings E2E adds, recolours, hides, refreshes, restores by touch, and removes a subscription while checking the visible Calendar layer. Screenshots use macOS emulation at 390/820/1440 px in light and dark.
The TLS Dovecot fixture now has a two-message RFC thread and a MIME attachment. Mail E2E opens the thread, expands the older reply, saves the attachment from the keyboard into Files, changes the sender category through the Mail menu, checks persistence and the Newsletters view, and opens the thread/attachment list by touch. Screenshot contexts use macOS emulation at all required widths and themes.
Photos zoom E2E now checks rendered tile height and the count of tiles in the visible timeline region. Command-scroll must enlarge tiles, show fewer tiles, then restore the prior scale and visible count. It no longer treats storage state or menu state as proof of the visible effect.
The Money fixture now accepts exactly 204, matching the documented success response.

Verification output:

bun run check (from apps/web):

$ node scripts/check-user-storage.mjs && node scripts/check-type-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json
User browser caches use userStorage; only documented device/public-link exceptions remain.
Text sizes and UI shape values use shared role tokens.
UI transitions and animation options use shared motion tokens or documented exceptions.
Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/testgaps/apps/web
Getting Svelte diagnostics...

svelte-check found 0 errors and 0 warnings

python3 tests/adversarial/test_xuser_classification.py:

........
----------------------------------------------------------------------
Ran 8 tests in 1.466s

OK

node --check on all four changed E2E files: exit 0, no output. Python AST parse for the three changed adversarial Python files: exit 0, no output. git diff --check: exit 0, no output. No Rust source changed, so Rust format, clippy and crate tests were not run. cargo clean output: Removed 1 file, 356B total. apps/web/build was removed. bun install --frozen-lockfile --filter '@calternal/web' installed the locked dependencies; bun.lock is unchanged.

The server-backed Calendar/Mail/Photos E2E and live XUser matrix were not run. The last host load snapshot was 56.79, 61.39, 64.05, so I’m listing the exact merge-round runs:

From the repository root:

CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=4 cargo build -p calternal-server

From apps/web after the server build:

bun run build
bun e2e/calendar-feeds.mjs
env -u CALTERNAL_SERVER_BIN bun e2e/mail-sync-613.mjs
bun e2e/motion-spring-evidence.mjs
bun e2e/photos.mjs --screenshots ../../artifacts/photos-890

From the repository root:

XUSER_MATRIX_ONLY=1 tests/adversarial/run.sh

Decisions not specified in DESIGN: I used two synthetic Northstar Accounts messages with RFC reply headers and invoice-2026.txt as the Mail fixture; the category flow assigns Newsletters. The Calendar matrix hosts one bounded local .ics fixture on the existing adversarial DNS-shim port and uses signed-in 404 versus anonymous 401 as the expected missing/foreign-ID result. Photos defines the visible region as rendered .tile elements intersecting both the timeline bounds and viewport.

UX gaps closed: the new flows exercise Calendar Settings by pointer, keyboard and touch, Mail thread/attachment/category by keyboard, pointer and touch, and verify server-backed changes in their visible destinations. Photos asserts the actual visible zoom result. UX gaps left: browser flows and Mac screenshot review remain unverified until the merge-round commands run. No product code changed.

#865 test gaps are implemented on `job/testgaps`. I merged `origin/dev` once before final checks. Head: `d3f2f8bdfaa3d5adac9bde6f32e030b97035d1ee`. Commits: - `0a2a79f79` — require Money enable to return the documented `204`. - `ab7077b40` — add Calendar subscription ownership matrix rows and Settings E2E. - `a1c0b7ce0` — add Mail thread, attachment and sender-category E2E. - `3b96113a7` — assert visible Photos zoom effects. - `202579e13` — merge `origin/dev` (the merge updates `docs/DESIGN.md`). - `514f853d5`, `d3f2f8bdf` — document the macOS screenshot contexts and Calendar fixture counter. Files changed for #865: - `tests/adversarial/xuser_matrix.py` - `tests/adversarial/test_xuser_classification.py` - `tests/adversarial/mail_sync_provider.py` - `apps/web/e2e/calendar-feeds.mjs` - `apps/web/e2e/mail-sync-613.mjs` - `apps/web/e2e/motion-spring-evidence.mjs` - `apps/web/e2e/photos.mjs` What changed: - Calendar's live cross-User matrix now creates a real A-owned URL subscription, checks update/remove/refresh against B, C, D, Admin and anonymous using both A's ID and a missing ID, and verifies no upstream fetch or change to A's saved subscription. The Settings E2E adds, recolours, hides, refreshes, restores by touch, and removes a subscription while checking the visible Calendar layer. Screenshots use macOS emulation at 390/820/1440 px in light and dark. - The TLS Dovecot fixture now has a two-message RFC thread and a MIME attachment. Mail E2E opens the thread, expands the older reply, saves the attachment from the keyboard into Files, changes the sender category through the Mail menu, checks persistence and the Newsletters view, and opens the thread/attachment list by touch. Screenshot contexts use macOS emulation at all required widths and themes. - Photos zoom E2E now checks rendered tile height and the count of tiles in the visible timeline region. Command-scroll must enlarge tiles, show fewer tiles, then restore the prior scale and visible count. It no longer treats storage state or menu state as proof of the visible effect. - The Money fixture now accepts exactly `204`, matching the documented success response. Verification output: `bun run check` (from `apps/web`): ```text $ node scripts/check-user-storage.mjs && node scripts/check-type-tokens.mjs && node scripts/check-motion-tokens.mjs && svelte-kit sync && svelte-check --tsconfig ./tsconfig.json User browser caches use userStorage; only documented device/public-link exceptions remain. Text sizes and UI shape values use shared role tokens. UI transitions and animation options use shared motion tokens or documented exceptions. Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/testgaps/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings ``` `python3 tests/adversarial/test_xuser_classification.py`: ```text ........ ---------------------------------------------------------------------- Ran 8 tests in 1.466s OK ``` `node --check` on all four changed E2E files: exit `0`, no output. Python AST parse for the three changed adversarial Python files: exit `0`, no output. `git diff --check`: exit `0`, no output. No Rust source changed, so Rust format, clippy and crate tests were not run. `cargo clean` output: `Removed 1 file, 356B total`. `apps/web/build` was removed. `bun install --frozen-lockfile --filter '@calternal/web'` installed the locked dependencies; `bun.lock` is unchanged. The server-backed Calendar/Mail/Photos E2E and live XUser matrix were not run. The last host load snapshot was `56.79, 61.39, 64.05`, so I’m listing the exact merge-round runs: From the repository root: ```sh CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=4 cargo build -p calternal-server ``` From `apps/web` after the server build: ```sh bun run build bun e2e/calendar-feeds.mjs env -u CALTERNAL_SERVER_BIN bun e2e/mail-sync-613.mjs bun e2e/motion-spring-evidence.mjs bun e2e/photos.mjs --screenshots ../../artifacts/photos-890 ``` From the repository root: ```sh XUSER_MATRIX_ONLY=1 tests/adversarial/run.sh ``` Decisions not specified in DESIGN: I used two synthetic `Northstar Accounts` messages with RFC reply headers and `invoice-2026.txt` as the Mail fixture; the category flow assigns `Newsletters`. The Calendar matrix hosts one bounded local `.ics` fixture on the existing adversarial DNS-shim port and uses signed-in `404` versus anonymous `401` as the expected missing/foreign-ID result. Photos defines the visible region as rendered `.tile` elements intersecting both the timeline bounds and viewport. UX gaps closed: the new flows exercise Calendar Settings by pointer, keyboard and touch, Mail thread/attachment/category by keyboard, pointer and touch, and verify server-backed changes in their visible destinations. Photos asserts the actual visible zoom result. UX gaps left: browser flows and Mac screenshot review remain unverified until the merge-round commands run. No product code changed.

Rows
Columns

Audit: test coverage gaps (routes without e2e or adversarial coverage; tests that assert internals) #865

Read-only job (LIGHT class)

Audit complete