kayg commented

Owner

Goal (owner decision 2026-10-02, #858)

Apple Notes (and later Mail clients) reach calternal IMAP on the public internet at calternal.cloud:993 and dev.calternal.com:993, secured with our Let's Encrypt certificate, through the same edge as HTTP.

Edge (done by the orchestrator)

Proxmox DNAT public 993/465 → k3s nodes (owner added; k3s_dnat_ports in homelab-private).
Traefik entry points imaps (:993) and submissions (:465) (homelab-private 16238a4, Argo-managed). externalTrafficPolicy: Local keeps the client IP at Traefik.
To do after this issue lands: IngressRouteTCP per host with TLS terminated at Traefik (cert-manager secrets calternal-tls, calternal-staging-tls), forwarding to the VM with PROXY protocol v2.

Server change (this issue)

crates/calternal-server/src/notes_imap.rs today only starts implicit-TLS listeners with certificate files in .system/secrets/. Add an edge mode:

A plain-TCP listener (edge_bind, edge_submission_bind, off by default) that accepts connections only from CALTERNAL_SERVER__TRUSTED_PROXIES and requires a PROXY protocol v2 (or v1) header on every connection; the header's source address becomes the client IP for every per-IP limit, rate limit, login-attempt limit and log line. Connections from other addresses, or without a valid header, are closed before any IMAP byte is sent.
Edge mode never runs without TLS somewhere: document that it is only for a TLS-terminating proxy, and refuse to start it if TRUSTED_PROXIES is empty.
The managed Apple profile advertises port 993 and 465 with SSL on for the public hostname.
Deploy: the Quadlet publishes the edge ports on the VM LAN address (rootless: ports ≥ 1024, for example 10993/10465), the VM firewall allows them from the k3s nodes only (same pattern as 8080).

Tests

Unit: PROXY v1/v2 parsing; missing/garbled header; untrusted peer; spoofed header from an untrusted peer is ignored (connection closed).
Integration: per-IP connection limit counts the PROXY source, not the proxy address; login rate limit per real client.
Adversarial: add the edge listener to tests/adversarial/ (oversized header, slowloris before the header, header then garbage).

## Goal (owner decision 2026-10-02, #858) Apple Notes (and later Mail clients) reach calternal IMAP on the public internet at `calternal.cloud:993` and `dev.calternal.com:993`, secured with our Let's Encrypt certificate, through the same edge as HTTP. ## Edge (done by the orchestrator) - Proxmox DNAT public 993/465 → k3s nodes (owner added; `k3s_dnat_ports` in homelab-private). - Traefik entry points `imaps` (:993) and `submissions` (:465) (homelab-private 16238a4, Argo-managed). `externalTrafficPolicy: Local` keeps the client IP at Traefik. - To do after this issue lands: IngressRouteTCP per host with TLS terminated at Traefik (cert-manager secrets `calternal-tls`, `calternal-staging-tls`), forwarding to the VM with PROXY protocol v2. ## Server change (this issue) `crates/calternal-server/src/notes_imap.rs` today only starts implicit-TLS listeners with certificate files in `.system/secrets/`. Add an **edge mode**: 1. A plain-TCP listener (`edge_bind`, `edge_submission_bind`, off by default) that accepts connections **only** from `CALTERNAL_SERVER__TRUSTED_PROXIES` and **requires** a PROXY protocol v2 (or v1) header on every connection; the header's source address becomes the client IP for every per-IP limit, rate limit, login-attempt limit and log line. Connections from other addresses, or without a valid header, are closed before any IMAP byte is sent. 2. Edge mode never runs without TLS somewhere: document that it is only for a TLS-terminating proxy, and refuse to start it if `TRUSTED_PROXIES` is empty. 3. The managed Apple profile advertises port 993 and 465 with SSL on for the public hostname. 4. Deploy: the Quadlet publishes the edge ports on the VM LAN address (rootless: ports ≥ 1024, for example 10993/10465), the VM firewall allows them from the k3s nodes only (same pattern as 8080). ## Tests - Unit: PROXY v1/v2 parsing; missing/garbled header; untrusted peer; spoofed header from an untrusted peer is ignored (connection closed). - Integration: per-IP connection limit counts the PROXY source, not the proxy address; login rate limit per real client. - Adversarial: add the edge listener to `tests/adversarial/` (oversized header, slowloris before the header, header then garbage).

kayg commented

2026-10-02 19:42:49 +00:00

Author

Owner

Edge status 2026-10-02 20:20 CEST: public 993 and 465 now reach Traefik on both compute01 (calternal.cloud) and backup01 (dev.calternal.com) — DNAT by the owner, entry points in homelab-private 16238a4, Traefik temporarily 2 replicas (c3f3161) because k3s-02 is NotReady. Remaining: the server edge mode (this issue), then the orchestrator adds IngressRouteTCP + PROXY protocol v2 to the VM.

kayg commented

2026-10-03 06:12:18 +00:00

Author

Owner

Server edge mode implemented (job imapedge, branch `job/imapedge`, `6f098f569`)

Server (crates/calternal-server)

New proxy_protocol.rs: strict PROXY v1/v2 reader. It reads exactly the header bytes and nothing after them. Accepted: the PROXY command with TCP4/TCP6 only. Refused: v2 LOCAL, v1 UNKNOWN, UDP, Unix, bad version, v1 line > 107 bytes, v2 payload > 512 bytes, non-canonical ports, family mismatch. IPv4-mapped IPv6 sources are folded to IPv4.
notes_imap.rs: new config edge_bind, edge_submission_bind (off by default), tls_listeners (default true; false = edge only, no certificate files), edge_public_port / edge_public_submission_port (993/465).
- The server does not start if an edge listener is set and CALTERNAL_SERVER__TRUSTED_PROXIES is empty.
- An untrusted TCP peer is closed in the accept loop. This happens before a permit is taken and before any byte is read or sent.
- A trusted peer must send a valid header within 5 s. Otherwise the server closes the connection before any IMAP or SMTP byte.
- The header source is the client for the per-IP cap, notes-login: and notes-submission-commands: limits, the App Password usage coarse IP, and the log.
The IMAP listeners now enforce max_connections_per_ip (default 8). The field existed, but only the submission stub used it. TLS IMAP and edge IMAP share one pool, so a client cannot double its allowance.
The managed Apple profile advertises 993/465 with SSL when the edge is on.

Deploy

deploy/cloud/calternal-cloud.container: PublishPort=10.70.3.162:10993:10993 and :10465:10465.
calternal-cloud.env.example: NOTES_IMAP__TLS_LISTENERS=false, EDGE_BIND=0.0.0.0:10993, EDGE_SUBMISSION_BIND=0.0.0.0:10465.
deploy/cloud/README.md: the chain and the nft rule (8080, 10993 and 10465 from the k3s nodes only).

Orchestrator follow-up: add the 3 env lines to the real ~calternal/calternal-cloud.env and the nft rule on the VM. Add the IngressRouteTCP with TLS and PROXY v2 to 10.70.3.162:10993/10465. Staging (10.70.3.158, #858) needs the same.

Gates (calternal-server)

cargo fmt --check -p calternal-server        -> exit 0
cargo clippy -p calternal-server --all-targets -- -D warnings -> exit 0, Finished `dev` profile
cargo test -p calternal-server               -> test result: ok. 115 passed; 0 failed; 4 ignored
wire::tests::notes_edge_limits_use_the_proxy_source (live app, run by live_apps_run_in_separate_processes) -> ok

Adversarial (ADVERSARIAL_NOTES_IMAP_ONLY=1 tests/adversarial/run.sh, new notes_edge.py)

Notes IMAP probe: 0 finding(s)
Notes submission regressions: 0 finding(s)
Notes edge probe: 0 finding(s)
Notes submission budget boundaries: 0 finding(s)

The edge probe uses 127.0.0.2 as the only trusted proxy. It covers untrusted peers with forged headers and without headers, 16 malformed or oversized headers on both ports, slowloris before the header (1 byte/s) and a 100-socket slowloris storm, header then garbage, and a header split over many packets. It also checks that the per-IP cap keys on the PROXY source, including the IPv4-mapped form.

Two earlier runs failed when /readyz timed out (10 s) inside notes_imap.py, before the edge probe ran. The timeouts happened at different points, and host load was 16–24 on 12 cores. Nothing in this change touches HTTP or the DB. The run with load near 11 was clean. This is a host-load flake, not an edge defect.
The IDLE storm still prints a few ConnectionError. This also happened with a per-IP cap of 48, so it does not come from the new cap. The likely cause is the shared per-User budget of 240 commands per minute. That is existing behavior, and the probe only prints it.

## Server edge mode implemented (job imapedge, branch `job/imapedge`, 6f098f569) **Server (`crates/calternal-server`)** - New `proxy_protocol.rs`: strict PROXY v1/v2 reader. It reads exactly the header bytes and nothing after them. Accepted: the PROXY command with TCP4/TCP6 only. Refused: v2 LOCAL, v1 UNKNOWN, UDP, Unix, bad version, v1 line > 107 bytes, v2 payload > 512 bytes, non-canonical ports, family mismatch. IPv4-mapped IPv6 sources are folded to IPv4. - `notes_imap.rs`: new config `edge_bind`, `edge_submission_bind` (off by default), `tls_listeners` (default true; false = edge only, no certificate files), `edge_public_port` / `edge_public_submission_port` (993/465). - The server does not start if an edge listener is set and `CALTERNAL_SERVER__TRUSTED_PROXIES` is empty. - An untrusted TCP peer is closed in the accept loop. This happens before a permit is taken and before any byte is read or sent. - A trusted peer must send a valid header within 5 s. Otherwise the server closes the connection before any IMAP or SMTP byte. - The header source is the client for the per-IP cap, `notes-login:` and `notes-submission-commands:` limits, the App Password usage coarse IP, and the log. - The IMAP listeners now enforce `max_connections_per_ip` (default 8). The field existed, but only the submission stub used it. TLS IMAP and edge IMAP share one pool, so a client cannot double its allowance. - The managed Apple profile advertises 993/465 with SSL when the edge is on. **Deploy** - `deploy/cloud/calternal-cloud.container`: `PublishPort=10.70.3.162:10993:10993` and `:10465:10465`. - `calternal-cloud.env.example`: `NOTES_IMAP__TLS_LISTENERS=false`, `EDGE_BIND=0.0.0.0:10993`, `EDGE_SUBMISSION_BIND=0.0.0.0:10465`. - `deploy/cloud/README.md`: the chain and the nft rule (8080, 10993 and 10465 from the k3s nodes only). **Orchestrator follow-up:** add the 3 env lines to the real `~calternal/calternal-cloud.env` and the nft rule on the VM. Add the IngressRouteTCP with TLS and PROXY v2 to `10.70.3.162:10993/10465`. Staging (10.70.3.158, #858) needs the same. **Gates (calternal-server)** ``` cargo fmt --check -p calternal-server -> exit 0 cargo clippy -p calternal-server --all-targets -- -D warnings -> exit 0, Finished `dev` profile cargo test -p calternal-server -> test result: ok. 115 passed; 0 failed; 4 ignored wire::tests::notes_edge_limits_use_the_proxy_source (live app, run by live_apps_run_in_separate_processes) -> ok ``` **Adversarial** (`ADVERSARIAL_NOTES_IMAP_ONLY=1 tests/adversarial/run.sh`, new `notes_edge.py`) ``` Notes IMAP probe: 0 finding(s) Notes submission regressions: 0 finding(s) Notes edge probe: 0 finding(s) Notes submission budget boundaries: 0 finding(s) ``` The edge probe uses 127.0.0.2 as the only trusted proxy. It covers untrusted peers with forged headers and without headers, 16 malformed or oversized headers on both ports, slowloris before the header (1 byte/s) and a 100-socket slowloris storm, header then garbage, and a header split over many packets. It also checks that the per-IP cap keys on the PROXY source, including the IPv4-mapped form. Two earlier runs failed when `/readyz` timed out (10 s) inside notes_imap.py, before the edge probe ran. The timeouts happened at different points, and host load was 16–24 on 12 cores. Nothing in this change touches HTTP or the DB. The run with load near 11 was clean. This is a host-load flake, not an edge defect. The IDLE storm still prints a few `ConnectionError`. This also happened with a per-IP cap of 48, so it does not come from the new cap. The likely cause is the shared per-User budget of 240 commands per minute. That is existing behavior, and the probe only prints it.

kayg referenced this issue from a commit

2026-10-03 06:16:46 +00:00

feat(notes): Notes IMAP behind the TLS edge with PROXY protocol (#941)

kayg referenced this issue

2026-10-03 06:53:29 +00:00

Log attachments: deleted file still shown, attachments not live, batch send not live, Log order, raw camera names #427

kayg commented

2026-10-03 07:02:04 +00:00

Author

Owner

Resuming #941 on job/imapedge, starting head 6f098f569356fa5269fc17386ab3b4383e02d0e1 (existing implementation based on dev). The worktree is clean. Reviewing the existing edge transport and tests before final gates. Found that PROXY v1 ignores the destination address and v2 ignores TLV structure; both can accept malformed headers contrary to the valid-header boundary in #941. I will add regression tests and reject these inputs. No dependency changes are planned.

Resuming #941 on `job/imapedge`, starting head `6f098f569356fa5269fc17386ab3b4383e02d0e1` (existing implementation based on dev). The worktree is clean. Reviewing the existing edge transport and tests before final gates. Found that PROXY v1 ignores the destination address and v2 ignores TLV structure; both can accept malformed headers contrary to the valid-header boundary in #941. I will add regression tests and reject these inputs. No dependency changes are planned.

kayg commented

2026-10-03 07:09:12 +00:00

Author

Owner

Header validation regressions reproduced on the prior parser:

test wire::proxy_protocol::tests::v1_invalid_destination_fails ... FAILED
test wire::proxy_protocol::tests::v2_truncated_tlvs_fail ... FAILED
test result: FAILED. 6 passed; 2 failed; 0 ignored; 0 measured; 113 filtered out; finished in 0.00s

The v1 destination field was not parsed. The v2 payload was consumed, but trailing TLV lengths were not checked. The fix validates the v1 destination against its declared family and requires complete TLV frames, with no allocation or extension-value interpretation. Valid unknown and zero-length TLVs remain accepted. The live-app regression and both-port adversarial probe now cover these malformed headers. No existing test expectations changed.

Fetched origin and merged origin/dev once (merge 512ed6c98), with only DESIGN documentation changes. Server crate gates are running. The full adversarial round and live staging Apple Notes check remain for the merge round under the current verification policy.

Header validation regressions reproduced on the prior parser: ``` test wire::proxy_protocol::tests::v1_invalid_destination_fails ... FAILED test wire::proxy_protocol::tests::v2_truncated_tlvs_fail ... FAILED test result: FAILED. 6 passed; 2 failed; 0 ignored; 0 measured; 113 filtered out; finished in 0.00s ``` The v1 destination field was not parsed. The v2 payload was consumed, but trailing TLV lengths were not checked. The fix validates the v1 destination against its declared family and requires complete TLV frames, with no allocation or extension-value interpretation. Valid unknown and zero-length TLVs remain accepted. The live-app regression and both-port adversarial probe now cover these malformed headers. No existing test expectations changed. Fetched origin and merged `origin/dev` once (merge `512ed6c98`), with only DESIGN documentation changes. Server crate gates are running. The full adversarial round and live staging Apple Notes check remain for the merge round under the current verification policy.

kayg commented

2026-10-03 08:43:49 +00:00

Author

Owner

Resuming #941 on job/imapedge, base/head 4a1c62056a23bdac666cf1982523bc0a2ca29754. Review covers the PROXY parser checkpoint and the Notes checkpoint 7fcc93174 from job/staging-imap. I will keep justified changes in clean atomic commits, run the three requested crate gates, and deploy and verify staging only. No production changes or pushes.

Resuming #941 on `job/imapedge`, base/head `4a1c62056a23bdac666cf1982523bc0a2ca29754`. Review covers the PROXY parser checkpoint and the Notes checkpoint `7fcc93174` from `job/staging-imap`. I will keep justified changes in clean atomic commits, run the three requested crate gates, and deploy and verify staging only. No production changes or pushes.

kayg commented

2026-10-03 08:47:42 +00:00

Author

Owner

Checkpoint review: retain the bounded PROXY v1 destination and v2 TLV validation. Retain the Notes creation-Date base recording and date-only timestamp conversion on top of 7a. The Notes change preserves the existing immutable-base retention (64 entries) and does not advance an old Date to a new hash. Added a regression that a Mac-created base cannot overwrite an overlapping server edit. Corrected the snapshot doc comment moved by the WIP extraction. No existing test expectation changes or new dependencies.

Dependency integration: merged 7a (a0f94231b) for the existing three-way merge; kept both DESIGN additions (§58 and §60/61). Fetched origin and merged origin/dev once for this resume. Targeted gates are running.

Checkpoint review: retain the bounded PROXY v1 destination and v2 TLV validation. Retain the Notes creation-Date base recording and date-only timestamp conversion on top of 7a. The Notes change preserves the existing immutable-base retention (64 entries) and does not advance an old Date to a new hash. Added a regression that a Mac-created base cannot overwrite an overlapping server edit. Corrected the snapshot doc comment moved by the WIP extraction. No existing test expectation changes or new dependencies. Dependency integration: merged 7a (`a0f94231b`) for the existing three-way merge; kept both DESIGN additions (§58 and §60/61). Fetched origin and merged `origin/dev` once for this resume. Targeted gates are running.

kayg commented

2026-10-03 08:51:34 +00:00

Author

Owner

Before deployment, the staging endpoint (dev.calternal.com:993) validates its public TLS certificate and accepts LOGIN, LIST and SELECT using the existing lab App Password. The normal-client regression then APPENDs an empty Note, a title, and a body with one creation Date. FETCH of the current identity does not contain the last body (serial edit readback), reproducing the missing creation-base bug on the existing 7a+edge image. No credentials or server responses are included in evidence. The reviewed checkpoint records the initial immutable base so the existing 7a three-way merge can keep serial additions.

Before deployment, the staging endpoint (`dev.calternal.com:993`) validates its public TLS certificate and accepts LOGIN, LIST and SELECT using the existing lab App Password. The normal-client regression then APPENDs an empty Note, a title, and a body with one creation Date. FETCH of the current identity does not contain the last body (`serial edit readback`), reproducing the missing creation-base bug on the existing 7a+edge image. No credentials or server responses are included in evidence. The reviewed checkpoint records the initial immutable base so the existing 7a three-way merge can keep serial additions.

kayg commented

2026-10-03 08:59:13 +00:00

Author

Owner

Client-probe correction: the first probe used SEARCH HEADER (unsupported by this Notes bridge) and inspected encoded MIME bytes. That run was not valid readback evidence. The corrected probe uses SEARCH ALL plus BODY.PEEK[HEADER] to locate the UUID, then decodes the MIME HTML body. On the current staging build it still reproduces the serial-save bug:

TLS certificate, greeting and LOGIN: PASS
LIST and SELECT Notes: PASS
Staging Notes edge verification: FAIL (serial edit readback)

The corrected run cleaned up its random test identity and recovery copies through UID STORE/UID EXPUNGE. This replaces the prior readback evidence. The fixed image must pass this exact probe.

Client-probe correction: the first probe used SEARCH HEADER (unsupported by this Notes bridge) and inspected encoded MIME bytes. That run was not valid readback evidence. The corrected probe uses SEARCH ALL plus BODY.PEEK[HEADER] to locate the UUID, then decodes the MIME HTML body. On the current staging build it still reproduces the serial-save bug: ``` TLS certificate, greeting and LOGIN: PASS LIST and SELECT Notes: PASS Staging Notes edge verification: FAIL (serial edit readback) ``` The corrected run cleaned up its random test identity and recovery copies through UID STORE/UID EXPUNGE. This replaces the prior readback evidence. The fixed image must pass this exact probe.

kayg commented

2026-10-03 10:06:03 +00:00

Author

Owner

The Notes checkpoint's date-only regression failed unchanged:

test imap::tests::served_date_accepts_a_date_only_created_stamp ... FAILED

Root cause: NoteMeta.created reads the Note date key. The Task builder writes a literal created: YYYY-MM-DD key (calternal-notes-core/src/tasks/build.rs). Parsing a date-only value was necessary but did not read the Task field. The fix uses the existing extract_note_properties scalar reader only when last edited and Note date are absent. The shared metadata contract stays intact; the existing fixture and expected timestamp stay unchanged. A focused rerun will verify it.

The serial-save regression and the new overlapping-server-edit regression passed in the full Notes run. The full run also found an unchanged nested Daily note Journal 404, filed separately. No test expectations were relaxed.

The Notes checkpoint's date-only regression failed unchanged: ``` test imap::tests::served_date_accepts_a_date_only_created_stamp ... FAILED ``` Root cause: `NoteMeta.created` reads the Note `date` key. The Task builder writes a literal `created: YYYY-MM-DD` key (`calternal-notes-core/src/tasks/build.rs`). Parsing a date-only value was necessary but did not read the Task field. The fix uses the existing `extract_note_properties` scalar reader only when `last edited` and Note `date` are absent. The shared metadata contract stays intact; the existing fixture and expected timestamp stay unchanged. A focused rerun will verify it. The serial-save regression and the new overlapping-server-edit regression passed in the full Notes run. The full run also found an unchanged nested Daily note Journal 404, filed separately. No test expectations were relaxed.

kayg referenced this issue

2026-10-03 10:06:18 +00:00

Journal read returns 404 after a Log write to a nested Daily note #1005

kayg referenced this issue from a commit

2026-10-03 10:42:29 +00:00

docs(server): distinguish edge certificates from direct TLS configuration (#941)

kayg commented

2026-10-03 11:04:14 +00:00

Author

Owner

The final Notes Clippy check passed. All 25 focused IMAP tests passed, including the new serial-creation and concurrent-server-edit cases and the unchanged date-only fixture. The unrelated Journal failure from the full crate run (#1005) passes in isolation without any source or expectation change; keep #1005 open for suite-level investigation. Clean committed head: 325b86e6dfbdb804a477f8d566904902f06d21ff. The staging image is building; server tests are queued. No pushes or production changes.

test result: ok. 25 passed; 0 failed; 0 ignored; 0 measured; 161 filtered out; finished in 17.20s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 185 filtered out; finished in 0.67s

The final Notes Clippy check passed. All 25 focused IMAP tests passed, including the new serial-creation and concurrent-server-edit cases and the unchanged date-only fixture. The unrelated Journal failure from the full crate run (#1005) passes in isolation without any source or expectation change; keep #1005 open for suite-level investigation. Clean committed head: `325b86e6dfbdb804a477f8d566904902f06d21ff`. The staging image is building; server tests are queued. No pushes or production changes. ```text test result: ok. 25 passed; 0 failed; 0 ignored; 0 measured; 161 filtered out; finished in 17.20s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 185 filtered out; finished in 0.67s ```

kayg referenced this issue from a commit

2026-10-03 11:14:39 +00:00

fix(server): reject incomplete PROXY headers before the greeting (#941)

kayg referenced this issue from a commit

2026-10-03 11:14:39 +00:00

docs(server): distinguish edge certificates from direct TLS configuration (#941)

kayg referenced this issue from a commit

2026-10-03 11:14:39 +00:00

fix(notes): record immutable creation bases for serial Mac saves (#941)

kayg referenced this issue from a commit

2026-10-03 11:14:39 +00:00

fix(notes): serve date-only Task creation stamps in the User time zone (#941)

kayg commented

2026-10-03 11:36:28 +00:00

Author

Owner

The staging build is still compiling the server binary after the focused Notes checks and atomic commits. Its compiler process has been alive for about 16 minutes; a thread is waiting on filesystem pages. The build log has no compiler error. The server test gate is queued on the target-directory lock. No staging restart has happened yet. The final report will mark any check that does not finish within the job time limit as incomplete.

kayg commented

2026-10-03 12:03:15 +00:00

Author

Owner

Staging now runs 325b86e6dfbdb804a477f8d566904902f06d21ff (running container revision verified). Real imaplib with normal TLS certificate and hostname checks passed every lifecycle step; only this run's random Note identity was deleted.

TLS certificate, greeting and LOGIN: PASS
LIST and SELECT Notes: PASS
APPEND, serial edits and FETCH readback: PASS
UID STORE, UID EXPUNGE and deletion readback: PASS

The prescribed deploy helper exited 141 after reporting a successful image load. I verified the loaded image revision, then ran the helper's remaining staging-only tag/Quadlet/restart/health steps from a worktree artifact. I did not change the shared helper or retransmit the image. Staging health passed. Server tests are compiling; the conditional existing-profile Mac readback is running under the nonblocking lock.

Staging now runs `325b86e6dfbdb804a477f8d566904902f06d21ff` (running container revision verified). Real imaplib with normal TLS certificate and hostname checks passed every lifecycle step; only this run's random Note identity was deleted. ```text TLS certificate, greeting and LOGIN: PASS LIST and SELECT Notes: PASS APPEND, serial edits and FETCH readback: PASS UID STORE, UID EXPUNGE and deletion readback: PASS ``` The prescribed deploy helper exited 141 after reporting a successful image load. I verified the loaded image revision, then ran the helper's remaining staging-only tag/Quadlet/restart/health steps from a worktree artifact. I did not change the shared helper or retransmit the image. Staging health passed. Server tests are compiling; the conditional existing-profile Mac readback is running under the nonblocking lock.

kayg commented

2026-10-03 12:07:51 +00:00

Author

Owner

The nonblocking Mac lock succeeded. The existing Notes profile was used without configuration or account changes. The AppleScript readback timed out, so native-client readback is NOT CONFIRMED. The original IMAP cleanup connection failed after the wait; a fresh TLS connection removed the exact saved fixture UUID and verified that it was absent.

Mac fixture fresh-connection cleanup: PASS

Owner check left: open Notes with the existing lab profile, confirm its saved login works, create a Note, save two edits, and confirm text and deletion sync with staging. Handle any OS password prompt as the owner. The job does not treat the passing Python TLS client result as native-client proof.

The nonblocking Mac lock succeeded. The existing Notes profile was used without configuration or account changes. The AppleScript readback timed out, so native-client readback is NOT CONFIRMED. The original IMAP cleanup connection failed after the wait; a fresh TLS connection removed the exact saved fixture UUID and verified that it was absent. ```text Mac fixture fresh-connection cleanup: PASS ``` Owner check left: open Notes with the existing lab profile, confirm its saved login works, create a Note, save two edits, and confirm text and deletion sync with staging. Handle any OS password prompt as the owner. The job does not treat the passing Python TLS client result as native-client proof.

kayg commented

2026-10-03 12:35:22 +00:00

Author

Owner

#941 final report

Built: reviewed both checkpoints, kept strict PROXY v1 address-family and v2 extension framing checks, and extracted the existing bounded served-base ledger for initial client-created Notes. Serial saves now retain a proven creation-Date base. A concurrent server edit remains a recovery Note instead of being lost. Date-only Task creation stamps use local midnight in the User time zone. Added a real TLS imaplib staging lifecycle probe with decoded readback, recovery-copy detection, and scoped deletion.

Files: crates/calternal-server/src/proxy_protocol.rs, crates/calternal-server/src/wire.rs, crates/calternal-server/src/notes_imap.rs, crates/plugins/notes/src/imap.rs, tests/adversarial/notes_edge.py, tests/staging/notes_edge.py. The finished #941 transport/deployment configuration and the required 7a dependency were retained. No web source was changed. All touched doc comments were read again.

Head: 325b86e6dfbdb804a477f8d566904902f06d21ff. Branch job/imapedge. The inherited WIP message was replaced without changing any tested tree. Merged 7a and then origin/dev once. No push, production access, or homelab change.

Decisions

Record the initial stored conversion under the client's creation Date. Keep this base immutable; do not advance the same Date to the current source after each edit.
NoteMeta.created reads the Note date key. Task builders use literal created. Reuse the existing scalar reader as a local fallback without changing shared metadata behavior. Retain the checkpoint fixture and expected timestamp.
Use the unchanged 7a runtime image with the checked unoptimized server and embedded production web assets for staging functional checks. Optimized release packaging remains a merge-round gate. No performance claim is made.

Known gaps

The server test gate was started but did not finish within the job time limit. Its compiler was waiting on filesystem pages. The job stopped the local build processes before cleanup. No server test success is claimed. The merge round must run cargo test -p calternal-server -- --test-threads=4, including the PROXY framing and live-listener regressions.

The first full Notes test run reported an unchanged Journal 404 regression and the checkpoint's date-only failure. The date-only failure is fixed and its unchanged fixture passes in the focused run. The Journal case passes in isolation without any code or expectation change; #1005 remains open for suite-level investigation. Do not describe the full Notes gate as green.

UX gaps closed: serial client edits keep their text on the original Note; date-only creation stamps do not fall back to the Unix epoch. No UI component was changed.

For the merge round

cargo build --release --locked -p calternal-server: verify optimized image packaging from the final combined tree.
cargo test -p calternal-plugin-notes -- --test-threads=4: verify the combined tree's full Notes suite and investigate #1005.
ADVERSARIAL_NOTES_IMAP_ONLY=1 tests/adversarial/run.sh: verify trusted-peer enforcement, malformed headers, connection limits, and cross-protocol behavior on a real local server. Full adversarial matrices run once on the combined branch.
Full web/e2e and deployed build gates run once in the merge round. Full Apple client create/edit/delete interop uses the existing profile under the Mac lock. This job's limited readback does not replace it.
Performance measurement is deferred under the latest owner policy because #941 is not a performance issue. The existing Notes bridge profile covers the edit hot path.

Gates (verbatim summary output)

cargo fmt --check: exit 0, no output. No web source change; a production web build succeeded for embedded staging assets.

cargo clippy -p calternal-plugin-notes --all-targets -- -D warnings

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 41m 39s

cargo clippy -p calternal-imap --all-targets -- -D warnings

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 30s

cargo clippy -p calternal-server --all-targets -- -D warnings

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 45m 15s

cargo test -p calternal-plugin-notes -- --test-threads=4 (initial full run)

    Finished `test` profile [unoptimized + debuginfo] target(s) in 37m 57s
test result: FAILED. 184 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 437.18s

cargo test -p calternal-plugin-notes imap::tests:: -- --test-threads=4

    Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 58s
test result: ok. 25 passed; 0 failed; 0 ignored; 0 measured; 161 filtered out; finished in 17.20s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

cargo test -p calternal-plugin-notes tests::daily_and_composer_preserve_unrelated_bytes -- --exact --test-threads=4

    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.48s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 185 filtered out; finished in 0.67s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s

cargo test -p calternal-imap -- --test-threads=4

    Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 17s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-server -- --test-threads=4
Incomplete: no test result was recorded. Last build output follows.

   Compiling calternal-plugin-calendar v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/calendar)
   Compiling calternal-plugin-video v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/video)
   Compiling calternal-plugin-ai v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/ai)
   Compiling calternal-plugin-photos v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/photos)
   Compiling calternal-plugin-analytics v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/analytics)
   Compiling calternal-plugin-notifications v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/notifications)

Staging verification

Image localhost/calternal-cloud:imapedge-941-325b86e6d, image ID d659824509f6db330bd3596daf992031594ec096243242f3754e892429589257. The running container's revision matches the reported head. TLS certificate and hostname validation were enabled on the real Python client.

TLS certificate, greeting and LOGIN: PASS
LIST and SELECT Notes: PASS
APPEND, serial edits and FETCH readback: PASS
UID STORE, UID EXPUNGE and deletion readback: PASS

The prescribed deploy helper exited 141 after the image load completed. The loaded image revision was verified, then the helper's remaining staging-only tag, Quadlet, restart and health steps were completed without retransmitting the image. The shared helper was not changed.

Use verified image already loaded on staging: localhost/calternal-cloud:imapedge-941-325b86e6d
staging deployed localhost/calternal-cloud:imapedge-941-325b86e6d: healthy

Mac: the nonblocking lock succeeded and the existing profile was used. AppleScript readback timed out, so native-client readback is not confirmed. The fixture was removed through a fresh TLS IMAP connection and its absence was verified. No Mac configuration or account was changed.

Mac fixture fresh-connection cleanup: PASS

UX gaps left: full native-client create/edit/delete confirmation. Owner steps: open Notes with the existing lab profile, confirm its saved login works, create a Note, save two edits, and verify that text and deletion sync with staging. The owner must handle any OS password prompt.

Exact staging edge manifest

Existing manifest; no homelab edit or push.

---
apiVersion: traefik.io/v1alpha1
kind: ServersTransportTCP
metadata:
  name: calternal-staging-edge
  namespace: calternal
spec:
  dialTimeout: 10s
  proxyProtocol:
    version: 2

---
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: calternal-staging-imaps
  namespace: calternal
spec:
  entryPoints:
    - imaps
  routes:
    - match: HostSNI(`dev.calternal.com`)
      services:
        - name: calternal-staging-edge
          port: 10993
          serversTransport: calternal-staging-edge
  tls:
    secretName: calternal-staging-tls

---
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: calternal-staging-submissions
  namespace: calternal
spec:
  entryPoints:
    - submissions
  routes:
    - match: HostSNI(`dev.calternal.com`)
      services:
        - name: calternal-staging-edge
          port: 10465
          serversTransport: calternal-staging-edge
  tls:
    secretName: calternal-staging-tls

---
apiVersion: v1
kind: Service
metadata:
  name: calternal-staging-edge
  namespace: calternal
spec:
  ports:
    - name: imap
      port: 10993
      targetPort: 10993
    - name: submission
      port: 10465
      targetPort: 10465
  clusterIP: None

---
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: calternal-staging-edge-vm
  namespace: calternal
  labels:
    kubernetes.io/service-name: calternal-staging-edge
    endpointslice.kubernetes.io/managed-by: homelab-private
addressType: IPv4
ports:
  - name: imap
    port: 10993
    protocol: TCP
  - name: submission
    port: 10465
    protocol: TCP
endpoints:
  - addresses:
      - 10.70.3.158
    conditions:
      ready: true

Cleanup

     Removed 17111 files, 8.9GiB total

Generated web and copied deployment output were removed. Review logs and the exact manifest remain in artifacts/.

# #941 final report Built: reviewed both checkpoints, kept strict PROXY v1 address-family and v2 extension framing checks, and extracted the existing bounded served-base ledger for initial client-created Notes. Serial saves now retain a proven creation-Date base. A concurrent server edit remains a recovery Note instead of being lost. Date-only Task creation stamps use local midnight in the User time zone. Added a real TLS imaplib staging lifecycle probe with decoded readback, recovery-copy detection, and scoped deletion. Files: `crates/calternal-server/src/proxy_protocol.rs`, `crates/calternal-server/src/wire.rs`, `crates/calternal-server/src/notes_imap.rs`, `crates/plugins/notes/src/imap.rs`, `tests/adversarial/notes_edge.py`, `tests/staging/notes_edge.py`. The finished #941 transport/deployment configuration and the required 7a dependency were retained. No web source was changed. All touched doc comments were read again. Head: `325b86e6dfbdb804a477f8d566904902f06d21ff`. Branch `job/imapedge`. The inherited WIP message was replaced without changing any tested tree. Merged 7a and then `origin/dev` once. No push, production access, or homelab change. ## Decisions - Record the initial stored conversion under the client's creation Date. Keep this base immutable; do not advance the same Date to the current source after each edit. - `NoteMeta.created` reads the Note `date` key. Task builders use literal `created`. Reuse the existing scalar reader as a local fallback without changing shared metadata behavior. Retain the checkpoint fixture and expected timestamp. - Use the unchanged 7a runtime image with the checked unoptimized server and embedded production web assets for staging functional checks. Optimized release packaging remains a merge-round gate. No performance claim is made. ## Known gaps The server test gate was started but did not finish within the job time limit. Its compiler was waiting on filesystem pages. The job stopped the local build processes before cleanup. No server test success is claimed. The merge round must run `cargo test -p calternal-server -- --test-threads=4`, including the PROXY framing and live-listener regressions. The first full Notes test run reported an unchanged Journal 404 regression and the checkpoint's date-only failure. The date-only failure is fixed and its unchanged fixture passes in the focused run. The Journal case passes in isolation without any code or expectation change; #1005 remains open for suite-level investigation. Do not describe the full Notes gate as green. UX gaps closed: serial client edits keep their text on the original Note; date-only creation stamps do not fall back to the Unix epoch. No UI component was changed. ## For the merge round - `cargo build --release --locked -p calternal-server`: verify optimized image packaging from the final combined tree. - `cargo test -p calternal-plugin-notes -- --test-threads=4`: verify the combined tree's full Notes suite and investigate #1005. - `ADVERSARIAL_NOTES_IMAP_ONLY=1 tests/adversarial/run.sh`: verify trusted-peer enforcement, malformed headers, connection limits, and cross-protocol behavior on a real local server. Full adversarial matrices run once on the combined branch. - Full web/e2e and deployed build gates run once in the merge round. Full Apple client create/edit/delete interop uses the existing profile under the Mac lock. This job's limited readback does not replace it. - Performance measurement is deferred under the latest owner policy because #941 is not a performance issue. The existing Notes bridge profile covers the edit hot path. ## Gates (verbatim summary output) `cargo fmt --check`: exit 0, no output. No web source change; a production web build succeeded for embedded staging assets. cargo clippy -p calternal-plugin-notes --all-targets -- -D warnings ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 41m 39s ``` cargo clippy -p calternal-imap --all-targets -- -D warnings ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 30s ``` cargo clippy -p calternal-server --all-targets -- -D warnings ```text Finished `dev` profile [unoptimized + debuginfo] target(s) in 45m 15s ``` cargo test -p calternal-plugin-notes -- --test-threads=4 (initial full run) ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 37m 57s test result: FAILED. 184 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 437.18s ``` cargo test -p calternal-plugin-notes imap::tests:: -- --test-threads=4 ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 7m 58s test result: ok. 25 passed; 0 failed; 0 ignored; 0 measured; 161 filtered out; finished in 17.20s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s ``` cargo test -p calternal-plugin-notes tests::daily_and_composer_preserve_unrelated_bytes -- --exact --test-threads=4 ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 0.48s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 185 filtered out; finished in 0.67s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 1 filtered out; finished in 0.00s ``` cargo test -p calternal-imap -- --test-threads=4 ```text Finished `test` profile [unoptimized + debuginfo] target(s) in 4m 17s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` cargo test -p calternal-server -- --test-threads=4 Incomplete: no test result was recorded. Last build output follows. ```text Compiling calternal-plugin-calendar v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/calendar) Compiling calternal-plugin-video v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/video) Compiling calternal-plugin-ai v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/ai) Compiling calternal-plugin-photos v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/photos) Compiling calternal-plugin-analytics v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/analytics) Compiling calternal-plugin-notifications v0.0.1 (/home/kayg/Developer/calternal-wt/imapedge/crates/plugins/notifications) ``` ## Staging verification Image `localhost/calternal-cloud:imapedge-941-325b86e6d`, image ID `d659824509f6db330bd3596daf992031594ec096243242f3754e892429589257`. The running container's revision matches the reported head. TLS certificate and hostname validation were enabled on the real Python client. ```text TLS certificate, greeting and LOGIN: PASS LIST and SELECT Notes: PASS APPEND, serial edits and FETCH readback: PASS UID STORE, UID EXPUNGE and deletion readback: PASS ``` The prescribed deploy helper exited 141 after the image load completed. The loaded image revision was verified, then the helper's remaining staging-only tag, Quadlet, restart and health steps were completed without retransmitting the image. The shared helper was not changed. ```text Use verified image already loaded on staging: localhost/calternal-cloud:imapedge-941-325b86e6d staging deployed localhost/calternal-cloud:imapedge-941-325b86e6d: healthy ``` Mac: the nonblocking lock succeeded and the existing profile was used. AppleScript readback timed out, so native-client readback is not confirmed. The fixture was removed through a fresh TLS IMAP connection and its absence was verified. No Mac configuration or account was changed. ```text Mac fixture fresh-connection cleanup: PASS ``` UX gaps left: full native-client create/edit/delete confirmation. Owner steps: open Notes with the existing lab profile, confirm its saved login works, create a Note, save two edits, and verify that text and deletion sync with staging. The owner must handle any OS password prompt. ## Exact staging edge manifest Existing manifest; no homelab edit or push. ```yaml --- apiVersion: traefik.io/v1alpha1 kind: ServersTransportTCP metadata: name: calternal-staging-edge namespace: calternal spec: dialTimeout: 10s proxyProtocol: version: 2 --- apiVersion: traefik.io/v1alpha1 kind: IngressRouteTCP metadata: name: calternal-staging-imaps namespace: calternal spec: entryPoints: - imaps routes: - match: HostSNI(`dev.calternal.com`) services: - name: calternal-staging-edge port: 10993 serversTransport: calternal-staging-edge tls: secretName: calternal-staging-tls --- apiVersion: traefik.io/v1alpha1 kind: IngressRouteTCP metadata: name: calternal-staging-submissions namespace: calternal spec: entryPoints: - submissions routes: - match: HostSNI(`dev.calternal.com`) services: - name: calternal-staging-edge port: 10465 serversTransport: calternal-staging-edge tls: secretName: calternal-staging-tls --- apiVersion: v1 kind: Service metadata: name: calternal-staging-edge namespace: calternal spec: ports: - name: imap port: 10993 targetPort: 10993 - name: submission port: 10465 targetPort: 10465 clusterIP: None --- apiVersion: discovery.k8s.io/v1 kind: EndpointSlice metadata: name: calternal-staging-edge-vm namespace: calternal labels: kubernetes.io/service-name: calternal-staging-edge endpointslice.kubernetes.io/managed-by: homelab-private addressType: IPv4 ports: - name: imap port: 10993 protocol: TCP - name: submission port: 10465 protocol: TCP endpoints: - addresses: - 10.70.3.158 conditions: ready: true ``` ## Cleanup ```text Removed 17111 files, 8.9GiB total ``` Generated web and copied deployment output were removed. Review logs and the exact manifest remain in `artifacts/`.

Rows
Columns

Notes IMAP behind the TLS edge: plain listener for trusted proxies with PROXY protocol #941

Goal (owner decision 2026-10-02, #858)

Edge (done by the orchestrator)

Server change (this issue)

Tests

Server edge mode implemented (job imapedge, branch job/imapedge, 6f098f569)

#941 final report

Decisions

Known gaps

For the merge round

Gates (verbatim summary output)

Staging verification

Exact staging edge manifest

Cleanup

Server edge mode implemented (job imapedge, branch `job/imapedge`, `6f098f569`)