kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 190 Pull requests 61 Projects Releases Packages Wiki Activity Actions 26

feat(photos): add library core and XMP organization #1020

Open
kayg wants to merge 12 commits from t3/photos-s1 into feat/files-s1-identity
pull from: t3/photos-s1
merge into: kayg:feat/files-s1-identity
Conversation 3 Commits 12 Files changed 7 +2205 -1
kayg commented 2026-11-08 22:02:34 +00:00
Owner
Copy link

Summary

  • Add a per-user Photos SQLite catalog keyed by shared Files IDs.
  • Read and write Favourite and Hidden in lossless XMP sidecars while preserving original bytes.
  • Serve bounded keyset timeline pages with year, month, week and day sections, including indexed group member counts.
  • Cover XMP preservation, per-user ownership, filters, section updates and a 200,000-photo metadata-only fixture.

This slice stacks on Files slice #1010. The backend design is in Photos plan #1018.

Validation

Gate results are still being collected and will be posted in a follow-up comment. This PR is open but is not ready for merge until the complete workspace and performance gates are recorded.

## Summary - Add a per-user Photos SQLite catalog keyed by shared Files IDs. - Read and write Favourite and Hidden in lossless XMP sidecars while preserving original bytes. - Serve bounded keyset timeline pages with year, month, week and day sections, including indexed group member counts. - Cover XMP preservation, per-user ownership, filters, section updates and a 200,000-photo metadata-only fixture. This slice stacks on Files slice #1010. The backend design is in Photos plan #1018. ## Validation Gate results are still being collected and will be posted in a follow-up comment. This PR is open but is not ready for merge until the complete workspace and performance gates are recorded.
kayg added 12 commits 2026-11-08 22:02:34 +00:00
test(photos): write fixture files atomically
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
afee4855cf
kayg commented 2026-11-09 03:40:23 +00:00
Author
Owner
Copy link

Gate update

  • cargo fmt --all -- --check: passed after the current slice head.
  • cargo clippy --workspace --all-targets -j4 -- -D warnings: passed after the current slice head.
  • Photos tests: 11/11 passed in the workspace run, including the sparse 200,000-photo timeline case.
  • Workspace tests are not clean: the workspace run recorded two unrelated Notes recovery failures. A separate Contacts actions run recorded kill9_import_after_history_transaction timing out. The isolated cargo test -p calternal-photos command also cannot compile unchanged calternal-core because tokio::select! is unavailable in that package-only feature graph.
  • Bun checks: not applicable; no web files changed.

Performance is still pending. The corrected ABAB run completed the five candidate/dev pairs, then stopped before controls when the 15 GiB free-space guard found 13,499,216 KiB available on /home. Those partial samples are not used for a performance conclusion. Both Photos perf holders have been released. I will rerun only after Mail's ordered perf-01 session has finished, using a harness that releases the candidate/dev fixture roots before building the dev/dev controls.

## Gate update - `cargo fmt --all -- --check`: passed after the current slice head. - `cargo clippy --workspace --all-targets -j4 -- -D warnings`: passed after the current slice head. - Photos tests: 11/11 passed in the workspace run, including the sparse 200,000-photo timeline case. - Workspace tests are not clean: the workspace run recorded two unrelated Notes recovery failures. A separate Contacts actions run recorded `kill9_import_after_history_transaction` timing out. The isolated `cargo test -p calternal-photos` command also cannot compile unchanged `calternal-core` because `tokio::select!` is unavailable in that package-only feature graph. - Bun checks: not applicable; no web files changed. Performance is still pending. The corrected ABAB run completed the five candidate/dev pairs, then stopped before controls when the 15 GiB free-space guard found 13,499,216 KiB available on `/home`. Those partial samples are not used for a performance conclusion. Both Photos perf holders have been released. I will rerun only after Mail's ordered perf-01 session has finished, using a harness that releases the candidate/dev fixture roots before building the dev/dev controls.
kayg commented 2026-11-09 06:08:35 +00:00
Author
Owner
Copy link

Code gates at 40ef1dc55 (takeover, 11-09)

Rebased onto the current Files slice 1 head (d00b48b6e, #1010) and added one commit, fix(core): enable tokio macros for select! (calternal-core uses tokio::select! but only got the macros feature from dev-dependencies, so cargo test -p calternal-photos could not compile alone).

Run on claude@calternal-dev-01-compute02 through ~/bin/slot:

  • cargo fmt --all -- --check: pass
  • cargo clippy --workspace --all-targets -j4 -- -D warnings: pass
  • cargo test -j4 -p calternal-photos: 11 passed, 0 failed (includes the sparse 200,000-photo timeline test)
  • cargo test -j4 -p calternal-notes --lib --no-fail-fast: 586 passed, 0 failed
  • cargo test -j4 --workspace --no-fail-fast: all targets pass

Earlier Notes failures (shutdown_settlement_state_machine, save_outside_retry) were reproduced on the Files slice 1 head without any Photos commit (52e7e54a1: same test failed there too), and are gone once the Files fix afb3d29b6 is in the base. They are not from Photos.

Perf ABAB x5 vs dev with dev-vs-dev controls is queued behind the Mail and p928 runs on the perf locks (perf-01 first, then perf-02). Note: nothing outside crates/photos depends on calternal-photos, so the server binary does not link it.

## Code gates at `40ef1dc55` (takeover, 11-09) Rebased onto the current Files slice 1 head (`d00b48b6e`, #1010) and added one commit, `fix(core): enable tokio macros for select!` (`calternal-core` uses `tokio::select!` but only got the `macros` feature from dev-dependencies, so `cargo test -p calternal-photos` could not compile alone). Run on `claude@calternal-dev-01-compute02` through `~/bin/slot`: - `cargo fmt --all -- --check`: pass - `cargo clippy --workspace --all-targets -j4 -- -D warnings`: pass - `cargo test -j4 -p calternal-photos`: 11 passed, 0 failed (includes the sparse 200,000-photo timeline test) - `cargo test -j4 -p calternal-notes --lib --no-fail-fast`: 586 passed, 0 failed - `cargo test -j4 --workspace --no-fail-fast`: all targets pass Earlier Notes failures (`shutdown_settlement_state_machine`, `save_outside_retry`) were reproduced on the Files slice 1 head without any Photos commit (`52e7e54a1`: same test failed there too), and are gone once the Files fix `afb3d29b6` is in the base. They are not from Photos. Perf ABAB x5 vs dev with dev-vs-dev controls is queued behind the Mail and `p928` runs on the perf locks (perf-01 first, then perf-02). Note: nothing outside `crates/photos` depends on `calternal-photos`, so the server binary does not link it.
kayg commented 2026-11-09 06:30:20 +00:00
Author
Owner
Copy link

State at pause (11-09, lane resumes after Sharing)

Pushed: #1020 head 40ef1dc5518ec556202e97834da5f6829eede867 (rebased on Files slice 1 d00b48b6e, plus fix(core): enable tokio macros for select!). #1018 plan unchanged (4b747f2ba). Nothing uncommitted.

Done: fmt, clippy -D warnings, cargo test -p calternal-photos (11/11 with the 200k sparse timeline), Notes lib 586/0, full workspace: all green on this head.

Left: perf ABAB x5 vs dev with dev-vs-dev controls. It was running on perf-02 when I stopped it on the orchestrator's call. The partial run is discarded, and no perf conclusion is claimed. Both perf holds and my run unit are released. Staged for the rerun on perf-02 in /home/claude/photos-s1-perf.y9nAeT: binaries (candidate sha256 fc3fe827..., dev built from d4b32af94) and the harness. About 5 GB of fixtures remain in its indexed/ and work/, and the rerun reuses or rebuilds them. Perf-01 had too little free disk (about 21 GB) for the 15 GB floor, so use perf-02. Candidate-vs-dev compares the Files stack too, because calternal-photos is not linked into the server.

Open questions (all have a recommendation in the plan):

  1. XMP namespace URI for calternal:hidden: recommend https://calternal.cloud/ns/photos/1.0/.
  2. Un-favourite when a rating exists: recommend set 0 only if it is 5. This slightly conflicts with the "never lowered" line in PH-B, which I read as applying to favouriting only. Please confirm.
  3. Upload folder date: recommend the capture local day, else arrival date.
  4. "Show in Photos" storage: recommend a hidden portable marker tied to the folder ID, recursive.
  5. Ambiguous pairs and missing offsets: recommend auto-pair only on strong agreement, otherwise leave separate.
  6. Sandbox and PP-OCRv6 weights: recommend a Landlock/seccomp re-exec worker, and validate redistribution terms before packaging.
  7. Pending from before: owner sign-off on the YuNet + AuraFace R100 (SFace fallback) picks.
## State at pause (11-09, lane resumes after Sharing) **Pushed:** #1020 head `40ef1dc5518ec556202e97834da5f6829eede867` (rebased on Files slice 1 `d00b48b6e`, plus `fix(core): enable tokio macros for select!`). #1018 plan unchanged (`4b747f2ba`). Nothing uncommitted. **Done:** fmt, clippy `-D warnings`, `cargo test -p calternal-photos` (11/11 with the 200k sparse timeline), Notes lib 586/0, full workspace: all green on this head. **Left:** perf ABAB x5 vs dev with dev-vs-dev controls. It was running on perf-02 when I stopped it on the orchestrator's call. The partial run is discarded, and no perf conclusion is claimed. Both perf holds and my run unit are released. Staged for the rerun on perf-02 in `/home/claude/photos-s1-perf.y9nAeT`: binaries (candidate sha256 `fc3fe827...`, dev built from `d4b32af94`) and the harness. About 5 GB of fixtures remain in its `indexed/` and `work/`, and the rerun reuses or rebuilds them. Perf-01 had too little free disk (about 21 GB) for the 15 GB floor, so use perf-02. Candidate-vs-dev compares the Files stack too, because `calternal-photos` is not linked into the server. **Open questions (all have a recommendation in the plan):** 1. XMP namespace URI for `calternal:hidden`: recommend `https://calternal.cloud/ns/photos/1.0/`. 2. Un-favourite when a rating exists: recommend set `0` only if it is `5`. This slightly conflicts with the "never lowered" line in PH-B, which I read as applying to favouriting only. Please confirm. 3. Upload folder date: recommend the capture local day, else arrival date. 4. "Show in Photos" storage: recommend a hidden portable marker tied to the folder ID, recursive. 5. Ambiguous pairs and missing offsets: recommend auto-pair only on strong agreement, otherwise leave separate. 6. Sandbox and PP-OCRv6 weights: recommend a Landlock/seccomp re-exec worker, and validate redistribution terms before packaging. 7. Pending from before: owner sign-off on the YuNet + AuraFace R100 (SFace fallback) picks.
Some checks failed
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
This pull request is broken due to missing fork information.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin t3/photos-s1:t3/photos-s1
git switch t3/photos-s1

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch feat/files-s1-identity
git merge --no-ff t3/photos-s1
git switch t3/photos-s1
git rebase feat/files-s1-identity
git switch feat/files-s1-identity
git merge --ff-only t3/photos-s1
git switch t3/photos-s1
git rebase feat/files-s1-identity
git switch feat/files-s1-identity
git merge --no-ff t3/photos-s1
git switch feat/files-s1-identity
git merge --squash t3/photos-s1
git switch feat/files-s1-identity
git merge --ff-only t3/photos-s1
git switch feat/files-s1-identity
git merge t3/photos-s1
git push origin feat/files-s1-identity
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!1020
No description provided.