#32 - Photos: video playback — direct play or cached HLS transcode - kayg/calternal

kayg commented

2026-09-24 14:45:10 +00:00

Owner

Owner decision (round 13, P7): play videos directly when the browser supports the codec; otherwise a background job transcodes to H.264 (720p and 1080p renditions, HLS) and caches it as derived data under /data/.system/cache/.

ffmpeg is added to the container image for the photos plugin (check license build flags: no GPL-only components unless the owner agrees).
Hardware acceleration auto-detected (VAAPI/QSV/NVENC/VideoToolbox not applicable server-side, V4L2 on arm) with software fallback; bounded concurrency shared with thumbnails/ML.
Seeking works before the whole transcode finishes (segmented HLS); transcodes are resumable and idempotent (keyed by content hash + profile).
Live Photo motion clips use the same path.

Design: DESIGN §12, §28.

Context for the owning job

Repo: kayg/calternal (~/Developer/calternal). Read CLAUDE.md, CONTEXT.md and docs/DESIGN.md first; this issue's section is cited below.
Owner rules that always apply: file over app (plain files are the truth, the DB is an index); the server is the single writer; data loss is unacceptable; performance first, never at the cost of finesse; UI is the calternal.js design system (copy components verbatim, compare side by side with calternal.js reference screenshots; Claude does visual review); never ship sample/mock data; atomic commits; adversarial testing after API work; good enough, not perfect (merge blockers: crash/DoS, data loss, security, sync collisions).
Comment on this issue when you start (branch, base SHA), on each finding, when blocked, and when finished (head SHA + gate output). Never close it.

Owner decision (round 13, P7): play videos directly when the browser supports the codec; otherwise a background job transcodes to H.264 (720p and 1080p renditions, HLS) and caches it as derived data under `/data/.system/cache/`. - ffmpeg is added to the container image for the photos plugin (check license build flags: no GPL-only components unless the owner agrees). - Hardware acceleration auto-detected (VAAPI/QSV/NVENC/VideoToolbox not applicable server-side, V4L2 on arm) with software fallback; bounded concurrency shared with thumbnails/ML. - Seeking works before the whole transcode finishes (segmented HLS); transcodes are resumable and idempotent (keyed by content hash + profile). - Live Photo motion clips use the same path. Design: DESIGN §12, §28. ## Context for the owning job - Repo: kayg/calternal (~/Developer/calternal). Read CLAUDE.md, CONTEXT.md and docs/DESIGN.md first; this issue's section is cited below. - Owner rules that always apply: file over app (plain files are the truth, the DB is an index); the server is the single writer; data loss is unacceptable; performance first, never at the cost of finesse; UI is the calternal.js design system (copy components verbatim, compare side by side with calternal.js reference screenshots; Claude does visual review); never ship sample/mock data; atomic commits; adversarial testing after API work; good enough, not perfect (merge blockers: crash/DoS, data loss, security, sync collisions). - Comment on this issue when you start (branch, base SHA), on each finding, when blocked, and when finished (head SHA + gate output). Never close it.

kayg commented

2026-09-24 17:56:36 +00:00

Author

Owner

Starting issue #32 in worktree branch job/video-transcode.

Prompt base: 57118d9648582e682f0a0e1997fc8ad9f84bab35 (main). Checkout starts at 734d41571d8512e266d2544ca4becd24ee19d1a1, a merge commit whose first parent is the prompt base. Working tree is clean.

Starting issue #32 in worktree branch `job/video-transcode`. Prompt base: `57118d9648582e682f0a0e1997fc8ad9f84bab35` (main). Checkout starts at `734d41571d8512e266d2544ca4becd24ee19d1a1`, a merge commit whose first parent is the prompt base. Working tree is clean.

kayg commented

2026-09-24 18:02:11 +00:00

Author

Owner

Finding: Debian trixie ffmpeg 7:7.1.5-0+deb13u1 does not meet the issue's no-GPL-component constraint. I downloaded the package without installing it and inspected /usr/bin/ffmpeg; its embedded configuration includes --enable-gpl and --enable-libx264. I will exclude that package from the container plan and verify an LGPL-compatible software H.264 path before finalizing the design.

Finding: Debian trixie `ffmpeg` 7:7.1.5-0+deb13u1 does not meet the issue's no-GPL-component constraint. I downloaded the package without installing it and inspected `/usr/bin/ffmpeg`; its embedded configuration includes `--enable-gpl` and `--enable-libx264`. I will exclude that package from the container plan and verify an LGPL-compatible software H.264 path before finalizing the design.

kayg commented

2026-09-24 18:06:33 +00:00

Author

Owner

Finding: this checkout has no crates/plugins/photos crate, so I will use the assigned crates/plugins/video option. The existing confined derived-data API in calternal-fs only covers thumbnails (Root::thumbnail_temp / publish_thumbnail / read_thumbnail); video HLS artifacts need a similarly narrow safe cache API. The Files plugin's share-aware read resolver is private, so a standalone video route also needs a small reusable authorization helper there. These are the minimal cross-crate additions required to keep the video plugin inside the server's confinement and authorization rules.

Finding: this checkout has no `crates/plugins/photos` crate, so I will use the assigned `crates/plugins/video` option. The existing confined derived-data API in `calternal-fs` only covers thumbnails (`Root::thumbnail_temp` / `publish_thumbnail` / `read_thumbnail`); video HLS artifacts need a similarly narrow safe cache API. The Files plugin's share-aware read resolver is private, so a standalone video route also needs a small reusable authorization helper there. These are the minimal cross-crate additions required to keep the video plugin inside the server's confinement and authorization rules.

kayg commented

2026-09-24 18:11:06 +00:00

Author

Owner

Status: implementation is paused for the architectural design review. The checkout has no crates/plugins/photos, and the proposed standalone video plugin would use an LGPL-only FFmpeg build with OpenH264 fallback plus narrow filesystem and Files authorization helpers. No repository files have changed. Awaiting design review.

Status: implementation is paused for the architectural design review. The checkout has no `crates/plugins/photos`, and the proposed standalone `video` plugin would use an LGPL-only FFmpeg build with OpenH264 fallback plus narrow filesystem and Files authorization helpers. No repository files have changed. Awaiting design review.

kayg commented

2026-09-24 18:20:20 +00:00

Author

Owner

Starting issue #32 in worktree branch job/video-transcode at prompt base 41aa77499e0a090371ff252458722fd80b4e6a44 (main). The current checkout is clean at that SHA. The approved direction is a standalone video core plugin, safe HLS derived-data cache, shared media concurrency of 2 with thumbnails, Debian trixie's packaged ffmpeg, and a configurable 20 GiB default cache cap.

Starting issue #32 in worktree branch `job/video-transcode` at prompt base `41aa77499e0a090371ff252458722fd80b4e6a44` (main). The current checkout is clean at that SHA. The approved direction is a standalone `video` core plugin, safe HLS derived-data cache, shared media concurrency of 2 with thumbnails, Debian trixie's packaged `ffmpeg`, and a configurable 20 GiB default cache cap.

kayg commented

2026-09-24 18:23:07 +00:00

Author

Owner

Finding: this checkout has no photos plugin crate, so this job will use the assigned standalone crates/plugins/video path. calternal-fs currently confines only thumbnail cache artifacts; HLS needs a typed helper that accepts only a content hash, approved rendition, and validated playlist/segment name. The Files share grant resolver is private; the video routes need a small public item-ID read helper so playback honors live share revocation and stable identity. The server currently starts workers before it builds plugin routes, so I will build plugin routes first to provide the video handler with initialized live state before worker registration.

Finding: this checkout has no photos plugin crate, so this job will use the assigned standalone `crates/plugins/video` path. `calternal-fs` currently confines only thumbnail cache artifacts; HLS needs a typed helper that accepts only a content hash, approved rendition, and validated playlist/segment name. The Files share grant resolver is private; the video routes need a small public item-ID read helper so playback honors live share revocation and stable identity. The server currently starts workers before it builds plugin routes, so I will build plugin routes first to provide the video handler with initialized live state before worker registration.

kayg commented

2026-09-24 18:53:41 +00:00

Author

Owner

Finding: I checked Debian trixie's current ffmpeg package from the configured trixie repositories (7:7.1.5-0+deb13u1) and ran its -buildconf. It includes --enable-gpl and --enable-libx264. The approved container uses this as a separate subprocess, not linked server code. I added the package to both runtime images and documented the package license and H.264/AAC patent responsibility for operators in docs/video-transcoding.md.

Finding: I checked Debian trixie's current `ffmpeg` package from the configured trixie repositories (`7:7.1.5-0+deb13u1`) and ran its `-buildconf`. It includes `--enable-gpl` and `--enable-libx264`. The approved container uses this as a separate subprocess, not linked server code. I added the package to both runtime images and documented the package license and H.264/AAC patent responsibility for operators in `docs/video-transcoding.md`.

kayg commented

2026-09-24 19:16:34 +00:00

Author

Owner

Finding: Debian ffmpeg 7.1.5 preserves completed segments and its last atomic playlist after SIGKILL. A local restart probe resumed from the accumulated EXTINF duration with -ss and HLS append_list; the resulting playlist kept the original segment, appended the remaining segments, and marked the timestamp discontinuity. I am implementing this restart path and validating playlist references before reuse.

Finding: Debian ffmpeg 7.1.5 preserves completed segments and its last atomic playlist after SIGKILL. A local restart probe resumed from the accumulated EXTINF duration with `-ss` and HLS `append_list`; the resulting playlist kept the original segment, appended the remaining segments, and marked the timestamp discontinuity. I am implementing this restart path and validating playlist references before reuse.

kayg commented

2026-09-24 19:58:20 +00:00

Author

Owner

Finished video-transcode on branch job/video-transcode.

Head SHA: b5e1ce2407a5bed9aa1175179e1633dd24c13ce7

Gate output (verbatim):

cargo fmt --all --check exited 0 with no output.

cargo clippy --workspace --all-targets -- -D warnings:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 13.11s

cargo test --workspace result lines:

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 36 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 79.77s
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 14.35s
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.01s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.48s
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.92s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 8 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.72s
test result: ok. 18 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.26s
test result: ok. 34 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.34s
test result: ok. 453 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.98s
test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.90s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.53s
test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 22.38s
test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.21s
test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.02s
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.07s
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.65s
test result: ok. 17 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.69s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

bash packages/api-client/check-generated.sh:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 28s
     Running `target/debug/calternal-server openapi`
$ bunx --package openapi-typescript@7.13.0 openapi-typescript ../../contracts/openapi.json -o src/generated.ts
✨ openapi-typescript 7.13.0
🚀 ../../contracts/openapi.json → src/generated.ts [732.4ms]

bash tests/adversarial/run.sh:

==== FINDINGS 0
server alive at end: True
==== ROUND 2 FINDINGS 0

The smoke transcode generated an HLS playlist and four segments. A restart probe killed FFmpeg after a committed segment, then resumed with append_list; the final playlist retained old segments and completed the remaining segments.

Decisions: use the standalone video core plugin because photos-core was not present; keep FFmpeg as Debian's separate process; expose source bytes with single-range support; return 202 with Retry-After: 2 while a requested rendition is not ready. The Video LRU limit accounts for completed renditions. Active or interrupted output is outside the LRU index until completion and can temporarily raise the cache directory above the configured limit. This behavior is documented in docs/video-transcoding.md.

The checked Debian package enables GPL components. The documentation assigns package distribution and codec patent review to the operator. No UI files were changed.

Finished video-transcode on branch `job/video-transcode`. Head SHA: `b5e1ce2407a5bed9aa1175179e1633dd24c13ce7` Gate output (verbatim): `cargo fmt --all --check` exited 0 with no output. `cargo clippy --workspace --all-targets -- -D warnings`: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 13.11s ``` `cargo test --workspace` result lines: ``` test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s test result: ok. 36 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 79.77s test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 14.35s test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.01s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.48s test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.92s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 8 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.72s test result: ok. 18 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.26s test result: ok. 34 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.34s test result: ok. 453 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.98s test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.90s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.28s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.53s test result: ok. 41 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 22.38s test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.21s test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.02s test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.07s test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.65s test result: ok. 17 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.69s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `bash packages/api-client/check-generated.sh`: ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 28s Running `target/debug/calternal-server openapi` $ bunx --package openapi-typescript@7.13.0 openapi-typescript ../../contracts/openapi.json -o src/generated.ts ✨ openapi-typescript 7.13.0 🚀 ../../contracts/openapi.json → src/generated.ts [732.4ms] ``` `bash tests/adversarial/run.sh`: ``` ==== FINDINGS 0 server alive at end: True ==== ROUND 2 FINDINGS 0 ``` The smoke transcode generated an HLS playlist and four segments. A restart probe killed FFmpeg after a committed segment, then resumed with `append_list`; the final playlist retained old segments and completed the remaining segments. Decisions: use the standalone `video` core plugin because `photos-core` was not present; keep FFmpeg as Debian's separate process; expose source bytes with single-range support; return 202 with `Retry-After: 2` while a requested rendition is not ready. The Video LRU limit accounts for completed renditions. Active or interrupted output is outside the LRU index until completion and can temporarily raise the cache directory above the configured limit. This behavior is documented in `docs/video-transcoding.md`. The checked Debian package enables GPL components. The documentation assigns package distribution and codec patent review to the operator. No UI files were changed.

kayg referenced this issue from a commit

2026-09-24 21:12:25 +00:00

Merge job/video-transcode: video transcoding: HLS playback, hardware encoders, LRU cache (#32)

kayg commented

2026-10-01 05:12:51 +00:00

Author

Owner

Hygiene review: the video transcode report says active or interrupted output is outside the LRU index and can temporarily exceed the configured cache limit. Keeping #32 open until that storage bound is addressed or confirmed safe.

kayg referenced this issue

2026-10-02 13:10:34 +00:00

BLOCKER: reserve and bound active HLS output and media snapshot disk use #779

kayg referenced this issue

2026-10-03 12:56:40 +00:00

Issue hygiene sweep 2026-10-03 #1008

Rows
Columns

Photos: video playback — direct play or cached HLS transcode #32

Context for the owning job