WebDAV: AppleDouble ._ writes get 403; cp of a file with extended attributes fails and leaves a 0-byte file; Finder tags dropped #648

Open
opened 2026-10-01 18:48:09 +00:00 by kayg · 17 comments
Owner

Summary

macOS stores extended attributes (Finder tags, download quarantine) on a WebDAV volume as AppleDouble ._name files. calternal answers every ._* request (PROPFIND, PUT) with 403 because hidden names are forbidden over WebDAV. Effects seen on the macOS 27 VM:

  • cp (Terminal) of a file that has extended attributes into the mounted volume fails (fcopyfile failed: Operation not permitted, exit 1) and leaves a 0-byte file on the server.
  • A Finder copy of the same file succeeds, but the Finder tag is silently dropped.
  • Finder also sends PUT /.DS_Store → 403 on every folder visit (harmless, but noisy logs and extra round trips; each 403 PROPFIND costs about 0.5 s on the debug lab build).

Any file downloaded from the internet has com.apple.quarantine, so this affects ordinary use. Not a merge blocker: the source file stays intact and Finder succeeds; file it and fix when nearby.

Found in the Apple interop run on the macOS 27 VM, 2026-10-01 (lab server from dev at 687ff7031, volume mounted with mount volume "https://calternal.lab:8443/dav/files/<user-id>/").

Repro

printf 'tagged\n' > tagged.txt
xattr -w com.apple.quarantine "0081;66f00000;Safari;" tagged.txt
cp tagged.txt "/Volumes/<user-id>/MDV Folder ✓/"     # exit 1

Wire: PUT …/tagged.txt 201 (empty body), LOCK 200, PROPFIND …/._tagged.txt 403 ×N, PUT …/._tagged.txt 403 ×3, UNLOCK 204. No PUT with the content follows. Server keeps tagged.txt, 0 bytes.

Expected (pick one, owner may decide)

  • Accept and discard AppleDouble/.DS_Store writes (answer PUT 201/204 and PROPFIND 404 without storing), so copies succeed; or
  • Store ._* sidecars in a hidden area and map Finder tags to the calternal tag store (DESIGN §31 K4 says Finder tags map to calternal tags on macOS via calternald; WebDAV could do the same).
    In all cases a failed copy must not leave a 0-byte file that looks like the real file.

Also seen (cosmetic)

The mounted volume is named after the User ID (01a0f89d-…) because the URL ends with the ID. A friendlier last path segment would give a readable volume name in Finder.

## Summary macOS stores extended attributes (Finder tags, download quarantine) on a WebDAV volume as AppleDouble `._name` files. calternal answers every `._*` request (PROPFIND, PUT) with **403** because hidden names are forbidden over WebDAV. Effects seen on the macOS 27 VM: - `cp` (Terminal) of a file that has extended attributes into the mounted volume fails (`fcopyfile failed: Operation not permitted`, exit 1) **and leaves a 0-byte file** on the server. - A Finder copy of the same file succeeds, but the Finder tag is silently dropped. - Finder also sends `PUT /.DS_Store` → 403 on every folder visit (harmless, but noisy logs and extra round trips; each 403 PROPFIND costs about 0.5 s on the debug lab build). Any file downloaded from the internet has `com.apple.quarantine`, so this affects ordinary use. Not a merge blocker: the source file stays intact and Finder succeeds; file it and fix when nearby. Found in the Apple interop run on the macOS 27 VM, 2026-10-01 (lab server from `dev` at `687ff7031`, volume mounted with `mount volume "https://calternal.lab:8443/dav/files/<user-id>/"`). ## Repro ``` printf 'tagged\n' > tagged.txt xattr -w com.apple.quarantine "0081;66f00000;Safari;" tagged.txt cp tagged.txt "/Volumes/<user-id>/MDV Folder ✓/" # exit 1 ``` Wire: `PUT …/tagged.txt` 201 (empty body), `LOCK` 200, `PROPFIND …/._tagged.txt` 403 ×N, `PUT …/._tagged.txt` 403 ×3, `UNLOCK` 204. No PUT with the content follows. Server keeps `tagged.txt`, 0 bytes. ## Expected (pick one, owner may decide) - Accept and discard AppleDouble/`.DS_Store` writes (answer PUT 201/204 and PROPFIND 404 without storing), so copies succeed; or - Store `._*` sidecars in a hidden area and map Finder tags to the calternal tag store (DESIGN §31 K4 says Finder tags map to calternal tags on macOS via `calternald`; WebDAV could do the same). In all cases a failed copy must not leave a 0-byte file that looks like the real file. ## Also seen (cosmetic) The mounted volume is named after the User ID (`01a0f89d-…`) because the URL ends with the ID. A friendlier last path segment would give a readable volume name in Finder.
Author
Owner

Extra measurement from the same run: cp -R of a 30-file tree (3 levels, Unicode names) into the mounted volume made 461 requests. 101 of them were PROPFIND …/._<name> answered 403, and 127 were PROPFIND 404 for names that did not exist yet. So about one request in five is an AppleDouble probe that can never succeed. The copy itself passed (diff -r identical). If the fix accepts-and-discards ._*, answer those PROPFINDs with a cheap 404 before any Files lookup.

Extra measurement from the same run: `cp -R` of a 30-file tree (3 levels, Unicode names) into the mounted volume made 461 requests. 101 of them were `PROPFIND …/._<name>` answered 403, and 127 were PROPFIND 404 for names that did not exist yet. So about one request in five is an AppleDouble probe that can never succeed. The copy itself passed (`diff -r` identical). If the fix accepts-and-discards `._*`, answer those PROPFINDs with a cheap 404 before any Files lookup.
Author
Owner

Started work on branch job/webdav-lock-476 at HEAD 3eb63adb2b54633c96d5064ef2c2ca17be03fe85, based on 3f258302a0f2d6418ff60c9ce22cbb33e008ca99.

I read CLAUDE.md, CONTEXT.md, docs/DESIGN.md, issue evidence, and macdav-lab/apple-interop-2026-10-02.md. DESIGN §31 K4 maps Finder tags to calternal tags on macOS, but DESIGN has no AppleDouble or .DS_Store DAV storage rule. The lab captured ._ PROPFIND/PUT 403 responses, a failed cp with a 0-byte target, and Finder tag loss. I am tracing the DAV storage/listing/move paths and will add replay coverage for the captured requests plus hostile ._ paths.

Started work on branch `job/webdav-lock-476` at HEAD `3eb63adb2b54633c96d5064ef2c2ca17be03fe85`, based on `3f258302a0f2d6418ff60c9ce22cbb33e008ca99`. I read `CLAUDE.md`, `CONTEXT.md`, `docs/DESIGN.md`, issue evidence, and `macdav-lab/apple-interop-2026-10-02.md`. DESIGN §31 K4 maps Finder tags to calternal tags on macOS, but DESIGN has no AppleDouble or `.DS_Store` DAV storage rule. The lab captured `._` PROPFIND/PUT 403 responses, a failed `cp` with a 0-byte target, and Finder tag loss. I am tracing the DAV storage/listing/move paths and will add replay coverage for the captured requests plus hostile `._` paths.
Author
Owner

The first captured-request replay regression is red. cargo test -p calternal-dav apple_finder_appledouble_probe_is_missing_not_forbidden -- --nocapture reached the assertion and got 403 for PROPFIND /dav/files/<user>/._tagged.txt with User-Agent: WebDAVFS/3.0 Darwin/25.0.0; expected 404 for an absent companion. The route rejects dot-prefixed paths before the Files provider. I will admit only validated AppleDouble leaves from WebDAVFS requests, while retaining the hidden-path refusal for other clients.

The first captured-request replay regression is red. `cargo test -p calternal-dav apple_finder_appledouble_probe_is_missing_not_forbidden -- --nocapture` reached the assertion and got `403` for `PROPFIND /dav/files/<user>/._tagged.txt` with `User-Agent: WebDAVFS/3.0 Darwin/25.0.0`; expected `404` for an absent companion. The route rejects dot-prefixed paths before the Files provider. I will admit only validated AppleDouble leaves from WebDAVFS requests, while retaining the hidden-path refusal for other clients.
Author
Owner

648 progress report (timebox reached)

Job time exceeded the four-hour limit. No push, deploy, or merge was done. Current branch head: 61e1fbef24cbc2373f955323cdf551f688576260.

Committed

  • 0bc1cbfe8 — allow Finder AppleDouble probes and add DAV replay coverage for the missing-sidecar PROPFIND response.
  • a39fa7f90 — add the AppleDouble traffic profile to the WebDAV benchmark.
  • 61e1fbef2 — record host load around the benchmark phases.

Previously completed crate gates for calternal-path and calternal-dav passed before the current filesystem/provider changes. Their exact captured terminal logs are not available in this report. No final gate run was completed for the current uncommitted changes.

Current uncommitted work

AppleDouble metadata storage is per Home under .calternal/appledouble, with bounded file and per-User sizes; Files listing hides it from non-Apple identities. Root operations attach metadata through writes, moves, trash, restore, and delete. The DAV provider stages bounded sidecar uploads and attempts to remove a newly-created zero-byte target if its associated sidecar upload fails. Provider persistence, isolation, move/trash, and oversized-upload tests are present. The adversarial WebDAV probe has Apple client path and payload cases.

The provider test command cargo test -p calternal-plugin-files webdav_finder_appledouble_is_hidden_persistent_and_follows_moves -- --nocapture was interrupted at the timebox while compiling dependencies and calternal-fs (exit 130 from interruption). The final fetch/merge, final crate gates, local server adversarial replay, performance measurement, final commits, and cleanup remain to be done.

Decisions not specified in DESIGN

  • Store Finder metadata below each User's reserved .calternal/appledouble root and apply independent 8 MiB per-resource and 64 MiB per-User limits.
  • Only expose companion metadata to WebDAVFS requests; keep it out of normal Files listings and indexing.
  • Preserve literal ._name components in the hidden store for bounded directory enumeration; use a hashed key for file companions moved to Trash.

Mac verification remains pending because the Mac VM is offline: mount the lab share in Finder; copy a downloaded file with its quarantine xattr and compare bytes; add a Finder tag, unmount/remount, and confirm the tag survives.

# 648 progress report (timebox reached) Job time exceeded the four-hour limit. No push, deploy, or merge was done. Current branch head: `61e1fbef24cbc2373f955323cdf551f688576260`. ## Committed - `0bc1cbfe8` — allow Finder AppleDouble probes and add DAV replay coverage for the missing-sidecar PROPFIND response. - `a39fa7f90` — add the AppleDouble traffic profile to the WebDAV benchmark. - `61e1fbef2` — record host load around the benchmark phases. Previously completed crate gates for `calternal-path` and `calternal-dav` passed before the current filesystem/provider changes. Their exact captured terminal logs are not available in this report. No final gate run was completed for the current uncommitted changes. ## Current uncommitted work AppleDouble metadata storage is per Home under `.calternal/appledouble`, with bounded file and per-User sizes; Files listing hides it from non-Apple identities. Root operations attach metadata through writes, moves, trash, restore, and delete. The DAV provider stages bounded sidecar uploads and attempts to remove a newly-created zero-byte target if its associated sidecar upload fails. Provider persistence, isolation, move/trash, and oversized-upload tests are present. The adversarial WebDAV probe has Apple client path and payload cases. The provider test command `cargo test -p calternal-plugin-files webdav_finder_appledouble_is_hidden_persistent_and_follows_moves -- --nocapture` was interrupted at the timebox while compiling dependencies and `calternal-fs` (exit 130 from interruption). The final fetch/merge, final crate gates, local server adversarial replay, performance measurement, final commits, and cleanup remain to be done. ## Decisions not specified in DESIGN - Store Finder metadata below each User's reserved `.calternal/appledouble` root and apply independent 8 MiB per-resource and 64 MiB per-User limits. - Only expose companion metadata to WebDAVFS requests; keep it out of normal Files listings and indexing. - Preserve literal `._name` components in the hidden store for bounded directory enumeration; use a hashed key for file companions moved to Trash. Mac verification remains pending because the Mac VM is offline: mount the lab share in Finder; copy a downloaded file with its quarantine xattr and compare bytes; add a Finder tag, unmount/remount, and confirm the tag survives.
Author
Owner

Independent read-only review started. Review branch: job/rev2-webdav-lock-476. Base SHA: 440e19dce2. Target: c9863ed9e. Scope: AppleDouble metadata, PUT staging, rollback, isolation, concurrency, and test history. No builds or tests will run (LIGHT job).

Independent read-only review started. Review branch: job/rev2-webdav-lock-476. Base SHA: 440e19dce23040ac8ebaae88f0469b6535b1afcb. Target: c9863ed9e. Scope: AppleDouble metadata, PUT staging, rollback, isolation, concurrency, and test history. No builds or tests will run (LIGHT job).
Author
Owner

Independent read-only review complete.

Review branch: job/rev2-webdav-lock-476.
Review head: 98608a2a3ecc9b83a1ae3fa9a640d4bf52d821c6.
Committed files: review-webdav-lock-476.md and audit-findings.md.
Reviewed product head: c9863ed9e3dfa68633a9486cad83ee7cd89a695e.

The report below contains five P1 findings and two P2 findings. Each has source evidence and a concrete fix. All belong to #648; no separate issue was filed. No product code changed.

Independent review of WebDAV AppleDouble changes (#648)

Recommendation: Do not merge this head. Fix the P1 findings first.

Reviewed head: c9863ed9e3dfa68633a9486cad83ee7cd89a695e.
Compared with fetched origin/dev: 440e19dce23040ac8ebaae88f0469b6535b1afcb.
Merge base: 3f258302a0f2d6418ff60c9ce22cbb33e008ca99.
All source line numbers below refer to the reviewed head.

This is a LIGHT, read-only review. It used the issue body, CLAUDE.md,
CONTEXT.md, DESIGN, branch diff, surrounding code, and commit history.
The author's report was not used. No product code changed.

Findings, ranked by severity

ID Severity Evidence and effect Concrete fix
F0 P1 crates/calternal-fs/src/write.rs:189 calls ensure_name_available on a first companion write. crates/calternal-fs/src/path.rs:142 constructs its reserved parent with public RelPath::new. The new reservation at crates/calternal-path/src/lib.rs:182 rejects that parent. First AppleDouble and .DS_Store PUTs fail before publication. Extend the shared name check to accept a validated internal parent for internal writes. Keep public reserved-path checks. Run the new persistence tests on an empty Home.
F1 P1 crates/calternal-fs/src/file_ops.rs:222 adds a companion step to every deletion. crates/calternal-fs/src/appledouble.rs:390 requires a Home. Existing Blob store collection (crates/calternal-fs/src/blob.rs:123) and expired archive removal (crates/calternal-fs/src/user_homes.rs:247) delete internal paths outside a Home. They now return InvalidPath before removal. Return no companion step for internal or non-Home paths. Apply the same restriction to the MOVE hook. Keep the existing collection and archive tests unchanged.
F5 P1 crates/plugins/files/src/dav.rs:110 treats an inode fingerprint as a revision check. The write path can hardlink the canonical empty blob (crates/calternal-fs/src/write.rs:357, line 408). A later empty revision can have the original fingerprint. Rollback at crates/plugins/files/src/dav.rs:114 permanently deletes that revision and its Versions. Bind rollback to the exact item revision and upload session. Check it under the mutation lock. Preserve all acknowledged later edits and Versions.
F2 P1 crates/calternal-dav/src/files.rs:1190 admits companion PUTs from upload-only access. The new provider branch at crates/plugins/files/src/dav.rs:394 bypasses the normal collision-renaming policy. crates/calternal-fs/src/appledouble.rs:119 always replaces metadata on the existing attached file. For both upload-only access levels, accept a companion only for a file created by the same restricted upload session. Use its installed, collision-renamed path. Refuse changes to older files and folder metadata.
F3 P1 crates/plugins/files/src/dav.rs:49 stores only the attached path. Start checks that item at line 135; finish writes to the current item at that path at line 628. A Files API move or delete-and-create between these steps can attach the old item's metadata to a new file. DAV gates do not cover the Files API. Capture the attached item's stable identity at start. Check it and companion preconditions again under the final publication lock. Refuse publication after the attached item changes.
F4 P2 crates/plugins/files/src/dav.rs:943 uses copy_tree, which copies visible data with Root::copy. The new metadata tree is outside that data. crates/calternal-fs/src/file_ops.rs:120 has no companion copy hook. COPY drops Finder metadata; overwrite can retain the old destination's metadata. Extend the shared confined copy operation to copy or clear companions with the content. Enforce the destination metadata cap. Cover files, folders, and overwrite from an untagged source.
F6 P2 crates/plugins/files/src/dav.rs:636 clears the rollback marker after metadata succeeds. The ordinary content abort at line 670 only terminates staging. A failed content PUT after an empty target and successful companion still leaves an empty target. Marker expiry and restart also lose rollback state. Keep a durable provisional-copy record through content completion. Clean up only that copy's provisional target on failure. Preserve real empty files and later edits. Fix this with F5 under one owner.

F2, F3, and F4 apply after F0 is fixed, or when companions already exist in
the hidden store. F0 must not conceal these later defects.

Detailed call chains and test ideas are in audit-findings.md. The most
important regression tests are:

  • First PUT and reopen of a companion and .DS_Store in an empty Home.
  • Existing orphan_blobs_are_collected and expired User archive removal.
  • An empty target that receives a nonempty revision and then an empty revision,
    followed by a failed companion. The file and its nonempty Version must remain.
  • Restricted upload access to an older companion and a collision-renamed file.
  • A paused companion upload while the Files API replaces its attached item.
  • File and folder COPY, including overwrite from a source without metadata.
  • A failed content upload after successful metadata, including a restart.

Other review results

Authorization: The HTTP adapter checks the authenticated User against the URL
User before the provider runs. The provider maps companions to the visible
item for Home-prefix checks. Hidden parents and reserved visible items remain
denied. In-memory companion chunks, finish, and abort check the upload's User.
No cross-User access defect was found in those checks. F2 is an access-level
defect within a User's Home.

Reuse: The branch reuses normal Files uploads for content and confined
filesystem writes for metadata. F0 shows that this reuse needs an internal
path contract. F1 shows that the new filesystem hooks need to preserve the
existing general deletion contract. Companion attachment parsing is repeated
in the DAV adapter and provider, but both use appledouble_resource; this
review did not treat those small scope conversions as a separate defect.

Errors: New failures use DAV status responses with empty bodies, not new UI
copy. Static failure logging avoids paths and request bodies. Source inspection
does not prove the runtime error mapping or cancellation cleanup.

Comments: The new modules explain hidden metadata and publication. The
rollback comment at crates/plugins/files/src/dav.rs:85 is incomplete: inode
and age do not establish the original revision (F5). Update that comment with
the fix. The storage is opaque AppleDouble data; it does not map Finder tags
into calternal Tags. Do not describe the tests as proof of that mapping.

Tests: git log -p was checked for the branch's test files and changed inline
tests. No existing assertion, expected status, or fixture was weakened. New
tests check bytes, hidden listing, restart, MOVE, Trash, restore, and rejected
metadata. These checks address behavior missing in the old code. The new
rollback test only checks an oversized companion immediately after an empty
PUT. It does not cover F5 or F6. F0 is in the new happy-path test call chain;
the test must run before any claim that the feature works.

Performance: The new companion write scans the full hidden metadata tree
while holding the shared writer lock (crates/calternal-fs/src/write.rs:196)
and scans it again under the operation lock (line 279). The provider also
holds the shared Files mutation lock during metadata publication. Cost grows
with all companion entries, including Trash. The new benchmark profile uses
30 visible files and a four-upload burst (bench/webdav-lock-476.py:302, line
367). It does not measure a large existing metadata tree or the 32-session
Finder case used elsewhere in this branch. Extend that profile before drawing
a worst-case performance conclusion. No new measurements were taken in this
LIGHT review.

Tracking

Existing issues were searched for AppleDouble, upload-only access, rollback,
and Blob store collection. All listed defects are introduced by this branch
and are recorded on #648. No separate issue was filed. F5 and F6 share one
rollback fix. No defect outside this branch was confirmed.

Validation and known gaps

git diff --check: exit 0; stdout and stderr were empty.
git fetch origin: exit 0; stdout and stderr were empty. origin/dev stayed
at the SHA above.

Rust and web gates were not run. The LIGHT instructions prohibit builds and
tests. There is no gate output to quote. No server, browser, Apple-client run,
adversarial round, or performance run was started. No build output was created.

DESIGN ends at §57 in both the reviewed head and fetched origin/dev.
The requested §58 is absent. This review used the available performance rules
and DESIGN §2, §24, §26, and §31 K4.

Decisions

  • The specific read-only job rules control this review. The general build,
    benchmark, cleanup, and merge instructions do not apply. The remote was
    fetched once; no merge was performed.
  • Branch defects stay on the owned issue #648, as the review brief requires.
  • No product design decision was made. Opaque companion storage can preserve
    Finder bytes, but this report makes no claim about unified Tag mapping.
  • UX gaps closed: none; this job changes review documents only. UX gaps left:
    runtime Apple-client copy, tag, failure, and recovery behavior is unverified.
Independent read-only review complete. Review branch: `job/rev2-webdav-lock-476`. Review head: `98608a2a3ecc9b83a1ae3fa9a640d4bf52d821c6`. Committed files: `review-webdav-lock-476.md` and `audit-findings.md`. Reviewed product head: `c9863ed9e3dfa68633a9486cad83ee7cd89a695e`. The report below contains five P1 findings and two P2 findings. Each has source evidence and a concrete fix. All belong to #648; no separate issue was filed. No product code changed. # Independent review of WebDAV AppleDouble changes (#648) Recommendation: Do not merge this head. Fix the P1 findings first. Reviewed head: `c9863ed9e3dfa68633a9486cad83ee7cd89a695e`. Compared with fetched `origin/dev`: `440e19dce23040ac8ebaae88f0469b6535b1afcb`. Merge base: `3f258302a0f2d6418ff60c9ce22cbb33e008ca99`. All source line numbers below refer to the reviewed head. This is a LIGHT, read-only review. It used the issue body, CLAUDE.md, CONTEXT.md, DESIGN, branch diff, surrounding code, and commit history. The author's report was not used. No product code changed. ## Findings, ranked by severity | ID | Severity | Evidence and effect | Concrete fix | | --- | --- | --- | --- | | F0 | P1 | `crates/calternal-fs/src/write.rs:189` calls `ensure_name_available` on a first companion write. `crates/calternal-fs/src/path.rs:142` constructs its reserved parent with public `RelPath::new`. The new reservation at `crates/calternal-path/src/lib.rs:182` rejects that parent. First AppleDouble and `.DS_Store` PUTs fail before publication. | Extend the shared name check to accept a validated internal parent for internal writes. Keep public reserved-path checks. Run the new persistence tests on an empty Home. | | F1 | P1 | `crates/calternal-fs/src/file_ops.rs:222` adds a companion step to every deletion. `crates/calternal-fs/src/appledouble.rs:390` requires a Home. Existing Blob store collection (`crates/calternal-fs/src/blob.rs:123`) and expired archive removal (`crates/calternal-fs/src/user_homes.rs:247`) delete internal paths outside a Home. They now return `InvalidPath` before removal. | Return no companion step for internal or non-Home paths. Apply the same restriction to the MOVE hook. Keep the existing collection and archive tests unchanged. | | F5 | P1 | `crates/plugins/files/src/dav.rs:110` treats an inode fingerprint as a revision check. The write path can hardlink the canonical empty blob (`crates/calternal-fs/src/write.rs:357`, line 408). A later empty revision can have the original fingerprint. Rollback at `crates/plugins/files/src/dav.rs:114` permanently deletes that revision and its Versions. | Bind rollback to the exact item revision and upload session. Check it under the mutation lock. Preserve all acknowledged later edits and Versions. | | F2 | P1 | `crates/calternal-dav/src/files.rs:1190` admits companion PUTs from upload-only access. The new provider branch at `crates/plugins/files/src/dav.rs:394` bypasses the normal collision-renaming policy. `crates/calternal-fs/src/appledouble.rs:119` always replaces metadata on the existing attached file. | For both upload-only access levels, accept a companion only for a file created by the same restricted upload session. Use its installed, collision-renamed path. Refuse changes to older files and folder metadata. | | F3 | P1 | `crates/plugins/files/src/dav.rs:49` stores only the attached path. Start checks that item at line 135; finish writes to the current item at that path at line 628. A Files API move or delete-and-create between these steps can attach the old item's metadata to a new file. DAV gates do not cover the Files API. | Capture the attached item's stable identity at start. Check it and companion preconditions again under the final publication lock. Refuse publication after the attached item changes. | | F4 | P2 | `crates/plugins/files/src/dav.rs:943` uses `copy_tree`, which copies visible data with `Root::copy`. The new metadata tree is outside that data. `crates/calternal-fs/src/file_ops.rs:120` has no companion copy hook. COPY drops Finder metadata; overwrite can retain the old destination's metadata. | Extend the shared confined copy operation to copy or clear companions with the content. Enforce the destination metadata cap. Cover files, folders, and overwrite from an untagged source. | | F6 | P2 | `crates/plugins/files/src/dav.rs:636` clears the rollback marker after metadata succeeds. The ordinary content abort at line 670 only terminates staging. A failed content PUT after an empty target and successful companion still leaves an empty target. Marker expiry and restart also lose rollback state. | Keep a durable provisional-copy record through content completion. Clean up only that copy's provisional target on failure. Preserve real empty files and later edits. Fix this with F5 under one owner. | F2, F3, and F4 apply after F0 is fixed, or when companions already exist in the hidden store. F0 must not conceal these later defects. Detailed call chains and test ideas are in `audit-findings.md`. The most important regression tests are: - First PUT and reopen of a companion and `.DS_Store` in an empty Home. - Existing `orphan_blobs_are_collected` and expired User archive removal. - An empty target that receives a nonempty revision and then an empty revision, followed by a failed companion. The file and its nonempty Version must remain. - Restricted upload access to an older companion and a collision-renamed file. - A paused companion upload while the Files API replaces its attached item. - File and folder COPY, including overwrite from a source without metadata. - A failed content upload after successful metadata, including a restart. ## Other review results Authorization: The HTTP adapter checks the authenticated User against the URL User before the provider runs. The provider maps companions to the visible item for Home-prefix checks. Hidden parents and reserved visible items remain denied. In-memory companion chunks, finish, and abort check the upload's User. No cross-User access defect was found in those checks. F2 is an access-level defect within a User's Home. Reuse: The branch reuses normal Files uploads for content and confined filesystem writes for metadata. F0 shows that this reuse needs an internal path contract. F1 shows that the new filesystem hooks need to preserve the existing general deletion contract. Companion attachment parsing is repeated in the DAV adapter and provider, but both use `appledouble_resource`; this review did not treat those small scope conversions as a separate defect. Errors: New failures use DAV status responses with empty bodies, not new UI copy. Static failure logging avoids paths and request bodies. Source inspection does not prove the runtime error mapping or cancellation cleanup. Comments: The new modules explain hidden metadata and publication. The rollback comment at `crates/plugins/files/src/dav.rs:85` is incomplete: inode and age do not establish the original revision (F5). Update that comment with the fix. The storage is opaque AppleDouble data; it does not map Finder tags into calternal Tags. Do not describe the tests as proof of that mapping. Tests: `git log -p` was checked for the branch's test files and changed inline tests. No existing assertion, expected status, or fixture was weakened. New tests check bytes, hidden listing, restart, MOVE, Trash, restore, and rejected metadata. These checks address behavior missing in the old code. The new rollback test only checks an oversized companion immediately after an empty PUT. It does not cover F5 or F6. F0 is in the new happy-path test call chain; the test must run before any claim that the feature works. Performance: The new companion write scans the full hidden metadata tree while holding the shared writer lock (`crates/calternal-fs/src/write.rs:196`) and scans it again under the operation lock (line 279). The provider also holds the shared Files mutation lock during metadata publication. Cost grows with all companion entries, including Trash. The new benchmark profile uses 30 visible files and a four-upload burst (`bench/webdav-lock-476.py:302`, line 367). It does not measure a large existing metadata tree or the 32-session Finder case used elsewhere in this branch. Extend that profile before drawing a worst-case performance conclusion. No new measurements were taken in this LIGHT review. ## Tracking Existing issues were searched for AppleDouble, upload-only access, rollback, and Blob store collection. All listed defects are introduced by this branch and are recorded on #648. No separate issue was filed. F5 and F6 share one rollback fix. No defect outside this branch was confirmed. ## Validation and known gaps `git diff --check`: exit 0; stdout and stderr were empty. `git fetch origin`: exit 0; stdout and stderr were empty. `origin/dev` stayed at the SHA above. Rust and web gates were not run. The LIGHT instructions prohibit builds and tests. There is no gate output to quote. No server, browser, Apple-client run, adversarial round, or performance run was started. No build output was created. DESIGN ends at §57 in both the reviewed head and fetched `origin/dev`. The requested §58 is absent. This review used the available performance rules and DESIGN §2, §24, §26, and §31 K4. ## Decisions - The specific read-only job rules control this review. The general build, benchmark, cleanup, and merge instructions do not apply. The remote was fetched once; no merge was performed. - Branch defects stay on the owned issue #648, as the review brief requires. - No product design decision was made. Opaque companion storage can preserve Finder bytes, but this report makes no claim about unified Tag mapping. - UX gaps closed: none; this job changes review documents only. UX gaps left: runtime Apple-client copy, tag, failure, and recovery behavior is unverified.
Author
Owner

Round 2 on staging c4a61e8cf090170f35b1bed3350d9de20c83ecd5 with the real macOS WebDAV mount:

A private 17-byte fixture with com.apple.metadata:kMDItemFinderComment copied twice through native cp. Both returned exit 1: could not copy extended attributes ... Operation not permitted. Both destination contents passed cmp (exit 0), and destination xattr -l was empty. Thus the metadata rejection persists, but this run did not reproduce the zero-byte destination. No claim that the earlier zero-byte defect is fixed.

Controls passed: Unicode/NFD filename copy (NFC on server), rename and move, append, 50,000,000-byte copy with independent DAV GET SHA-256 equality, 30-file/three-level tree copy with diff -r, tree rename and delete, file delete, and DAV-created/renamed folder visible on the Mac. Finder still names the volume after the User ID.

Round 2 on staging `c4a61e8cf090170f35b1bed3350d9de20c83ecd5` with the real macOS WebDAV mount: A private 17-byte fixture with `com.apple.metadata:kMDItemFinderComment` copied twice through native `cp`. Both returned exit 1: `could not copy extended attributes ... Operation not permitted`. Both destination contents passed `cmp` (exit 0), and destination `xattr -l` was empty. Thus the metadata rejection persists, but **this run did not reproduce the zero-byte destination**. No claim that the earlier zero-byte defect is fixed. Controls passed: Unicode/NFD filename copy (NFC on server), rename and move, append, 50,000,000-byte copy with independent DAV GET SHA-256 equality, 30-file/three-level tree copy with `diff -r`, tree rename and delete, file delete, and DAV-created/renamed folder visible on the Mac. Finder still names the volume after the User ID.
Author
Owner

Started next round on job/webdav-lock-476, checkpoint c9863ed9e, base 3f258302a. Merged fetched origin/dev once. Fixing independent review F0–F6 before gates. Latest verification policy defers full adversarial matrix, benchmark and Mac interop to the merge round. No deployment or push.

Started next round on `job/webdav-lock-476`, checkpoint `c9863ed9e`, base `3f258302a`. Merged fetched `origin/dev` once. Fixing independent review F0–F6 before gates. Latest verification policy defers full adversarial matrix, benchmark and Mac interop to the merge round. No deployment or push.
Author
Owner

Fixed F0/F1/F4 filesystem hooks in ee49867ce928fe81551007f88dd2d08a6a8b3808: validated internal parent for first writes; no metadata hook for non-Home/internal moves and deletes; confined metadata COPY and overwrite clearing. New regression tests cover COPY/reopen and internal MOVE/delete. Existing Blob store collection, expired archive and public listing tests remain unchanged and pass. The checkpoint metadata parent assertion conflicted with the established .calternal listing contract; the new store now uses reserved .calternal-appledouble so both contracts hold.

cargo clippy -p calternal-fs --all-targets -- -D warnings:

    Checking calternal-path v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-path)
    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 11.35s

cargo test -p calternal-fs result lines (verbatim):

test result: ok. 54 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 26.60s
test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 7.64s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The benchmark profile now seeds 2,048 existing companions and adds a 32-session burst. Measurements remain deferred to the merge round under the current verification policy. Provider gates are still in progress; this is not a deployment-ready claim.

Fixed F0/F1/F4 filesystem hooks in `ee49867ce928fe81551007f88dd2d08a6a8b3808`: validated internal parent for first writes; no metadata hook for non-Home/internal moves and deletes; confined metadata COPY and overwrite clearing. New regression tests cover COPY/reopen and internal MOVE/delete. Existing Blob store collection, expired archive and public listing tests remain unchanged and pass. The checkpoint metadata parent assertion conflicted with the established `.calternal` listing contract; the new store now uses reserved `.calternal-appledouble` so both contracts hold. `cargo clippy -p calternal-fs --all-targets -- -D warnings`: ``` Checking calternal-path v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-path) Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 11.35s ``` `cargo test -p calternal-fs` result lines (verbatim): ``` test result: ok. 54 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 26.60s test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 7.64s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` The benchmark profile now seeds 2,048 existing companions and adds a 32-session burst. Measurements remain deferred to the merge round under the current verification policy. Provider gates are still in progress; this is not a deployment-ready claim.
Author
Owner

Provider fixes and regression tests for F2/F3/F5/F6 are in place. Rollback now uses a durable item revision and operation ID under the mutation lock, and moves failed provisional copies to Trash. A metadata attempt distinguishes a provisional copy from an ordinary Apple empty file. Content upload abort and expiry keep their bound record through restart and past the initial copy window. New loopback HTTP coverage verifies body staging for collision-renamed companion PUTs.

The Files clippy gate failed in a dependency before checking project code because the shared compiler cache retained a deleted temp directory:

sccache: error: Failed to create temp dir
sccache: caused by: Failed to create temp dir

Retrying the remaining crate gates with RUSTC_WRAPPER= for this job only. The required target directory and low-memory settings stay in use. No shared cache service or other worktree was changed.

Additional filesystem review found that a long visible destination could discard a source companion, and metadata refusal could leave a content temporary. Added guards and a regression test before the final filesystem gate.

Provider fixes and regression tests for F2/F3/F5/F6 are in place. Rollback now uses a durable item revision and operation ID under the mutation lock, and moves failed provisional copies to Trash. A metadata attempt distinguishes a provisional copy from an ordinary Apple empty file. Content upload abort and expiry keep their bound record through restart and past the initial copy window. New loopback HTTP coverage verifies body staging for collision-renamed companion PUTs. The Files clippy gate failed in a dependency before checking project code because the shared compiler cache retained a deleted temp directory: ``` sccache: error: Failed to create temp dir sccache: caused by: Failed to create temp dir ``` Retrying the remaining crate gates with `RUSTC_WRAPPER=` for this job only. The required target directory and low-memory settings stay in use. No shared cache service or other worktree was changed. Additional filesystem review found that a long visible destination could discard a source companion, and metadata refusal could leave a content temporary. Added guards and a regression test before the final filesystem gate.
Author
Owner

Filesystem follow-up committed as 673d28683: reject tagged COPY/MOVE destinations that cannot represent the companion; clean only unpublished COPY staging on pre-journal failures. Existing storage assertions remain unchanged. Provider verification is still running; no deployment-ready claim.

cargo clippy -p calternal-fs --all-targets -- -D warnings:

    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 46s

cargo test -p calternal-fs result lines, verbatim:

test result: ok. 55 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 162.03s
test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 163.13s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
Filesystem follow-up committed as `673d28683`: reject tagged COPY/MOVE destinations that cannot represent the companion; clean only unpublished COPY staging on pre-journal failures. Existing storage assertions remain unchanged. Provider verification is still running; no deployment-ready claim. `cargo clippy -p calternal-fs --all-targets -- -D warnings`: ``` Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 46s ``` `cargo test -p calternal-fs` result lines, verbatim: ``` test result: ok. 55 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 162.03s test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 163.13s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ```
Author
Owner

Additional F2/F6 finding: a restricted Finder sequence (empty target PUT → companion PUT → content PUT) retained the provisional empty item and collision-renamed the final content a second time. The provider now continues only the trusted App Password copy claim, checks its captured Files item ID again inside upload creation, and persists the original request for completion/recovery. A focused test covers success, subsequent metadata, abort, and preservation of the older colliding file for both restricted access levels. Also fixed Home-relative provenance handling in upload completion and rollback, and retired rollback records on acknowledged metadata DELETE. These provider changes remain uncommitted until the Files gates pass; dependency compilation is in progress. No staging-ready claim.

Additional F2/F6 finding: a restricted Finder sequence (empty target PUT → companion PUT → content PUT) retained the provisional empty item and collision-renamed the final content a second time. The provider now continues only the trusted App Password copy claim, checks its captured Files item ID again inside upload creation, and persists the original request for completion/recovery. A focused test covers success, subsequent metadata, abort, and preservation of the older colliding file for both restricted access levels. Also fixed Home-relative provenance handling in upload completion and rollback, and retired rollback records on acknowledged metadata DELETE. These provider changes remain uncommitted until the Files gates pass; dependency compilation is in progress. No staging-ready claim.
Author
Owner

F4/UX completion finding: visible_is_attached rejected ._Folder because it required a regular file, and the directory hooks handled only the mirrored subtree (.DS_Store and child companions). The existing resource hooks now compose the folder's own companion and subtree for COPY, MOVE, Trash, restore and delete. New Filesystem regression covers the complete lifecycle/reopen/stale clearing; a provider regression covers Depth:0 COPY. The latest FS gates are rerunning for this code change before the Files/DAV/Path/server gates. Cached dependency compilation uses offline resolution to avoid additional shared-cache traffic; no dependency versions changed.

F4/UX completion finding: `visible_is_attached` rejected `._Folder` because it required a regular file, and the directory hooks handled only the mirrored subtree (.DS_Store and child companions). The existing resource hooks now compose the folder's own companion and subtree for COPY, MOVE, Trash, restore and delete. New Filesystem regression covers the complete lifecycle/reopen/stale clearing; a provider regression covers Depth:0 COPY. The latest FS gates are rerunning for this code change before the Files/DAV/Path/server gates. Cached dependency compilation uses offline resolution to avoid additional shared-cache traffic; no dependency versions changed.
Author
Owner

Folder companion lifecycle is committed as 10737e1e6. The revised Filesystem gates pass; Files/Path/DAV/server gates remain pending. This is not yet staging-ready.

cargo clippy -p calternal-fs --all-targets -- -D warnings:

    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 22s

cargo test -p calternal-fs result lines, verbatim:

test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 156.80s
test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.25s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s
Folder companion lifecycle is committed as `10737e1e6`. The revised Filesystem gates pass; Files/Path/DAV/server gates remain pending. This is not yet staging-ready. `cargo clippy -p calternal-fs --all-targets -- -D warnings`: ``` Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 22s ``` `cargo test -p calternal-fs` result lines, verbatim: ``` test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 156.80s test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.25s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s ```
Author
Owner

Files clippy passes with the provider and durable upload fixes. Files tests are queued next, then Path/DAV/server gates. The provider slice is not yet committed and this is not a staging-ready claim.

cargo clippy -p calternal-plugin-files --all-targets -- -D warnings output, verbatim:

    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs)
    Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-plugin)
    Checking calternal-location v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-location)
    Checking calternal-imap v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-imap)
    Checking calternal-tags v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-tags)
    Checking calternal-plugin-notes v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/notes)
    Checking calternal-plugin-files v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/files)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 17m 04s
Files clippy passes with the provider and durable upload fixes. Files tests are queued next, then Path/DAV/server gates. The provider slice is not yet committed and this is not a staging-ready claim. `cargo clippy -p calternal-plugin-files --all-targets -- -D warnings` output, verbatim: ``` Blocking waiting for file lock on package cache Blocking waiting for file lock on package cache Blocking waiting for file lock on package cache Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs) Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-plugin) Checking calternal-location v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-location) Checking calternal-imap v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-imap) Checking calternal-tags v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-tags) Checking calternal-plugin-notes v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/notes) Checking calternal-plugin-files v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/files) Finished `dev` profile [unoptimized + debuginfo] target(s) in 17m 04s ```
Author
Owner

#648 verification report

Status: not ready for deployment or merge. The four-hour job limit ends this round with verification incomplete. No push or deployment was done. The checkpoint and follow-up work are split into three concern commits with an unchanged final tree:

  • eff4ae2c202584af32210f8a13a59607c135c22b — confined storage and item lifecycle.
  • 7b81d22d73772abd4c09d2f91e767bf2823fb467 — provider publication and durable rollback.
  • 7ca53a061f0b08bd6f74c114468929ccc2031adf — probes/profile. HEAD.

Merged the fetched origin/dev (c4faf184d) before gates. Migration 19 was free in that fetched tree.

Verification:

cargo fmt --check: exit 0; stdout and stderr empty.

cargo clippy -p calternal-fs --all-targets -- -D warnings, output verbatim:

    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 22s

cargo test -p calternal-fs, result lines verbatim:

test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 156.80s
test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.25s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s

Latest cargo clippy -p calternal-plugin-files --all-targets -- -D warnings, output verbatim:

    Checking calternal-path v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-path)
    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs)
    Checking calternal-notes-core v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-notes-core)
    Checking calternal-auth v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-auth)
    Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-plugin)
    Checking calternal-location v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-location)
    Checking calternal-dav v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-dav)
    Checking calternal-imap v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-imap)
    Checking calternal-tags v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-tags)
    Checking calternal-plugin-notes v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/notes)
    Checking calternal-plugin-files v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/files)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 8m 28s

cargo test -p calternal-plugin-files, result lines verbatim (before the final pending-install fixes):

test result: FAILED. 147 passed; 15 failed; 1 ignored; 0 measured; 0 filtered out; finished in 442.02s

The failing run had 12 Finder failures caused by resolving the item through the public pending-install guard inside the install itself. The final fix reads the just-verified indexed ID under the publication lock. Cleanup now retires only an unpublished failed intent before checking identity, and has a regression test. The focused cargo test -p calternal-plugin-files finder_ -- --test-threads=2 rebuild was stopped at the job time limit, before any test ran. These final runtime fixes are not verified.

Existing failures left unchanged:

  • dev_files_schema_upgrades_through_share_log_and_sidecar_migrations: left: 19, right: 18. The job adds migration 19 for the durable copy record. The owner rule forbids changing an old expectation merely to pass; the orchestrator must decide how to update this migration coverage.
  • filename_search_filters_current_shares: Sqlx(Database(SqliteError { code: 5, message: "database is locked" })). Needs a focused rerun/triage.
  • internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm: writes, reconcile scans, and watcher adoption complete within five minutes: Elapsed(()), followed by an entry not found writer failure after timeout. Needs a focused rerun/triage; no expectation was relaxed.

Path, DAV and server per-crate clippy/tests remain pending. Full probe matrices, benchmark measurements and real Finder remain for the merge round under the current verification policy. Python syntax checks passed for both scripts. No web code changed.

Required next verification (same small-build Cargo environment, wrapper disabled):

cargo clippy -p calternal-plugin-files --all-targets -- -D warnings
cargo test -p calternal-plugin-files finder_ -- --test-threads=2
cargo test -p calternal-plugin-files filename_search_filters_current_shares -- --test-threads=2
cargo test -p calternal-plugin-files internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm -- --test-threads=2
cargo clippy -p calternal-path --all-targets -- -D warnings
cargo test -p calternal-path
cargo clippy -p calternal-dav --all-targets -- -D warnings
cargo test -p calternal-dav
cargo clippy -p calternal-server --all-targets -- -D warnings
cargo test -p calternal-server

Resolve the migration expectation with the orchestrator, then run the Files crate suite once. The final Tree is 7e1a6d37c4d897d0ae4310b749c6aaa3d7c52196.

Built: confined per-User AppleDouble storage; upload body staging; safe internal maintenance hooks; companion COPY/overwrite; App Password copy claims; stable item and conditional checks; durable provisional rollback through restart/expiry; restricted Finder content continuation. Added focused provider and loopback HTTP regressions. Extended the profile to 2,048 companions, four maximum-size uploads, and 32 sessions. No existing assertion was weakened.

Files: crates/calternal-fs/src/{appledouble,file_ops,lib,path,quota,trash,write}.rs, crates/calternal-path/src/lib.rs, crates/calternal-dav/src/files.rs, crates/calternal-server/src/wire.rs, crates/plugins/files/src/{dav,uploads,lib}.rs, crates/plugins/files/migrations/0019_dav_copies.sql, bench/webdav-lock-476.py; checkpoint probe changes remain in tests/adversarial/webdav.py.

Decisions:

  • Keep opaque metadata in reserved .calternal-appledouble, preserving the established .calternal listing contract. The unverified checkpoint was not deployed, so no store migration is required.
  • Use the non-secret App Password ID as the Installation boundary. A claim lasts 15 minutes; a content upload already bound to its exact operation survives age and restart.
  • An ordinary empty PUT is a real empty file until a metadata attempt starts a provisional copy. Time alone never deletes it. Failed provisional copies go to recoverable Trash.
  • Refuse COPY/MOVE when the destination cannot represent a tagged companion, rather than dropping acknowledged metadata.

UX gaps closed: first companion PUT; metadata on renamed restricted copies; false empty targets after observed upload failure/expiry; later content/metadata/deletion protected against old aborts; metadata COPY and stale overwrite clearing; folder attribute lifecycle; companion body receipt through the HTTP adapter; restricted empty→metadata→content sequence.

Known gaps / UX gaps left:

  • Real Finder behavior and native tag display require the staging check. No claim of Finder-to-calternal Tags mapping; metadata bytes are opaque.
  • When the client stops after an acknowledged empty target/metadata without starting or failing content, the server has no reliable failure signal; it preserves the file.
  • Requests with overlong companion components are rejected at path validation; Finder handling of such names needs the staging check.
  • Simultaneous copies of the same request name using the same App Password share the latest recent claim; DAV provides no explicit copy transaction identifier.
  • No performance numbers were collected under the latest verification policy.

For the merge round:

  1. CALTERNAL_WEBDAV_PROBE_ONLY=1 cargo test -p calternal-plugin-files local_server_scripted_probe_and_rclone_ten_thousand_small_files -- --ignored --nocapture — real loopback protocol matrix, including opaque companion persistence and rejected uploads. The broad authz/XUser/robustness matrices also belong to the combined branch.
  2. In the assigned perf VM slot, with private DAV environment already set: flock /root/perf.lock bash -c 'cat /proc/loadavg; python3 bench/webdav-lock-476.py --appledouble-profile --pid "$CALTERNAL_WEBDAV_SERVER_PID" --output artifacts/webdav-648.json' — collect p50/p95, CPU and RSS for populated metadata and bursts; compare with docs/perf/baseline.json. Do not compile on the VM.
  3. Orchestrator deploys this build to https://dev.calternal.com and runs the real Finder check. This job does not deploy.

Cleanup: final cargo clean returned exit 0. Its output, verbatim:

     Removed 0 files

The job target directory was removed. No web build output existed. The worktree is clean. The initial shared sccache failure came from its deleted temporary directory; Rust gates then used a disabled wrapper. Later native compiler cache shutdowns fell back to local compilation.

# #648 verification report Status: **not ready for deployment or merge**. The four-hour job limit ends this round with verification incomplete. No push or deployment was done. The checkpoint and follow-up work are split into three concern commits with an unchanged final tree: - `eff4ae2c202584af32210f8a13a59607c135c22b` — confined storage and item lifecycle. - `7b81d22d73772abd4c09d2f91e767bf2823fb467` — provider publication and durable rollback. - `7ca53a061f0b08bd6f74c114468929ccc2031adf` — probes/profile. **HEAD**. Merged the fetched `origin/dev` (`c4faf184d`) before gates. Migration 19 was free in that fetched tree. Verification: `cargo fmt --check`: exit 0; stdout and stderr empty. `cargo clippy -p calternal-fs --all-targets -- -D warnings`, output verbatim: ``` Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 22s ``` `cargo test -p calternal-fs`, result lines verbatim: ``` test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 156.80s test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 12.25s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s ``` Latest `cargo clippy -p calternal-plugin-files --all-targets -- -D warnings`, output verbatim: ``` Checking calternal-path v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-path) Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-fs) Checking calternal-notes-core v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-notes-core) Checking calternal-auth v0.1.0 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-auth) Checking calternal-plugin v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-plugin) Checking calternal-location v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-location) Checking calternal-dav v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-dav) Checking calternal-imap v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-imap) Checking calternal-tags v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/calternal-tags) Checking calternal-plugin-notes v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/notes) Checking calternal-plugin-files v0.0.1 (/home/kayg/Developer/calternal-wt/webdav-lock-476/crates/plugins/files) Finished `dev` profile [unoptimized + debuginfo] target(s) in 8m 28s ``` `cargo test -p calternal-plugin-files`, result lines verbatim (before the final pending-install fixes): ``` test result: FAILED. 147 passed; 15 failed; 1 ignored; 0 measured; 0 filtered out; finished in 442.02s ``` The failing run had 12 Finder failures caused by resolving the item through the public pending-install guard inside the install itself. The final fix reads the just-verified indexed ID under the publication lock. Cleanup now retires only an unpublished failed intent before checking identity, and has a regression test. The focused `cargo test -p calternal-plugin-files finder_ -- --test-threads=2` rebuild was stopped at the job time limit, before any test ran. These final runtime fixes are **not verified**. Existing failures left unchanged: - `dev_files_schema_upgrades_through_share_log_and_sidecar_migrations`: `left: 19`, `right: 18`. The job adds migration 19 for the durable copy record. The owner rule forbids changing an old expectation merely to pass; the orchestrator must decide how to update this migration coverage. - `filename_search_filters_current_shares`: `Sqlx(Database(SqliteError { code: 5, message: "database is locked" }))`. Needs a focused rerun/triage. - `internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm`: `writes, reconcile scans, and watcher adoption complete within five minutes: Elapsed(())`, followed by an `entry not found` writer failure after timeout. Needs a focused rerun/triage; no expectation was relaxed. Path, DAV and server per-crate clippy/tests remain pending. Full probe matrices, benchmark measurements and real Finder remain for the merge round under the current verification policy. Python syntax checks passed for both scripts. No web code changed. Required next verification (same small-build Cargo environment, wrapper disabled): ```sh cargo clippy -p calternal-plugin-files --all-targets -- -D warnings cargo test -p calternal-plugin-files finder_ -- --test-threads=2 cargo test -p calternal-plugin-files filename_search_filters_current_shares -- --test-threads=2 cargo test -p calternal-plugin-files internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm -- --test-threads=2 cargo clippy -p calternal-path --all-targets -- -D warnings cargo test -p calternal-path cargo clippy -p calternal-dav --all-targets -- -D warnings cargo test -p calternal-dav cargo clippy -p calternal-server --all-targets -- -D warnings cargo test -p calternal-server ``` Resolve the migration expectation with the orchestrator, then run the Files crate suite once. The final Tree is `7e1a6d37c4d897d0ae4310b749c6aaa3d7c52196`. Built: confined per-User AppleDouble storage; upload body staging; safe internal maintenance hooks; companion COPY/overwrite; App Password copy claims; stable item and conditional checks; durable provisional rollback through restart/expiry; restricted Finder content continuation. Added focused provider and loopback HTTP regressions. Extended the profile to 2,048 companions, four maximum-size uploads, and 32 sessions. No existing assertion was weakened. Files: `crates/calternal-fs/src/{appledouble,file_ops,lib,path,quota,trash,write}.rs`, `crates/calternal-path/src/lib.rs`, `crates/calternal-dav/src/files.rs`, `crates/calternal-server/src/wire.rs`, `crates/plugins/files/src/{dav,uploads,lib}.rs`, `crates/plugins/files/migrations/0019_dav_copies.sql`, `bench/webdav-lock-476.py`; checkpoint probe changes remain in `tests/adversarial/webdav.py`. Decisions: - Keep opaque metadata in reserved `.calternal-appledouble`, preserving the established `.calternal` listing contract. The unverified checkpoint was not deployed, so no store migration is required. - Use the non-secret App Password ID as the Installation boundary. A claim lasts 15 minutes; a content upload already bound to its exact operation survives age and restart. - An ordinary empty PUT is a real empty file until a metadata attempt starts a provisional copy. Time alone never deletes it. Failed provisional copies go to recoverable Trash. - Refuse COPY/MOVE when the destination cannot represent a tagged companion, rather than dropping acknowledged metadata. UX gaps closed: first companion PUT; metadata on renamed restricted copies; false empty targets after observed upload failure/expiry; later content/metadata/deletion protected against old aborts; metadata COPY and stale overwrite clearing; folder attribute lifecycle; companion body receipt through the HTTP adapter; restricted empty→metadata→content sequence. Known gaps / UX gaps left: - Real Finder behavior and native tag display require the staging check. No claim of Finder-to-calternal Tags mapping; metadata bytes are opaque. - When the client stops after an acknowledged empty target/metadata without starting or failing content, the server has no reliable failure signal; it preserves the file. - Requests with overlong companion components are rejected at path validation; Finder handling of such names needs the staging check. - Simultaneous copies of the same request name using the same App Password share the latest recent claim; DAV provides no explicit copy transaction identifier. - No performance numbers were collected under the latest verification policy. For the merge round: 1. `CALTERNAL_WEBDAV_PROBE_ONLY=1 cargo test -p calternal-plugin-files local_server_scripted_probe_and_rclone_ten_thousand_small_files -- --ignored --nocapture` — real loopback protocol matrix, including opaque companion persistence and rejected uploads. The broad authz/XUser/robustness matrices also belong to the combined branch. 2. In the assigned perf VM slot, with private DAV environment already set: `flock /root/perf.lock bash -c 'cat /proc/loadavg; python3 bench/webdav-lock-476.py --appledouble-profile --pid "$CALTERNAL_WEBDAV_SERVER_PID" --output artifacts/webdav-648.json'` — collect p50/p95, CPU and RSS for populated metadata and bursts; compare with `docs/perf/baseline.json`. Do not compile on the VM. 3. Orchestrator deploys this build to `https://dev.calternal.com` and runs the real Finder check. This job does not deploy. Cleanup: final `cargo clean` returned exit 0. Its output, verbatim: ``` Removed 0 files ``` The job target directory was removed. No web build output existed. The worktree is clean. The initial shared sccache failure came from its deleted temporary directory; Rust gates then used a disabled wrapper. Later native compiler cache shutdowns fell back to local compilation.
Author
Owner

Orchestrator check of job/webdav-lock-476 (local only; no push, no deploy).

Head: 51a9a5c80971354a49edf8ce37b2c97967f24255

Changes in this pass

  • 51a9a5c80 moves Files migration 0019_dav_copies to 0022_dav_copies. Version 19 is taken by 0019_directory_parent_fingerprint (merge-round-7a, files-631) and 0019_directory_birth (dirid-627). 20 and 21 are taken on merge-round-7a and noext-851. calternal-db needs only ascending versions, so the gap is safe. The schema-upgrade test now expects MAX(version) = 22, because this branch adds that migration. No data assertion changed.
  • Note: dirid-627 and perf-495 (0019_photo_media_scan) also clash on 19. They must renumber on their own branches.

The 15 earlier failures

  • 12 Finder failures (147 passed; 15 failed): that run was before 7b81d22d7. Root cause: complete_install resolved the new item through the public pending-install guard during the install itself. The head reads the indexed item ID under the publication lock. All Finder tests pass at the head without more code changes.
  • dev_files_schema_upgrades_through_share_log_and_sidecar_migrations (left: 19, right: 18): fixed by the renumber above.
  • filename_search_filters_current_shares (SQLite busy) and internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm (5 min timeout): not reproduced. They passed in 2 full runs and 3 focused reruns. The failing run took 442 s at load average about 40, so host contention is the probable cause. No expectation changed.

Review P1/P2 and their regression tests (all pass)

  • F0 first companion write: finder_root_directory_metadata_is_persistent, webdav_finder_appledouble_is_hidden_persistent_and_follows_moves, finder_http_companion_put_stages_the_body_for_renamed_copy
  • F1 internal deletion: internal_delete_and_move_do_not_require_a_home (fs). The existing Blob store and archive tests are unchanged.
  • F2 upload-only companions: finder_restricted_companions_follow_their_own_collision_renamed_copy, finder_restricted_content_continues_only_its_provisional_copy
  • F3 item identity: finder_companion_rechecks_item_identity_and_preconditions
  • F5 rollback data loss: finder_rollback_preserves_later_empty_revision_and_versions, finder_old_real_empty_file_survives_failed_metadata_and_content_edits, finder_old_metadata_abort_preserves_*
  • F4 COPY (P2): finder_copy_preserves_folder_and_file_metadata, finder_folder_attributes_survive_shallow_copy, fs copy_preserves_metadata_and_untagged_overwrite_clears_it
  • F6 (P2): finder_content_abort_after_metadata_and_restart_removes_only_provisional_copy, finder_expired_content_upload_rolls_back_after_restart, finder_failed_install_intent_rolls_back_its_unchanged_target

Gates (verbatim)

cargo fmt --check: exit 0, no output.

Clippy --all-targets -- -D warnings, each exit 0:

calternal-fs:           Finished `dev` profile [unoptimized + debuginfo] target(s) in 28.43s
calternal-path:         Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.80s
calternal-dav:          Finished `dev` profile [unoptimized + debuginfo] target(s) in 49.64s
calternal-plugin-files: Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 07s

cargo test -p calternal-fs:

test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s
test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-path:

test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-dav:

test result: ok. 45 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s
test result: ok. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

cargo test -p calternal-plugin-files:

test result: ok. 163 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 79.05s
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Still open for the merge round: calternal-server clippy and tests, the ignored loopback probe, the adversarial round, perf, and the real Finder check.

Orchestrator check of `job/webdav-lock-476` (local only; no push, no deploy). Head: `51a9a5c80971354a49edf8ce37b2c97967f24255` ## Changes in this pass - `51a9a5c80` moves Files migration `0019_dav_copies` to `0022_dav_copies`. Version 19 is taken by `0019_directory_parent_fingerprint` (merge-round-7a, files-631) and `0019_directory_birth` (dirid-627). 20 and 21 are taken on merge-round-7a and noext-851. calternal-db needs only ascending versions, so the gap is safe. The schema-upgrade test now expects `MAX(version) = 22`, because this branch adds that migration. No data assertion changed. - Note: dirid-627 and perf-495 (`0019_photo_media_scan`) also clash on 19. They must renumber on their own branches. ## The 15 earlier failures - 12 Finder failures (`147 passed; 15 failed`): that run was before `7b81d22d7`. Root cause: `complete_install` resolved the new item through the public pending-install guard during the install itself. The head reads the indexed item ID under the publication lock. All Finder tests pass at the head without more code changes. - `dev_files_schema_upgrades_through_share_log_and_sidecar_migrations` (`left: 19, right: 18`): fixed by the renumber above. - `filename_search_filters_current_shares` (SQLite busy) and `internal_temp_paths_never_enter_index_during_atomic_write_reconcile_storm` (5 min timeout): not reproduced. They passed in 2 full runs and 3 focused reruns. The failing run took 442 s at load average about 40, so host contention is the probable cause. No expectation changed. ## Review P1/P2 and their regression tests (all pass) - F0 first companion write: `finder_root_directory_metadata_is_persistent`, `webdav_finder_appledouble_is_hidden_persistent_and_follows_moves`, `finder_http_companion_put_stages_the_body_for_renamed_copy` - F1 internal deletion: `internal_delete_and_move_do_not_require_a_home` (fs). The existing Blob store and archive tests are unchanged. - F2 upload-only companions: `finder_restricted_companions_follow_their_own_collision_renamed_copy`, `finder_restricted_content_continues_only_its_provisional_copy` - F3 item identity: `finder_companion_rechecks_item_identity_and_preconditions` - F5 rollback data loss: `finder_rollback_preserves_later_empty_revision_and_versions`, `finder_old_real_empty_file_survives_failed_metadata_and_content_edits`, `finder_old_metadata_abort_preserves_*` - F4 COPY (P2): `finder_copy_preserves_folder_and_file_metadata`, `finder_folder_attributes_survive_shallow_copy`, fs `copy_preserves_metadata_and_untagged_overwrite_clears_it` - F6 (P2): `finder_content_abort_after_metadata_and_restart_removes_only_provisional_copy`, `finder_expired_content_upload_rolls_back_after_restart`, `finder_failed_install_intent_rolls_back_its_unchanged_target` ## Gates (verbatim) `cargo fmt --check`: exit 0, no output. Clippy `--all-targets -- -D warnings`, each exit 0: ``` calternal-fs: Finished `dev` profile [unoptimized + debuginfo] target(s) in 28.43s calternal-path: Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.80s calternal-dav: Finished `dev` profile [unoptimized + debuginfo] target(s) in 49.64s calternal-plugin-files: Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 07s ``` `cargo test -p calternal-fs`: ``` test result: ok. 56 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s test result: ok. 42 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo test -p calternal-path`: ``` test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo test -p calternal-dav`: ``` test result: ok. 45 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s test result: ok. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.05s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` `cargo test -p calternal-plugin-files`: ``` test result: ok. 163 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 79.05s test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Still open for the merge round: calternal-server clippy and tests, the ignored loopback probe, the adversarial round, perf, and the real Finder check.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#648
No description provided.