Core: calternal-fs safe storage layer (single writer, dedup, trash, versions, quota) #2

Closed
opened 2026-09-24 07:05:39 +00:00 by kayg · 17 comments
Owner

Codex job fs. Task brief lives with the orchestrator; the job comments progress here.

Base: 5f15b1e66a

Codex job `fs`. Task brief lives with the orchestrator; the job comments progress here. Base: 5f15b1e66ac9f1e781add30ed77a2594dda4472e
Author
Owner

Started job/fs at base 5f15b1e66a. Read CLAUDE.md, docs/DESIGN.md, and CONTEXT.md. Implementing only crates/calternal-fs/**; standalone crate, Linux handle-relative filesystem access, and required gates.

Started job/fs at base 5f15b1e66ac9f1e781add30ed77a2594dda4472e. Read CLAUDE.md, docs/DESIGN.md, and CONTEXT.md. Implementing only crates/calternal-fs/**; standalone crate, Linux handle-relative filesystem access, and required gates.
Author
Owner

Finding: this branch has no crate or root workspace. Verified standalone dependency versions with cargo search (rustix 1.1.5, Tokio 1.53.1, proptest 1.11.0). Implemented the initial handle-relative Root and RelPath API; the first integration tests now pass (2 passed; 0 failed). Remaining work includes trash, versions thinning, concurrency and race tests, and quota edge cases.

Finding: this branch has no crate or root workspace. Verified standalone dependency versions with cargo search (rustix 1.1.5, Tokio 1.53.1, proptest 1.11.0). Implemented the initial handle-relative Root and RelPath API; the first integration tests now pass (`2 passed; 0 failed`). Remaining work includes trash, versions thinning, concurrency and race tests, and quota edge cases.
Author
Owner

Finding: openat2 with BENEATH|NO_SYMLINKS|NO_MAGICLINKS rejects a planted symlink and a thread repeatedly swapping a directory for a symlink; the race test observed no outside write across 300 attempts. Identical concurrent writes produced the same inode, and replacing one hardlinked path preserved the other's content. The full gate run exposed only formatting and a Clippy argument-count warning so far; addressing those while completing quota and version behavior.

Finding: openat2 with BENEATH|NO_SYMLINKS|NO_MAGICLINKS rejects a planted symlink and a thread repeatedly swapping a directory for a symlink; the race test observed no outside write across 300 attempts. Identical concurrent writes produced the same inode, and replacing one hardlinked path preserved the other's content. The full gate run exposed only formatting and a Clippy argument-count warning so far; addressing those while completing quota and version behavior.
Author
Owner

Review finding: copy and cross-home move could bypass home quotas; trash did not carry a file's path-keyed versions; zero-byte overwrites skipped version capture; the thinning function was not yet applied. I am adding regression tests and fixing these before commit.

Review finding: copy and cross-home move could bypass home quotas; trash did not carry a file's path-keyed versions; zero-byte overwrites skipped version capture; the thinning function was not yet applied. I am adding regression tests and fixing these before commit.
Author
Owner

Finished on job/fs at head dc75eafcc7. Built standalone calternal-fs within crates/calternal-fs/**. cargo clean removed 781 files, 357.0MiB; worktree is clean. Review findings on quota, trash/version transfer, zero-byte versions, overwrite history, and thinning were fixed and covered by tests.

Gate commands and output verbatim (all exit 0):

CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 cargo fmt --manifest-path crates/calternal-fs/Cargo.toml --check

CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 cargo clippy --manifest-path crates/calternal-fs/Cargo.toml --all-targets -- -D warnings

    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.15s

CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 cargo test --manifest-path crates/calternal-fs/Cargo.toml

   Compiling calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1.94s
     Running unittests src/lib.rs (crates/calternal-fs/target/debug/deps/calternal_fs-f4bd95ffd2e65b49)

running 2 tests
test tests::relative_paths_reject_ambiguous_components ... ok
test tests::thinning_keeps_every_version_in_first_day ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/storage.rs (crates/calternal-fs/target/debug/deps/storage-68cd26c02aee4a19)

running 19 tests
test configured_quota_covers_copy_and_cross_home_move ... ok
test copy_of_deduped_file_reuses_blob ... ok
test dedup_concurrent_writes ... ok
test orphan_blobs_are_collected ... ok
test replace_does_not_modify_another_hardlink ... ok
test symlink_parent_is_never_followed ... ok
test overwriting_move_carries_source_versions_and_preserves_target ... ok
test directory_pages_advance_with_cursor ... ok
test overwriting_with_copy_or_move_keeps_destination_version ... ok
test thinning_keeps_recent_and_bucket_representatives ... ok
test trash_collisions_get_distinct_names_and_empty ... ok
test moving_file_carries_versions ... ok
test trash_restores_versions_without_mixing_new_file ... ok
test traversal_is_rejected ... ok
test version_counts_against_quota ... ok
test zero_byte_file_gets_version_and_thins ... ok
test trash_round_trip_and_quota ... ok
test symlink_swap_race_never_writes_outside ... ok
test arbitrary_paths_never_escape ... ok

test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

   Doc-tests calternal_fs

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Known limits: a move refuses with Exists if the destination has an existing version-key directory from a different prior file; a caller must resolve that history collision. Home quota limits are configured in memory via Root::set_quota and must be reapplied by the server after restart. The in-process writer mutex does not coordinate separate server processes.

Decisions for owner confirmation: UTF-8-only RelPath with literal percent rejected; freedesktop Trash under each home with a private .Trash/versions archive; version key is path hash plus first content hash; retention tiers are all for 1 day, hourly through 1 week, daily through 4 weeks, weekly through 12 weeks, then monthly.

Finished on job/fs at head dc75eafcc78c4c2d608025c56585093eddef2781. Built standalone calternal-fs within crates/calternal-fs/**. cargo clean removed 781 files, 357.0MiB; worktree is clean. Review findings on quota, trash/version transfer, zero-byte versions, overwrite history, and thinning were fixed and covered by tests. Gate commands and output verbatim (all exit 0): `CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 cargo fmt --manifest-path crates/calternal-fs/Cargo.toml --check` ``` ``` `CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 cargo clippy --manifest-path crates/calternal-fs/Cargo.toml --all-targets -- -D warnings` ``` Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.15s ``` `CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_INCREMENTAL=0 cargo test --manifest-path crates/calternal-fs/Cargo.toml` ``` Compiling calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs) Finished `test` profile [unoptimized + debuginfo] target(s) in 1.94s Running unittests src/lib.rs (crates/calternal-fs/target/debug/deps/calternal_fs-f4bd95ffd2e65b49) running 2 tests test tests::relative_paths_reject_ambiguous_components ... ok test tests::thinning_keeps_every_version_in_first_day ... ok test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Running tests/storage.rs (crates/calternal-fs/target/debug/deps/storage-68cd26c02aee4a19) running 19 tests test configured_quota_covers_copy_and_cross_home_move ... ok test copy_of_deduped_file_reuses_blob ... ok test dedup_concurrent_writes ... ok test orphan_blobs_are_collected ... ok test replace_does_not_modify_another_hardlink ... ok test symlink_parent_is_never_followed ... ok test overwriting_move_carries_source_versions_and_preserves_target ... ok test directory_pages_advance_with_cursor ... ok test overwriting_with_copy_or_move_keeps_destination_version ... ok test thinning_keeps_recent_and_bucket_representatives ... ok test trash_collisions_get_distinct_names_and_empty ... ok test moving_file_carries_versions ... ok test trash_restores_versions_without_mixing_new_file ... ok test traversal_is_rejected ... ok test version_counts_against_quota ... ok test zero_byte_file_gets_version_and_thins ... ok test trash_round_trip_and_quota ... ok test symlink_swap_race_never_writes_outside ... ok test arbitrary_paths_never_escape ... ok test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s Doc-tests calternal_fs running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` Known limits: a move refuses with Exists if the destination has an existing version-key directory from a different prior file; a caller must resolve that history collision. Home quota limits are configured in memory via Root::set_quota and must be reapplied by the server after restart. The in-process writer mutex does not coordinate separate server processes. Decisions for owner confirmation: UTF-8-only RelPath with literal percent rejected; freedesktop Trash under each home with a private .Trash/versions archive; version key is path hash plus first content hash; retention tiers are all for 1 day, hourly through 1 week, daily through 4 weeks, weekly through 12 weeks, then monthly.
Author
Owner

Starting read-only audit on branch job/fs. Base SHA: 5f15b1e66a. Head SHA: dc75eafcc7. Reviewing confinement, data integrity, crash safety, quota, and contract coverage.

Starting read-only audit on branch job/fs. Base SHA: 5f15b1e66ac9f1e781add30ed77a2594dda4472e. Head SHA: dc75eafcc78c4c2d608025c56585093eddef2781. Reviewing confinement, data integrity, crash safety, quota, and contract coverage.
Author
Owner

Read-only audit findings at dc75eaf: (1) Root::write accepts reserved .cas paths; a digest-named blob can be populated with different bytes, and subsequent uploads trust EEXIST without verifying content (src/lib.rs:65-75, 483-535). (2) collect_blobs shares only a per-Root lock with write; copy takes no lock, and separate Root handles have independent locks (121, 742-813, 1058-1075), allowing GC/link races. (3) quota maps are per Root and write's explicit quota overrides the configured limit (122-123, 383-410, 423-449). I am preparing concrete scratch cases outside the worktree. Findings are being appended to the requested state file.

Read-only audit findings at dc75eaf: (1) Root::write accepts reserved .cas paths; a digest-named blob can be populated with different bytes, and subsequent uploads trust EEXIST without verifying content (src/lib.rs:65-75, 483-535). (2) collect_blobs shares only a per-Root lock with write; copy takes no lock, and separate Root handles have independent locks (121, 742-813, 1058-1075), allowing GC/link races. (3) quota maps are per Root and write's explicit quota overrides the configured limit (122-123, 383-410, 423-449). I am preparing concrete scratch cases outside the worktree. Findings are being appended to the requested state file.
Author
Owner

Additional read-only findings at dc75eaf: delete unlinks the live path but leaves its previous versions on disk (src/lib.rs:814-824, 617-624); the scratch probe observed deleted path exists=false version bytes remain=1. The live-file and version-directory renames in move, trash, and restore are separate operations, so a process crash between them leaves inconsistent state (705-735, 944-982, 1012-1025). The symlink-swap test discards every write result and can pass if all writes fail (tests/storage.rs:137-167). A crafted .trashinfo traversal path was rejected by RelPath validation in the probe. The full evidence and command outputs are in /home/kayg/.local/state/codex-jobs/calternal/fs-review/findings.md.

Additional read-only findings at dc75eaf: `delete` unlinks the live path but leaves its previous versions on disk (src/lib.rs:814-824, 617-624); the scratch probe observed `deleted path exists=false version bytes remain=1`. The live-file and version-directory renames in move, trash, and restore are separate operations, so a process crash between them leaves inconsistent state (705-735, 944-982, 1012-1025). The symlink-swap test discards every write result and can pass if all writes fail (tests/storage.rs:137-167). A crafted `.trashinfo` traversal path was rejected by RelPath validation in the probe. The full evidence and command outputs are in /home/kayg/.local/state/codex-jobs/calternal/fs-review/findings.md.
Author
Owner

Read-only audit complete. Branch job/fs, head dc75eafcc7; worktree clean, no commit. Findings with file:line evidence and probes: /home/kayg/.local/state/codex-jobs/calternal/fs-review/findings.md. Confirmed blob poisoning through public .cas paths, per-Root quota bypass/isolation, surviving versions after delete, and literal-percent rejection. Found a GC/link handoff race and multi-operation crash windows from source interleavings; these need fault-injection coverage. Crafted .trashinfo traversal was rejected. No direct symlink escape or ordinary in-place shared-blob mutation found. Separate Cargo target was deleted after cargo clean.

Gate output verbatim (trailing command summaries):
cargo fmt --manifest-path crates/calternal-fs/Cargo.toml --check: no output, exit 0.
cargo clippy --manifest-path crates/calternal-fs/Cargo.toml --all-targets -- -D warnings:
Finished dev profile [unoptimized + debuginfo] target(s) in 8.02s
cargo test --manifest-path crates/calternal-fs/Cargo.toml:
Doc-tests calternal_fs

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Earlier in the same test output:
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

Read-only audit complete. Branch job/fs, head dc75eafcc78c4c2d608025c56585093eddef2781; worktree clean, no commit. Findings with file:line evidence and probes: /home/kayg/.local/state/codex-jobs/calternal/fs-review/findings.md. Confirmed blob poisoning through public .cas paths, per-Root quota bypass/isolation, surviving versions after delete, and literal-percent rejection. Found a GC/link handoff race and multi-operation crash windows from source interleavings; these need fault-injection coverage. Crafted .trashinfo traversal was rejected. No direct symlink escape or ordinary in-place shared-blob mutation found. Separate Cargo target was deleted after cargo clean. Gate output verbatim (trailing command summaries): `cargo fmt --manifest-path crates/calternal-fs/Cargo.toml --check`: no output, exit 0. `cargo clippy --manifest-path crates/calternal-fs/Cargo.toml --all-targets -- -D warnings`: Finished `dev` profile [unoptimized + debuginfo] target(s) in 8.02s `cargo test --manifest-path crates/calternal-fs/Cargo.toml`: Doc-tests calternal_fs running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s Earlier in the same test output: test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
Author
Owner

Started audit fixes on branch job/fs at base dc75eafcc7. Read the audit and main:docs/DESIGN.md. Scope is crates/calternal-fs/** only. I will report regression failures before fixes and gate output with the final head SHA.

Started audit fixes on branch job/fs at base dc75eafcc78c4c2d608025c56585093eddef2781. Read the audit and main:docs/DESIGN.md. Scope is crates/calternal-fs/** only. I will report regression failures before fixes and gate output with the final head SHA.
Author
Owner

Regression tests added and run before fixes. reserved_internals_are_not_public_paths fails at .cas/x; quota_is_shared_and_override_only_lowers accepts a 2-byte write under a 1-byte configured limit with Some(u64::MAX); existing_blob_with_wrong_bytes_is_replaced reads poison rather than wanted; traversal_is_rejected fails because the literal %2e%2e filename is rejected. Each command exited 101. These reproduce audit findings 1, 3, and 4 with the current implementation.

Regression tests added and run before fixes. `reserved_internals_are_not_public_paths` fails at `.cas/x`; `quota_is_shared_and_override_only_lowers` accepts a 2-byte write under a 1-byte configured limit with Some(u64::MAX); `existing_blob_with_wrong_bytes_is_replaced` reads `poison` rather than `wanted`; `traversal_is_rejected` fails because the literal `%2e%2e` filename is rejected. Each command exited 101. These reproduce audit findings 1, 3, and 4 with the current implementation.
Author
Owner

Further pre-fix regressions: mirrored_versions_follow_directory_and_delete exits 101 because permanent deletion leaves users/u/.calternal/versions/renamed/a; percent_name_survives_trash_round_trip exits 101 with InvalidPath on restore. The former also confirms that directory rename can move the mirrored subtree, but cleanup is missing.

Further pre-fix regressions: `mirrored_versions_follow_directory_and_delete` exits 101 because permanent deletion leaves `users/u/.calternal/versions/renamed/a`; `percent_name_survives_trash_round_trip` exits 101 with InvalidPath on restore. The former also confirms that directory rename can move the mirrored subtree, but cleanup is missing.
Author
Owner

Implemented and verified shared per-data-directory coordination, bounded quota overrides, blob hash/mode verification on reuse, mirrored versions, reserved public-path filtering, and recoverable namespace steps. The deterministic GC test pauses an upload while it holds the shared blob-store lock and runs GC through a second Root; GC waits and the installed file remains readable. A fault-injection test fails after each move, trash, restore, and write journal step, opens a fresh Root, calls recover, and checks the resulting file and versions.

Implemented and verified shared per-data-directory coordination, bounded quota overrides, blob hash/mode verification on reuse, mirrored versions, reserved public-path filtering, and recoverable namespace steps. The deterministic GC test pauses an upload while it holds the shared blob-store lock and runs GC through a second Root; GC waits and the installed file remains readable. A fault-injection test fails after each move, trash, restore, and write journal step, opens a fresh Root, calls recover, and checks the resulting file and versions.
Author
Owner

Additional recovery work: overwrite Version links now appear as journal steps before the content rename. The write journal also records blob publication (new link or replacement of a corrupt digest entry) before installing the destination. Fault injection now covers each overwrite step for move, copy, and write; a fresh Root replays the journal and finds the expected content and Version. The GC interleaving test uses a pre-existing one-link canonical blob and pauses the writer before linking it into the home, while a second Root tries GC.

Additional recovery work: overwrite Version links now appear as journal steps before the content rename. The write journal also records blob publication (new link or replacement of a corrupt digest entry) before installing the destination. Fault injection now covers each overwrite step for move, copy, and write; a fresh Root replays the journal and finds the expected content and Version. The GC interleaving test uses a pre-existing one-link canonical blob and pauses the writer before linking it into the home, while a second Root tries GC.
Author
Owner

The folder-move work exposed another quota bypass: move_path charged a directory's stat size rather than its contained file bytes when crossing homes. The new cross_home_folder_move_checks_all_file_bytes regression exits 101 before the fix, accepting a 16,384-byte folder move into a home limited to 100 bytes. I am changing the growth calculation to walk the source folder.

The folder-move work exposed another quota bypass: `move_path` charged a directory's stat size rather than its contained file bytes when crossing homes. The new `cross_home_folder_move_checks_all_file_bytes` regression exits 101 before the fix, accepting a 16,384-byte folder move into a home limited to 100 bytes. I am changing the growth calculation to walk the source folder.
Author
Owner

Finished on branch job/fs at head b64882b77c. Committed only crates/calternal-fs/**.

Built: public user-path/internal-path separation, canonical blob verification and repair, shared per-data-directory quota limits and blob operation lock, mirrored Versions, folder moves and Version lifecycle, journal + Root::recover() for move/trash/restore/write/copy/delete steps, percent names, stronger race coverage, and source split. The independent audit's planted-hardlink case remains bounded by the documented single-writer/OS ownership assumption.

Known gaps: quota limits live in the process-wide Root registry and must be reloaded by the server from its authoritative config after process restart. An abrupt crash before an operation intent is published can leave an unreferenced temporary file in a home; the old or new content path remains readable and the journal covers subsequent installation steps.

Design choices not specified in main: Unicode control characters are rejected; Versions mirror relative paths per this job prompt despite the older text; trashed Versions live at .Trash/versions/; permanent delete recursively removes a folder and its Versions.

Required gate output verbatim (all exit 0; fmt produced no output):

cargo fmt --check --manifest-path crates/calternal-fs/Cargo.toml

cargo clippy --manifest-path crates/calternal-fs/Cargo.toml --all-targets -- -D warnings

    Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.57s

cargo test --manifest-path crates/calternal-fs/Cargo.toml

   Compiling calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 4.61s
     Running unittests src/lib.rs (crates/calternal-fs/target/debug/deps/calternal_fs-f4bd95ffd2e65b49)

running 9 tests
test tests::thinning_keeps_every_version_in_first_day ... ok
test tests::recovery_replaces_poisoned_blob_at_each_step ... ok
test tests::write_future_can_run_on_server_workers ... ok
test tests::relative_paths_reject_ambiguous_components ... ok
test tests::recovery_discards_uncommitted_intent_file ... ok
test tests::gc_waits_for_in_flight_blob_link_across_roots ... ok
test tests::gc_waits_for_dedup_copy_after_source_unlink ... ok
test tests::recovery_replays_each_move_trash_restore_and_install_step ... ok
test tests::recovery_replays_overwrites_with_versions ... ok

test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s

     Running tests/storage.rs (crates/calternal-fs/target/debug/deps/storage-68cd26c02aee4a19)

running 27 tests
test configured_quota_covers_copy_and_cross_home_move ... ok
test existing_blob_with_wrong_bytes_is_replaced ... ok
test mirrored_versions_follow_directory_and_delete ... ok
test orphan_blobs_are_collected ... ok
test copy_of_deduped_file_reuses_blob ... ok
test copied_file_is_read_only_with_reflink_preference ... ok
test cross_home_folder_move_checks_all_file_bytes ... ok
test directory_pages_advance_with_cursor ... ok
test public_listing_hides_reserved_names ... ok
test moving_file_carries_versions ... ok
test quota_is_shared_and_override_only_lowers ... ok
test percent_name_survives_trash_round_trip ... ok
test overwriting_with_copy_or_move_keeps_destination_version ... ok
test overwriting_move_carries_source_versions_and_preserves_target ... ok
test reserved_internals_are_not_public_paths ... ok
test thinning_keeps_recent_and_bucket_representatives ... ok
test dedup_concurrent_writes ... ok
test symlink_parent_is_never_followed ... ok
test trash_restores_versions_without_mixing_new_file ... ok
test trash_round_trip_and_quota ... ok
test replace_does_not_modify_another_hardlink ... ok
test traversal_is_rejected ... ok
test trash_collisions_get_distinct_names_and_empty ... ok
test arbitrary_paths_never_escape ... ok
test zero_byte_file_gets_version_and_thins ... ok
test version_counts_against_quota ... ok
test symlink_swap_race_never_writes_outside ... ok

test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.23s

   Doc-tests calternal_fs

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

After gates: cargo clean --manifest-path crates/calternal-fs/Cargo.toml removed 761 files, 324.7MiB total. Issue remains open.

Finished on branch job/fs at head b64882b77c1cb447c9861b2f7d5d841d2a5c32a8. Committed only crates/calternal-fs/**. Built: public user-path/internal-path separation, canonical blob verification and repair, shared per-data-directory quota limits and blob operation lock, mirrored Versions, folder moves and Version lifecycle, journal + Root::recover() for move/trash/restore/write/copy/delete steps, percent names, stronger race coverage, and source split. The independent audit's planted-hardlink case remains bounded by the documented single-writer/OS ownership assumption. Known gaps: quota limits live in the process-wide Root registry and must be reloaded by the server from its authoritative config after process restart. An abrupt crash before an operation intent is published can leave an unreferenced temporary file in a home; the old or new content path remains readable and the journal covers subsequent installation steps. Design choices not specified in main: Unicode control characters are rejected; Versions mirror relative paths per this job prompt despite the older <file-id> text; trashed Versions live at .Trash/versions/<item>; permanent delete recursively removes a folder and its Versions. Required gate output verbatim (all exit 0; fmt produced no output): cargo fmt --check --manifest-path crates/calternal-fs/Cargo.toml ``` ``` cargo clippy --manifest-path crates/calternal-fs/Cargo.toml --all-targets -- -D warnings ``` Checking calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs) Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.57s ``` cargo test --manifest-path crates/calternal-fs/Cargo.toml ``` Compiling calternal-fs v0.1.0 (/home/kayg/Developer/calternal-wt/fs/crates/calternal-fs) Finished `test` profile [unoptimized + debuginfo] target(s) in 4.61s Running unittests src/lib.rs (crates/calternal-fs/target/debug/deps/calternal_fs-f4bd95ffd2e65b49) running 9 tests test tests::thinning_keeps_every_version_in_first_day ... ok test tests::recovery_replaces_poisoned_blob_at_each_step ... ok test tests::write_future_can_run_on_server_workers ... ok test tests::relative_paths_reject_ambiguous_components ... ok test tests::recovery_discards_uncommitted_intent_file ... ok test tests::gc_waits_for_in_flight_blob_link_across_roots ... ok test tests::gc_waits_for_dedup_copy_after_source_unlink ... ok test tests::recovery_replays_each_move_trash_restore_and_install_step ... ok test tests::recovery_replays_overwrites_with_versions ... ok test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.07s Running tests/storage.rs (crates/calternal-fs/target/debug/deps/storage-68cd26c02aee4a19) running 27 tests test configured_quota_covers_copy_and_cross_home_move ... ok test existing_blob_with_wrong_bytes_is_replaced ... ok test mirrored_versions_follow_directory_and_delete ... ok test orphan_blobs_are_collected ... ok test copy_of_deduped_file_reuses_blob ... ok test copied_file_is_read_only_with_reflink_preference ... ok test cross_home_folder_move_checks_all_file_bytes ... ok test directory_pages_advance_with_cursor ... ok test public_listing_hides_reserved_names ... ok test moving_file_carries_versions ... ok test quota_is_shared_and_override_only_lowers ... ok test percent_name_survives_trash_round_trip ... ok test overwriting_with_copy_or_move_keeps_destination_version ... ok test overwriting_move_carries_source_versions_and_preserves_target ... ok test reserved_internals_are_not_public_paths ... ok test thinning_keeps_recent_and_bucket_representatives ... ok test dedup_concurrent_writes ... ok test symlink_parent_is_never_followed ... ok test trash_restores_versions_without_mixing_new_file ... ok test trash_round_trip_and_quota ... ok test replace_does_not_modify_another_hardlink ... ok test traversal_is_rejected ... ok test trash_collisions_get_distinct_names_and_empty ... ok test arbitrary_paths_never_escape ... ok test zero_byte_file_gets_version_and_thins ... ok test version_counts_against_quota ... ok test symlink_swap_race_never_writes_outside ... ok test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.23s Doc-tests calternal_fs running 0 tests test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s ``` After gates: cargo clean --manifest-path crates/calternal-fs/Cargo.toml removed 761 files, 324.7MiB total. Issue remains open.
Author
Owner

Completed on dev in 9aeb451de9 (Merge job/fs: calternal-fs safe storage layer, audit fixes (#2)).

Completed on dev in 9aeb451de9bc915039934fb3373c402cbbd18663 (Merge job/fs: calternal-fs safe storage layer, audit fixes (#2)).
kayg closed this issue 2026-10-01 05:08:18 +00:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#2
No description provided.