kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 190 Pull requests 61 Projects Releases Packages Wiki Activity Actions 26

plan: Files (Nextcloud replacement) #962

Closed
kayg wants to merge 0 commits from docs/files-plan into dev
pull from: docs/files-plan
merge into: kayg:dev
Conversation 2 Commits - Files changed - +118
kayg commented 2026-11-05 11:45:25 +00:00
Owner
Copy link

Design-only Files replacement plan based on the 11-05 owner grill. Replaces the Files portion of proposed ADR 0051 and records scope, identity, transfer, shared history and quota, dedupe gates, scale gates, failures, migration, backend-first slices and open recommendations. Builds on #409, #641, #761 and #865. PR #953 measures the Notes identity projection; no result is claimed here. No product code changed.

Design-only Files replacement plan based on the 11-05 owner grill. Replaces the Files portion of proposed ADR 0051 and records scope, identity, transfer, shared history and quota, dedupe gates, scale gates, failures, migration, backend-first slices and open recommendations. Builds on #409, #641, #761 and #865. PR #953 measures the Notes identity projection; no result is claimed here. No product code changed.
docs: plan Files replacement from 11-05 grill
Some checks failed
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details
06487d9771 
Record the latest Files owner answers as a design-only plan replacing the proposed Files ADR. Cover shared identity, resumable transfer, retention, quota, scale gates, failure recovery, manual migration, slices and open recommendations.
kayg commented 2026-11-05 13:47:06 +00:00
Author
Owner
Copy link

Perf-VM sparse quota-math spike (2026-11-05)

Control arm Sparse files SQLite logical sum du -s -B1 du -s --apparent-size -B1
dev-a 1,024 1,125,899,906,842,624 bytes (1 PiB) 0 bytes 1,125,899,906,842,624 bytes
dev-b 1,024 1,125,899,906,842,624 bytes (1 PiB) 0 bytes 1,125,899,906,842,624 bytes

Both arms used 1 TiB sparse files (stat sample: 1,099,511,627,776 bytes each). The independent dev-vs-dev fixture arms matched. du reports no allocated payload blocks; inode and filesystem metadata overhead is not included in that number. Both shared locks were held in order, perf-01 then perf-02, while measuring on perf-01. df -hT / before and after was unchanged at displayed precision: perf-01 ext4 had 74G available; perf-02 ext4 had 26G available.

Commands for dev-a: seq 1 1024 | xargs -I{} truncate -s 1099511627776 /tmp/calternal-files-quota-962-dev-a/{}; sqlite3 -list /tmp/calternal-files-quota-962-dev-a.sqlite "CREATE TABLE f(nbytes INTEGER); WITH RECURSIVE n(i) AS (VALUES(1) UNION ALL SELECT i+1 FROM n WHERE i<1024) INSERT INTO f SELECT 1099511627776 FROM n; SELECT count(*),sum(nbytes) FROM f;"; du -s -B1 /tmp/calternal-files-quota-962-dev-a; du -s --apparent-size -B1 /tmp/calternal-files-quota-962-dev-a. The dev-b arm used the same commands and its dev-b paths. Tools came from pinned Nixpkgs 1e7da2489c0ba3a06bceec1b444fbc02c5223ab1 via nix shell with sqlite, coreutils and findutils.

This confirms the 1 PiB logical-size fixture can use sparse placeholders without writing file payloads. It does not measure the Files index, a 50M-entry account, folder listing, search or sync latency; those remain separate gates. No product server or real user bytes were used.

Perf-VM sparse quota-math spike (2026-11-05) | Control arm | Sparse files | SQLite logical sum | `du -s -B1` | `du -s --apparent-size -B1` | |---|---:|---:|---:|---:| | dev-a | 1,024 | 1,125,899,906,842,624 bytes (1 PiB) | 0 bytes | 1,125,899,906,842,624 bytes | | dev-b | 1,024 | 1,125,899,906,842,624 bytes (1 PiB) | 0 bytes | 1,125,899,906,842,624 bytes | Both arms used 1 TiB sparse files (`stat` sample: 1,099,511,627,776 bytes each). The independent dev-vs-dev fixture arms matched. `du` reports no allocated payload blocks; inode and filesystem metadata overhead is not included in that number. Both shared locks were held in order, perf-01 then perf-02, while measuring on perf-01. `df -hT /` before and after was unchanged at displayed precision: perf-01 ext4 had 74G available; perf-02 ext4 had 26G available. Commands for dev-a: `seq 1 1024 | xargs -I{} truncate -s 1099511627776 /tmp/calternal-files-quota-962-dev-a/{}`; `sqlite3 -list /tmp/calternal-files-quota-962-dev-a.sqlite "CREATE TABLE f(nbytes INTEGER); WITH RECURSIVE n(i) AS (VALUES(1) UNION ALL SELECT i+1 FROM n WHERE i<1024) INSERT INTO f SELECT 1099511627776 FROM n; SELECT count(*),sum(nbytes) FROM f;"`; `du -s -B1 /tmp/calternal-files-quota-962-dev-a`; `du -s --apparent-size -B1 /tmp/calternal-files-quota-962-dev-a`. The dev-b arm used the same commands and its `dev-b` paths. Tools came from pinned Nixpkgs `1e7da2489c0ba3a06bceec1b444fbc02c5223ab1` via `nix shell` with `sqlite`, `coreutils` and `findutils`. This confirms the 1 PiB logical-size fixture can use sparse placeholders without writing file payloads. It does not measure the Files index, a 50M-entry account, folder listing, search or sync latency; those remain separate gates. No product server or real user bytes were used.
kayg commented 2026-11-07 17:53:04 +00:00
Author
Owner
Copy link

Landed in dev via b5e8dcb289 — closing.

Landed in dev via b5e8dcb289bc5e8ee695689877d9551d82332100 — closing.
kayg closed this pull request 2026-11-07 17:53:05 +00:00
Some checks failed
Markdown extraction contract / markdown-native-contract (pull_request) Has been cancelled
Details
CI / server (pull_request) Has been cancelled
Details
CI / rust (pull_request) Has been cancelled
Details
CI / web (pull_request) Has been cancelled
Details
Markdown extraction contract / markdown-editor-contract (pull_request) Has been cancelled
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!962
No description provided.