fix(storage): keep live private writes out of recovery #759
Closed
kayg
wants to merge 7 commits from
fix/storage-owner-liveness-642 into dev
pull from: fix/storage-owner-liveness-642
merge into: kayg:dev
kayg:main
kayg:feat/contrast
kayg:feat/style-sheet-fixes
kayg:storage/wal-checkpoint-off-save-1051
kayg:t3/paged-search-reconcile-final
kayg:feat/pass8
kayg:review/calendar-2
kayg:fix/shared-row-swipes
kayg:dev
kayg:feat/collections-db
kayg:fix/instant-keys-1056
kayg:feat/money-round
kayg:perf/stalls-1054
kayg:wip/tiering-v3-cold4
kayg:wip/tiering-v3-cold5
kayg:port/rows-actions
kayg:port/rows-drop
kayg:port/hidden-blocks
kayg:port/rows
kayg:port/port-tables
kayg:port/port-tables-2
kayg:t3/mail-1011-liveness
kayg:port/devices
kayg:port/port-attachments
kayg:port/port-attachments-2
kayg:port/port-attachments-3
kayg:docs/design-consistency
kayg:docs/engineering-standards
kayg:fix/rusqlite-0.41-rustsec-2026-0459
kayg:feat/iphone-fixes
kayg:shared/restore-and-stale-918-919
kayg:fix/phone-top-band-contrast
kayg:feat/notes-approve
kayg:perf/soak-max-gate
kayg:feat/files-s1-identity
kayg:fix/lifecycle-event-id
kayg:feat/quickfix
kayg:wip/tiering-v3-cold
kayg:perf/web-mockup-frame-fixes
kayg:port/contacts-merge
kayg:port/contacts-edit
kayg:test/shared-room-no-deleted-text
kayg:perf/first-paint-credits
kayg:fix/shared-history-after-grant
kayg:fix/phone-swipe-regressions
kayg:feat/contrast-r1
kayg:review/sharing-pr3-review
kayg:feat/tiering-v3-s2-tags
kayg:feat/folder-drop
kayg:fix/deskpills-lazy-share
kayg:feat/sharing-s1-pr3
kayg:tmp/money-round-before
kayg:fix/phone-pane-backdrop-roots
kayg:fix/palette-preload-chips
kayg:fix/invite-defaults
kayg:port/tasks-matrix
kayg:fix/restore-green-gates
kayg:fix/sharing-fast-path-hardening
kayg:t3/photos-s1
kayg:feat/tiering-v3-s1-namespace
kayg:notes-archive-hide
kayg:feat/tasks-actions
kayg:t3/photos-plan-slice-one
kayg:feat/whiteboard-s1
kayg:t3/paged-search-reconcile
kayg:t3/smooth-pane-transitions
kayg:t3/index-position-startup-scan
kayg:fix/private-staging-759b-r2
kayg:feat/dav-transport
kayg:feat/imap-notes-fetch
kayg:feat/imap-notes-projection
kayg:feat/imap-notes-readonly
kayg:feat/imap-notes-auth
kayg:feat/imap-notes-server-foundation
kayg:docs/undo-block-ids-plan
kayg:fix/undo-768
kayg:proto/block-identity-v7d
kayg:issue/carddav-read-perf-fix-r2
kayg:issue/carddav-read-r2
kayg:docs/imap-766-plan-v2
kayg:docs/imap-notes-plan
kayg:docs/quota-plan-2
kayg:fix/parallel-test-flakes
kayg:feat/contacts-companion-pure
kayg:docs/calendar-backend-v2-plan
kayg:feat/calendar-ical-converter
kayg:docs/tasks-writes-v2-plan
kayg:docs/carddav-writes-v2-plan
kayg:docs/contacts-import-v2-plan
kayg:issue/610-pm-midnight
kayg:feat/calendar-v2-s1-containment
kayg:docs/dav-recorder-v2-plan
kayg:plan/previews-865
kayg:plan/save-log-durable-entry
kayg:fix/e2e-data-on-ram
kayg:feat/appwindow3
kayg:feat/appwindow2
kayg:feat/appwindow
kayg:feat/money-p1
kayg:feat/keys-sheet
kayg:review/calendar-2-themes
kayg:feat/smooth-cursor
kayg:fix/float-style
kayg:review/calendar-2-float
kayg:release/2026-11-06-hotfix
kayg:archive/port/rows-drop-code
kayg:archive/port/rows-actions
kayg:archive/port/rows
kayg:scratch/tasks-894-ui-gates
kayg:review/calendar-2-night-wip
kayg:release/2026-11-05
kayg:archive/feat/tiering-v3-s1-namespace
kayg:archive/flaky/rows-drop-fixed
kayg:archive/flaky/rows-actions
kayg:archive/feat/tiering-v3-reader
kayg:archive/feat/tiering-v3-roots
kayg:archive/flaky/rows
kayg:sim/bm15-final
kayg:sim/bm15-fix
kayg:sim/bm15-bis-932
kayg:sim/bm15-bis-909
kayg:sim/bm15
kayg:sim/bm15-bis-887
kayg:perf/tiering-option-b
kayg:port/style-3
kayg:port/style-2
kayg:port/style
kayg:fix/sw-lazy-precache
kayg:fix/vim-w-waits-for-save
kayg:fix/signout-test-races
kayg:fp-scratch/all
kayg:sim/bm15-first-paint
kayg:archive/bm15/rows-drop
kayg:archive/bm15/rows-actions
kayg:fix/palette-snippet-contrast
kayg:archive/bm15/rows
kayg:archive/bm15/hidden
kayg:archive/plan/save-log-durable-entry
kayg:mock/money-fix2
kayg:archive/fix/e2e-data-on-ram
kayg:mock/appearance-all
kayg:mock/pill-press
kayg:archive/docs/design-consistency
kayg:mock/auto-swipes
kayg:mock/mail-actions
kayg:archive/feat/mail-p1
kayg:mock/money-static
kayg:archive/review/standards-917
kayg:archive/docs/engineering-standards
kayg:perf/tiering-761-fix
kayg:mock/type-audit
kayg:mock/third-pane
kayg:feat/contacts-import-v2-s1
kayg:perf/executor-hot-path
kayg:archive/port/hidden-blocks
kayg:archive/port/devices
kayg:scratch/executor-on-894
kayg:scratch/executor-spans2
kayg:scratch/executor-spans
kayg:train/2026-11-03-bm14c
kayg:archive/port/rows-drop
kayg:feat/tiering-761-d-scale
kayg:feat/tiering-761-c-index
kayg:feat/tiering-761-b-reader
kayg:archive/review/sharing-pr3-review-9fec3d1f
kayg:train/2026-11-03-bm14b
kayg:perf/ports-lazy-budgets
kayg:feat/tiering-761-a-migration
kayg:train/2026-11-03-bm14
kayg:train/2026-11-03-bm12
kayg:archive/review/sharing-pr3-review
kayg:archive/feat/sharing-s1-pr3
kayg:train/2026-11-03-bm11
kayg:archive/feat/imap-notes-fetch
kayg:train/2026-11-03-bm10
kayg:archive/review/sharing-pr3-review-code
kayg:archive/plan/previews-865
kayg:train/2026-11-02-bm9
kayg:archive/port/port-attachments-3
kayg:archive/port/contacts-merge
kayg:fix/motion-audit-859
kayg:train/2026-11-02-bm8
kayg:archive/scratch/bm8-prep
kayg:archive/port/port-pickers
kayg:archive/port/port-tables-2
kayg:archive/port/port-tables
kayg:archive/feat/sharing-s1-pr3-code
kayg:archive/scratch/bm7-fixed
kayg:train/2026-11-02-bm7
kayg:archive/fix/parallel-test-flakes-code-minipc01
kayg:train/2026-11-02-bm6
kayg:docs/imap-766-readonly-slice
kayg:feat/dav-recorder-v3
kayg:research/788-diagnosis
kayg:archive/fix/parallel-test-flakes
kayg:archive/perf/pr792-cold-index-edit
kayg:docs/sharing-slice-1
kayg:archive/docs/imap-766-plan-v2
kayg:feat/dav-recorder-v2
kayg:archive/docs/dav-recorder-v2-plan
kayg:spike/imap-766-s0-cases
kayg:archive/fix/undo-768
kayg:feat/dav-recorder
kayg:archive/docs/contacts-import-v2-plan
kayg:archive/feat/calendar-ical-converter
kayg:feat/contacts-import
kayg:perf/cold-growth-on-764
kayg:perf/cold-growth-on-tiering
kayg:archive/docs/tasks-writes-v2-plan
kayg:archive/docs/calendar-backend-v2-plan
kayg:archive/feat/contacts-companion-pure
kayg:feat/tasks-set-status
kayg:feat/tasks-move-reorder
kayg:archive/docs/carddav-writes-v2-plan
kayg:archive/feat/calendar-v2-s1-containment
kayg:feat/tasks-delete-promote-duplicate
kayg:archive/feat/dav-transport
kayg:archive/fix/carddav-rebuild-r1
kayg:archive/fix/carddav-retention-r1
kayg:feat/carddav-writes
kayg:perf/quota-on-tiering
kayg:perf/quota-simple
kayg:perf/cold-growth-on-quota-stack
kayg:feat/calendar-identity
kayg:feat/calendar-event-parser
kayg:feat/calendar-props-maps
kayg:port/glass-when-ring
kayg:feat/tiering-761-s1
kayg:wip/10by32-01-impl-759b-scratch-20261030
kayg:wip/6by32-01-bm3-webkit-pause-20261030
kayg:wip/10by32-01-bm3-chromium-pause-20261030
kayg:wip/6by32-01-tier-761-s1-pause-20261030
kayg:spike/imap-766-s0
kayg:wip/6by32-02-calternal-pause-20261030
kayg:wip/8by64-01-review-688-pause-20261030
kayg:wip/6by32-01-verify-744-pause-20261030
kayg:wip/6by32-01-verify-742-pause-20261030
kayg:fix/private-staging-759b
kayg:wip/6by32-01-review-748-pause-20261030
kayg:wip/6by32-01-review-733c-pause-20261030
kayg:wip/6by32-01-review-733b-pause-20261030
kayg:wip/6by32-01-perf-idle-8-pause-20261030
kayg:wip/6by32-01-perf767-base-pause-20261030
kayg:wip/6by32-01-perf767b-767-pause-20261030
kayg:wip/6by32-01-perf767b-764-pause-20261030
kayg:wip/6by32-01-perf767-767-pause-20261030
kayg:wip/6by32-01-perf767-764-pause-20261030
kayg:wip/6by32-01-lists-b-pause-20261030
kayg:wip/6by32-01-ids-verify5-pause-20261030
kayg:wip/6by32-01-ids-verify4-pause-20261030
kayg:review/calendar-month
kayg:wip/10by32-02-undo-768-20261030
kayg:test/ids-verify3
kayg:wip/10by32-02-bm3-chromium-20261030
kayg:wip/10by32-01-review-753-20261030
kayg:wip/10by32-01-perf-idle-2-20261030
kayg:wip/10by32-01-bm2-webkit2-20261030
kayg:plan/759b-staging
kayg:train/2026-10-30-bm2
kayg:review/758b-tests
kayg:review/758-tests
kayg:review/764d-tests
kayg:review/767b-tests
kayg:review/preview-2026-10-29
kayg:review/767-tests
kayg:docs/tiering-731-plan
kayg:wip/c6b02-when2
kayg:wip/c6b02-tasks-create-fix5
kayg:wip/c6b02-settings-frost-mock
kayg:wip/c6b02-settings-frost-base
kayg:wip/c6b02-settings-frost
kayg:wip/c6b02-ring
kayg:wip/c6b02-review-733d
kayg:wip/c6b02-bm1-full-a
kayg:wip/compute02-when-field
kayg:wip/compute02-webkit-lists-repro
kayg:wip/compute02-train-bl2-dev
kayg:wip/compute02-dev-green-pw164
kayg:docs/deploy-plan
kayg:review/764c-tests
kayg:review/764b-tests
kayg:review/748g-tests
kayg:review/764-tests
kayg:review/759b-tests
kayg:fix/storage-fix-7
kayg:wip/compute02-storage-fix-7
kayg:review/759-tests
kayg:feat/tiering-731
kayg:wip/tiering-731-first-attempt
kayg:wip/compute02-storage-fix-1
kayg:review/754g-tests
kayg:perf/idle-honest-2
kayg:review/754f-tests
kayg:review/754e-tests
kayg:review/755-tests
kayg:review/754d-tests
kayg:review/754c-tests
kayg:review/754b-tests
kayg:review/754-tests
kayg:fix/storage-private-tmp-642-dev
kayg:wip/compute02-verify-750
kayg:test/ids-verify5
kayg:test/ids-verify4
kayg:fix/storage-exact-file-quarantine-642
kayg:wip/compute02-verify-740
kayg:perf/one-flush-save
kayg:review/calendar-static
kayg:wip/compute02-verify-737
kayg:review/indexing-ring
kayg:review/when-field-2
kayg:wip/compute02-review-733
kayg:proto/block-identity-2-review
kayg:proto/block-identity-review
kayg:wip/c6b02-tc-design
kayg:review/when-field
kayg:issue/tasks-create-ids
kayg:issue/tasks-create
kayg:review/combined
kayg:wip/compute02-review-combined
kayg:review/task-polish
kayg:review/top-buttons
kayg:review/q1-q5
kayg:train/2026-10-26-bl2
kayg:archive/wt-398h
kayg:archive/train-bm-proptest-seed
kayg:archive/integ325-293
kayg:archive/integ325-ci
kayg:archive/client-auth-review-baseline
kayg:archive/wt-sim-4853
kayg:archive/wt-sim-4811
kayg:archive/wt-448m
kayg:archive/wt-398a-allocator
kayg:issue/calendar-adr
kayg:archive/old/train/2026-10-23bk2
kayg:archive/old/train/2026-10-23bk
kayg:issue/469-mockup-9a
kayg:issue/smart-tasks-mockup
kayg:issue/469-menus
kayg:issue/652-signout-flake
kayg:issue/652-history-fix7
kayg:issue/652-history-fix6
kayg:archive/old/issue/spring-check-perf
kayg:issue/spring-check-perf
kayg:issue/659-rename-fast
kayg:archive/old/train/2026-10-21ar
kayg:issue/search-hardening-3
kayg:issue/652-history-fix5
kayg:issue/660-port-sheets-glass
kayg:issue/652-history-fix4
kayg:issue/652-history-fix3
kayg:issue/652-history-fix2
kayg:issue/652-history-flake
kayg:train/2026-10-21an
kayg:issue/settings-admin-port
kayg:issue/search-port
kayg:train/2026-10-20am
kayg:issue/storage-split-fix2
kayg:issue/storage-split
kayg:archive/old/issue/folder-ids-fix
kayg:train/2026-10-20al
kayg:archive/old/issue/folder-ids
kayg:train/2026-10-20ak
kayg:issue/tables-mockup
kayg:train/2026-10-19aj
kayg:train/2026-10-19ai
kayg:issue/469-merge-fixes
kayg:issue/sidebar-band
kayg:archive/old/issue/608-settings-spec
kayg:train/2026-10-19ah
kayg:issue/swipe-offscreen
kayg:train/2026-10-19ag
kayg:archive/old/issue/590-nlp-adr
kayg:archive/old/issue/587-event-after-write-2
kayg:train/2026-10-19af
kayg:archive/old/issue/587-event-after-write
kayg:issue/469-pill-room-c
kayg:issue/469-pill-room
kayg:train/2026-10-19ae
kayg:issue/469-narrow-handle
kayg:issue/397-ios-swipe-2
kayg:archive/old/issue/adrs-photos-files-tasks
kayg:issue/469-desk-fixes
kayg:archive/old/issue/auto-update-adr
kayg:issue/469-filter-gap
kayg:issue/469-port-1b
kayg:issue/442-opacity-flake
kayg:archive/397-ios-swipe-2
kayg:issue/327-web-fast-3
kayg:archive/old/issue/469-port-1
kayg:issue/469-port-1
kayg:issue/327-web-fast-2
kayg:archive/old/issue/327-web-fast
kayg:archive/old/issue/425-stt-bench-vq1
kayg:archive/old/issue/445-no-bleed
kayg:archive/old/issue/400-swipe-mockup-2
kayg:issue/426-client-review
kayg:issue/426-client-2
kayg:issue/426-client
kayg:archive/old/issue/476-safe-area-lab
kayg:archive/445-top-light-2
kayg:archive/old/issue/445-top-light
kayg:issue/471-blocks-mockup
kayg:archive/old/issue/448-links-app-2
kayg:archive/old/issue/448-links-app
kayg:archive/old/issue/398-cold-links
kayg:issue/434-442-small-2
kayg:issue/434-442-small
kayg:archive/old/issue/398-unload
kayg:archive/old/issue/445-bottom-band
kayg:issue/445-bottom-band
kayg:issue/351-rem-3
kayg:archive/312-views-mockup
kayg:issue/351-rem-2
kayg:issue/351-rem
kayg:archive/old/issue/327-first-note
kayg:issue/395-width-phone
kayg:archive/old/issue/327-typing
kayg:issue/396-ios-edges
kayg:issue/397-ios-swipe
kayg:archive/398-allocator
kayg:archive/old/issue/327-notes-index
kayg:issue/345-347-contrast-empty-2
kayg:archive/old/issue/44-arenas
kayg:issue/44-arenas
kayg:archive/integ325
kayg:archive/old/issue/327-perf-harness
kayg:archive/old/issue/44-thp
kayg:issue/44-thp
kayg:archive/old/issue/285-sticky-pills
kayg:archive/old/issue/44-trim-3
kayg:issue/44-trim-3
kayg:archive/old/issue/165-cards
kayg:archive/old/issue/165-picker-2
kayg:archive/old/issue/212-multi-select
No reviewers
Labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
kayg/calternal!759
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/storage-owner-liveness-642"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Opening a cold Notes Home could expose another plugin's partial upload and break its completion. A staging scan could also remove a cancellation directory before its owner lock existed. This branch includes the private-folder work from #750 and addresses those findings for #642.
Private operations hold a directory lock. A short shared root lock covers directory creation through owner-lock acquisition. Recovery checks ownership and keeps its guard through journal proof and moves; enumeration leaves directories intact. The root module identifies entries by their registered root, so a directory named
tmpcannot bypass ownership checking. Prepared imports use the same protocol. Creator receipts and exclusive file leases continue to guard disposal of bytes.The failing gates also exposed two application races. Startup now identifies completed creates from their planned paths and embedded note identities before replay attempts another exclusive create. Contact cache sweeping waits for selected card copies, preventing a copier from recreating a stale card after import undo. Private name allocation advances a process-wide logical clock, preventing repeated clock values from reusing a name while preserving the previous filename budget.
Permanent regressions cover live uploads during cold Home recovery, scans during cancellation and construction, ownership through journal proof, prepared import ownership, locked entries named
tmp, repeated clocks, long Unicode destinations, create placement recovery, and a paused copier racing import undo. The cancellation and live-upload reproductions failed before the fix. Negative controls also demonstrate the copier and basename regressions. Upload readiness tests observe live files directly and retain the empty-private-root assertion; note crash fixtures release owner locks through a real child-process exit.Validation passed:
-D warnings.git diff --check.Tests use the disk-backed job-local temporary directory, one test thread per binary, cargo -j3 and a descriptor limit of 8192. The reviewer's Notes tag test reported EMFILE before its SQLite failure; other CannotOpen failures did not record their OS cause and have not recurred in the controlled runs. Evidence is in issue comments and is excluded from the branch. Related to #642; follow-up to #750.
Verdict: fix first. Independent verification of PR #759 at
257a1af579932a78b22599bda86d03ec062d98df. Tests only were pushed toreview/759-tests, headdcd997011cdc8a27a81a43546f9c6f74cac1dd73, and the remote head was verified. The worktree is clean. No product fixes, merges, deployments or issue closures.Two P2 failures remain under the supplied contract. Both mechanisms are unchanged from #750; these are inherited contract failures, not claims that the ownership changes introduced them.
P2: interrupted private upload becomes a public stray. A real child creates a TempFile and writes
partial owner bytes. A cfg(test) pipe handshake pauses immediately after the payload write and before its receipt update. SIGKILL that child, wait for exit, then run production recovery. Recovery publisheslive.conflict-1containing that unacknowledged partial upload. The receipt still describes the previous bytes, so recovery treats the image as unknown and restores it publicly. This preserves bytes but violates the no-public-stray requirement. The existing attachment crash test explicitly permits conflict copies; it does not gate the stricter requirement. No acknowledged-byte loss is claimed. Finding and evidence. Permanent failing regression.P2: continuous edits miss the one-second Home catch-up limit. With production defaults and no save-delay override, the first acknowledged marker was still absent from the canonical Home file after 1.505 seconds and 144 further acknowledged appends. SIGKILL/restart recovered it and explicit save placed it. The independent Rust regression also fails: 1.50522005 seconds, marker absent, 165 further acknowledged edits. The default 750ms quiet delay can keep moving until the five-second maximum. Twenty discrete edits caught up within 0.786 seconds locally; the continuous case is the counterexample. Finding and ordinary-binary reproduction. Permanent failing regression.
P3 verification qualification: the original #750 cancellation reproducer passes unchanged. The original cold-Home source fails unchanged at its setup assertion, expecting staging_files to return one live image; the corrected scanner returns zero. The committed adapted test uses direct private enumeration and passes its lifecycle checks. I verified the original copies byte for byte and recorded their hashes. This is a fixture-contract change, not evidence that the earlier live-upload bug persists. Evidence.
Independent checks that passed:
Required gates pass on the exact reviewed PR head, with reviewer registrations/hooks removed for those gates and restored afterward. Contacts: 131 passed; fsio: 82 passed; Notes: 434 passed, six existing ignored cases. Full workspace and documentation tests pass. The Tasks abrupt-stop test from #757 passed in the full run and once in the requested separate repeat; the flake did not occur. Final formatting and Clippy also pass with the pushed reviewer tests. The two new contract regression assertions fail as reported; they are not safety passes.
Gate output, verbatim:
Complete required gate stdout/stderr, verbatim, is recorded in these issue comments:
The first final inventory was intentionally recorded as failed: it found one private transaction in the latency fixture killed immediately after observing its last file catch-up, before a cleanup barrier. Ordinary restart drained it. All 20 latency acknowledgments remained exactly once in the canonical file. After discovery/save barriers and a graceful fixture stop, the six-root inventory found zero public temporary entries and zero private entries. The successful settled scan does not erase the earlier failed raw scan.
Evidence, logs, run JSON, seeds, original-source copies and reproduction instructions remain outside git in
/home/agent/codex-jobs/verify-759. Only test sources and cfg(test) hooks were committed. This verifies Linux process-stop behavior and bounded ENOSPC, not power loss, non-Linux execution or physical-HDD performance.Starting storage-fix-7 in /home/agent/wt/storage-fix-7. PR head/base for this fix is
257a1af579on fix/storage-owner-liveness-642. Local branch fix/storage-fix-7 now includes origin/review/759-tests atdcd997011c(fast-forward). Reviewer tests will stay unchanged.Failure table before code (intended outcomes):
The final task-specific instruction says push to the PR branch only; I will follow that over the general instruction to open a replacement PR. No main/dev push, PR merge or deployment. Unknown bytes in the payload/receipt gap cannot safely be classified as app-only, so retaining them privately is the conservative choice.
Confirmed both reviewer failures before product changes. Payload/receipt SIGKILL test exits 101 and publishes live.conflict-1. Continuous-edit test exits 101: first acknowledgment age=1.506176074s; marker in Home=false; further acknowledged edits=173. RecordedFile currently writes payload before updating its receipt; recovery routes rejected receipts to public conflicts. Options::default sets flush_max to five seconds, and the actor services deferred commands before checking its timer. The fix will retain ambiguous in-flight writes privately, lower the default coalescing bound, and check overdue saves before deferred work.
Verbatim regression output:
Targeted checks after the fix: all six fsio reviewer tests pass, including the real SIGKILL receipt gap (public entries=[]), owner liveness, outside writers and the other Home. The unchanged continuous-edit regression passes: first acknowledgment age=870.314691ms; marker in Home=true; further acknowledged edits=75. Before the fix it was absent after 1.506176074s and 173 edits. Formatting and workspace Clippy with -D warnings pass. Full workspace tests and the ordinary-binary 1,000-stop gate are next.
The interrupted-write marker never grants deletion authority. If a creation receipt rejects a marked image, cleanup syncs it and leaves it private. This intentionally retains ambiguous partial/outside bytes rather than deleting them or publishing them as user files. Normal successful writes remove the marker; publication retains the existing verified fsync and exclusive rename. The marker adds two xattr operations per storage write, without rescanning the document on edits. The actor uses the existing first-unsaved timestamp and no vault scan.
workspace-initial.log, part 1/3. Output verbatim.
workspace-initial.log, part 2/3. Output verbatim.
workspace-initial.log, part 3/3. Output verbatim.
The initial full workspace and documentation gate passed: Contacts 131 passed, fsio 97 passed, Notes 435 passed with six existing ignored tests. Tasks #757 abrupt-stop test passed in that run. Full initial output is in comments 69436, 69437 and 69438.
Two edge regressions identified during review of this fix now fail before their refinements:
These new tests supplement the unchanged reviewer tests; neither reviewer test is weakened. Failure evidence, verbatim:
edge-red-fsio exit=101
edge-red-notes exit=101
The two refinements are implemented, but final checks are currently waiting for the shared build-slot wrapper to grant a slot. The queued command is /usr/local/bin/build-slot.sh bash /home/agent/codex-jobs/storage-fix-7/fault-gates.sh. It has not produced fmt-final.log or begun another gate yet. I am keeping one queued build-slot job and preserving scheduler priority. The initial full workspace gate passed; the two new edge tests were confirmed failing before their refinements. This is a resource wait, not a final completion claim.
Both added edge regressions now pass. The unchanged fsio reviewer tests pass, including public entries=[] after the real receipt-gap SIGKILL. The unchanged Notes regression passes: first acknowledgment age=930.031131ms; marker in Home=true; further acknowledged edits=57.
The next gate caught one test notation error: Clippy requires an octal literal for Unix mode zero. I changed from_mode(0) to from_mode(0o0), preserving the same permissions. No lint suppression or test assertion changed. The failed gate remains recorded and Clippy will be rerun before continuing.
Clippy output, verbatim:
edge-fsio-final.log, part 1/1. Output verbatim.
edge-notes-final.log, part 1/1. Output verbatim.
review-fsio-final.log, part 1/1. Output verbatim.
review-notes-final.log, part 1/1. Output verbatim.
clippy-final-retry.log, part 1/1. Output verbatim.
build-final.log, part 1/1. Output verbatim.
tasks-abrupt-repeat.log, part 1/1. Output verbatim.
continuous-catchup.log, part 1/1. Output verbatim.
disk-full.log, part 1/1. Output verbatim.
durability-1000.log, part 1/3. Output verbatim.
durability-1000.log, part 2/3. Output verbatim.
durability-1000.log, part 3/3. Output verbatim.
Resuming on 6by32-compute01 in /home/agent/wt/storage-fix-8, branch fix/storage-fix-7, base
96b9c0cf(review testsdcd99701). Rerunning prior workspace failures before changing code. The latest job instruction authorizes a normal merge into this PR head branch and a non-force push.Failure table for the inherited implementation under review:
The two named tests are Engine tests, not Daily tests. Both pass here on the inherited code: tests::a_huge_job_never_delays_anyone (0.08 s) and tests::a_user_with_a_live_view_or_a_running_action_is_kept (0.19 s). All 22 Daily tests and its doc tests pass. The previous failures do not reproduce on this host; the filesystem has 173 GiB available. I am leaving unrelated Engine/Daily code unchanged and running the full workspace gate to check for broader failures.
Review of git diff dcd99701..HEAD confirms the intended private-write marker and bounded healthy-save scheduling are already implemented. Reviewer tests are unchanged. Removed accidentally committed generated Python bytecode in
833517c2. Ambiguous interrupted writes remain private; the marker never grants deletion authority. Unreadable regular images remain private too. Healthy overdue saves take priority over deferred commands; failed retries retain fair request selection.Reviewer gate finding: with the repository default RUST_TEST_THREADS=4, reviewer_759::misleading_pid_metadata_never_substitutes_for_a_kernel_owner fails at its immediate staging_files assertion (line 211). Reproduced twice in the reviewer group, while all five sibling tests pass. The exact same test passes alone, and all six reviewer tests pass with --test-threads=1. Output from both failed runs and the serial pass is retained outside the repo.
The failure is consistent with a sibling process spawn transiently inheriting the raw test File's advisory lock between fork and exec after the parent drops it. Recovery correctly skips an apparently live kernel lock; this is not the partial-upload P2. I am running the final workspace gates with RUST_TEST_THREADS=1 to isolate test harness processes. Test code and assertions remain unchanged, including their own explicit concurrent writers and real child processes. This follows the serial reviewer invocation used by the preceding job (comments 69483 and 69484). The concurrent-harness failure is disclosed rather than treated as a clean default-parallel pass.
The first full workspace run on this new worktree reached Notes with all earlier crates passing, then failed one real SyncClient test because Bun could not resolve y-protocols/sync from packages/editor/src/sync/protocol.ts. Notes summary: 435 passed, 1 failed, 6 existing ignored. The failing test was independent_652_real_syncclient_joining_snapshot_is_not_saved; its harness tried to parse an empty child reply after the missing-module error. This is a missing worktree dependency, not a storage behavior failure. Installing dependencies from bun.lock with bun install --frozen-lockfile, then rerunning that exact test and the full workspace gate. No source or lockfile changes are intended.
workspace.log, part 1/3. Output verbatim.
workspace.log, part 2/3. Output verbatim.
workspace.log, part 3/3. Output verbatim.
reviewer-fsio.log, part 1/1. Output verbatim.
reviewer-notes.log, part 1/1. Output verbatim.
disk-full.log, part 1/1. Output verbatim.
clippy.log, part 1/1. Output verbatim.
stop-loop.log, part 1/1. Output verbatim.
reviewer-fsio-retry.log, part 1/1. Output verbatim.
syncclient-retry.log, part 1/1. Output verbatim.
durability-1000.log, part 1/1. Output verbatim.
Finished and pushed PR #759 head
c129eb38a0to fix/storage-owner-liveness-642 with a normal merge. Parents:257a1af579and833517c2e9. Forgejo's PR API and branch API both confirm this head. The merge tree is identical to tested source833517c2. Worktree /home/agent/wt/storage-fix-8 is clean and back on fix/storage-fix-7.The inherited product fixes are complete: interrupted payload/receipt writes stay private, unreadable regular images cannot become public conflicts, and healthy overdue note saves cannot be starved by continuous commands. The default coalescing bound is 750 ms; failed saves retain fair request handling. I removed generated Python bytecode in the small cleanup commit
833517c2. No reviewer assertion or source was changed.Gates passed through /usr/local/bin/build-slot.sh, with cargo -j3:
Validation caveats are preserved: the default four-thread reviewer group failed its immediate stale-PID lock assertion twice while the same test passed alone and the full reviewer group passed serially. This is consistent with transient inherited advisory locks around sibling process spawn; it is not claimed to be a clean default-parallel pass. Final workspace execution used RUST_TEST_THREADS=1 without changing tests or their explicit concurrency. The first workspace run also found missing y-protocols/sync; bun install --frozen-lockfile fixed the worktree dependencies, the exact test passed, and the entire workspace then passed. No tracked dependency files changed.
Complete clean workspace output is verbatim in part 1, part 2, and part 3. Failed reviewer output and the dependency retry are verbatim in comments 69575 and 69576. The following other gate output is verbatim.
clippy.log
reviewer-fsio.log
reviewer-notes.log
disk-full.log
stop-loop.log
durability-1000.log
Independent verification at
c129eb38:gate-evidence.log, part 1/1. Output verbatim.Independent verification at
c129eb38:workspace.log, part 1/2. Output verbatim.Independent verification at
c129eb38:workspace.log, part 2/2. Output verbatim.Independent verification at
c129eb38:workspace-serial.log, part 1/3. Output verbatim.Independent verification at
c129eb38:workspace-serial.log, part 2/3. Output verbatim.Independent verification at
c129eb38:workspace-serial.log, part 3/3. Output verbatim.Verdict: ship at
c129eb38a098f4e33e50f14c7c8bc43eb12e7323. No P1/P2 product finding observed. One P3 test-isolation finding is confirmed; the default-parallel workspace failure remains disclosed.P3 — sibling test forks transiently inherit locks and upload descriptors. I ran the unchanged
reviewer_759::group 100 times with four test threads and 100 times serially. Parallel: 8/100 failures, all the immediate stale-PID assertion atreviewer_759.rs:211; serial: 100/100 passed. Reproducer:/usr/local/bin/build-slot.sh cargo test -p calternal-fsio -j3 reviewer_759:: -- --nocapture --test-threads=4, repeated with the committed support script. The default-parallel full workspace also failed a separate fixture atprivate_tmp_tests.rs:49, where upload publication could not obtain an exclusive lease.The added
actual_fork_before_exec_explains_transient_stale_pid_assertiondeterministically reproduces both mechanisms using a real child paused before exec and descriptor handshakes, without timing sleeps. The inherited directory descriptor keeps its kernel lock alive after the parent drops its guard; the inherited upload writer prevents an exclusive lease. Exec closes those descriptors and recovery proceeds while preserving outside bytes. These are test-isolation failures: the shipped storage paths do not launch these sibling child processes, and real competing servers remain excluded. Coordinate test spawns with lock/lease assertions; retain the conservative production ownership checks.Independent contract checks passed:
Gates, through the build-slot wrapper with cargo
-j3:The full serial run includes documentation tests: Contacts 131 passed, fsio 97 passed, Notes 436 passed with six existing ignored cases. Tasks #757 passes in the full run and its separate requested repeat; that flake did not occur. Final fmt/Clippy also pass with the pushed reviewer sources. No test assertion was weakened.
Verbatim evidence: targeted checks, stress results, final lint and failure signatures; original default-parallel workspace part 1, part 2; passing serial workspace part 1, part 2, part 3.
Tests only are pushed to
review/759b-tests, headc61d48766ed28257b83eae02c2ba5d73784d2cb7; remote SHA verified. Only two new test source files changed; worktree clean. Findings, per-run logs and machine-readable results remain in/home/agent/codex-jobs/verify-759b, with findings appended during the review. PR head was rechecked and remainsc129eb38.No merges, deployments or perf-test-* VM access. These checks cover Linux process stops and bounded ENOSPC on 6by32-compute01; they do not establish power-loss or physical-HDD timing guarantees.
Paused pending the simpler private staging redesign plan: #771.
Plan document on
plan/759b-stagingatc7464621(freshorigin/devbase). This is documentation only; no product changes, builds, merges or deployments.The plan separates disposable app-created uploads from adopted user bytes, removes per-chunk receipt/intent work, bounds private retention, and requires performance plus SIGKILL gates in the first implementation PR. It explicitly asks kayg to decide the same-UID private-write boundary and flags that OFD/flock explicit unlock cannot alone release inherited locks after SIGKILL. No further patching of #759 pending that decision.
Baseline upload throughput on
origin/devd83e54db(release binary), perf-01, usinghandover/759b/upload_bench.pyfromwip/10by32-01-impl-759b-scratch-20261030@1e6e32d7, unmodified. 20 repetitions (7 + 13, two sessions), each on a fresh data dir. The numbers are the median over reps of each rep's value, with min and max. Run viaperf/vm/upload-bench.sh(PR #787); no 759b PR existed yet, so this is posted here.Noise: the results are bimodal. 3 of 20 reps ran in a fast state (4 KiB p50 about 28 ms, 4 MiB 64-67 MiB/s, 128 MiB 115 MiB/s: reps 1 and 2 of the first session and rep 13 of the second); the other 17 sit at 37-40 ms / 41-48 MiB/s. Nothing on the guest changed between states (same MHz shown, steal ~0.1 %), so it is host-side. The 128 MiB case is disk-bound and varies 67-116 MiB/s even in the slow state. Compare a branch by running both in the same session, alternating, and use medians. Setup:
claude@calternal-perf-0{1,2}-backup01, AMD Ryzen 7 3700X guests (KVM), 4 vCPUs, 7.7 GiB, ext4 on a non-rotational virtual disk (uncached), no cpufreq interface (MHz constant at 3593; frequency cannot be observed). Binaries:origin/devd83e54db, release, built on dev-01 (sha256 ofcalternal2ac46c10...). Steal time 0.07-0.12 % per run. One measurement at a time viaflock /tmp/perf.lock.Raw files on perf-01:
~/perf/out-upload-dev/andout-upload-dev2/.ADR 0003 evidence migration:
perf/README-perf-vms.mdSource commit: 8c066c7353da75ff8fe5535c2d3f9e8a4373184a; snapshot from dev
55f35fcd. Full original source preserved below before repository removal/stripping. Design, code and test inputs are retained where applicable.kayg referenced this pull request2026-11-05 23:21:11 +00:00
housekeeping #1009: needs-decision.
Explicitly paused after repeated review in favor of the simpler private-staging redesign (#771/#788). Retain old crash/race evidence; #788 is the active rebase lane.
original head:
c129eb38a098f4e33e50f14c7c8bc43eb12e7323. against devcd7f84432febfc63b788fbb9b9f9950452982d74, git cherry reports 10 unmatched and 0 equivalent patches. the landing audit lists each candidate missing commit and stable patch match inpr-759.md; unmatched patches can include superseded behavior.left open, source branch unchanged. no rebase or new test claim for this decision lane.
Superseded by the private-staging redesign #788. The branch and original tips are archived under
archive/(see #1009), so nothing is lost; useful regression vectors are noted there for the active lanes. Closing.Pull request closed