A Home purge can slow another User's file write #174

Closed
opened 2026-09-26 13:25:49 +00:00 by kayg · 5 comments
Owner

An adversarial run measured a write to one User's Home while a different User's Home was being purged.

Evidence from 2026-09-26:

  • The probe started a purge for User D's Home after preparing 20,000 files in the test Home.
  • A concurrent upload to User A returned HTTP 201 in 2.89 seconds. The probe's expected limit was under 2 seconds.
  • The purge completed within its 120-second bound. No failed write, 5xx response, or data loss was reported.
  • Several builds, servers, and other adversarial probes were active on the shared host. This single run does not isolate whether the delay came from the purge, host contention, or both.

Please profile Home purge and concurrent writes for another User on an otherwise idle server. Check whether filesystem mutation serialization or purge work can block unrelated Users. No behavior change is proposed from this timing result alone.

An adversarial run measured a write to one User's Home while a different User's Home was being purged. Evidence from 2026-09-26: - The probe started a purge for User D's Home after preparing 20,000 files in the test Home. - A concurrent upload to User A returned HTTP 201 in 2.89 seconds. The probe's expected limit was under 2 seconds. - The purge completed within its 120-second bound. No failed write, 5xx response, or data loss was reported. - Several builds, servers, and other adversarial probes were active on the shared host. This single run does not isolate whether the delay came from the purge, host contention, or both. Please profile Home purge and concurrent writes for another User on an otherwise idle server. Check whether filesystem mutation serialization or purge work can block unrelated Users. No behavior change is proposed from this timing result alone.
Author
Owner

Additional evidence from the ask-page branch's one post-merge adversarial run on 2026-09-26 (head 32f9d1a9): the concurrent purge probe reported write during user purge: upload took 3.58s; expected under 2 s. Several independent builds and adversarial probes were active on the shared host, so this is not a controlled-load measurement. The server remained alive.

Additional evidence from the `ask-page` branch's one post-merge adversarial run on 2026-09-26 (head `32f9d1a9`): the concurrent purge probe reported `write during user purge: upload took 3.58s; expected under 2 s`. Several independent builds and adversarial probes were active on the shared host, so this is not a controlled-load measurement. The server remained alive.
Author
Owner

Additional evidence from the #176 post-merge adversarial run on 2026-09-27:

  • While User D’s Home purge was active, an upload to User A returned HTTP 201 in 2.23 seconds. The probe expects under 2 seconds.
  • No failed write, 5xx response, or data loss was reported.
  • Several other worktrees were running builds and adversarial probes on the shared host. This run does not isolate purge cost from host contention.

This adds one measurement to the existing finding. It does not establish a behavior change or a new cause.

Additional evidence from the #176 post-merge adversarial run on 2026-09-27: - While User D’s Home purge was active, an upload to User A returned HTTP 201 in 2.23 seconds. The probe expects under 2 seconds. - No failed write, 5xx response, or data loss was reported. - Several other worktrees were running builds and adversarial probes on the shared host. This run does not isolate purge cost from host contention. This adds one measurement to the existing finding. It does not establish a behavior change or a new cause.
Author
Owner

Duplicate and already-fixed evidence: #78 identifies the instance-wide deletion lock, and #330 tracks the cross-User upload latency with the same under-2-second acceptance check. The merged origin/dev commits f8e93b7e7 and 211dd4577 move heavy deletion work to a restart-safe background Worker and retain the probe. Recommend link this 2.89 s run to #330 as historical evidence; do not close it in this audit.

Duplicate and already-fixed evidence: #78 identifies the instance-wide deletion lock, and #330 tracks the cross-User upload latency with the same under-2-second acceptance check. The merged origin/dev commits f8e93b7e7 and 211dd4577 move heavy deletion work to a restart-safe background Worker and retain the probe. Recommend link this 2.89 s run to #330 as historical evidence; do not close it in this audit.
Author
Owner

Merge-round 7a evidence from the large Home purge probe: while purging a 50,000-file User Home, 20 uploads by another User had p95 latency 66.391 s (probe threshold 2 s); request 15 timed out. The host load average recorded during this phase was 47.19, 44.71, 43.12, so the result is load-contaminated but the purge interference did not meet the probe's latency bound. No file loss or authorization failure was observed. Full output is attached to #427's worktree log target/tmp/verify-7a-adversarial.log.

Merge-round 7a evidence from the large Home purge probe: while purging a 50,000-file User Home, 20 uploads by another User had p95 latency 66.391 s (probe threshold 2 s); request 15 timed out. The host load average recorded during this phase was 47.19, 44.71, 43.12, so the result is load-contaminated but the purge interference did not meet the probe's latency bound. No file loss or authorization failure was observed. Full output is attached to #427's worktree log `target/tmp/verify-7a-adversarial.log`.
Author
Owner

Duplicate of the User Home purge contention tracked by #330. The #330 fix is on origin/dev; #330 remains open for its unverified quiet-host latency target.

Duplicate of the User Home purge contention tracked by #330. The #330 fix is on origin/dev; #330 remains open for its unverified quiet-host latency target.
kayg closed this issue 2026-10-03 11:55:42 +00:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#174
No description provided.