Photo upload burst drops items under adversarial load #565

Open
opened 2026-10-01 02:42:08 +00:00 by kayg · 1 comment
Owner

During the local API adversarial round for calternal#465, the fresh server received 120 synthetic 1x1 JPEG Tus uploads with 8 concurrent workers. The setup returned 429 responses without upload Locations and request timeouts. The probe then listed only 36 of the expected 120 Calendar Items. Several Calendar, Files, and Reminder requests also timed out during the same sustained burst. The server stayed alive.

Reproduce with ADVERSARIAL_API_ONLY=1 tests/adversarial/run.sh. The unchanged probe is in tests/adversarial/attack.py under the Tags shared sources and Calendar items burst sections. This indicates that an import-like burst can leave most items absent; please investigate rate limits, backpressure, and recovery behavior.

Evidence was collected on 2026-10-01 from the local dev server. The Inspector change does not touch upload or Calendar routes.

During the local API adversarial round for calternal#465, the fresh server received 120 synthetic 1x1 JPEG Tus uploads with 8 concurrent workers. The setup returned 429 responses without upload Locations and request timeouts. The probe then listed only 36 of the expected 120 Calendar Items. Several Calendar, Files, and Reminder requests also timed out during the same sustained burst. The server stayed alive. Reproduce with `ADVERSARIAL_API_ONLY=1 tests/adversarial/run.sh`. The unchanged probe is in `tests/adversarial/attack.py` under the Tags shared sources and Calendar items burst sections. This indicates that an import-like burst can leave most items absent; please investigate rate limits, backpressure, and recovery behavior. Evidence was collected on 2026-10-01 from the local dev server. The Inspector change does not touch upload or Calendar routes.
Author
Owner

#427 comparison: production cc25c441b and merged a6fd7f704 each passed ten 120-photo bursts with eight workers. Every POST was 201, every PATCH was 204, and paged Calendar reads returned exactly 120 unique paths each time. Upload rejection is recorded separately from missing accepted writes.

This issue already shows the problem on dev: 429 responses and timeouts prevented uploads from being accepted. The original round-4 report's 0/120 has no preserved POST/PATCH status witness, so its precise cause remains unreproduced. Severity: pre-existing setup/load rejection confirmed here; lost accepted uploads not reproduced. No evidence that round 4 makes production worse. Keep the issue open; do not remove rate controls or weaken the 120-item assertion.

#427 comparison: production cc25c441b and merged a6fd7f704 each passed ten 120-photo bursts with eight workers. Every POST was 201, every PATCH was 204, and paged Calendar reads returned exactly 120 unique paths each time. Upload rejection is recorded separately from missing accepted writes. This issue already shows the problem on dev: 429 responses and timeouts prevented uploads from being accepted. The original round-4 report's 0/120 has no preserved POST/PATCH status witness, so its precise cause remains unreproduced. Severity: pre-existing setup/load rejection confirmed here; lost accepted uploads not reproduced. No evidence that round 4 makes production worse. Keep the issue open; do not remove rate controls or weaken the 120-item assertion.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#565
No description provided.