Money register: four concurrent 200k reads OOM-kill the server #1364

Open
opened 2026-10-07 14:08:41 +00:00 by kayg · 0 comments
Owner

Parent: #688. Blocks merge.

The locked perf-VM run used the production Money backend at 2829bc799 (binary SHA-256 666b602a4d128ef603390bd30b56d41884c6014f43b360c82cfff410676112be). A 200,000-Transaction synthetic Budget completed import. Warm page p95 was 22344 ms; Account summary p95 was 22263.8 ms (50 reads each). A burst of four register reads then failed with TypeError: Failed to fetch.

At 2026-10-07 13:54:25 UTC, the kernel OOM-killed PID 1666079 (calternal-serve), with anonymous RSS 7285028 KiB. The cold page peak RSS was 2841628672 bytes.

The source-byte admission rejects retention for this Budget. Each request can therefore build its own full Ledger and SQL generation. Overlapping reads multiply memory. Fix admission and generation sharing, and replace cold replay with persistent maintained aggregates. Do not merely raise the memory limit. Add a regression for overlapping reads when retention is denied, then repeat the four-read 200k burst on a real server.

Profile: bench/money-register-688.mjs; numbers are in docs/perf/baseline.json. No real User data was used.

Parent: #688. Blocks merge. The locked perf-VM run used the production Money backend at `2829bc799` (binary SHA-256 `666b602a4d128ef603390bd30b56d41884c6014f43b360c82cfff410676112be`). A 200,000-Transaction synthetic Budget completed import. Warm page p95 was 22344 ms; Account summary p95 was 22263.8 ms (50 reads each). A burst of four register reads then failed with `TypeError: Failed to fetch`. At 2026-10-07 13:54:25 UTC, the kernel OOM-killed PID 1666079 (`calternal-serve`), with anonymous RSS 7285028 KiB. The cold page peak RSS was 2841628672 bytes. The source-byte admission rejects retention for this Budget. Each request can therefore build its own full Ledger and SQL generation. Overlapping reads multiply memory. Fix admission and generation sharing, and replace cold replay with persistent maintained aggregates. Do not merely raise the memory limit. Add a regression for overlapping reads when retention is denied, then repeat the four-read 200k burst on a real server. Profile: `bench/money-register-688.mjs`; numbers are in `docs/perf/baseline.json`. No real User data was used.
Commenting is not possible because the repository is archived.
No labels
No milestone
No project
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal-v0#1364
No description provided.