Memory: glibc stays; trim after bursts, no huge pages (#398)

kayg commented

2026-10-15 09:29:52 +00:00

Owner

Part of #398. Supersedes #314 and #358: it is the same two commits, rebased onto #417 (the whole-Home list is gone, so its after_burst call there is too). It also replaces #386, because two heaps now come from the tunable in #418. Merge after #417 and #418.

Allocator verdict: glibc stays

I tried mimalloc (v2 and v3) and jemalloc against glibc. glibc keeps the server smallest, as long as it has three things:

two heaps (GLIBC_TUNABLES=glibc.malloc.arena_max=2, #418);
malloc_trim after bursts (calternal_memory::after_burst, from #314);
no transparent huge pages (prctl(PR_SET_THP_DISABLE), from #358).

This PR adds the last two. Both are unsafe calls in calternal-memory, the one crate kayg allowed it.

A/B on the perf VM, 100k notes

All on top of #417, with caches dropped and the lock taken per run. Each cell is the median over runs (run count in brackets) of each run's p50, or p95 where the row says so.

mi3: mimalloc v3 for Rust only.
mi3o: mimalloc v3 replacing malloc for SQLite too (override). The suffix -nothp means MIMALLOC_ALLOW_THP=0. -p0 means MIMALLOC_PURGE_DELAY=0.
jeo: jemalloc replacing malloc. jet adds narenas:2,dirty_decay_ms:1000,muzzy_decay_ms:0, and jetn adds thp:never on top.

metric	glibc defaults (1)	2 heaps (1)	2 heaps + trim + no THP (4)	mi3 (1)	mi3o (1)	mi3o-nothp (1)	mi3o-nothp-p0 (3)	mi2o (1)	jeo (1)	jet (1)	jetn (3)
idle.rss MB	183	130	66.8	188	190	78.6	72.0	125	193	127	74.2
mem.peak MB p50	142	108	110	178	179	121	108	178	216	130	118
mem.peak MB p95	180	131	121	695	670	368	291	489	224	132	131
cold.mem_peak MB	546	310	299	614	772	500	354	870	646	314	342
cold.index s	22.0	22.7	25.2	24.5	18.1	21.6	21.4	33.4	21.8	20.9	21.2
search.query ms p95	17.9	12.8	23.3	105	25.7	17.4	17.0	21.7	14.1	51.9	16.8
open.note ms p95	4.38	3.42	4.94	7.46	4.13	4.05	3.88	5.54	2.95	4.24	8.81
sync.save_ack ms p95	15.7	8.10	16.6	52.2	8.75	13.2	12.1	11.3	13.5	15.9	13.4
idle CPU, wake-ups/s	0, 0	0, 0	0, 0	0, 0	0, 0	0, 0	0, 0	0, 0	0, 0	0, 0	0, 0

The latency columns overlap run to run on this VM (its disk has no host cache). dev itself, run 3 times, gives open.note p95 2.5 to 5.2 ms and first_page p95 19 to 72 ms. No allocator is reliably faster.

Why mimalloc and jemalloc lose here

A cold open runs about 75 threads. 64 file readers plus search's 64 side-by-side document reads share tokio's blocking pool. Per-thread heaps hold what each thread freed:
- mimalloc's cold peak is 500 to 870 MB, against 299 MB for glibc with two heaps;
- a warm run with mimalloc sometimes peaks near 300 MB (p95), even tuned.
Transparent huge pages decide idle memory. The VM runs THP always, Debian's default. Every allocator's idle RSS roughly halves without THP.
- mimalloc can switch THP off itself with MIMALLOC_ALLOW_THP=0, which calls prctl(PR_SET_THP_DISABLE).
- jemalloc has thp:never.
- glibc needs the prctl this PR adds.
Tuned, the others come close but don't win. mimalloc with no THP and immediate purge reaches 72 MB idle, and tuned jemalloc reaches 74 MB, against 67 MB for glibc. glibc's cold peak is lower than both.
Nothing here wakes an idle server.
- mimalloc has no thread of its own. Its sources (libmimalloc-sys 0.1.49, which bundles mimalloc 2.3.2 and 3.3.2) never create one, and purging runs inside allocator calls (_mi_arenas_collect from page.c). As a result, with the default purge_delay (10 ms in v2, 1000 ms in v3), memory freed just before going idle stays resident until the next allocation.
- jemalloc's background_thread is off by default. Decay without it also runs only on allocator activity.
- glibc's trim here runs once, 2 s after the last burst, on a short-lived thread.
- Every configuration measured 0 idle CPU and 0 wake-ups/s.

Sources

mimalloc options and defaults (purge_delay, purge_decommits, arena_eager_commit, allow_thp): https://microsoft.github.io/mimalloc/environment.html. Bundled defaults were checked in c_src/mimalloc/v{2,3}/src/options.c.
mimalloc releases (v3 "tends to use less memory than v2"): https://github.com/microsoft/mimalloc/releases
The Rust crate: v3 is the default, v2 and override are features, and override defines MI_MALLOC_OVERRIDE: https://github.com/purpleprotocol/mimalloc_rust (README, libmimalloc-sys/build.rs).
jemalloc options (background_thread, dirty_decay_ms, muzzy_decay_ms, narenas, thp): https://jemalloc.net/jemalloc.3.html. The background thread sleeps indefinitely when no arena has deferred work: https://github.com/jemalloc/jemalloc/blob/dev/src/background_thread.c
jemalloc's upstream was archived in June 2025, and Meta resumed work on it in March 2026: https://engineering.fb.com/2026/03/02/data-infrastructure/investing-in-infrastructure-metas-renewed-commitment-to-jemalloc/
glibc malloc tunables: https://sourceware.org/glibc/manual/latest/html_node/Memory-Allocation-Tunables.html
SQLite uses the system malloc unless configured otherwise: https://www.sqlite.org/malloc.html
PR_SET_THP_DISABLE: https://man7.org/linux/man-pages/man2/PR_SET_THP_DISABLE.2const.html, and khugepaged: https://docs.kernel.org/admin-guide/mm/transhuge.html

Also tried

glibc.malloc.mmap_threshold=131072, on top of the floor fixes, 3 runs. It lowered the cold peak a little (196 to 180 MB) and left idle unchanged. It made cold.index slower, 24.5 to 33.4 s, so it is not used.

Checks

cargo fmt, clippy -D warnings and the workspace tests are clean. Leaks: see the 30-minute soak on #398 and #44.

🤖 Generated with Claude Code

https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T

Part of #398. **Supersedes #314 and #358:** it is the same two commits, rebased onto #417 (the whole-Home list is gone, so its `after_burst` call there is too). It also replaces #386, because two heaps now come from the tunable in #418. **Merge after #417 and #418.** ## Allocator verdict: glibc stays I tried mimalloc (v2 and v3) and jemalloc against glibc. glibc keeps the server smallest, as long as it has three things: - two heaps (`GLIBC_TUNABLES=glibc.malloc.arena_max=2`, #418); - `malloc_trim` after bursts (`calternal_memory::after_burst`, from #314); - no transparent huge pages (`prctl(PR_SET_THP_DISABLE)`, from #358). This PR adds the last two. Both are `unsafe` calls in `calternal-memory`, the one crate kayg allowed it. ### A/B on the perf VM, 100k notes All on top of #417, with caches dropped and the lock taken per run. Each cell is the median over runs (run count in brackets) of each run's p50, or p95 where the row says so. - **mi3**: mimalloc v3 for Rust only. - **mi3o**: mimalloc v3 replacing `malloc` for SQLite too (`override`). The suffix `-nothp` means `MIMALLOC_ALLOW_THP=0`. `-p0` means `MIMALLOC_PURGE_DELAY=0`. - **jeo**: jemalloc replacing `malloc`. **jet** adds `narenas:2,dirty_decay_ms:1000,muzzy_decay_ms:0`, and **jetn** adds `thp:never` on top. | metric | glibc defaults (1) | 2 heaps (1) | **2 heaps + trim + no THP (4)** | mi3 (1) | mi3o (1) | mi3o-nothp (1) | mi3o-nothp-p0 (3) | mi2o (1) | jeo (1) | jet (1) | jetn (3) | |---|---|---|---|---|---|---|---|---|---|---|---| | idle.rss MB | 183 | 130 | **66.8** | 188 | 190 | 78.6 | 72.0 | 125 | 193 | 127 | 74.2 | | mem.peak MB p50 | 142 | 108 | **110** | 178 | 179 | 121 | 108 | 178 | 216 | 130 | 118 | | mem.peak MB p95 | 180 | 131 | **121** | 695 | 670 | 368 | 291 | 489 | 224 | 132 | 131 | | cold.mem_peak MB | 546 | 310 | **299** | 614 | 772 | 500 | 354 | 870 | 646 | 314 | 342 | | cold.index s | 22.0 | 22.7 | 25.2 | 24.5 | 18.1 | 21.6 | 21.4 | 33.4 | 21.8 | 20.9 | 21.2 | | search.query ms p95 | 17.9 | 12.8 | 23.3 | 105 | 25.7 | 17.4 | 17.0 | 21.7 | 14.1 | 51.9 | 16.8 | | open.note ms p95 | 4.38 | 3.42 | 4.94 | 7.46 | 4.13 | 4.05 | 3.88 | 5.54 | 2.95 | 4.24 | 8.81 | | sync.save_ack ms p95 | 15.7 | 8.10 | 16.6 | 52.2 | 8.75 | 13.2 | 12.1 | 11.3 | 13.5 | 15.9 | 13.4 | | idle CPU, wake-ups/s | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | 0, 0 | The latency columns overlap run to run on this VM (its disk has no host cache). `dev` itself, run 3 times, gives open.note p95 2.5 to 5.2 ms and first_page p95 19 to 72 ms. No allocator is reliably faster. ### Why mimalloc and jemalloc lose here - **A cold open runs about 75 threads.** 64 file readers plus search's 64 side-by-side document reads share tokio's blocking pool. Per-thread heaps hold what each thread freed: - mimalloc's cold peak is 500 to 870 MB, against 299 MB for glibc with two heaps; - a warm run with mimalloc sometimes peaks near 300 MB (p95), even tuned. - **Transparent huge pages decide idle memory.** The VM runs THP `always`, Debian's default. Every allocator's idle RSS roughly halves without THP. - mimalloc can switch THP off itself with `MIMALLOC_ALLOW_THP=0`, which calls `prctl(PR_SET_THP_DISABLE)`. - jemalloc has `thp:never`. - glibc needs the `prctl` this PR adds. - **Tuned, the others come close but don't win.** mimalloc with no THP and immediate purge reaches 72 MB idle, and tuned jemalloc reaches 74 MB, against 67 MB for glibc. glibc's cold peak is lower than both. - **Nothing here wakes an idle server.** - mimalloc has no thread of its own. Its sources (`libmimalloc-sys` 0.1.49, which bundles mimalloc 2.3.2 and 3.3.2) never create one, and purging runs inside allocator calls (`_mi_arenas_collect` from `page.c`). As a result, with the default `purge_delay` (10 ms in v2, 1000 ms in v3), memory freed just before going idle stays resident until the next allocation. - jemalloc's `background_thread` is off by default. Decay without it also runs only on allocator activity. - glibc's trim here runs once, 2 s after the last burst, on a short-lived thread. - Every configuration measured 0 idle CPU and 0 wake-ups/s. ### Sources - mimalloc options and defaults (`purge_delay`, `purge_decommits`, `arena_eager_commit`, `allow_thp`): https://microsoft.github.io/mimalloc/environment.html. Bundled defaults were checked in `c_src/mimalloc/v{2,3}/src/options.c`. - mimalloc releases (v3 "tends to use less memory than v2"): https://github.com/microsoft/mimalloc/releases - The Rust crate: v3 is the default, `v2` and `override` are features, and `override` defines `MI_MALLOC_OVERRIDE`: https://github.com/purpleprotocol/mimalloc_rust (README, `libmimalloc-sys/build.rs`). - jemalloc options (`background_thread`, `dirty_decay_ms`, `muzzy_decay_ms`, `narenas`, `thp`): https://jemalloc.net/jemalloc.3.html. The background thread sleeps indefinitely when no arena has deferred work: https://github.com/jemalloc/jemalloc/blob/dev/src/background_thread.c - jemalloc's upstream was archived in June 2025, and Meta resumed work on it in March 2026: https://engineering.fb.com/2026/03/02/data-infrastructure/investing-in-infrastructure-metas-renewed-commitment-to-jemalloc/ - glibc malloc tunables: https://sourceware.org/glibc/manual/latest/html_node/Memory-Allocation-Tunables.html - SQLite uses the system `malloc` unless configured otherwise: https://www.sqlite.org/malloc.html - `PR_SET_THP_DISABLE`: https://man7.org/linux/man-pages/man2/PR_SET_THP_DISABLE.2const.html, and khugepaged: https://docs.kernel.org/admin-guide/mm/transhuge.html ### Also tried `glibc.malloc.mmap_threshold=131072`, on top of the floor fixes, 3 runs. It lowered the cold peak a little (196 to 180 MB) and left idle unchanged. It made `cold.index` slower, 24.5 to 33.4 s, so it is not used. ## Checks `cargo fmt`, `clippy -D warnings` and the workspace tests are clean. Leaks: see the 30-minute soak on #398 and #44. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T

kayg added 3 commits

2026-10-15 09:29:52 +00:00

Memory: remove the whole-Home notes.list (#398 )

CI / server (pull_request) Successful in 33s

Details

CI / rust (pull_request) Successful in 2m35s

Details

CI / web (pull_request) Failing after 44m28s

Details

44306033b1

`notes.list` without input built every note of the Home into one answer
(100,000 objects at 100k notes, hundreds of megabytes at once). The app
only asks for pages, and the form was deprecated "for one release".
Without input it now returns the first page of 50, like `{}`.

The perf harness no longer calls it: `home.first_list`, `home.list` and
`mem.after_list` are gone (`mem.after_first_page` instead), and the cold
run opens the Home with a first page (`cold.first_page`).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T

Memory: hand freed memory back after bursts 401d55ed06

glibc keeps what a burst of work freed (the startup scan, the list store
catching up, a search index build or catch-up, the whole-Home list)
inside its heaps, a heap per thread, and rarely returns it: at 100,000
notes the server sat at 150 MB or more with about 30 MB in use (#44).

A new small crate, `calternal-memory`, calls glibc's `malloc_trim` two
seconds after the last burst (a burst arms one wait on a thread of its
own: nothing runs on a timer, an idle server never wakes up for it). It
is the one crate allowed an `unsafe` call, for that one call; the
workspace still forbids `unsafe` everywhere else. Hooked after the
startup scan, big list refreshes, search catch-up and updates, and the
whole-Home list.

Local, 100k notes, system allocator: settled 82 -> 60 MB, after searches
152-160 -> 84 MB.

Part of #44

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T

Memory: no transparent huge pages

CI / server (pull_request) Successful in 23s

Details

CI / rust (pull_request) Successful in 2m26s

Details

CI / web (pull_request) Failing after 41m1s

Details

d5cebeb198

With transparent huge pages on `always` (Debian's default, and the perf
VM's), the kernel's background thread merges heap regions into whole
2 MB pages even when most of the region is free. A trimmed heap is
mostly free pages, so an idle server at 100,000 notes grew from 126 to
155 MB while doing nothing, 84 MB of it in huge pages that trimming
can't hand back.

The server now asks the kernel at start (`prctl(PR_SET_THP_DISABLE)`)
to keep it on normal pages, in calternal-memory, the one crate allowed
an `unsafe` call. (#44)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T

kayg referenced this pull request

2026-10-15 10:02:23 +00:00

Memory: hand freed memory back after bursts #314

kayg referenced this pull request

2026-10-15 10:02:25 +00:00

Memory: no transparent huge pages #358

kayg referenced this pull request

2026-10-15 11:20:37 +00:00

Perf: a leak soak scenario (#398) #435

kayg referenced this pull request

2026-10-15 11:21:16 +00:00

Memory: allocator A/B (mimalloc), tunables, drop whole-Home list, lower the floor #398