kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 191 Pull requests 61 Projects Releases Packages Wiki Activity Actions 20

Deploy: two allocator heaps via GLIBC_TUNABLES (#398) #418

Closed
kayg wants to merge 0 commits from issue/398-tunables into dev
pull from: issue/398-tunables
merge into: kayg:dev
Conversation 1 Commits - Files changed -
kayg commented 2026-10-15 07:39:01 +00:00
Owner
Copy link

Part of #398. kayg asked for this on 2026-10-15: "set the 2 heap limit in the tunables".

What

  • Environment=GLIBC_TUNABLES=glibc.malloc.arena_max=2 goes in both quadlets, deploy/next/calternal-next.container and deploy/cloud/calternal.container. Two allocator heaps for the whole server, instead of one per busy thread (up to 8 per core). This replaces the mallopt(M_ARENA_MAX, 2) call in #386 without code or unsafe.
  • The perf harness passes the same setting to the server it starts, so perf numbers describe what is deployed. A run that sets its own GLIBC_TUNABLES keeps it; set it empty to get glibc's defaults.

Quadlet syntax checked

I checked this on calternal-staging (podman 5.4.2), remembering the earlier EnvironmentFile=- outage:

  • QUADLET_UNIT_DIRS=… /usr/libexec/podman/quadlet -dryrun converts both units with exit 0, and ExecStart gets --env GLIBC_TUNABLES=glibc.malloc.arena_max=2.
  • In a container started that way, ld.so --list-tunables prints glibc.malloc.arena_max: 0x2.
  • The tunable value has two =. The first one splits Environment=, and podman passes the rest as is.

Numbers

Perf VM, 100k notes, on top of #398's removal of the whole-Home list. Both runs use the same binary, with glibc's defaults against arena_max=2. Caches were dropped; 3 warm runs and 1 cold run each. Values are p50 / p95.

metric glibc defaults arena_max=2
idle.rss MB 183 130
mem.peak MB 142 / 180 108 / 131
cold.mem_peak MB 546 310
cold.index s 22.0 22.7
search.query ms 4.73 / 17.9 4.34 / 12.8
open.note ms 1.71 / 4.38 1.25 / 3.42
home.first_page ms 20.4 / 35.8 14.0 / 17.6
sync.save_ack ms 8.62 / 15.7 5.93 / 8.10
idle CPU, wake-ups/s 0, 0 0, 0

#44 round 23 ran interleaved A/B tests of the same setting through mallopt, at 1k and 10k with 5 runs each. They found no request slower.

Supersedes #386. The rest of the memory work (trim, no huge pages, allocator choice) is in the next #398 PRs.

🤖 Generated with Claude Code

https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T

Part of #398. kayg asked for this on 2026-10-15: "set the 2 heap limit in the tunables". ## What - **`Environment=GLIBC_TUNABLES=glibc.malloc.arena_max=2`** goes in both quadlets, `deploy/next/calternal-next.container` and `deploy/cloud/calternal.container`. Two allocator heaps for the whole server, instead of one per busy thread (up to 8 per core). This replaces the `mallopt(M_ARENA_MAX, 2)` call in #386 without code or `unsafe`. - **The perf harness passes the same setting to the server it starts**, so perf numbers describe what is deployed. A run that sets its own `GLIBC_TUNABLES` keeps it; set it empty to get glibc's defaults. ## Quadlet syntax checked I checked this on calternal-staging (podman 5.4.2), remembering the earlier `EnvironmentFile=-` outage: - `QUADLET_UNIT_DIRS=… /usr/libexec/podman/quadlet -dryrun` converts both units with exit 0, and `ExecStart` gets `--env GLIBC_TUNABLES=glibc.malloc.arena_max=2`. - In a container started that way, `ld.so --list-tunables` prints `glibc.malloc.arena_max: 0x2`. - The tunable value has two `=`. The first one splits `Environment=`, and podman passes the rest as is. ## Numbers Perf VM, 100k notes, on top of #398's removal of the whole-Home list. Both runs use the same binary, with glibc's defaults against `arena_max=2`. Caches were dropped; 3 warm runs and 1 cold run each. Values are p50 / p95. | metric | glibc defaults | arena_max=2 | |---|---|---| | idle.rss MB | 183 | **130** | | mem.peak MB | 142 / 180 | **108 / 131** | | cold.mem_peak MB | 546 | **310** | | cold.index s | 22.0 | 22.7 | | search.query ms | 4.73 / 17.9 | 4.34 / 12.8 | | open.note ms | 1.71 / 4.38 | 1.25 / 3.42 | | home.first_page ms | 20.4 / 35.8 | 14.0 / 17.6 | | sync.save_ack ms | 8.62 / 15.7 | 5.93 / 8.10 | | idle CPU, wake-ups/s | 0, 0 | 0, 0 | #44 round 23 ran interleaved A/B tests of the same setting through `mallopt`, at 1k and 10k with 5 runs each. They found no request slower. Supersedes #386. The rest of the memory work (trim, no huge pages, allocator choice) is in the next #398 PRs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T 
`glibc.malloc.arena_max=2` in the next and cloud quadlets, instead of a
`mallopt` call in the server (#386). With a heap per busy thread, what a
burst freed sat in many half-empty heaps; two heaps took the idle server
at 100,000 notes from 102 to 80 MB and the peak from 451 to 263 MB on
the perf VM (#44 round 19), with no request slower.

Checked on calternal-staging: `quadlet -dryrun` turns both units into
`--env GLIBC_TUNABLES=glibc.malloc.arena_max=2` (exit 0), and glibc in
a container started that way lists `glibc.malloc.arena_max: 0x2`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T
Perf: start the server with the deploys' allocator settings (#398)
Some checks failed
CI / server (pull_request) Successful in 30s
Details
CI / rust (pull_request) Failing after 30m2s
Details
CI / web (pull_request) Failing after 43m28s
Details
6076b15249 
The harness runs the binary itself, not the container, so it now passes
the same `GLIBC_TUNABLES` as the quadlets (two heaps), unless the run
sets its own (empty for glibc's defaults). Perf numbers then describe
what is deployed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T
kayg commented 2026-10-15 16:19:45 +00:00
Author
Owner
Copy link

Landed in dev via train m.

Landed in dev via train m.
kayg closed this pull request 2026-10-15 16:19:46 +00:00
Some checks failed
CI / server (pull_request) Successful in 30s
Details
CI / rust (pull_request) Failing after 30m2s
Details
CI / web (pull_request) Failing after 43m28s
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!418
No description provided.