kayg/calternal
Watch
1
0
Fork
You've already forked calternal
 0
Code Issues 191 Pull requests 61 Projects Releases Packages Wiki Activity Actions 20

Notes: a cold open keeps its announcements small until they go out #388

Closed
kayg wants to merge 1 commit from issue/44-cold-events into dev
pull from: issue/44-cold-events
merge into: kayg:dev
Conversation 1 Commits 1 Files changed 1 +12 -9
kayg commented 2026-10-15 00:55:17 +00:00
Owner
Copy link

Part of #44 (peak memory). Based on dev, independent.

cold.mem_peak at 100k is about 530 MB on the perf VM. One part of it: a cold open announces every note it found, a thousand at a time, once the id index is on disk. Until then it held a full EventDraft, serde_json path object included, for every note. It also re-split the remaining vector for every thousand it sent.

Now it keeps (kind, id, path) per note and builds each thousand events only as they're recorded. The order is unchanged, and so is the wait for the index flush.

Measured, 100,000 events with realistic ids and paths, RSS delta, one heap:

held at once
before, full events 85.5 MB
after, (kind, id, path) 22.8 MB

cargo test -p calternal-notes (169) and clippy are clean. A perf-VM cold.mem_peak follows on #44.

🤖 Generated with Claude Code

Part of #44 (peak memory). Based on dev, independent. `cold.mem_peak` at 100k is about 530 MB on the perf VM. One part of it: a cold open announces every note it found, a thousand at a time, once the id index is on disk. Until then it held a full `EventDraft`, `serde_json` path object included, for every note. It also re-split the remaining vector for every thousand it sent. Now it keeps `(kind, id, path)` per note and builds each thousand events only as they're recorded. The order is unchanged, and so is the wait for the index flush. **Measured**, 100,000 events with realistic ids and paths, RSS delta, one heap: | | held at once | |---|---| | before, full events | 85.5 MB | | after, (kind, id, path) | 22.8 MB | `cargo test -p calternal-notes` (169) and clippy are clean. A perf-VM `cold.mem_peak` follows on #44. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Notes: a cold open keeps its announcements small until they go out
Some checks failed
CI / server (pull_request) Successful in 26s
Details
CI / rust (pull_request) Successful in 2m9s
Details
CI / web (pull_request) Failing after 35m45s
Details
f916805eca 
A cold open announces every note it found, a thousand at a time, once
the id index is safely on disk. Until then it held a full event (with
its JSON) per note: 85 MB at once for 100,000 notes. It now keeps just
the kind, id and path (23 MB) and makes each thousand events as they're
recorded; the order and the wait for the index are unchanged. (#44)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Yyy8fQGxPLkLa8v8NXz7T
kayg commented 2026-10-15 01:43:43 +00:00
Author
Owner
Copy link

Closing: on the perf VM (round 22) it didn't move cold.mem_peak at 100k (300 → 300 MB, with two heaps from #386). The cold peak happens somewhere else, after these events have gone out. A heap profile of a cold 10k fill is running to find it. I'll open a new PR if this turns out to matter.

Closing: on the perf VM (round 22) it didn't move `cold.mem_peak` at 100k (300 → 300 MB, with two heaps from #386). The cold peak happens somewhere else, after these events have gone out. A heap profile of a cold 10k fill is running to find it. I'll open a new PR if this turns out to matter.
kayg closed this pull request 2026-10-15 01:43:43 +00:00
Some checks failed
CI / server (pull_request) Successful in 26s
Details
CI / rust (pull_request) Successful in 2m9s
Details
CI / web (pull_request) Failing after 35m45s
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
backend
Rust server

  
design
Mockups, UI and TASTE

  
docs
ADRs, CONTEXT, TASTE

  
idea
Parked idea, never worked until promoted

  
infra
Deploy, CI, VMs

  
perf
Performance and budgets

  
reliability
Data integrity, corruption traps

  
web
Web client

No labels
backend
design
docs
idea
infra
perf
reliability
web
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal!388
No description provided.