Profile live log serialization against the 50 MiB/s gate #81
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem and evidence
In the 306-Docker / 306-Podman stream workload, completed log bursts carried 32,768 records and 33,947,648 encoded bytes. Remote Docker throughput was 8.949 / 11.776 / 12.862 MiB/s at 256 KiB / 1 MiB / 4 MiB windows; remote Podman measured 6.220 / 9.491 MiB/s in its successful 256 KiB / 1 MiB trials, while the second 1 MiB follow trial exceeded 60 seconds after several 32 MiB bursts. Load1 was roughly 25–34. The fixture used 64 MB rotation, so rotation/follow behavior remains a confounder. The audit found that larger windows alone do not reach the 50 MiB/s product target.
The shared output profile is named-stream-output.svg; exact trials are in continuation-results.json.
The current path decodes engine LogOutput into bounded log records (crates/provider-bollard/src/logs.rs:117-150), encodes each record (crates/proto/src/stream/records.rs:57-71), then batches and flushes records into stream frames (crates/agent/src/lib.rs:1916-2014). These are the stages to profile; the audit does not prove which one dominates.
Recommendation
Measure live log source serialization separately from receive-window capacity using a real, byte-counted rotating/follow fixture. Profile provider reads, record encoding and server/browser delivery, and distinguish a fixture rotation stall from a product stall. Keep the simultaneous paced logs/keystrokes gate as the acceptance test.
Payoff, effort and risk
Expected gain: identify the bottleneck preventing the existing 50 MiB/s gate; no numerical gain estimate is credible until the log source and fixture are isolated.
Quick win vs deeper change: validate rotation and reproduce the >60 s follow trial in ≤1 day; profiling and fixing source serialization is estimated at 1–3 days.
Risk: medium; log follow, rotation, record boundaries and bounded-memory behavior must remain correct.
Priority: Next milestone.
2026-10-04 owner decisions: Matches the live, multi-tail logs product and the custom stream protocol. Do not lower the settled 50 MiB/s target based on these unshaped diagnostics. Related: #20 and #60.
Report: perf-audit report
Source: perf-audit 2026-10-04