S1 spike: browser transport and wire format benchmark

kayg commented

2026-10-05 15:32:08 +00:00

Owner

Spike to choose the leg-A transport and the wire format with numbers. Decisions 5, 11, 12 and 41.

A Rust echo server with WebTransport (for example wtransport) and a WebSocket fallback; a browser client that echoes keystrokes
Measure the overhead from keystroke to echo (p50, p99) on the LAN and with simulated loss and latency (netem); the budget is < 5 ms p99 of overhead
Check WebTransport in Safari (iOS and macOS): certificates, what the server must advertise
Benchmark encode/decode for the hot-path frames: hand-written framing against protobuf (prost) against flatbuffers. Decide by the numbers.
Write a recommendation in the issue. The spike code is thrown away and is not merged.

Spike to choose the leg-A transport and the wire format with numbers. Decisions 5, 11, 12 and 41. - [ ] A Rust echo server with WebTransport (for example `wtransport`) and a WebSocket fallback; a browser client that echoes keystrokes - [ ] Measure the overhead from keystroke to echo (p50, p99) on the LAN and with simulated loss and latency (netem); the budget is < 5 ms p99 of overhead - [ ] Check WebTransport in Safari (iOS and macOS): certificates, what the server must advertise - [ ] Benchmark encode/decode for the hot-path frames: hand-written framing against protobuf (prost) against flatbuffers. Decide by the numbers. - [ ] Write a recommendation in the issue. The spike code is thrown away and is not merged.

kayg commented

2026-10-05 15:33:47 +00:00

Author

Owner

Starting S1 transport and wire-format spike on branch spike-transport, base 4560d9dd64. Probe: build a temporary Tokio WT/WS echo server, browser and Chromium benchmark client, and Rust/JS codecs; measure LAN and netem behavior, record evidence, recommend a transport and wire format.

Starting S1 transport and wire-format spike on branch spike-transport, base 4560d9dd64cec2da88254eb5014aa180728f1b5f. Probe: build a temporary Tokio WT/WS echo server, browser and Chromium benchmark client, and Rust/JS codecs; measure LAN and netem behavior, record evidence, recommend a transport and wire format.

kayg commented

2026-10-05 15:36:39 +00:00

Author

Owner

Finding: this worktree contains only the initial docs and license (rg --files returned no scripts/verify), so the required fast gate is absent. rustup has no installed toolchain; I will install stable under $HOME. sudo -n true succeeds, and 10.69.69.125 is assigned to wt0. Browser constraint: WebTransport requires a secure context, so the requested http:// URL can exercise WebSocket, while WT needs an HTTPS-served page; documenting both URLs and certificate requirements.

Finding: this worktree contains only the initial docs and license (rg --files returned no scripts/verify), so the required fast gate is absent. rustup has no installed toolchain; I will install stable under $HOME. `sudo -n true` succeeds, and 10.69.69.125 is assigned to wt0. Browser constraint: WebTransport requires a secure context, so the requested http:// URL can exercise WebSocket, while WT needs an HTTPS-served page; documenting both URLs and certificate requirements.

kayg commented

2026-10-05 15:43:30 +00:00

Author

Owner

Finding: WebKit documents WebTransport as shipping in Safari 26.4, and WebKit changes show serverCertificateHashes support. The W3C spec limits pinned certificate validity to two weeks. Safari still needs a secure page origin; pinning authenticates only the WebTransport connection, not the HTTPS document. I cannot run Safari from this Linux worktree. References: https://webkit.org/blog/17862/webkit-features-for-safari-26-4/ https://www.w3.org/TR/webtransport/ https://results.webkit.org/commit?uuid=175946337900

kayg referenced this issue

2026-10-05 16:03:24 +00:00

S1: contracts (wire, plugin/action, shim, bootstrap) #10

kayg referenced this issue

2026-10-05 16:04:56 +00:00

S1: contracts (wire, plugin/action, shim, bootstrap) #10

kayg commented

2026-10-05 16:16:56 +00:00

Author

Owner

Finding: the release server now serves the page over HTTPS and both echo transports work in headless Chromium 153. On the 10.69.69.125 listener, a one-sample forced WebTransport run and forced WSS run each echoed the byte; the observed RTT was 0.600 ms for each (smoke only, n=1; Chromium timer resolution is visible here), and server echo time was 17,964 ns for WT vs 50,604 ns for WS. The six-fixture browser decoder smoke measured handwritten decode at 273–299 ns/op and prost decode at 591–766 ns/op; wire sizes were 14/77/4,109/65,549/17/17 B vs 19/82/4,116/65,558/18/18 B. Full-sample and Criterion runs are next. Evidence: spikes/transport/static/bench.js and /tmp/spike-transport-smoke.json.

kayg commented

2026-10-05 16:21:02 +00:00

Author

Owner

Finding: WebKit’s public tracker records a Safari 26.4/macOS 26.x regression where createBidirectionalStream() can remain pending against non-Apple servers, while the WebTransport session itself becomes ready (Bug 312697, marked INVALID as a framework issue: https://bugs.webkit.org/show_bug.cgi?id=312697). Since the iPhone Safari path cannot be tested from this Linux host, I’m adding a 5-second stream-open timeout and Auto-mode WSS fallback, plus documenting that Safari support and successful interoperability are separate claims.

kayg commented

2026-10-05 16:22:02 +00:00

Author

Owner

Finding: full HeadlessChrome 153 local-host runs completed with 40 human-paced and 500 burst samples per transport (evidence: /tmp/spike-transport-lan.json). RTT p50/p95/p99 ms: WT human 0.700/0.800/1.400, WT burst 0.400/0.700/0.900, WS human 0.600/0.700/0.900, WS burst 0.400/0.700/0.700. Server echo p50/p95/p99 ns: WT human 5,148/5,943/39,107; WT burst 1,612/5,343/19,885; WS human 40,621/70,398/76,334; WS burst 27,039/60,355/77,504. These are local-interface results, and RTT quantization is 0.1 ms, so they do not establish a meaningful RTT winner. A simulated pending WT stream opened WSS after 5 s. Browser decoder: handwritten/prost ns-op for key, 64B, 4KiB, 64KiB PTY, resume, ack was 287/660, 283/610, 300/736, 305/706, 288/643, 306/667; wire sizes 14/19, 77/82, 4109/4116, 65549/65558, 17/18, 17/18 bytes.

kayg commented

2026-10-05 16:24:23 +00:00

Author

Owner

Finding: the disposable veth/netns run successfully applied 50 ms delay in each direction plus 1% netem loss on each egress (about 100 ms nominal added RTT; per-direction configured loss is 1%). Under 500 burst echoes, WT RTT p50/p95/p99 was 100.6/200.8/216.1 ms, while WebSocket was 190.2/401.3/489.1 ms. Under 40 human-paced echoes, WT was 101.0/102.5/330.0 ms and WebSocket 100.8/631.9/1032.5 ms. Server echo time stayed in the microsecond range (burst p50 1,070 ns WT, 5,141 ns WS), so the added tail is transport/network recovery rather than server processing. Evidence: /tmp/spike-transport-netem-1.json; script: spikes/transport/scripts/netem-bench.sh.

kayg commented

2026-10-05 16:29:40 +00:00

Author

Owner

Finding: a repeat after the cleanup fix hit Chromium ERR_NETWORK_CHANGED before loading the page. I added a namespace-side curl preflight plus one navigation retry for that transient error; a reduced 50 ms/1% WT+WS control then completed, and the script exited with no network namespace or veth left (evidence: /tmp/spike-transport-netem-smoke.log; ip netns list and the temporary-link check were empty). The original full 1% and 5% matrices remain in /tmp/spike-transport-netem-1-full.json and /tmp/spike-transport-netem-5.json.

Finding: a repeat after the cleanup fix hit Chromium ERR_NETWORK_CHANGED before loading the page. I added a namespace-side curl preflight plus one navigation retry for that transient error; a reduced 50 ms/1% WT+WS control then completed, and the script exited with no network namespace or veth left (evidence: /tmp/spike-transport-netem-smoke.log; `ip netns list` and the temporary-link check were empty). The original full 1% and 5% matrices remain in /tmp/spike-transport-netem-1-full.json and /tmp/spike-transport-netem-5.json.

kayg commented

2026-10-05 16:31:40 +00:00

Author

Owner

Finding: with 50 ms delay in each direction and 5% netem loss on each egress, WT human RTT p50/p95/p99 was 101.0/330.1/346.1 ms and WebSocket 101.0/578.3/828.4 ms. At 500 burst samples, WT was 159.4/216.7/304.6 ms versus WebSocket 189.1/266.6/287.8 ms. WT improved burst median and p95; WebSocket had a slightly lower burst p99 in this single run. Server echo p99 remained 5,610 ns WT and 24,226 ns WS. Evidence: /tmp/spike-transport-netem-5.json. The corrected cleanup preflight/control run completed with no residual namespace or veth.

kayg commented

2026-10-05 16:42:46 +00:00

Author

Owner

Finding: the current Criterion run used 20 samples/case (200 ms warmup, 500 ms measurement). Across all six requested frame shapes the length-prefixed codec had the lowest central encode and decode estimates; e.g. the 64 KiB PTY frame was 2,876 ns encode / 4.69 ns decode at 65,549 B, prost 4,154 ns / 2,570 ns at 65,558 B, and the safe FlatBuffers-layout codec 3,447 ns / 20.01 ns at 65,608 B. Tiny control frames were 17 B handwritten, 18 B protobuf, and 60 B FlatBuffers; key frame was 14/19/73 B. Full table and JS decoder costs are in spikes/transport/RESULTS.md. Recommendation: keep custom framing for the next comparison among these three, but Decision 12’s Cap’n Proto comparison is still open.

kayg commented

2026-10-05 16:50:02 +00:00

Author

Owner

Complete: commit 43db3b880d on branch spike-transport; working tree clean. Full methods, tables, limitations, and reproduction commands: spikes/transport/RESULTS.md. Recommendation: WebTransport first with WebSocket fallback. Under 1% loss on each egress and 50 ms each direction, 500-key burst WT RTT p50/p95/p99 was 100.6/200.8/216.1 ms vs WS 190.2/401.3/489.1 ms; at 5% loss WT was 159.4/216.7/304.6 ms vs WS 189.1/266.6/287.8 ms. Recommend the custom length-prefixed wire format among the three measured: it was smallest and fastest; Cap’n Proto remains unmeasured under Decision 12, and the FlatBuffers result uses a safe hand-written wire-layout codec rather than generated runtime code.

The enabled user unit spike-transport.service is active. Chromium 153 verified WT on HTTPS, WS on HTTP, and the copy button on both; HTTP/HTTPS routes return 200. Exact URLs:
HTTPS page: https://10.69.69.125:47331/
WebTransport: https://10.69.69.125:47331/wt
HTTP page: http://10.69.69.125:47332/
WebSocket: ws://10.69.69.125:47332/ws
Secure WebSocket: wss://10.69.69.125:47331/ws

Safari 26.4 adds WebTransport, but the HTTPS page must be trusted separately from the WebTransport certificate hash; Auto mode falls back to WSS/WS after 5 seconds if bidirectional stream setup stalls. Physical iPhone Safari was not available here. References: W3C two-week certificate limit https://www.w3.org/TR/webtransport/#server-authentication-using-certificate-hashes ; Apple Safari 26.4 support https://developer.apple.com/documentation/safari-release-notes/safari-26_4-release-notes ; WebKit tracker report https://bugs.webkit.org/show_bug.cgi?id=312697 .

Gate output, verbatim: scripts/verify fast returned zsh:1: no such file or directory: scripts/verify because this base has no repository fast-gate script. Spike gates passed: test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s; all-target cargo check; Clippy pedantic with warnings denied; release build; frozen Bun install; systemd unit verification.

Complete: commit 43db3b880d8c0b0515589934daf9c1ce0be181e6 on branch spike-transport; working tree clean. Full methods, tables, limitations, and reproduction commands: spikes/transport/RESULTS.md. Recommendation: WebTransport first with WebSocket fallback. Under 1% loss on each egress and 50 ms each direction, 500-key burst WT RTT p50/p95/p99 was 100.6/200.8/216.1 ms vs WS 190.2/401.3/489.1 ms; at 5% loss WT was 159.4/216.7/304.6 ms vs WS 189.1/266.6/287.8 ms. Recommend the custom length-prefixed wire format among the three measured: it was smallest and fastest; Cap’n Proto remains unmeasured under Decision 12, and the FlatBuffers result uses a safe hand-written wire-layout codec rather than generated runtime code. The enabled user unit spike-transport.service is active. Chromium 153 verified WT on HTTPS, WS on HTTP, and the copy button on both; HTTP/HTTPS routes return 200. Exact URLs: HTTPS page: https://10.69.69.125:47331/ WebTransport: https://10.69.69.125:47331/wt HTTP page: http://10.69.69.125:47332/ WebSocket: ws://10.69.69.125:47332/ws Secure WebSocket: wss://10.69.69.125:47331/ws Safari 26.4 adds WebTransport, but the HTTPS page must be trusted separately from the WebTransport certificate hash; Auto mode falls back to WSS/WS after 5 seconds if bidirectional stream setup stalls. Physical iPhone Safari was not available here. References: W3C two-week certificate limit https://www.w3.org/TR/webtransport/#server-authentication-using-certificate-hashes ; Apple Safari 26.4 support https://developer.apple.com/documentation/safari-release-notes/safari-26_4-release-notes ; WebKit tracker report https://bugs.webkit.org/show_bug.cgi?id=312697 . Gate output, verbatim: `scripts/verify fast` returned `zsh:1: no such file or directory: scripts/verify` because this base has no repository fast-gate script. Spike gates passed: test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s; all-target cargo check; Clippy pedantic with warnings denied; release build; frozen Bun install; systemd unit verification.

kayg commented

2026-10-05 16:51:22 +00:00

Author

Owner

Orchestrator review of the spike (43db3b8, RESULTS.md). It is good work, with careful caveats.

Conclusions for the contract:

Leg A: WebTransport first, WebSocket as the fallback (confirmed). The server-side echo overhead is < 80 µs p99 for both. That is far under the 5 ms budget, so the transport tails come from the network. With 1% loss, WT burst p99 is 216 ms against 489 ms for WS: this is TCP head-of-line blocking. Keep "auto mode": if the WT stream does not open in 5 s, use WS (see the WebKit 26.4 bidi-stream report).
Wire format: custom length-prefixed framing. It is the smallest frame, and decode is ~4.6 ns (Rust) and ~290 ns (JS), against 35–2570 ns for prost and ~650 ns in JS. Cap'n Proto was not measured. I do not add another spike for it: a zero-copy format cannot decode faster than ~4.6 ns for a borrowed slice, and it adds schema and runtime weight in TS. Proposal: close decision 12 as "custom". The parser is fuzzed as decision 40 requires.
New S1 question (for the owner): TLS. WebTransport needs a secure context with a trusted page certificate. The certificate-hash pin only covers the WT connection. Passkeys also need HTTPS (except on localhost). So the product needs a real certificate story (see S1 grill).

Still open: a test on a real iPhone (Safari 26.4+) at https://10.69.69.125:47331/ (trust the self-signed cert first) and http://10.69.69.125:47332/. Use "copy results" and paste the numbers here.

Orchestrator review of the spike (`43db3b8`, RESULTS.md). It is good work, with careful caveats. Conclusions for the contract: - **Leg A: WebTransport first, WebSocket as the fallback (confirmed).** The server-side echo overhead is < 80 µs p99 for both. That is far under the 5 ms budget, so the transport tails come from the network. With 1% loss, WT burst p99 is 216 ms against 489 ms for WS: this is TCP head-of-line blocking. Keep "auto mode": if the WT stream does not open in 5 s, use WS (see the WebKit 26.4 bidi-stream report). - **Wire format: custom length-prefixed framing.** It is the smallest frame, and decode is ~4.6 ns (Rust) and ~290 ns (JS), against 35–2570 ns for prost and ~650 ns in JS. Cap'n Proto was not measured. I do not add another spike for it: a zero-copy format cannot decode faster than ~4.6 ns for a borrowed slice, and it adds schema and runtime weight in TS. Proposal: close decision 12 as "custom". The parser is fuzzed as decision 40 requires. - **New S1 question (for the owner): TLS.** WebTransport needs a secure context with a trusted page certificate. The certificate-hash pin only covers the WT connection. **Passkeys also need HTTPS** (except on localhost). So the product needs a real certificate story (see S1 grill). Still open: a test on a real iPhone (Safari 26.4+) at https://10.69.69.125:47331/ (trust the self-signed cert first) and http://10.69.69.125:47332/. Use "copy results" and paste the numbers here.

kayg referenced this issue from a commit

2026-10-05 16:52:15 +00:00

Draft the wire protocol contract for legs A and B

kayg referenced this issue from a commit

2026-10-06 05:24:11 +00:00

Drop WebTransport from S1: WebSocket only on the browser leg

kayg commented

2026-10-06 05:24:32 +00:00

Author

Owner

Decision by the owner (2026-10-06): WebSocket only on leg A in S1 (DECISIONS 63, main 5304b4c). WebTransport returns only if real-phone numbers justify it. The spike's wire format result stands (custom codec). contracts/wire.md is updated (fd9a498).

Decision by the owner (2026-10-06): **WebSocket only on leg A in S1** (DECISIONS 63, `main` `5304b4c`). WebTransport returns only if real-phone numbers justify it. The spike's wire format result stands (custom codec). `contracts/wire.md` is updated (`fd9a498`).

kayg referenced this issue

2026-10-06 10:10:48 +00:00

S1 lane: protocol codec (Rust + TS), fuzzing, golden vectors #15

kayg referenced this issue

2026-10-06 10:47:45 +00:00

S1 lane: protocol codec (Rust + TS), fuzzing, golden vectors #15

kayg referenced this issue

2026-10-06 11:18:09 +00:00

S1 lane: protocol codec (Rust + TS), fuzzing, golden vectors #15

kayg referenced this issue from a commit

2026-10-10 09:21:11 +00:00

spike: measure transport and frame codecs

kayg commented

2026-10-28 17:52:06 +00:00

Author

Owner

Closed by the orchestrator during the 2026-10-28 handover (#254), on the owner's request to close finished issues. Evidence: Spike result delivered and owner decided WebSocket-only on leg A in D63 (custom codec stands); wire.md updated fd9a498. Reopen if anything is still missing.

Closed by the orchestrator during the 2026-10-28 handover (#254), on the owner's request to close finished issues. Evidence: Spike result delivered and owner decided WebSocket-only on leg A in D63 (custom codec stands); wire.md updated fd9a498. Reopen if anything is still missing.

kayg closed this issue

2026-10-28 17:52:07 +00:00

~~kayg referenced this issue 2026-10-28 17:53:54 +00:00~~

Orchestrator handover 2026-10-28: moving to a new machine #254

kayg referenced this issue

2026-10-28 18:00:45 +00:00

Orchestrator handover 2026-10-28: moving to a new machine #254

Rows
Columns

S1 spike: browser transport and wire format benchmark #12