PERF: make MCP crazy fast (measure initialize/tools.list/tools.call, cache, in-process dispatch) #492

Open
opened 2026-09-30 07:16:42 +00:00 by kayg · 10 comments
Owner

MCP performance: "crazy fast" (owner, 2026-09-30)

Measure first, then optimise. Measurement protocol (perf VM, flock /root/perf.lock, the shared release binary, interleaved runs, decision rules written down before running):

  • End-to-end latency for initialize, tools/list (with the full #484 tool set, which will be 400+ tools), and tools/call for representative reads (list today's events, search notes, a Money month) and writes (create a task, log a transaction): p50/p95/p99, cold and warm.
  • Payload sizes: the tools/list response bytes and the tokens a model spends reading it; tool result sizes.
  • Throughput: 50 concurrent MCP sessions; memory per session.
    Optimisation targets (verify each with numbers):
  • Transport: Streamable HTTP with keep-alive; no per-call re-authentication cost (cache the App Password verification with revoke-aware invalidation); HTTP/2.
  • tools/list: precomputed and cached per scope set, ETag and compression, pagination, and short but clear descriptions. Consider grouping tools with an inputSchema discriminator if the tool count hurts model latency, and measure the token cost both ways.
  • Calls: a direct in-process dispatch to the same handlers as the API (no loopback HTTP hop), streaming results for big lists, and field selection so models can ask for less.
  • Target to verify (not assume): warm tools/call p95 under 20 ms server-side for simple reads; tools/list under 5 ms from cache.
    Deliver docs/perf/runs/<date>-mcp.md, a baseline in docs/perf/, and an issue per bottleneck. The measurement can start now. Code changes to the MCP layer must wait until the #484 registry core merges (same files), then land as their own commits.
## MCP performance: "crazy fast" (owner, 2026-09-30) Measure first, then optimise. Measurement protocol (perf VM, `flock /root/perf.lock`, the shared release binary, interleaved runs, decision rules written down before running): - End-to-end latency for `initialize`, `tools/list` (with the full #484 tool set, which will be 400+ tools), and `tools/call` for representative reads (list today's events, search notes, a Money month) and writes (create a task, log a transaction): p50/p95/p99, cold and warm. - Payload sizes: the `tools/list` response bytes and the tokens a model spends reading it; tool result sizes. - Throughput: 50 concurrent MCP sessions; memory per session. Optimisation targets (verify each with numbers): - **Transport:** Streamable HTTP with keep-alive; no per-call re-authentication cost (cache the App Password verification with revoke-aware invalidation); HTTP/2. - **tools/list:** precomputed and cached per scope set, ETag and compression, pagination, and short but clear descriptions. Consider grouping tools with an `inputSchema` discriminator if the tool count hurts model latency, and measure the token cost both ways. - **Calls:** a direct in-process dispatch to the same handlers as the API (no loopback HTTP hop), streaming results for big lists, and field selection so models can ask for less. - **Target to verify (not assume):** warm tools/call p95 under 20 ms server-side for simple reads; tools/list under 5 ms from cache. Deliver `docs/perf/runs/<date>-mcp.md`, a baseline in `docs/perf/`, and an issue per bottleneck. The measurement can start now. Code changes to the MCP layer must wait until the #484 registry core merges (same files), then land as their own commits.
Author
Owner

Starting #492 on branch job/mcp-fast-492, base SHA 7e847c2ffc5b76b2bb7cd790ddf51bf414b6f898. I read CLAUDE.md, CONTEXT.md, and DESIGN.md §§41, 48. I will measure the available MCP build first and wait for #484 registry core before changing MCP implementation files.

Starting #492 on branch `job/mcp-fast-492`, base SHA `7e847c2ffc5b76b2bb7cd790ddf51bf414b6f898`. I read CLAUDE.md, CONTEXT.md, and DESIGN.md §§41, 48. I will measure the available MCP build first and wait for #484 registry core before changing MCP implementation files.
Author
Owner

Finding: #484 remains open, and the current crates/calternal-server/src/mcp.rs has 9 tools while docs/mcp.md groups multiple Mail actions under one tool. The current API cannot exercise Calendar event listing or Money month/transaction tools through MCP. I will mark this interim baseline as incomplete and repeat the requested full matrix after the registry core merges.

Finding: #484 remains open, and the current `crates/calternal-server/src/mcp.rs` has 9 tools while `docs/mcp.md` groups multiple Mail actions under one tool. The current API cannot exercise Calendar event listing or Money month/transaction tools through MCP. I will mark this interim baseline as incomplete and repeat the requested full matrix after the registry core merges.
Author
Owner

Finding: every incoming Bearer request reaches wire.rs and calls verify_app_password(None, token). calternal-auth/src/store.rs does a SQL lookup, Argon2id verification on spawn_blocking, then wire.rs records app-password use in a database update. This gives #492 a concrete auth hot path to measure; any cache must keep revocation effective on the next request. mcp.rs already dispatches API calls in process with Router::oneshot; there is no loopback HTTP hop to remove. I will profile Router/body handling as the remaining dispatch cost.

Finding: every incoming Bearer request reaches `wire.rs` and calls `verify_app_password(None, token)`. `calternal-auth/src/store.rs` does a SQL lookup, Argon2id verification on `spawn_blocking`, then `wire.rs` records app-password use in a database update. This gives #492 a concrete auth hot path to measure; any cache must keep revocation effective on the next request. `mcp.rs` already dispatches API calls in process with `Router::oneshot`; there is no loopback HTTP hop to remove. I will profile Router/body handling as the remaining dispatch cost.
Author
Owner

Correction from the shared release artifact: its source is commit 369ab6a2f9fc673e3564b94857fbecfeb04df404 (recorded by the perf snapshot), and its mcp.rs/wire.rs hashes match that commit. That build has 8 MCP tools; the checked out origin/dev source has 9. The interim measurements will identify the release commit and tool count. #484 is still open, so Calendar and Money tool timings remain pending.

Correction from the shared release artifact: its source is commit `369ab6a2f9fc673e3564b94857fbecfeb04df404` (recorded by the perf snapshot), and its `mcp.rs`/`wire.rs` hashes match that commit. That build has 8 MCP tools; the checked out `origin/dev` source has 9. The interim measurements will identify the release commit and tool count. #484 is still open, so Calendar and Money tool timings remain pending.
Author
Owner

Measurement harness finding on the shared release server (369ab6a2f9fc673e3564b94857fbecfeb04df404): its MCP_PROTOCOLS list contains only 2026-07-28, and rmcp defines that version as NO_INITIALIZE. Sending an initialize request therefore returns HTTP 400 / JSON-RPC -32022 (unsupported protocol version); a successful initialize latency does not exist for this build. The 2026 protocol also requires Mcp-Method; omitting it returns HTTP 400 / -32020 with “missing required Mcp-Method header”. The profile will record the unsupported legacy initialize response and benchmark the supported server/discover, tools/list, and tools/call paths instead. This is protocol behavior in the measured build, not an application-code change.

Measurement harness finding on the shared release server (`369ab6a2f9fc673e3564b94857fbecfeb04df404`): its `MCP_PROTOCOLS` list contains only `2026-07-28`, and rmcp defines that version as `NO_INITIALIZE`. Sending an `initialize` request therefore returns HTTP 400 / JSON-RPC `-32022` (unsupported protocol version); a successful initialize latency does not exist for this build. The 2026 protocol also requires `Mcp-Method`; omitting it returns HTTP 400 / `-32020` with “missing required Mcp-Method header”. The profile will record the unsupported legacy initialize response and benchmark the supported `server/discover`, `tools/list`, and `tools/call` paths instead. This is protocol behavior in the measured build, not an application-code change.
Author
Owner

Correction to my previous protocol comment: the valid legacy initialize probe uses version 2025-11-25 (the latest MCP version with an initialize handshake). On the measured release it returns HTTP 200 with JSON-RPC error -32022; unsupported negotiation is carried in the JSON-RPC envelope. The profile now records the HTTP status and error code separately. The server advertises only 2026-07-28, which has no initialize handshake, so server/discover is the supported capability-discovery path.

Correction to my previous protocol comment: the valid legacy initialize probe uses version `2025-11-25` (the latest MCP version with an initialize handshake). On the measured release it returns HTTP 200 with JSON-RPC error `-32022`; unsupported negotiation is carried in the JSON-RPC envelope. The profile now records the HTTP status and error code separately. The server advertises only `2026-07-28`, which has no initialize handshake, so `server/discover` is the supported capability-discovery path.
Author
Owner

Full current-release baseline completed under /root/perf.lock on the perf VM. The runner commit is 35248923f; the release source is 369ab6a2f9fc673e3564b94857fbecfeb04df404, binary SHA-256 bb3dde668120fd848ef6e31997293701e0de8332d1d615f4f41f0e30b9e2324d. The profile used 30 cold samples, 100 warm samples, and 50 clients × 20 read calls. Every server start restored the same seeded Home snapshot.

Evidence: warm tools/list p95 was 21.051 ms; its 8-tool schema was 2,889 bytes / 654 o200k_base tokens. A grouped dispatcher measured 681 tokens. There was no compression, ETag, cache policy, or pagination cursor. Warm Search Notes p95 was 23.130 ms. Warm create Task p95 was 666.881 ms; create Log entry p95 was 221.447 ms. These are client-side loopback results: the response had no Server-Timing header. The 50-client read burst measured p95 441.895 ms, 127.42 requests/s, and an estimated 2,854,257 bytes per idle connection; peak server RSS was 461,324,288 bytes. Host load average rose from 0.22 / 0.14 / 0.07 to 2.52 / 0.99 / 0.39 during the run.

The current release exposes protocol 2026-07-28, so legacy initialize is unsupported (2025-11-25 probe: HTTP 200 / JSON-RPC -32022); server/discover is the supported discovery path. The current release has 8 tools, and Calendar and Money operations are still unavailable pending #484. The server listener supports HTTP/1.1; HTTP/2 at the reverse proxy was not measured. Follow-up findings are tracked in #515 (tool-list caching), #516 (password verification cost), and #517 (write latency and memory growth). Full results are in docs/perf/runs/2026-09-30-mcp-current.json and docs/perf/mcp-baseline.json; the run report is docs/perf/runs/2026-09-30-mcp.md.

Full current-release baseline completed under `/root/perf.lock` on the perf VM. The runner commit is `35248923f`; the release source is `369ab6a2f9fc673e3564b94857fbecfeb04df404`, binary SHA-256 `bb3dde668120fd848ef6e31997293701e0de8332d1d615f4f41f0e30b9e2324d`. The profile used 30 cold samples, 100 warm samples, and 50 clients × 20 read calls. Every server start restored the same seeded Home snapshot. Evidence: warm `tools/list` p95 was 21.051 ms; its 8-tool schema was 2,889 bytes / 654 `o200k_base` tokens. A grouped dispatcher measured 681 tokens. There was no compression, ETag, cache policy, or pagination cursor. Warm Search Notes p95 was 23.130 ms. Warm create Task p95 was 666.881 ms; create Log entry p95 was 221.447 ms. These are client-side loopback results: the response had no `Server-Timing` header. The 50-client read burst measured p95 441.895 ms, 127.42 requests/s, and an estimated 2,854,257 bytes per idle connection; peak server RSS was 461,324,288 bytes. Host load average rose from 0.22 / 0.14 / 0.07 to 2.52 / 0.99 / 0.39 during the run. The current release exposes protocol `2026-07-28`, so legacy initialize is unsupported (`2025-11-25` probe: HTTP 200 / JSON-RPC `-32022`); `server/discover` is the supported discovery path. The current release has 8 tools, and Calendar and Money operations are still unavailable pending #484. The server listener supports HTTP/1.1; HTTP/2 at the reverse proxy was not measured. Follow-up findings are tracked in #515 (tool-list caching), #516 (password verification cost), and #517 (write latency and memory growth). Full results are in `docs/perf/runs/2026-09-30-mcp-current.json` and `docs/perf/mcp-baseline.json`; the run report is `docs/perf/runs/2026-09-30-mcp.md`.
Author
Owner

Decision from the baseline: keep the current 8 individual tool schemas. The hypothetical grouped schema uses 681 tokens versus 654 for the individual schemas. Re-measure grouping when the full #484 registry is available. The runner uses a separate docs/perf/mcp-baseline.json so it does not replace the shared web baseline in docs/perf/baseline.json.

Decision from the baseline: keep the current 8 individual tool schemas. The hypothetical grouped schema uses 681 tokens versus 654 for the individual schemas. Re-measure grouping when the full #484 registry is available. The runner uses a separate `docs/perf/mcp-baseline.json` so it does not replace the shared web baseline in `docs/perf/baseline.json`.
Author
Owner

Adversarial finding: the one-time real-server API-only run reported no response on POST /api/v1/tags/rename (old_tag=travel, new_tag=journey), then connection refused; the server process exited as Aborted. The probe is tag rename across sources near tests/adversarial/attack.py:2608. Host load reached about 42, and runner cleanup removed its temporary logs; no core or stack was available. Timing implicates the probe sequence but does not establish the cause. I filed #525 with the evidence and follow-up. The job's single adversarial round is complete; I did not rerun it.

Adversarial finding: the one-time real-server API-only run reported no response on `POST /api/v1/tags/rename` (`old_tag=travel`, `new_tag=journey`), then connection refused; the server process exited as `Aborted`. The probe is `tag rename across sources` near `tests/adversarial/attack.py:2608`. Host load reached about 42, and runner cleanup removed its temporary logs; no core or stack was available. Timing implicates the probe sequence but does not establish the cause. I filed [#525](https://git.kayg.org/kayg/calternal/issues/525) with the evidence and follow-up. The job's single adversarial round is complete; I did not rerun it.
Author
Owner

Completed

Branch: job/mcp-fast-492
Base before merge: cd3cea757 (origin/dev)
Final HEAD: 0457a2bf701ce577c026b0fc0a0d8c1a6fd88da8

This job adds a repeatable, credential-safe MCP profile harness and records the current release baseline. It does not change MCP server behavior: #484's registry core is still Open, and the issue requires MCP-layer code changes to wait for that merge.

Files added or updated:

  • bench/mcp-profile.py, bench/mcp-bootstrap.mjs, bench/mcp-prepare.sh, bench/mcp-seed-current.py
  • bench/mcp-requirements.txt, bench/mcp-scenarios-current.json
  • docs/perf/README.md, docs/perf/mcp-baseline.json
  • docs/perf/runs/2026-09-30-mcp.md, docs/perf/runs/2026-09-30-mcp-current.json

Measurements

The shared release binary (source 369ab6a2f9fc673e3564b94857fbecfeb04df404) exposes 8 tools and MCP protocol 2026-07-28. Its legacy initialize probe returns HTTP 200 with JSON-RPC -32022; server/discover is the supported discovery request. The 30-cold / 100-warm samples used 10 warmups and a restored fixture snapshot per server start, under /root/perf.lock on the perf VM.

Client-loopback p50/p95/p99 in ms:

  • tools/list: cold 22.178 / 26.825 / 29.305; warm 18.270 / 21.051 / 22.089.
  • Search Notes: cold 23.975 / 26.124 / 28.426; warm 19.894 / 23.130 / 25.193.
  • Create Task: cold 70.793 / 95.193 / 96.658; warm 370.743 / 666.881 / 725.149.
  • Create Log entry: cold 74.706 / 89.746 / 96.446; warm 105.001 / 221.447 / 311.932.

The tool array was 2,889 bytes / 654 o200k_base tokens. The measured grouped dispatcher was 2,969 bytes / 681 tokens (+27), so the current 8-tool set keeps individual schemas. tools/list had no compression, ETag, cache policy, or pagination cursor. The server sent no Server-Timing, so server-side latency targets are unmeasured. Calendar Events, Money operations, the full #484 registry, and HTTP/2 were unavailable in this release profile. calternal_create_log is not a Money transaction.

The 50-client burst ran 1,000 Search Notes calls at 127.42 requests/s; p50/p95/p99 were 388.402 / 441.895 / 464.072 ms. Estimated idle connection memory was 2,854,257 bytes per client; peak RSS was 461,324,288 bytes. The warm Task and Log write tails and RSS growth are recorded in the report.

Follow-ups filed: #515 (tools/list caching), #516 (App Password verification cost), and #517 (write latency and memory growth).

Adversarial finding

The one-time API-only real-server adversarial round reached POST /api/v1/tags/rename (travel → journey) after the rename preview. That probe received no response, later requests received connection refused, and the server process exited as Aborted. The timing points to the tag rename across sources probe, but does not prove that request caused the abort. Host load reached about 42; cleanup removed the server log, and no core/stack was available. I filed #525 with the evidence. The one allowed round is complete; I did not rerun it or change the tag expectation.

Gates

Commands used the required Cargo environment (CARGO_PROFILE_DEV_DEBUG=line-tables-only, CARGO_INCREMENTAL=0, CARGO_BUILD_JOBS=4, TMPDIR=$PWD/target/tmp). The required origin/dev merge is included at HEAD.

  • cargo fmt --check: exit 0, no output.
  • cargo clippy -p calternal-server --all-targets -- -D warnings (after restoring generated web assets):
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 31.93s
    
  • cargo test -p calternal-server:
    test result: ok. 85 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 26.27s
    
  • bun run check:
    Text sizes and UI shape values use shared role tokens.
    Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/mcp-fast-492/apps/web
    Getting Svelte diagnostics...
    
    svelte-check found 0 errors and 0 warnings
    
  • bun run test ended with:
    Error: Test timed out in 5000ms.
    
    The affected tests were src/lib/date-formatting.test.ts (shared date formatter reuse gate) and src/lib/styles/font-weight-token.test.ts (font weight token guard). Final summary:
     Test Files  2 failed | 135 passed (137)
          Tests  2 failed | 897 passed (899)
       Duration  137.98s (transform 43%, import 20%, tests 16%, environment 16%, setup 4%)
    
    The cause of these two timeouts is unverified. I did not change expectations or rerun the gate.
  • Final cleanup:
         Removed 12965 files, 5.5GiB total
    
    Removed generated apps/web/build and apps/web/.svelte-kit. git diff --check and the Python, Node, shell, and JSON syntax checks passed.

An initial clippy attempt, before restoring apps/web/build, failed because the server embeds that generated directory. bun run build succeeded; after building, clippy and the server tests passed. The generated web output was deleted after gates.

Decisions not covered by DESIGN

  • Use server/discover for the current no-initialize protocol and keep a legacy initialize request only as a compatibility probe.
  • When Server-Timing is absent, report loopback client latency separately and leave server-side SLOs unmeasured.
  • Keep the MCP baseline in docs/perf/mcp-baseline.json, separate from the shared docs/perf/baseline.json.
  • Keep individual schemas while the release has 8 tools because grouping costs 27 additional tokens; remeasure with the full #484 registry.
## Completed Branch: `job/mcp-fast-492` Base before merge: `cd3cea757` (`origin/dev`) Final HEAD: `0457a2bf701ce577c026b0fc0a0d8c1a6fd88da8` This job adds a repeatable, credential-safe MCP profile harness and records the current release baseline. It does not change MCP server behavior: #484's registry core is still Open, and the issue requires MCP-layer code changes to wait for that merge. Files added or updated: - `bench/mcp-profile.py`, `bench/mcp-bootstrap.mjs`, `bench/mcp-prepare.sh`, `bench/mcp-seed-current.py` - `bench/mcp-requirements.txt`, `bench/mcp-scenarios-current.json` - `docs/perf/README.md`, `docs/perf/mcp-baseline.json` - `docs/perf/runs/2026-09-30-mcp.md`, `docs/perf/runs/2026-09-30-mcp-current.json` ## Measurements The shared release binary (source `369ab6a2f9fc673e3564b94857fbecfeb04df404`) exposes 8 tools and MCP protocol `2026-07-28`. Its legacy `initialize` probe returns HTTP 200 with JSON-RPC `-32022`; `server/discover` is the supported discovery request. The 30-cold / 100-warm samples used 10 warmups and a restored fixture snapshot per server start, under `/root/perf.lock` on the perf VM. Client-loopback p50/p95/p99 in ms: - `tools/list`: cold `22.178 / 26.825 / 29.305`; warm `18.270 / 21.051 / 22.089`. - Search Notes: cold `23.975 / 26.124 / 28.426`; warm `19.894 / 23.130 / 25.193`. - Create Task: cold `70.793 / 95.193 / 96.658`; warm `370.743 / 666.881 / 725.149`. - Create Log entry: cold `74.706 / 89.746 / 96.446`; warm `105.001 / 221.447 / 311.932`. The tool array was 2,889 bytes / 654 `o200k_base` tokens. The measured grouped dispatcher was 2,969 bytes / 681 tokens (+27), so the current 8-tool set keeps individual schemas. `tools/list` had no compression, ETag, cache policy, or pagination cursor. The server sent no `Server-Timing`, so server-side latency targets are unmeasured. Calendar Events, Money operations, the full #484 registry, and HTTP/2 were unavailable in this release profile. `calternal_create_log` is not a Money transaction. The 50-client burst ran 1,000 Search Notes calls at 127.42 requests/s; p50/p95/p99 were `388.402 / 441.895 / 464.072` ms. Estimated idle connection memory was 2,854,257 bytes per client; peak RSS was 461,324,288 bytes. The warm Task and Log write tails and RSS growth are recorded in the report. Follow-ups filed: #515 (tools/list caching), #516 (App Password verification cost), and #517 (write latency and memory growth). ## Adversarial finding The one-time API-only real-server adversarial round reached `POST /api/v1/tags/rename` (`travel` → `journey`) after the rename preview. That probe received no response, later requests received connection refused, and the server process exited as `Aborted`. The timing points to the `tag rename across sources` probe, but does not prove that request caused the abort. Host load reached about 42; cleanup removed the server log, and no core/stack was available. I filed #525 with the evidence. The one allowed round is complete; I did not rerun it or change the tag expectation. ## Gates Commands used the required Cargo environment (`CARGO_PROFILE_DEV_DEBUG=line-tables-only`, `CARGO_INCREMENTAL=0`, `CARGO_BUILD_JOBS=4`, `TMPDIR=$PWD/target/tmp`). The required `origin/dev` merge is included at HEAD. - `cargo fmt --check`: exit 0, no output. - `cargo clippy -p calternal-server --all-targets -- -D warnings` (after restoring generated web assets): ``` Finished `dev` profile [unoptimized + debuginfo] target(s) in 31.93s ``` - `cargo test -p calternal-server`: ``` test result: ok. 85 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out; finished in 26.27s ``` - `bun run check`: ``` Text sizes and UI shape values use shared role tokens. Loading svelte-check in workspace: /home/kayg/Developer/calternal-wt/mcp-fast-492/apps/web Getting Svelte diagnostics... svelte-check found 0 errors and 0 warnings ``` - `bun run test` ended with: ``` Error: Test timed out in 5000ms. ``` The affected tests were `src/lib/date-formatting.test.ts` (shared date formatter reuse gate) and `src/lib/styles/font-weight-token.test.ts` (font weight token guard). Final summary: ``` Test Files 2 failed | 135 passed (137) Tests 2 failed | 897 passed (899) Duration 137.98s (transform 43%, import 20%, tests 16%, environment 16%, setup 4%) ``` The cause of these two timeouts is unverified. I did not change expectations or rerun the gate. - Final cleanup: ``` Removed 12965 files, 5.5GiB total ``` Removed generated `apps/web/build` and `apps/web/.svelte-kit`. `git diff --check` and the Python, Node, shell, and JSON syntax checks passed. An initial clippy attempt, before restoring `apps/web/build`, failed because the server embeds that generated directory. `bun run build` succeeded; after building, clippy and the server tests passed. The generated web output was deleted after gates. ## Decisions not covered by DESIGN - Use `server/discover` for the current no-initialize protocol and keep a legacy `initialize` request only as a compatibility probe. - When `Server-Timing` is absent, report loopback client latency separately and leave server-side SLOs unmeasured. - Keep the MCP baseline in `docs/perf/mcp-baseline.json`, separate from the shared `docs/perf/baseline.json`. - Keep individual schemas while the release has 8 tools because grouping costs 27 additional tokens; remeasure with the full #484 registry.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#492
No description provided.