Add superfast local BM25 chat search with Command-K #8

New issue

Open

opened 2026-08-21 08:34:37 +00:00 by kayg · 2 comments

kayg commented

2026-08-21 08:34:37 +00:00

Owner

Goal

Add near-instant local chat/message search, opened with ⌘K, backed by an incremental full-text index over the local transcript cache.

Direction

Spotlight-style command/search palette; keyboard-first, immediately focused, dismissible with Escape.
Search chat titles and message bodies locally. Results show chat, role, timestamp/context snippet, and highlighted terms.
Selecting a result opens the cached chat instantly and scrolls to the stable message identity.
Start with SQLite FTS5 bm25() if the deployed macOS SQLite supports it; benchmark before considering a custom BM25 implementation.
Incrementally index cache writes/deletes instead of rebuilding on every launch.
Keep the index private and local; clearing/disabling Cache clears/disables the search index too.

Performance bar

Palette visible and accepting input in one frame.
First result for a warm index within 50 ms on the current transcript corpus.
Query work stays off the main actor and is cancellable as the query changes.

Dependencies

Message-level navigation needs the stable server-backed identity tracked in the deep-link issue. Chat-level results can ship first.

## Goal Add near-instant local chat/message search, opened with **⌘K**, backed by an incremental full-text index over the local transcript cache. ## Direction - Spotlight-style command/search palette; keyboard-first, immediately focused, dismissible with Escape. - Search chat titles and message bodies locally. Results show chat, role, timestamp/context snippet, and highlighted terms. - Selecting a result opens the cached chat instantly and scrolls to the stable message identity. - Start with SQLite FTS5 `bm25()` if the deployed macOS SQLite supports it; benchmark before considering a custom BM25 implementation. - Incrementally index cache writes/deletes instead of rebuilding on every launch. - Keep the index private and local; clearing/disabling Cache clears/disables the search index too. ## Performance bar - Palette visible and accepting input in one frame. - First result for a warm index within 50 ms on the current transcript corpus. - Query work stays off the main actor and is cancellable as the query changes. ## Dependencies Message-level navigation needs the stable server-backed identity tracked in the deep-link issue. Chat-level results can ship first.

kayg referenced this issue

2026-08-21 08:36:17 +00:00

Index chats and messages in macOS Spotlight #11

kayg referenced this issue

2026-08-22 04:17:30 +00:00

Outstanding: Cross-conversation Cmd-K local search panel #21

kayg commented

2026-08-22 04:23:12 +00:00

Author

Owner

Design-session details to carry into implementation:

⌘⇧K opens the same panel prefilled with > for an action/command mode listing settings panes and app commands. Action labels match by substring; message text matches by token prefix.
Panel geometry: its top edge is at one third of the window height, horizontally centred, and it grows downward until its bottom edge is one third up from the bottom; beyond that it scrolls internally.
The backdrop dims everything beneath at maximum frost, regardless of the user's frost slider setting.
Result rows represent each matching message. The conversation title is secondary, with role and relative date also shown.
Proposed alternative to a per-conversation hit cap: take the best hit from each conversation first, then the second best from each, so one long chat cannot monopolise the top of the list. This may or may not be compatible with pure bm25 ordering; leave it as an open question for the issue author, not a decision.

Design-session details to carry into implementation: - ⌘⇧K opens the same panel prefilled with `> ` for an action/command mode listing settings panes and app commands. Action labels match by substring; message text matches by token prefix. - Panel geometry: its top edge is at one third of the window height, horizontally centred, and it grows downward until its bottom edge is one third up from the bottom; beyond that it scrolls internally. - The backdrop dims everything beneath at maximum frost, regardless of the user's frost slider setting. - Result rows represent each matching message. The conversation title is secondary, with role and relative date also shown. - Proposed alternative to a per-conversation hit cap: take the best hit from each conversation first, then the second best from each, so one long chat cannot monopolise the top of the list. This may or may not be compatible with pure bm25 ordering; leave it as an open question for the issue author, not a decision.

kayg commented

2026-08-22 04:23:37 +00:00

Author

Owner

Clarification from the issue author: #8 is authoritative over a competing design produced in a separate design session. Implement FTS5 with bm25() and an incremental index as #8 states, ship chat-level results first, and treat the 50 ms warm-result bar as the target. A proposed 120 ms input debounce was rejected as incompatible with that bar.

Diversity ranking remains an open question layered on top of bm25, not a decision: measure strict bm25 first, and only add a diversity pass if one long conversation visibly monopolises the top of the results.

Clarification from the issue author: #8 is authoritative over a competing design produced in a separate design session. Implement FTS5 with `bm25()` and an incremental index as #8 states, ship chat-level results first, and treat the 50 ms warm-result bar as the target. A proposed 120 ms input debounce was rejected as incompatible with that bar. Diversity ranking remains an open question layered on top of bm25, not a decision: measure strict bm25 first, and only add a diversity pass if one long conversation visibly monopolises the top of the results.

kayg referenced this issue

2026-08-22 04:23:56 +00:00

Outstanding: Cross-conversation Cmd-K local search panel #21

kayg referenced this issue

2026-08-22 05:07:02 +00:00

Add fast, interruptible native motion throughout the app #9

kayg referenced this issue

2026-08-22 09:47:29 +00:00

Add accounts and profiles with active gateway switching #32

kayg referenced this issue

2026-08-22 12:55:20 +00:00

Decide whether local search indexes compaction-preserved rows #34