MAIL (later): index attachment text in Search (PDF, Office, text) with cost and deletion rules #293

Open
opened 2026-09-28 06:31:58 +00:00 by kayg · 0 comments
Owner

Owner (2026-09-28, mail grill Q19): the first Mail release indexes sender, subject, recipients and full body text. Attachment text comes later. File it so it is not forgotten.

Scope when picked up (after the Mail plugin #257 ships):

  • Extract text from mail attachments (PDF, DOCX/ODT/XLSX, plain text, HTML, calendar .ics) and index it in the same Tantivy index as the message, so the global palette finds a message by words inside its attachments.
  • Attachments are fetched on demand today. Decide whether indexing forces a fetch (bandwidth; Gmail's ~2.5 GB/day IMAP cap) or indexes only attachments already opened or saved. Measure the storage and CPU cost on a large mailbox first.
  • Deletion: when a message or account goes away, its attachment text leaves the index at once.
  • Reuse: Files may already extract text for its own search. Share one extractor (reuse gate), and never run a second pipeline.
  • Security: extractors run on hostile input. Enforce size and time limits, run out of process or in a sandbox, and fuzz them.
  • Search speed must not regress (#258 targets; the owner wants Spotlight-level speed).
Owner (2026-09-28, mail grill Q19): the first Mail release indexes sender, subject, recipients and full body text. **Attachment text comes later. File it so it is not forgotten.** Scope when picked up (after the Mail plugin #257 ships): - Extract text from mail attachments (PDF, DOCX/ODT/XLSX, plain text, HTML, calendar .ics) and index it in the same Tantivy index as the message, so the global palette finds a message by words inside its attachments. - Attachments are fetched on demand today. Decide whether indexing forces a fetch (bandwidth; Gmail's ~2.5 GB/day IMAP cap) or indexes only attachments already opened or saved. Measure the storage and CPU cost on a large mailbox first. - Deletion: when a message or account goes away, its attachment text leaves the index at once. - Reuse: Files may already extract text for its own search. Share one extractor (reuse gate), and never run a second pipeline. - Security: extractors run on hostile input. Enforce size and time limits, run out of process or in a sandbox, and fuzz them. - Search speed must not regress (#258 targets; the owner wants Spotlight-level speed).
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/calternal#293
No description provided.