Docs: accept ADR 0053 (the language engine reads dates with its own rules) #650

Closed

kayg wants to merge 0 commits from docs/adr-0053-accept into dev

kayg commented

2026-10-20 20:44:47 +00:00

Owner

kayg (2026-10-21) accepted 0053, renamed: it is the date/time part of the one language engine, not a separate parser. Docs only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk

kayg (2026-10-21) accepted 0053, renamed: it is the date/time part of the one language engine, not a separate parser. Docs only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk

kayg added 1 commit

2026-10-20 20:44:48 +00:00

Accept ADR 0053: the language engine reads dates with its own rules

CI / web (pull_request) Failing after 7s

Details

CI / server (pull_request) Successful in 24s

Details

CI / rust (pull_request) Failing after 1m30s

Details

4e97e468ed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U5oTUycqAh7DT72mdSUztk

kayg commented

2026-10-21 11:32:26 +00:00

Author

Owner

This PR's commits are on dev (landed). Forgejo kept it open and its merge endpoint answers 405, so it is closed by hand.

kayg closed this pull request

2026-10-21 11:32:27 +00:00

kayg commented

2026-11-02 12:42:34 +00:00

Author

Owner

ADR 0003 evidence migration: docs/adr/0053-the-language-engine-reads-dates-with-its-own-rules.md

Source commit: 4e97e468eda92c8b5792f55a4a30bdc94015316d; snapshot from dev 55f35fcd. Full original source preserved below before repository removal/stripping. Design, code and test inputs are retained where applicable.

# 0053. The language engine reads dates with its own rules

Status: accepted, 2026-10-21. This is the date and time part of the one language engine (the Composer's, used by tasks, contacts and money), not a separate parser.

Updated 2026-10-19

## Context

People put dates inside task phrases. The Composer must understand the phrase as a whole, infer whether a date-shaped word is scheduling intent, and leave title text alone when the word is only part of a name. The browser parser runs on each keystroke, so it must stay fast and predictable.

The 823-case English corpus has 160 title traps. Chrono-node was the best tested library, but it got only 61/114 authored positive cases exactly right, parsed 41/160 traps, and returned no exact recurrence rule for any of 15 recurrence cases. Its source-suite score is not independent: chrono-node supplied those expected values. The measurements do not support using a library as our task parser. Browser/WASM measurements are recorded in `/home/kayg/Developer/calternal-research/wasm-bench/RESULTS.md`.

## Decision

The browser client parser is TypeScript. Do not call Rust through WASM in the browser for this per-keystroke work. The server may use Rust. When implementations differ, they must agree on the shared 823-case golden test vector corpus; the vectors define behavior without requiring one implementation everywhere. English is the only language in v1. Chrono-node 2.10.3 is a test oracle and comparison tool only; it is not a product dependency.

The Composer reads intent from the whole phrase rather than requiring a date prefix or a kind-cycling step. It previews a recognized date before applying it. When a phrase has more than one plausible interpretation, show the preview and let the person choose; do not silently commit one. The settled bare-hour rule in N2 remains a deterministic guess and never prompts.

### Grammar

Use a tokenizer and a small, explicit set of rules. Tokenize once, keep the original text and UTF-16 offsets, then recognize bounded sequences for dates, weekdays, relative dates and durations, clocks, ranges, deadlines, and recurrence. Use deterministic precedence: a complete date or range with its time first, then relative phrases and durations, then recurrence and standalone time forms. An incomplete prefix is simply not a match yet. Invalid input never panics.

This fits the corpus better than a general parser crate. Most useful cases are short, repeated forms; the hard part is deciding whether a token is a date at all. Explicit rules let us require scheduling context around ambiguous words and numbers, avoid backtracking on every growing input prefix, and add a focused regression when a rule changes. There is no model call or network work.

### Product rules

- Tasks have `When` and `Deadline`. An unmarked date fills `When`; `by`, `due`, or `deadline` cues fill `Deadline`.
- A bare weekday means its next occurrence, including today. `This Friday` means Friday in the current calendar week, or next week's Friday if this week's Friday has passed. `Next Friday` means Friday of the following week.
- A bare hour such as `5` is guessed from the current local time and never prompts. In a recognized task-action slot, choose the next occurrence of that hour: at 09:00, `Call 5` means 17:00 today; after 17:00, it means 05:00 tomorrow. A number joined to a noun or identifier is not a time. An explicit `at 5` uses the same rule.
- All-number dates follow the app's date locale, which defaults to the system locale. A yearless month and day means its next occurrence on or after the reference date.
- A wall time skipped by a clock change moves forward by the full gap (`02:30` becomes `03:30`). In a repeated hour, use the first occurrence. Elapsed durations such as `in 1 hour` remain elapsed time.
- One-off times are stored as ISO 8601 date-times with an offset. Date-only values remain dates. Ranges have inclusive date endpoints.
- Recurrence is parsed into an iCal `RRULE`; `DTSTART` is a separate value. V1 covers daily and weekly rules, intervals, weekdays, ordinal weekdays, month days, year dates, and clock times. Tasks follow iCal recurrence behavior.

### Trap policy

A date word or digit alone is not enough evidence. Require a complete rule and a scheduling-shaped boundary. Month names need a day, year, or a temporal cue such as `in May`; do not take the `May` in `Call May` as a date. A bare hour is accepted alone, after `at`, or at the end of a recognized task action such as `Call 5`; a number followed by a noun, or embedded in a version or identifier, is not a date. A weekday that continues into a title such as `Friday deploy notes` is not a date. Accept only the corpus's common short forms (`tmr`, `tmrw`, `tomorow`, `nxt wk`, `fri`, `mon`, `tues`, `thur`, `tonite`, and `yday`); there is no fuzzy matching.

The v1 grammar includes relative days and durations, weekdays, month and numeric dates, times, ranges, explicit UTC offsets and IANA zone names, deadline cues, and the recurrence forms above. Later work may add other languages, business or holiday calendars, vague phrases such as “sometime soon,” and ambiguous zone abbreviations such as `ET` and `PST`. `UTC`, `GMT`, numeric offsets, and full IANA names are accepted.

Each implementation may use the timezone facilities available to its runtime. The shared golden vectors include skipped and repeated local times so both implementations must return the same product result for those cases. Keep the vectors pinned and review any change that alters their expected values.

### Result shape

Return every accepted match with its exact source span and semantic result:

- `span`: UTF-16 start and end offsets plus the matched text, including a `by`, `due`, or `deadline` cue when present. The editor can highlight the text or turn it into a chip.
- `kind`: `When`, `Deadline`, or `Recurrence`.
- `value`: an ISO date, an ISO 8601 date-time with offset, or an inclusive range. Recurrences return the `RRULE` and a separate optional `DTSTART`; keep the IANA zone with recurring local times.

The caller supplies the reference date-time, timezone, and date locale. Parsing does not rewrite the title. Unknown or unsafe text returns no match.

## Why

The TypeScript parser keeps the common browser path small and avoids a WASM startup and string-boundary cost on each growing input. The benchmark measured TypeScript at 0.9–3.1 µs per keystroke and WASM at 0.6–2.2 µs, with less than 1.3 µs saved and about 29% of WASM time spent crossing the string boundary. WASM cold start measured 240 ms in Chromium, 8.7 s in WebKit and 1.4 s in iPhone simulator Safari, with a 273 KB brotli module. Shared vectors protect agreement with a Rust server implementation. The title traps make false matches a product failure, not an acceptable trade-off for finding more date-like words.

Chrono-node remains useful for a side-by-side diff and for coverage of common date syntax. Its task extraction errors and missing recurrence support make it a poor runtime dependency. The recurrence grammar is ours; the result uses the iCal format the task model follows.

## Rejected

- Chrono-node or another tested natural-language parser as the product grammar: none is accurate enough on authored task text, and the broadest extractor matched 41 of 160 title traps.
- Rust-to-WASM for the browser parser: the measured per-keystroke gain is under 1.3 µs, while the cold start and bundle cost are material.
- A combinator crate or a general free-text grammar: most v1 inputs are short fixed patterns, and this adds work on the path that runs for every growing prefix without solving the intent problem.
- Guessing from any month, weekday, or number token: that recreates false positives such as names, release titles, counts, and model numbers.

## Test plan

- Use all 823 corpus rows as shared golden test vectors for the TypeScript browser parser and any server parser. Keep the 545 chrono-node-derived rows labeled as compatibility coverage, not independent truth. Product expectations override oracle output when they conflict with these rules. Mark every unsupported form explicitly; do not silently drop it from the report.
- The 255 authored cases and 23 documented examples are the independent accuracy set. Require zero matches for all 160 title traps. Require no panics, invalid dates, or out-of-range spans. Compare span text, UTF-16 offsets, kind, and value.
- Add boundary fixtures for N1–N4: weekday phrases before and after Friday; bare hours around 05:00 and 17:00; `3/4` with US and day-first locales; and both Berlin clock-change transitions. Reclassify the corpus's four ambiguous examples using the settled number and locale rules.
- Emit a pinned chrono-node 2.10.3 diff report by category. Keep its oracle-derived source results separate from authored accuracy; do not fail solely because the product correctly rejects an oracle match.
- Require exact `RRULE` output for all 15 recurrence cases, with `DTSTART` separate when the phrase supplies it. Check that a recurrence is not reduced to a one-off date.
- Parse every growing prefix in the corpus. On the oldest supported phone, budget at most 1.0 ms p95 and 3.0 ms p99 per parse for inputs up to 512 UTF-16 units. The 4× benchmark estimate is not a phone pass.
- Keep the browser implementation on the TypeScript path. If a future measured proposal adds WASM for a coarse batch operation, benchmark the full boundary and startup cost against TypeScript before changing this decision.

## Failure cases

These show where v1 will miss or make a deterministic choice:

| Text | Result |
| --- | --- |
| `Call May` | No match; May can be a person's name. |
| `5 apples` | No match; 5 is a quantity. |
| `March on` | No match; March is a verb. |
| `iPhone 17` | No match; 17 is part of a model name. |
| `Room 101` | No match; 101 is an identifier. |
| `Sprint 5` | No match; 5 labels a sprint. |
| `Model 5pm` | No match; 5pm is part of a model name. |
| `Friday deploy notes` | No match; Friday starts a title, not a schedule phrase. |
| `Call 5` | Guesses the next 05:00 or 17:00 occurrence. `Meeting 5` stays unmatched because 5 follows a title noun. |
| `10-3 9` | Locale decides whether the date is October 3 or March 10; the same text can mean the wrong date if the device locale differs from the writer's intent. |
| `PST tomorrow at 9` | No match; `PST` is not a unique IANA timezone. |
| `the first business day after a holiday` | No match; business and holiday calendars are outside v1. |
| `every weekday except holidays` | No match; do not silently discard the exception. |
| `every month on the 31st` | Emits the iCal day-31 rule; shorter months have no occurrence, rather than moving to month-end. |
| `tomorrow at 02:30` in a spring gap | Moves the time forward by the gap, for example to 03:30. |
| `02:30` in a repeated fall hour | Chooses the first occurrence; there is no v1 phrase for choosing the second. |

## Open questions

- Which minimum-supported phone is the latency reference? Recommended: the oldest phone Calternal supports.
- What is the default `DTSTART` when a recurrence has no written start date? Recommended: the first matching local occurrence on or after the task's reference date.
- Should v1 accept common zone abbreviations such as `ET`, `PST`, and `CET`? Recommended: no, until each has a tested mapping and daylight rule.
- If a line contains two unconnected matches for the same field, should the editor ask the user to choose or leave that field unset? Recommended: show both spans in the preview and do not silently choose.
- If a date cue and recurrence cue appear in the same phrase, which chip should the editor offer first? Recommended: show both spans and let the task model keep the date and rule separately.

## Changelog

- 2026-10-19: Set the browser parser to TypeScript, kept server Rust optional, and aligned intent, previews, N1–N4 and shared golden vectors with kayg's decisions.

ADR 0003 evidence migration: `docs/adr/0053-the-language-engine-reads-dates-with-its-own-rules.md` Source commit: [4e97e468eda92c8b5792f55a4a30bdc94015316d](https://git.kayg.org/kayg/calternal/commit/4e97e468eda92c8b5792f55a4a30bdc94015316d); snapshot from dev `55f35fcd`. Full original source preserved below before repository removal/stripping. Design, code and test inputs are retained where applicable. ```text # 0053. The language engine reads dates with its own rules Status: accepted, 2026-10-21. This is the date and time part of the one language engine (the Composer's, used by tasks, contacts and money), not a separate parser. Updated 2026-10-19 ## Context People put dates inside task phrases. The Composer must understand the phrase as a whole, infer whether a date-shaped word is scheduling intent, and leave title text alone when the word is only part of a name. The browser parser runs on each keystroke, so it must stay fast and predictable. The 823-case English corpus has 160 title traps. Chrono-node was the best tested library, but it got only 61/114 authored positive cases exactly right, parsed 41/160 traps, and returned no exact recurrence rule for any of 15 recurrence cases. Its source-suite score is not independent: chrono-node supplied those expected values. The measurements do not support using a library as our task parser. Browser/WASM measurements are recorded in `/home/kayg/Developer/calternal-research/wasm-bench/RESULTS.md`. ## Decision The browser client parser is TypeScript. Do not call Rust through WASM in the browser for this per-keystroke work. The server may use Rust. When implementations differ, they must agree on the shared 823-case golden test vector corpus; the vectors define behavior without requiring one implementation everywhere. English is the only language in v1. Chrono-node 2.10.3 is a test oracle and comparison tool only; it is not a product dependency. The Composer reads intent from the whole phrase rather than requiring a date prefix or a kind-cycling step. It previews a recognized date before applying it. When a phrase has more than one plausible interpretation, show the preview and let the person choose; do not silently commit one. The settled bare-hour rule in N2 remains a deterministic guess and never prompts. ### Grammar Use a tokenizer and a small, explicit set of rules. Tokenize once, keep the original text and UTF-16 offsets, then recognize bounded sequences for dates, weekdays, relative dates and durations, clocks, ranges, deadlines, and recurrence. Use deterministic precedence: a complete date or range with its time first, then relative phrases and durations, then recurrence and standalone time forms. An incomplete prefix is simply not a match yet. Invalid input never panics. This fits the corpus better than a general parser crate. Most useful cases are short, repeated forms; the hard part is deciding whether a token is a date at all. Explicit rules let us require scheduling context around ambiguous words and numbers, avoid backtracking on every growing input prefix, and add a focused regression when a rule changes. There is no model call or network work. ### Product rules - Tasks have `When` and `Deadline`. An unmarked date fills `When`; `by`, `due`, or `deadline` cues fill `Deadline`. - A bare weekday means its next occurrence, including today. `This Friday` means Friday in the current calendar week, or next week's Friday if this week's Friday has passed. `Next Friday` means Friday of the following week. - A bare hour such as `5` is guessed from the current local time and never prompts. In a recognized task-action slot, choose the next occurrence of that hour: at 09:00, `Call 5` means 17:00 today; after 17:00, it means 05:00 tomorrow. A number joined to a noun or identifier is not a time. An explicit `at 5` uses the same rule. - All-number dates follow the app's date locale, which defaults to the system locale. A yearless month and day means its next occurrence on or after the reference date. - A wall time skipped by a clock change moves forward by the full gap (`02:30` becomes `03:30`). In a repeated hour, use the first occurrence. Elapsed durations such as `in 1 hour` remain elapsed time. - One-off times are stored as ISO 8601 date-times with an offset. Date-only values remain dates. Ranges have inclusive date endpoints. - Recurrence is parsed into an iCal `RRULE`; `DTSTART` is a separate value. V1 covers daily and weekly rules, intervals, weekdays, ordinal weekdays, month days, year dates, and clock times. Tasks follow iCal recurrence behavior. ### Trap policy A date word or digit alone is not enough evidence. Require a complete rule and a scheduling-shaped boundary. Month names need a day, year, or a temporal cue such as `in May`; do not take the `May` in `Call May` as a date. A bare hour is accepted alone, after `at`, or at the end of a recognized task action such as `Call 5`; a number followed by a noun, or embedded in a version or identifier, is not a date. A weekday that continues into a title such as `Friday deploy notes` is not a date. Accept only the corpus's common short forms (`tmr`, `tmrw`, `tomorow`, `nxt wk`, `fri`, `mon`, `tues`, `thur`, `tonite`, and `yday`); there is no fuzzy matching. The v1 grammar includes relative days and durations, weekdays, month and numeric dates, times, ranges, explicit UTC offsets and IANA zone names, deadline cues, and the recurrence forms above. Later work may add other languages, business or holiday calendars, vague phrases such as “sometime soon,” and ambiguous zone abbreviations such as `ET` and `PST`. `UTC`, `GMT`, numeric offsets, and full IANA names are accepted. Each implementation may use the timezone facilities available to its runtime. The shared golden vectors include skipped and repeated local times so both implementations must return the same product result for those cases. Keep the vectors pinned and review any change that alters their expected values. ### Result shape Return every accepted match with its exact source span and semantic result: - `span`: UTF-16 start and end offsets plus the matched text, including a `by`, `due`, or `deadline` cue when present. The editor can highlight the text or turn it into a chip. - `kind`: `When`, `Deadline`, or `Recurrence`. - `value`: an ISO date, an ISO 8601 date-time with offset, or an inclusive range. Recurrences return the `RRULE` and a separate optional `DTSTART`; keep the IANA zone with recurring local times. The caller supplies the reference date-time, timezone, and date locale. Parsing does not rewrite the title. Unknown or unsafe text returns no match. ## Why The TypeScript parser keeps the common browser path small and avoids a WASM startup and string-boundary cost on each growing input. The benchmark measured TypeScript at 0.9–3.1 µs per keystroke and WASM at 0.6–2.2 µs, with less than 1.3 µs saved and about 29% of WASM time spent crossing the string boundary. WASM cold start measured 240 ms in Chromium, 8.7 s in WebKit and 1.4 s in iPhone simulator Safari, with a 273 KB brotli module. Shared vectors protect agreement with a Rust server implementation. The title traps make false matches a product failure, not an acceptable trade-off for finding more date-like words. Chrono-node remains useful for a side-by-side diff and for coverage of common date syntax. Its task extraction errors and missing recurrence support make it a poor runtime dependency. The recurrence grammar is ours; the result uses the iCal format the task model follows. ## Rejected - Chrono-node or another tested natural-language parser as the product grammar: none is accurate enough on authored task text, and the broadest extractor matched 41 of 160 title traps. - Rust-to-WASM for the browser parser: the measured per-keystroke gain is under 1.3 µs, while the cold start and bundle cost are material. - A combinator crate or a general free-text grammar: most v1 inputs are short fixed patterns, and this adds work on the path that runs for every growing prefix without solving the intent problem. - Guessing from any month, weekday, or number token: that recreates false positives such as names, release titles, counts, and model numbers. ## Test plan - Use all 823 corpus rows as shared golden test vectors for the TypeScript browser parser and any server parser. Keep the 545 chrono-node-derived rows labeled as compatibility coverage, not independent truth. Product expectations override oracle output when they conflict with these rules. Mark every unsupported form explicitly; do not silently drop it from the report. - The 255 authored cases and 23 documented examples are the independent accuracy set. Require zero matches for all 160 title traps. Require no panics, invalid dates, or out-of-range spans. Compare span text, UTF-16 offsets, kind, and value. - Add boundary fixtures for N1–N4: weekday phrases before and after Friday; bare hours around 05:00 and 17:00; `3/4` with US and day-first locales; and both Berlin clock-change transitions. Reclassify the corpus's four ambiguous examples using the settled number and locale rules. - Emit a pinned chrono-node 2.10.3 diff report by category. Keep its oracle-derived source results separate from authored accuracy; do not fail solely because the product correctly rejects an oracle match. - Require exact `RRULE` output for all 15 recurrence cases, with `DTSTART` separate when the phrase supplies it. Check that a recurrence is not reduced to a one-off date. - Parse every growing prefix in the corpus. On the oldest supported phone, budget at most 1.0 ms p95 and 3.0 ms p99 per parse for inputs up to 512 UTF-16 units. The 4× benchmark estimate is not a phone pass. - Keep the browser implementation on the TypeScript path. If a future measured proposal adds WASM for a coarse batch operation, benchmark the full boundary and startup cost against TypeScript before changing this decision. ## Failure cases These show where v1 will miss or make a deterministic choice: | Text | Result | | --- | --- | | `Call May` | No match; May can be a person's name. | | `5 apples` | No match; 5 is a quantity. | | `March on` | No match; March is a verb. | | `iPhone 17` | No match; 17 is part of a model name. | | `Room 101` | No match; 101 is an identifier. | | `Sprint 5` | No match; 5 labels a sprint. | | `Model 5pm` | No match; 5pm is part of a model name. | | `Friday deploy notes` | No match; Friday starts a title, not a schedule phrase. | | `Call 5` | Guesses the next 05:00 or 17:00 occurrence. `Meeting 5` stays unmatched because 5 follows a title noun. | | `10-3 9` | Locale decides whether the date is October 3 or March 10; the same text can mean the wrong date if the device locale differs from the writer's intent. | | `PST tomorrow at 9` | No match; `PST` is not a unique IANA timezone. | | `the first business day after a holiday` | No match; business and holiday calendars are outside v1. | | `every weekday except holidays` | No match; do not silently discard the exception. | | `every month on the 31st` | Emits the iCal day-31 rule; shorter months have no occurrence, rather than moving to month-end. | | `tomorrow at 02:30` in a spring gap | Moves the time forward by the gap, for example to 03:30. | | `02:30` in a repeated fall hour | Chooses the first occurrence; there is no v1 phrase for choosing the second. | ## Open questions - Which minimum-supported phone is the latency reference? Recommended: the oldest phone Calternal supports. - What is the default `DTSTART` when a recurrence has no written start date? Recommended: the first matching local occurrence on or after the task's reference date. - Should v1 accept common zone abbreviations such as `ET`, `PST`, and `CET`? Recommended: no, until each has a tested mapping and daylight rule. - If a line contains two unconnected matches for the same field, should the editor ask the user to choose or leave that field unset? Recommended: show both spans in the preview and do not silently choose. - If a date cue and recurrence cue appear in the same phrase, which chip should the editor offer first? Recommended: show both spans and let the task model keep the date and rule separately. ## Changelog - 2026-10-19: Set the browser parser to TypeScript, kept server Rust optional, and aligned intent, previews, N1–N4 and shared golden vectors with kayg's decisions. ```

kayg referenced this pull request

2026-11-02 12:47:22 +00:00

chore: move evidence out of the repo (ADR 0003) #858