VT: lossless state export/import in our libghostty-vt fork (D110)

kayg commented

2026-10-06 18:25:39 +00:00

Owner

DECISIONS 110. Build an engine-level terminal state export/import for libghostty-vt in our own fork. Prove it with the vt-ghostty differential suite (20k nightly corpus plus the reviewer's real-program captures: less, vim, htop, tmux), aiming for zero rejections and zero divergences. Only then propose it upstream as a PR, after the owner sees it.

Inputs: the "engine-level state export/import" section that fix-vt2 writes in docs/S1-VT-GHOSTTY.md, and the restoration-gap table there.

Measure:

export and import time p99;
blob size per 80x24 and 200x60 session, with and without scrollback;
the version and compatibility story, since a blob must survive an engine upgrade or be rejected cleanly.

Blocked by: fix-vt2 (same worktree).

DECISIONS 110. Build an engine-level terminal state export/import for libghostty-vt in **our own fork**. Prove it with the vt-ghostty differential suite (20k nightly corpus plus the reviewer's real-program captures: less, vim, htop, tmux), aiming for zero rejections and zero divergences. Only then propose it upstream as a PR, after the owner sees it. Inputs: the "engine-level state export/import" section that fix-vt2 writes in docs/S1-VT-GHOSTTY.md, and the restoration-gap table there. Measure: - export and import time p99; - blob size per 80x24 and 200x60 session, with and without scrollback; - the version and compatibility story, since a blob must survive an engine upgrade or be rejected cleanly. **Blocked by:** fix-vt2 (same worktree).

kayg referenced this issue

2026-10-06 18:25:45 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue

2026-10-06 21:31:01 +00:00

Status board: work in flight (kept current) #24

kayg commented

2026-10-06 21:31:06 +00:00

Author

Owner

Starting vt-export at base ec38f9521d. Read CLAUDE.md, DECISIONS.md, architecture, S1-VT-GHOSTTY and all #37/#26/#22 comments. D110 supplied by owner; checked-in decisions end at 86 and docs/PROCESS.md is absent. Proposal before implementation: versioned transactional native state codec; additive capability and bounded chunks for complete history exceeding frozen 1 MiB frames, painter retained for peers without native state support. Browser and frozen VT Snapshot layouts stay unchanged. No upstream action, push or deploy.

Starting vt-export at base ec38f9521d588e0d110e3bc31089a8f211dc923f. Read CLAUDE.md, DECISIONS.md, architecture, S1-VT-GHOSTTY and all #37/#26/#22 comments. D110 supplied by owner; checked-in decisions end at 86 and docs/PROCESS.md is absent. Proposal before implementation: versioned transactional native state codec; additive capability and bounded chunks for complete history exceeding frozen 1 MiB frames, painter retained for peers without native state support. Browser and frozen VT Snapshot layouts stay unchanged. No upstream action, push or deploy.

kayg commented

2026-10-06 21:41:27 +00:00

Author

Owner

Inventory finding: Tabstops.resize retains custom stops outside the current width (src/terminal/Tabstops.zig:136-164 in pinned a887df42). Exporting visible-width stops alone would diverge after shrink/grow. Codec therefore includes retained preallocated and dynamic stops. Pages are reconstructed from validated logical records, preserving source capacities and populated page boundaries; native pointers and allocator metadata are excluded. Initial build/gate logs are under target/vt-export; no completed step claimed yet.

kayg commented

2026-10-06 21:47:02 +00:00

Author

Owner

Additive transport detail before codec implementation: capability bit 2 NATIVE_STATE; 0x1C NativeStateRequest (empty), 0x1D NativeStateChunk (u64 captured seq, u32 total envelope bytes, u32 contiguous offset, nonempty bytes). Total limit 64 MiB; each frame remains <=1 MiB. Native checkpoint requests freeze PTY reads through chunk delivery. Existing SnapshotRequest/0x15/0x1B remain VT. An explicit infraternal-agent checkpoint connection selects this capability and validates/imports the complete envelope; ordinary browser relays keep their existing path. Invalid/incomplete transfers are never published.

kayg commented

2026-10-06 21:58:29 +00:00

Author

Owner

Preliminary real-engine evidence (uncommitted draft): native expanded RIS smoke passes 1000 restored / 0 rejected / 0 divergent, seed 22601006, independent continuation and shrink/grow branches. target/vt-export/native-tests-3.log. Full-history oracle includes all modes/defaults, retained tabs, colors, title/pwd and terminal flags independently of the codec. Original smoke and targeted tests are still running; 20k proof and gates are not yet claimed.

kayg referenced this issue

2026-10-06 21:59:48 +00:00

S1 vt: second review: FFI soundness, carry observer, build and licences #26

kayg commented

2026-10-06 22:19:37 +00:00

Author

Owner

Draft evidence (uncommitted): target/vt-export/native-tests-3.log: native original and expanded RIS smoke each 1,000 restored, 0 rejected, 0 divergent; eight real/review-2 captures plus continuation/history-pruning/hidden-tab/parser-boundary/atomic-rejection checks passed (10 passed, 2 ignored). Preliminary debug timing, 10 samples on shared i9-9900K host, target/vt-export/{before,after}-timings-debug.log: sizing reference maps to actual page capacities reduced full-history import p50 80x24 ~77.7 to 28.0 ms and 200x60 ~68.9 to 24.7 ms. Full corpus/release/scale gates remain outstanding. Inventory audit found enabled Kitty graphics and glyph glossary; extending codec/oracle so successful export cannot silently lose these fields.

kayg commented

2026-10-06 22:24:44 +00:00

Author

Owner

Graphics audit finding: pinned src/terminal/Screen.zig:396-399 resets kitty_images to ImageStorage defaults (graphics_storage.zig total_limit=320,000,000), whereas fresh lib Screen.Options sets 10,000,000. Draft importer rejected RIS states by incorrectly treating the fresh limit as the only reachable value: target/vt-export/native-release.log has 4 failing corpus/capture tests, while 10 other checks passed. Add explicit RIS regression and preserve the existing reset-reachable limit; decoded checkpoint allocations remain capped at 64 MiB and file/shared-memory media remain forbidden. This job does not alter upstream graphics reset behavior.

kayg commented

2026-10-06 22:37:12 +00:00

Author

Owner

Draft release proof: target/vt-export/native-release-2.log: 14 passed, 0 failed, 5 ignored; original and expanded RIS smoke each 1,000 restored / 0 rejected / 0 divergent, eight captures, graphics/glyph/chunked-loading checks included. target/vt-export/review2-prefixes.log: vim 1,116 + htop 1,661 + less 1,407 + tmux 843 = 5,027 prefixes, each checked immediately and after remaining capture plus recorded continuation; 0 rejected / 0 divergent. Real socket test target/vt-export/checkpoint-integration-3.log: 1 passed, interrupted multi-frame transfer then authenticated infraternal-agent checkpoint import and sequenced input/raw-offset resume. Four 20k native corpus contexts and parser fuzz are running. Atomic commit gates are being isolated from subsequent draft files.

kayg referenced this issue

2026-10-06 22:39:00 +00:00

S1 vt: second review: FFI soundness, carry observer, build and licences #26

kayg commented

2026-10-06 22:43:50 +00:00

Author

Owner

Native release RIS 20k results: original 20,000 restored / 0 rejected / 0 divergent (464.64 s), expanded 20,000 / 0 / 0 (471.12 s), target/vt-export/{original,expanded}-20k-RIS.log. Fresh-C contexts are running. Scope for the absent server terminal relay: add a bounded server-owned import/sequence-resume API and exercise it in the real socket test with the infraternal-agent consumer; keep browser VT and network wire types unchanged. This is local checkpoint recovery, not restoration of a dead PTY. Upgrade consumers use the existing painter on Unsupported; malformed state fails closed.

kayg commented

2026-10-06 22:47:36 +00:00

Author

Owner

Run 2 starting on vt-export: base ec38f9521d, saved draft 79bfad9. Read #37 and parent #26/#22 history. Splitting draft into independently gated commits. Fresh-C logs also show original and expanded 20,000 restored / 0 rejected / 0 divergent each. Found unreported fuzz failure in target/vt-export/fuzz-import.log; will reproduce with a regression before fixing. docs/PROCESS.md remains absent; D110 comes from owner instructions.

Run 2 starting on vt-export: base ec38f9521d588e0d110e3bc31089a8f211dc923f, saved draft 79bfad9. Read #37 and parent #26/#22 history. Splitting draft into independently gated commits. Fresh-C logs also show original and expanded 20,000 restored / 0 rejected / 0 divergent each. Found unreported fuzz failure in target/vt-export/fuzz-import.log; will reproduce with a regression before fixing. docs/PROCESS.md remains absent; D110 comes from owner instructions.

kayg commented

2026-10-06 22:55:06 +00:00

Author

Owner

vt-export parked for tonight (orchestrator). Runs 1 and 2 were both stopped by the AI tooling's content filter: run 2 while it was testing malformed input to import_state. All work is saved as WIP commit on branch vt-export (on top of ec38f95), to be split into atomic commits when the work resumes. It blocks no deploy. Morning: decide whether to continue with a narrower prompt (no fuzzing in the job; fuzz targets run by the orchestrator) or another approach.

vt-export parked for tonight (orchestrator). Runs 1 and 2 were both stopped by the AI tooling's content filter: run 2 while it was testing malformed input to import_state. All work is saved as WIP commit on branch vt-export (on top of ec38f95), to be split into atomic commits when the work resumes. It blocks no deploy. Morning: decide whether to continue with a narrower prompt (no fuzzing in the job; fuzz targets run by the orchestrator) or another approach.

kayg referenced this issue

2026-10-06 23:55:59 +00:00

S1 vt: second review: FFI soundness, carry observer, build and licences #26

kayg referenced this issue

2026-10-07 00:53:34 +00:00

S1 vt: second review: FFI soundness, carry observer, build and licences #26

kayg referenced this issue

2026-10-07 00:55:09 +00:00

S1 vt: second review: FFI soundness, carry observer, build and licences #26

kayg commented

2026-10-07 00:55:10 +00:00

Author

Owner

Add to scope (from review-vt4): transactional native resize, so allocation refusal leaves the terminal unchanged, retains nothing, and never ends the shim. This replaces the retire-and-rebuild workaround in vt-ghostty 565105f.

Add to scope (from review-vt4): transactional native resize, so allocation refusal leaves the terminal unchanged, retains nothing, and never ends the shim. This replaces the retire-and-rebuild workaround in vt-ghostty 565105f.

kayg referenced this issue

2026-10-07 00:55:11 +00:00

Status board: work in flight (kept current) #24

kayg referenced this issue from a commit

2026-10-10 09:20:34 +00:00

fix(shim): retire native ownership after every resize error

kayg referenced this issue from a commit

2026-10-10 09:20:34 +00:00

docs(vt): record third-review fixes, scale budgets and resize ownership limits

kayg referenced this issue

2026-10-28 17:52:13 +00:00

S1: switch the shim to libghostty-vt #22