kayg/infraternal

Watch

Fork 0

libghostty-vt 1.0 upgrade #415

New issue

Open

opened 2026-11-07 06:21:10 +00:00 by kayg · 8 comments

kayg commented

2026-11-07 06:21:10 +00:00

Owner

Plan and estimate: (1) inspect current packaging/integration and upstream release artifacts/API (1–2 h); (2) source/tarball and adapter migration, atomic commits (3–5 h); (3) platform and feature flags (1–2 h); (4) baselines and after measurements plus tests on Linux/Mac (4–8 h, subject to VM slots and upstream artifacts); (5) final fast gate and evidence (1–3 h). If a required API or signed/offline artifact is unavailable, or a migration would require a different engine/design, report evidence and stop for owner direction.

Owner request from kayg (2026-11-07): upgrade the existing libghostty-vt integration to the 1.0 release. Inventory the current patches, extracted internals, adapter symbols and types; migrate to the stable C API where it covers our use; build offline from signed source plus deps tarball; trim unused features; update Zig, Nix and macOS handling; measure before/after; run terminal suites, real-server specs, shim gate, Linux and macOS validation; verify SPDX licences. Do not change the foundational engine decision (D94), merge, or deploy. Plan and estimate: (1) inspect current packaging/integration and upstream release artifacts/API (1–2 h); (2) source/tarball and adapter migration, atomic commits (3–5 h); (3) platform and feature flags (1–2 h); (4) baselines and after measurements plus tests on Linux/Mac (4–8 h, subject to VM slots and upstream artifacts); (5) final fast gate and evidence (1–3 h). If a required API or signed/offline artifact is unavailable, or a migration would require a different engine/design, report evidence and stop for owner direction.

kayg commented

2026-11-07 06:21:54 +00:00

Author

Owner

Starting J339 on branch libghostty-vt-1.0, based on origin/dev at 3644950b8e4b5dddeefba350f97a3892189ca193. Worktree: /home/kayg/Developer/Worktrees/infraternal-libghostty-vt-1.0. I will inventory the integration and artifacts first, then establish pre-change measurements before editing.

Starting J339 on branch `libghostty-vt-1.0`, based on `origin/dev` at `3644950b8e4b5dddeefba350f97a3892189ca193`. Worktree: `/home/kayg/Developer/Worktrees/infraternal-libghostty-vt-1.0`. I will inventory the integration and artifacts first, then establish pre-change measurements before editing.

kayg commented

2026-11-07 06:25:58 +00:00

Author

Owner

Finding 1 (source inventory): scripts/build-vt pins source a887df42c56f6de86c0fe6da9c4eeca37931e083 by GitHub-generated codeload checksum and requires Zig 0.15.2; nix/package.nix repeats that pin and currently packages a separate extracted Zig package cache. The shim’s Rust FFI uses seven upstream terminal calls (new, free, vt_write, resize, mode_get, get) plus four project exports (compare, approximate, snapshot, compact, screen_pages; feature-gated calls are noted separately). Native adapters are snapshot.zig, memory.zig, history.zig, and session_log.zig. The build script contains two sed -i invocations and multiple guarded Python source rewrites; I am enumerating each rewrite and private field before marking which the release API replaces.

Finding 1 (source inventory): `scripts/build-vt` pins source `a887df42c56f6de86c0fe6da9c4eeca37931e083` by GitHub-generated codeload checksum and requires Zig 0.15.2; `nix/package.nix` repeats that pin and currently packages a separate extracted Zig package cache. The shim’s Rust FFI uses seven upstream terminal calls (`new`, `free`, `vt_write`, `resize`, `mode_get`, `get`) plus four project exports (`compare`, `approximate`, `snapshot`, `compact`, `screen_pages`; feature-gated calls are noted separately). Native adapters are `snapshot.zig`, `memory.zig`, `history.zig`, and `session_log.zig`. The build script contains two `sed -i` invocations and multiple guarded Python source rewrites; I am enumerating each rewrite and private field before marking which the release API replaces.

kayg commented

2026-11-07 06:26:12 +00:00

Author

Owner

Correction to Finding 1: the adapter declares six upstream calls (not seven) and five project exports (not four). The build has three sed -i commands total: one callback/log rewrite plus two build-definition edits. I will list their exact edits and conditional scope in the final inventory.

Correction to Finding 1: the adapter declares six upstream calls (not seven) and five project exports (not four). The build has three `sed -i` commands total: one callback/log rewrite plus two build-definition edits. I will list their exact edits and conditional scope in the final inventory.

kayg commented

2026-11-07 06:33:18 +00:00

Author

Owner

Finding 2 (upstream release validated): the official vt-1.0.0 release directory exists. On dev-02, minisign 0.12 verified signatures on SHA256SUMS, the source tarball, and the deps tarball with the published public key; the two SHA-256 entries in the signed sums also match. Evidence: ~/.cache/codex-scratch/J339/vt-release/verification.log. PACKAGING-LIBGHOSTTY.md requires Zig 0.17.0 and provides the exact --system offline dependency layout and -Dvt-features flags. Its manifest lists: MIT AND Unicode-3.0; Apache-2.0 OR BSD-3-Clause; Apache-2.0 OR MIT; and MIT. The source and deps archives are 4.49 MB and 5.87 MB. No alternative engine or design is needed.

Finding 2 (upstream release validated): the official `vt-1.0.0` release directory exists. On dev-02, minisign 0.12 verified signatures on `SHA256SUMS`, the source tarball, and the deps tarball with the published public key; the two SHA-256 entries in the signed sums also match. Evidence: `~/.cache/codex-scratch/J339/vt-release/verification.log`. `PACKAGING-LIBGHOSTTY.md` requires Zig 0.17.0 and provides the exact `--system` offline dependency layout and `-Dvt-features` flags. Its manifest lists: MIT AND Unicode-3.0; Apache-2.0 OR BSD-3-Clause; Apache-2.0 OR MIT; and MIT. The source and deps archives are 4.49 MB and 5.87 MB. No alternative engine or design is needed.

kayg commented

2026-11-07 06:34:57 +00:00

Author

Owner

Finding 3 (reference mismatch): this repository’s #259 is the sidebar pill and host rename request; its comments contain no Zig/macOS SDK failure. The existing libghostty-vt macOS path is tracked under #328: commit be2ee211a records a successful cold macOS 26.5.2 arm64 build using Zig 0.15.2 and aarch64-macos. I’ll carry the explicit target mapping and validate the new Zig 0.17 path under the required Mac lock. If #259 referred to a different report, please link it here; I’ll continue with this documented path and record any SDK failure with the command and output.

Finding 3 (reference mismatch): this repository’s #259 is the sidebar pill and host rename request; its comments contain no Zig/macOS SDK failure. The existing libghostty-vt macOS path is tracked under #328: commit `be2ee211a` records a successful cold macOS 26.5.2 arm64 build using Zig 0.15.2 and `aarch64-macos`. I’ll carry the explicit target mapping and validate the new Zig 0.17 path under the required Mac lock. If `#259` referred to a different report, please link it here; I’ll continue with this documented path and record any SDK failure with the command and output.

kayg commented

2026-11-07 06:45:13 +00:00

Author

Owner

Finding 4 (public API coverage): the signed 1.0 header supplies ghostty_terminal_new/free/vt_write/resize, enum-keyed ghostty_terminal_get (including mode, scrollback rows, and memory usage), ghostty_formatter_* for VT output, ghostty_render_state_* for visible cells, and ghostty_terminal_compress. These cover the lifecycle/getters and the custom serializer/compressor interfaces. The 1.0 GhosttyTerminalMemoryUsage reports primary/alternate page counts, resident bytes, and compressed bytes. Limit: src/terminal/PageList.zig:592-603 still selects std.heap.page_allocator on Linux; the public C allocator does not account for that OS page pool. The PPU page-pool adapter therefore remains necessary for admission/floor accounting, subject to the existing resize tests.

Finding 4 (public API coverage): the signed 1.0 header supplies `ghostty_terminal_new/free/vt_write/resize`, enum-keyed `ghostty_terminal_get` (including mode, scrollback rows, and memory usage), `ghostty_formatter_*` for VT output, `ghostty_render_state_*` for visible cells, and `ghostty_terminal_compress`. These cover the lifecycle/getters and the custom serializer/compressor interfaces. The 1.0 `GhosttyTerminalMemoryUsage` reports primary/alternate page counts, resident bytes, and compressed bytes. Limit: `src/terminal/PageList.zig:592-603` still selects `std.heap.page_allocator` on Linux; the public C allocator does not account for that OS page pool. The PPU page-pool adapter therefore remains necessary for admission/floor accounting, subject to the existing resize tests.

kayg commented

2026-11-07 07:08:18 +00:00

Author

Owner

Finding 5 (complete source/adapter inventory, evidence from scripts/build-vt, terminal_native.rs, and the 1.0 headers/source):

sed -i is used three times. (1) In CARGO_FEATURE_SESSION_IMAGE builds, remove the ghostty_sys_set and ghostty_sys_log_stderr exports and redirect sys.logFn to native/session_log.zig; this keeps callback-free session images from exposing setters that silently discard callbacks. (2) Add link_function_sections and link_data_sections to the VT library build so the static linker can discard unused functions/data. (3) For ReleaseFast only, set root_module.strip = true; the Rust release output is stripped and the native source-symbol reader need not be linked. The latter two reasons are already adjacent to the edits and need to be remeasured with the 1.0 feature flags.
Explicitly extracted/restored internal Zig files are src/lib_vt.zig (always), src/terminal/PageList.zig, src/terminal/page.zig, src/terminal/Screen.zig, and src/terminal/compress/lz4.zig (shared-host build), src/terminal/compress/Page.zig (compact-history), and src/build/GhosttyLibVt.zig plus build.zig (build adjustments). The source archive itself is unpacked before these exact upstream files are restored for guarded patching.
The Rust adapter currently calls six upstream symbols: ghostty_terminal_new/free/vt_write/resize/mode_get/get. mode_get is test-only; get is used for shared-host data. Its five project exports are infraternal_vt_compare, infraternal_vt_approximate, infraternal_vt_snapshot, feature-gated infraternal_vt_compact, and shared-host infraternal_vt_screen_pages. The old Options { cols, rows, max_scrollback } Rust layout is not a stable C type.
Private adapter types/fields are concentrated in native/snapshot.zig, memory.zig, and history.zig: snapshot unwraps terminal/c/terminal.zig's Terminal and reads terminal/main.zig's Terminal, Screen, PageList.Pin, Style, CharsetSlot, modes.Mode/ModeState/ModeTag, page.Cell, Screen.CharsetState/SavedCursor, hyperlink records, and Unicode width/grapheme tables. It reads terminal dimensions/modes/flags/colors/screens/previous character/scrolling region/tabstops/charsets and raw screen cursor/page/cell/hyperlink/grapheme state. Memory routing uses terminal/lib.zig's allocator bridge/vtable and terminal/mem.zig, and calculates page layout using terminal/page.zig's Page/Cell/Row. Compact history uses that raw page layout plus terminal/compress/lz4.zig's HashTable, compress, and decompress. The compact driver also unwraps the C handle and reaches each screen's page compressor.
The 1.0 ABI now supplies opaque terminal/formatter/render handles; stable create/free/write/resize calls; enum-keyed get/set (including mode, geometry, scrollback rows, and memory usage); VT formatting with extras; visible render-state rows/cells; and terminal scrollback compression. This replaces the private Options layout, the test-only mode getter, and potentially the snapshot/comparison/compression exports, pending the repaint, continuation, retention and resize suites. GHOSTTY_TERMINAL_DATA_MEMORY_USAGE reports live usage and scans pages; it does not replace the geometry-only screen_pages pre-admission estimate. PageList still routes pages through std.heap.page_allocator on 1.0 Linux, so the shared-host page-pool accounting patch remains a candidate until the resize and process-per-user gates prove otherwise.

No source changes yet. Baseline measurement is queued on dev-02; current status has its slot free but 39 GB free against a 40 GB build-slot floor. compute01 has all five slots occupied and 110 GB free. I will retry when a VM can take the job without crossing its floor.

Finding 5 (complete source/adapter inventory, evidence from `scripts/build-vt`, `terminal_native.rs`, and the 1.0 headers/source): - `sed -i` is used three times. (1) In `CARGO_FEATURE_SESSION_IMAGE` builds, remove the `ghostty_sys_set` and `ghostty_sys_log_stderr` exports and redirect `sys.logFn` to `native/session_log.zig`; this keeps callback-free session images from exposing setters that silently discard callbacks. (2) Add `link_function_sections` and `link_data_sections` to the VT library build so the static linker can discard unused functions/data. (3) For ReleaseFast only, set `root_module.strip = true`; the Rust release output is stripped and the native source-symbol reader need not be linked. The latter two reasons are already adjacent to the edits and need to be remeasured with the 1.0 feature flags. - Explicitly extracted/restored internal Zig files are `src/lib_vt.zig` (always), `src/terminal/PageList.zig`, `src/terminal/page.zig`, `src/terminal/Screen.zig`, and `src/terminal/compress/lz4.zig` (shared-host build), `src/terminal/compress/Page.zig` (compact-history), and `src/build/GhosttyLibVt.zig` plus `build.zig` (build adjustments). The source archive itself is unpacked before these exact upstream files are restored for guarded patching. - The Rust adapter currently calls six upstream symbols: `ghostty_terminal_new/free/vt_write/resize/mode_get/get`. `mode_get` is test-only; `get` is used for shared-host data. Its five project exports are `infraternal_vt_compare`, `infraternal_vt_approximate`, `infraternal_vt_snapshot`, feature-gated `infraternal_vt_compact`, and shared-host `infraternal_vt_screen_pages`. The old `Options { cols, rows, max_scrollback }` Rust layout is not a stable C type. - Private adapter types/fields are concentrated in `native/snapshot.zig`, `memory.zig`, and `history.zig`: snapshot unwraps `terminal/c/terminal.zig`'s `Terminal` and reads `terminal/main.zig`'s `Terminal`, `Screen`, `PageList.Pin`, `Style`, `CharsetSlot`, `modes.Mode/ModeState/ModeTag`, `page.Cell`, `Screen.CharsetState/SavedCursor`, hyperlink records, and Unicode width/grapheme tables. It reads terminal dimensions/modes/flags/colors/screens/previous character/scrolling region/tabstops/charsets and raw screen cursor/page/cell/hyperlink/grapheme state. Memory routing uses `terminal/lib.zig`'s allocator bridge/vtable and `terminal/mem.zig`, and calculates page layout using `terminal/page.zig`'s `Page/Cell/Row`. Compact history uses that raw page layout plus `terminal/compress/lz4.zig`'s `HashTable`, `compress`, and `decompress`. The compact driver also unwraps the C handle and reaches each screen's page compressor. - The 1.0 ABI now supplies opaque terminal/formatter/render handles; stable create/free/write/resize calls; enum-keyed get/set (including mode, geometry, scrollback rows, and memory usage); VT formatting with extras; visible render-state rows/cells; and terminal scrollback compression. This replaces the private `Options` layout, the test-only mode getter, and potentially the snapshot/comparison/compression exports, pending the repaint, continuation, retention and resize suites. `GHOSTTY_TERMINAL_DATA_MEMORY_USAGE` reports live usage and scans pages; it does not replace the geometry-only `screen_pages` pre-admission estimate. `PageList` still routes pages through `std.heap.page_allocator` on 1.0 Linux, so the shared-host page-pool accounting patch remains a candidate until the resize and process-per-user gates prove otherwise. No source changes yet. Baseline measurement is queued on dev-02; current status has its slot free but 39 GB free against a 40 GB build-slot floor. compute01 has all five slots occupied and 110 GB free. I will retry when a VM can take the job without crossing its floor.

kayg commented

2026-11-07 07:15:59 +00:00

Author

Owner

Finding 6 (baseline runner status): the base worktree's scripts/verify quick passed on dev-02 at 07:06 host time after correcting the shell working directory; its scope was empty against origin/dev, so it ran the repository pre-check rows but no package clippy/unit tests. Current build-slot --status shows dev-02 at 1/1 occupied with 43 GB free (40 GB floor) and compute01 at 5/5 occupied with a queue and 93 GB free (25 GB floor). The before performance build and static-size receipts are still pending. I will keep one target per VM job and remove it before releasing its slot.

Finding 6 (baseline runner status): the base worktree's `scripts/verify quick` passed on dev-02 at 07:06 host time after correcting the shell working directory; its scope was empty against `origin/dev`, so it ran the repository pre-check rows but no package clippy/unit tests. Current `build-slot --status` shows dev-02 at 1/1 occupied with 43 GB free (40 GB floor) and compute01 at 5/5 occupied with a queue and 93 GB free (25 GB floor). The before performance build and static-size receipts are still pending. I will keep one target per VM job and remove it before releasing its slot.

No labels

launch-qa

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

kayg/infraternal#415

No description provided.

Rows
Columns