fuzz: host_name_parse found a failing input #353

Open
opened 2026-11-04 19:32:21 +00:00 by kayg · 25 comments
Owner

Nightly parser fuzzing (D351, #344).

Target: host_name_parse
Source: 6ee12794de510c62907d1615bcdb935e0e364464
Input SHA-256: 456fa477dd35a89e1bf6af9a8b7090428306c5b88df0a0abbe2b890548463445

Reproduce with cargo +nightly fuzz run host_name_parse <attached-input>. The exact failing input is attached to the report comment.

Nightly parser fuzzing (D351, #344). Target: `host_name_parse` Source: `6ee12794de510c62907d1615bcdb935e0e364464` Input SHA-256: `456fa477dd35a89e1bf6af9a8b7090428306c5b88df0a0abbe2b890548463445` Reproduce with `cargo +nightly fuzz run host_name_parse <attached-input>`. The exact failing input is attached to the report comment.
Author
Owner

Failing input for host_name_parse, SHA-256 456fa477dd35a89e1bf6af9a8b7090428306c5b88df0a0abbe2b890548463445.

Failing input for `host_name_parse`, SHA-256 `456fa477dd35a89e1bf6af9a8b7090428306c5b88df0a0abbe2b890548463445`.
Author
Owner

Context from J208c (#344): this is the first issue filed by the D351 nightly fuzz runner, during its install smoke run on dev-02 (36 s budget). The harness's idempotence assertion fails: machine_name("0X.A-1") returns Some("0x"), but machine_name("0x") returns None (fuzz/fuzz_targets/host_name_parse.rs:24). The zip on comment 78870 holds the exact 6 bytes (0X.A-1). Reproduce: unzip it, then cargo +nightly fuzz run host_name_parse <file>. Not fixed in J208c.

Context from J208c (#344): this is the first issue filed by the D351 nightly fuzz runner, during its install smoke run on dev-02 (36 s budget). The harness's idempotence assertion fails: `machine_name("0X.A-1")` returns `Some("0x")`, but `machine_name("0x")` returns `None` (`fuzz/fuzz_targets/host_name_parse.rs:24`). The zip on comment 78870 holds the exact 6 bytes (`0X.A-1`). Reproduce: unzip it, then `cargo +nightly fuzz run host_name_parse <file>`. Not fixed in J208c.
Author
Owner

Failing input for host_name_parse from source 100bd91f6dd966b55409f0dd53dd9b3d468c359b, SHA-256 2a8e8b6f0945d597d1c4fccce4bce1aa4f9353ab34b9a9b212a78b1545318641.

Input (base64): MFg0MC4yLhI=

Failing input for `host_name_parse` from source `100bd91f6dd966b55409f0dd53dd9b3d468c359b`, SHA-256 `2a8e8b6f0945d597d1c4fccce4bce1aa4f9353ab34b9a9b212a78b1545318641`. Input (base64): `MFg0MC4yLhI=`
Author
Owner

Failing input for host_name_parse from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 a16a2869969aeaa8f67d32acd6c12fb7eaa406ea55b6452a4054d30c00663a5b.

Input (base64): MFguADAxTi4S

Failing input for `host_name_parse` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `a16a2869969aeaa8f67d32acd6c12fb7eaa406ea55b6452a4054d30c00663a5b`. Input (base64): `MFguADAxTi4S`
Author
Owner

Failing input for host_name_parse from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 db0284ab184f9bcb70a9be0c4ae9fd821b49e761a42f012f13ee96ec69079f6f.

Input (base64): MFgyMjIyMi4SMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjI=

Failing input for `host_name_parse` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `db0284ab184f9bcb70a9be0c4ae9fd821b49e761a42f012f13ee96ec69079f6f`. Input (base64): `MFgyMjIyMi4SMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjI=`
Author
Owner

J264 starting on branch fuzz-fixes-j264, base 159edcedc9978f0370ea8cb02a937082d631c9b6 (origin/dev). Plan: reproduce every reported input on a build VM, add failing regression coverage, fix the demonstrated cause, run each target for 600 seconds, run scripts/verify quick through build-slot in the foreground, and push one atomic commit per issue. Estimate: 90 minutes including compilation/slot queues; no merge, deploy, or issue closure.

J264 starting on branch `fuzz-fixes-j264`, base `159edcedc9978f0370ea8cb02a937082d631c9b6` (origin/dev). Plan: reproduce every reported input on a build VM, add failing regression coverage, fix the demonstrated cause, run each target for 600 seconds, run `scripts/verify quick` through build-slot in the foreground, and push one atomic commit per issue. Estimate: 90 minutes including compilation/slot queues; no merge, deploy, or issue closure.
Author
Owner

J264b (taking over from J264): finding, reproduced, fix pushed.

Failure type: the target's oracle disagrees. The idempotence assertion machine_name(name) == Some(name) at fuzz/fuzz_targets/host_name_parse.rs:24 panics. There's no overflow and no timeout. All four filed inputs reproduce against the nightly's own host_name_parse binary on dev-02, built from 521cb8990:

input (SHA-256) bytes first call second call
456fa477… 0X.A-1 Some("0x") None
2a8e8b6f… 0X40.2.\x12 Some("0x40") None
a16a2869… 0X.\x0001N.\x12 Some("0x") None
db0284ab… 0X22222.\x12 + 29×2 Some("0x22222") None

There are two causes in crates/infraternal-plugin-api/src/host_name.rs:

  1. is_ipv4_spelling counts a bare 0x/0X (no digits) as a hex number. glibc's inet_aton rejects it. On dev-02, Python's socket.inet_aton rejects '0x', '0X' and '0x.1', and accepts '0x40' → 00000040 and '0x22222' → 00022222.
  2. The first-label check refuses only labels made of digits. A hex label (0x40) is returned from a multi-label host, but on its own it's an address.

Fix (f839112ba on fuzz-fixes-j264):

  • A hex part needs at least one digit.
  • The first label is refused when is_ipv4_spelling(label) holds, which also covers the old digits-only rule.

After this, a returned label passes every check the second call makes, so idempotence holds by construction. One side effect for callers (mesh store, remote inventory): a host such as 0x40.lan now gets the D339 fallback name instead of "0x40".

Regression: host_name_tests::fuzz_inputs_from_353_are_idempotent holds the exact bytes (hashes checked against the table above), next to numeric_first_labels_give_no_name_but_a_bare_hex_prefix_does. On compute01, cargo test -p infraternal-plugin-api --lib host_name:

  • before the fix: test result: FAILED. 7 passed; 2 failed (left: None, right: Some("0x") and left: Some("0x40"), right: None)
  • after the fix: test result: ok. 9 passed; 0 failed

Logs: compute01 ~/.cache/codex-scratch/j264b/353-red.log and 353-green.log. Nightly-binary replay: dev-02 /tmp/j264b-353/. The 600 s fuzz run follows.

J264b (taking over from J264): finding, reproduced, fix pushed. **Failure type:** the target's oracle disagrees. The idempotence assertion `machine_name(name) == Some(name)` at `fuzz/fuzz_targets/host_name_parse.rs:24` panics. There's no overflow and no timeout. All four filed inputs reproduce against the nightly's own `host_name_parse` binary on dev-02, built from 521cb8990: | input (SHA-256) | bytes | first call | second call | |---|---|---|---| | 456fa477… | `0X.A-1` | `Some("0x")` | `None` | | 2a8e8b6f… | `0X40.2.\x12` | `Some("0x40")` | `None` | | a16a2869… | `0X.\x0001N.\x12` | `Some("0x")` | `None` | | db0284ab… | `0X22222.\x12` + 29×`2` | `Some("0x22222")` | `None` | **There are two causes in `crates/infraternal-plugin-api/src/host_name.rs`:** 1. `is_ipv4_spelling` counts a bare `0x`/`0X` (no digits) as a hex number. glibc's `inet_aton` rejects it. On dev-02, Python's `socket.inet_aton` rejects `'0x'`, `'0X'` and `'0x.1'`, and accepts `'0x40'` → `00000040` and `'0x22222'` → `00022222`. 2. The first-label check refuses only labels made of digits. A hex label (`0x40`) is returned from a multi-label host, but on its own it's an address. **Fix (f839112ba on `fuzz-fixes-j264`):** - A hex part needs at least one digit. - The first label is refused when `is_ipv4_spelling(label)` holds, which also covers the old digits-only rule. After this, a returned label passes every check the second call makes, so idempotence holds by construction. One side effect for callers (mesh store, remote inventory): a host such as `0x40.lan` now gets the D339 fallback name instead of "0x40". **Regression:** `host_name_tests::fuzz_inputs_from_353_are_idempotent` holds the exact bytes (hashes checked against the table above), next to `numeric_first_labels_give_no_name_but_a_bare_hex_prefix_does`. On compute01, `cargo test -p infraternal-plugin-api --lib host_name`: - before the fix: `test result: FAILED. 7 passed; 2 failed` (`left: None, right: Some("0x")` and `left: Some("0x40"), right: None`) - after the fix: `test result: ok. 9 passed; 0 failed` Logs: compute01 `~/.cache/codex-scratch/j264b/353-red.log` and `353-green.log`. Nightly-binary replay: dev-02 `/tmp/j264b-353/`. The 600 s fuzz run follows.
Author
Owner

J264b progress: fuzz-fixes-j264 now has f839112ba (#353 parser fix) and b1e8d164e (#370 runner fix), both pushed.

  • #370 regression red/green on compute01: before, FAILED (errors=2) with AttributeError: ... no attribute 'confirmed_inputs'. After, Ran 13 tests ... OK (~/.cache/codex-scratch/j264b/370-{red,green}.log).
  • Queued on dev-02: the fuzz build with the nightly's toolchain, a replay of the #353 inputs and the empty input, the runner's exact re-check command, then 600 s of host_name_parse and 600 s of ssh_auth_once_decode, seeded from copies of the nightly corpora (~/.cache/codex-scratch/j264b/fuzz-run1.log).
  • Queued on compute01: scripts/verify quick (~/.cache/codex-scratch/j264b/verify-quick.log).
J264b progress: `fuzz-fixes-j264` now has f839112ba (#353 parser fix) and b1e8d164e (#370 runner fix), both pushed. - #370 regression red/green on compute01: before, `FAILED (errors=2)` with `AttributeError: ... no attribute 'confirmed_inputs'`. After, `Ran 13 tests ... OK` (`~/.cache/codex-scratch/j264b/370-{red,green}.log`). - Queued on dev-02: the fuzz build with the nightly's toolchain, a replay of the #353 inputs and the empty input, the runner's exact re-check command, then 600 s of `host_name_parse` and 600 s of `ssh_auth_once_decode`, seeded from copies of the nightly corpora (`~/.cache/codex-scratch/j264b/fuzz-run1.log`). - Queued on compute01: `scripts/verify quick` (`~/.cache/codex-scratch/j264b/verify-quick.log`).
Author
Owner

J264b finish.

Branch fuzz-fixes-j264, head d1450e1a0b. It has two commits on base 159edcedc:

  • 271913dc0 fix(plugin-api): machine_name refuses numeric first labels, accepts bare 0x (#353)
  • d1450e1a0 fix(fuzz): re-check timeouts in isolation before filing them (#370). This also covers #374 and #358.

I have not merged, deployed or closed anything.

Gate: scripts/verify quick through build-slot on compute01 at d1450e1a0 (~/.cache/codex-scratch/j264b/verify-quick2.log), verbatim:

quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0
+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| scope guard               | PASS         |
| migration names           | PASS         |
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| clippy (workspace)        | PASS         |
| unit tests (changed)      | PASS         |
+---------------------------+--------------+

QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done.
VERIFY_EXIT=0
  • An earlier quick run at the pre-amend head b1e8d164e failed only on fmt (one assert_eq! in the #353 test). I folded the fix into the #353 commit and force-with-lease pushed the branch, so each commit passes the gate.
  • quick scoped only infraternal-plugin-api. The runner's Python tests (the nightly fuzz reporter row, in fast) passed separately on compute01: Ran 13 tests ... OK (370-green.log). The scripts are byte-identical between b1e8d164e and d1450e1a0.
  • As tasked, I did not run scripts/verify fast.

Fuzz runs: one batched build on dev-02, with the nightly's own toolchain (rustc 1.101.0-nightly 708c94ef0, cargo-fuzz 0.13.2, ASan) and its flags (-timeout=10 -rss_limit_mb=2048 -max_len=1048580), seeded from copies of the nightly corpora. 600 s per target:

target executions rc artifacts
host_name_parse 102 384 284 in 601 s 0 none
ssh_auth_once_decode 13 828 284 in 601 s 0 none
release_channel 57 369 130 in 601 s 0 none
remote_standby_decode 10 143 408 in 601 s 0 none

Logs on dev-02: ~/.cache/codex-scratch/j264b/fuzz-run1.log, fuzz-run2.log and fuzz-<target>.log.

Runner re-check, per input (fuzz-run2.log). I called the runner's own confirmed_inputs() with the exact cargo fuzz run <target> <input> -- -timeout=10 -rss_limit_mb=2048 -artifact_prefix=... command that main() builds, on copies of the nightly's four timeout artifacts:

re-check release_channel/timeout-3f4bde2f…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check remote_standby_decode/timeout-1c839c82…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check remote_standby_decode/timeout-941a98f0…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check ssh_auth_once_decode/timeout-da39a3ee…: rc=0 Executed … in 0 ms  -> set aside (not filed)

On the fixed build, the four #353 inputs replay with rc=0 (Executed … in 31/5/1/5 ms).

Once this merges and the nightly checkout moves to it, its first run re-checks the four timeout artifacts still in the nightly's artifact directory and moves them to unconfirmed/. These issues can then be closed by the owner.

J264b finish. **Branch** `fuzz-fixes-j264`, **head d1450e1a0bc234a2acf8fc346c6cb521f06eebb9**. It has two commits on base 159edcedc: - 271913dc0 `fix(plugin-api): machine_name refuses numeric first labels, accepts bare 0x (#353)` - d1450e1a0 `fix(fuzz): re-check timeouts in isolation before filing them (#370)`. This also covers #374 and #358. I have not merged, deployed or closed anything. **Gate:** `scripts/verify quick` through build-slot on compute01 at d1450e1a0 (`~/.cache/codex-scratch/j264b/verify-quick2.log`), verbatim: ``` quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0 +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | scope guard | PASS | | migration names | PASS | | Nix Cargo vendoring | PASS | | fmt | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | clippy (workspace) | PASS | | unit tests (changed) | PASS | +---------------------------+--------------+ QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done. VERIFY_EXIT=0 ``` - An earlier quick run at the pre-amend head b1e8d164e failed only on `fmt` (one `assert_eq!` in the #353 test). I folded the fix into the #353 commit and force-with-lease pushed the branch, so each commit passes the gate. - `quick` scoped only `infraternal-plugin-api`. The runner's Python tests (the `nightly fuzz reporter` row, in `fast`) passed separately on compute01: `Ran 13 tests ... OK` (`370-green.log`). The scripts are byte-identical between b1e8d164e and d1450e1a0. - As tasked, I did **not** run `scripts/verify fast`. **Fuzz runs:** one batched build on dev-02, with the nightly's own toolchain (rustc 1.101.0-nightly 708c94ef0, cargo-fuzz 0.13.2, ASan) and its flags (`-timeout=10 -rss_limit_mb=2048 -max_len=1048580`), seeded from copies of the nightly corpora. 600 s per target: | target | executions | rc | artifacts | |---|---|---|---| | host_name_parse | 102 384 284 in 601 s | 0 | none | | ssh_auth_once_decode | 13 828 284 in 601 s | 0 | none | | release_channel | 57 369 130 in 601 s | 0 | none | | remote_standby_decode | 10 143 408 in 601 s | 0 | none | Logs on dev-02: `~/.cache/codex-scratch/j264b/fuzz-run1.log`, `fuzz-run2.log` and `fuzz-<target>.log`. **Runner re-check, per input** (`fuzz-run2.log`). I called the runner's own `confirmed_inputs()` with the exact `cargo fuzz run <target> <input> -- -timeout=10 -rss_limit_mb=2048 -artifact_prefix=...` command that `main()` builds, on copies of the nightly's four timeout artifacts: ``` re-check release_channel/timeout-3f4bde2f…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check remote_standby_decode/timeout-1c839c82…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check remote_standby_decode/timeout-941a98f0…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check ssh_auth_once_decode/timeout-da39a3ee…: rc=0 Executed … in 0 ms -> set aside (not filed) ``` On the fixed build, the four #353 inputs replay with rc=0 (`Executed … in 31/5/1/5 ms`). **Once this merges and the nightly checkout moves to it**, its first run re-checks the four timeout artifacts still in the nightly's artifact directory and moves them to `unconfirmed/`. These issues can then be closed by the owner.
Author
Owner

J274 review of fuzz-fixes-j264 @ d1450e1a0 (base 159edcedc): in progress.

  • Parser (271913dc0): differential check against glibc inet_aton over all 5 229 071 strings up to 6 chars from 0189xXafg.-@:, plus 28 hand cases. 0 failures. Log: dev-02 ~/.cache/codex-scratch/j274/hn_check.log.
  • Runner (d1450e1a0): mutation check of scripts/test-nightly-fuzz.py. Summary: compute01 ~/.cache/codex-scratch/j274/mut/summary.txt.
  • Next: a 600 s host_name_parse run from an empty corpus. It's queued in build-slot on dev-02 (~/.cache/codex-scratch/j274/fuzz-run.log). The verdict follows.
J274 review of `fuzz-fixes-j264` @ d1450e1a0 (base 159edcedc): in progress. - Parser (271913dc0): differential check against glibc `inet_aton` over all 5 229 071 strings up to 6 chars from `0189xXafg.-@:`, plus 28 hand cases. 0 failures. Log: dev-02 `~/.cache/codex-scratch/j274/hn_check.log`. - Runner (d1450e1a0): mutation check of `scripts/test-nightly-fuzz.py`. Summary: compute01 `~/.cache/codex-scratch/j274/mut/summary.txt`. - Next: a 600 s `host_name_parse` run from an empty corpus. It's queued in build-slot on dev-02 (`~/.cache/codex-scratch/j274/fuzz-run.log`). The verdict follows.
Author
Owner

J274 review of fuzz-fixes-j264 @ d1450e1a0b (base 159edcedc). This covers #353, #370, #374 and #358.

Verdict: REQUEST CHANGES

The parser commit 271913dc0 is approved as is. The changes requested are only in the runner commit d1450e1a0: two P2s.

I did not run verify fast or verify quick (reviewer rule). I read the author's quick output in 81353.


1. 271913dc0, the host-name parser (#353): approve

Against inet_aton. I included the branch's host_name.rs byte for byte in a harness and compared it with glibc inet_aton through FFI. It enumerated all 5 229 071 strings up to 6 chars over 0189xXafg.-@:, which covers decimal, octal, hex, 1 to 5 parts, empty parts, trailing dots, users, ports and IPv6-style colons. It also ran 28 hand cases (0x100000000, 1.2.3.256, 089.1.2.3, 1.2.3.4 junk, 0xdeadbeef.lan, a-0x40.lan, deadbeef, and so on). Three properties were checked on every string:

property failures
self-mapping: machine_name(s)=Some(n) ⇒ machine_name(n)=Some(n) 0
a returned name is never accepted by inet_aton on its own 0
a host part that inet_aton accepts never gives a name 0

Log: dev-02 ~/.cache/codex-scratch/j274/hn_check.log. The harness is hn_check.rs next to it.

  • Is self-mapping now total? Yes, by construction as well as by test. A returned label has no @, :, [, %, . or whitespace. On the second call it is therefore the whole host, and is_ipv4_spelling(host) and is_ipv4_spelling(label) are the same check the first call already passed. Every other check is idempotent on [a-z0-9-].
  • Can an IPv4 address still become a name? No. Every part of an inet_aton address is numeric, so its first label is refused (third row above).
  • Can a legal name now be refused? The new rule refuses only first labels of the form 0x<hex digits>.
    • Where glibc also reads that label as an address, the refusal is required. If 0x40 were returned, ssh 0x40 would go to 0.0.0.64.
    • The harness found only two kinds of over-refusal compared with glibc. One is all-digit labels that glibc rejects (08, 0009), which the old rule already refused. The other is hex labels too large for 32 bits (0x100000000), which is new and harmless.
    • Names such as 0xygen, 0x1g, deadbeef and a-0x40 stay names.
  • 0x40.lan getting the D339 fallback is acceptable. The function also takes addresses (root@0x40 is an address), so the bare label must be refused, and then idempotence forces refusing it as a first label. 10.lan already worked this way.
    • Strictly, D339 says only empty and localhost fall back. Refusing numeric first labels predates this branch, so I'm only mentioning it.
  • The bare 0x now being a name follows glibc, musl and BSD inet_aton. The WHATWG URL parser is the exception: it reads http://0x/ as 0.0.0.0. Callers (remote_inventory.rs:173, mesh/src/store.rs:288) use the name as a label, not as a bare URL host, so this is not a finding.
  • Fresh-corpus fuzz run. 600 s from an empty corpus on dev-02, with the nightly flags (-timeout=10 -rss_limit_mb=2048 -max_len=1048580):
    • Result: Done 578382211 runs in 601 second(s), rc=0, 0 artifacts, peak RSS 29 MB, corpus 281 units / cov 136.
    • Build: the nightly toolchain lives under the fuzz service user, so I built the branch's exact host_name.rs and fuzz_targets/host_name_parse.rs body in a standalone crate (libfuzzer-sys =0.4.13). It used cargo-fuzz's -s none sancov flags plus debug assertions and overflow checks, without ASan. ASan adds nothing for this safe-Rust assertion oracle.
    • Logs: dev-02 ~/.cache/codex-scratch/j274/fuzz-host_name_parse.log and fuzz-run.log.
  • The doc comments and the regression tests (exact fuzz bytes, plus 0x and 0x40 cases) are good.

2. d1450e1a0, the nightly runner (#370, #374, #358): request changes

What's right:

  • Crash inputs are filed exactly as before. confirmed_inputs touches only names starting with timeout-. The order (sorted) and report_crash are unchanged.
  • A re-check that crashes is filed. Its exit code is non-zero, so it counts as reproduced, and the timeout input is filed.
  • A re-check that times out again is filed. libFuzzer exits with code 70.
  • A re-check that can't run is filed. Any non-zero exit counts as reproduced, so a tool error never drops an input.
  • Fixing the runner and not the decoders is right. The #370, #374 and #358 replays are convincing.

P2-1: the tests don't cover the code that can drop a real hang

I mutated nightly-fuzz.py at d1450e1a0 and ran test-nightly-fuzz.py against each version (compute01 ~/.cache/codex-scratch/j274/mut/summary.txt, one log per mutant). The one error that appears in every run, baseline included (test_existing_targets_and_budget, needs fuzz/Cargo.toml), comes from my scratch layout, not the branch.

mutant caught?
M1: the re-check's rule inverted (returncode == 0), so every real hang is set aside no
M2: -timeout removed from the re-check command no
M3: confirmed_inputs not wired into main() (crashes = before) no
M4: crashes re-checked too yes

The two new tests check confirmed_inputs with a stubbed reproduces. Their red was "the function doesn't exist", which proves little. The rule "non-zero counts as reproduced" is the part whose failure loses real hangs, and it sits in an untested closure inside main().

Requested:

  • Move reproduces to module level, taking the runner as a parameter (default subprocess.run).
  • Test it with a fake runner: exit 0 → not reproduced; 70, 77 and 1 → reproduced. The command must carry -timeout={TIMEOUT_SECONDS} and the input path, and must not carry -max_total_time.
  • The red has to come from wrong behaviour, not from a missing name: show M1 and M2 failing.

P2-2: a slow but real input under the bar is set aside permanently

The re-check uses the same 10 s bar and runs once.

Take an input that runs in about 7–9 s alone, for example quadratic work on a near-1 MiB input. Under load it trips 10 s and gets written as timeout-*. On its own it passes, so it moves to unconfirmed/ and is never re-checked. Before this commit it was filed.

That's a real performance bug (CLAUDE.md priority 1), and its only trace is a log line. libFuzzer already prints Executed <input> in N ms for the re-check, into the same log.

Requested: count the re-check as reproduced when it fails or when it ran for at least TIMEOUT_SECONDS/2. One option is to parse Executed … in N ms; another is to re-check with -timeout={TIMEOUT_SECONDS//2}.

  • This still sets aside the #370, #374 and #358 artifacts, which ran in 0–1 ms.
  • If the host is still loaded during the re-check, the input is filed. That errs on the safe side.

On intermittent hangs: this runner's targets are deterministic for a given input. A timeout that needs state built up by earlier inputs in the same process can't be reproduced from the filed input anyway. Keeping it in unconfirmed/ is a reasonable outcome, as long as P2-2 stops the slow-input case from going there.

P3 (do now or in a follow-up)

  • P3-1: set-aside timeouts are still reported as new findings, and the target's shortened run goes unnoticed. found is computed before the re-check, so a set-aside timeout still counts in new_inputs and in FUZZ_DONE new_inputs=. A libFuzzer timeout also ends that target's run, so with #370's opening-call stall the target fuzzed for about 10 s instead of its budget, and the night now reports failed=False with no issue filed. Subtract the unconfirmed inputs from found, and flag cut_short (seconds well under the budget) in last-run.json so lost coverage shows up.
  • P3-2: state/recheck/<target>/ keeps growing. It collects the re-check's own artifacts and is never cleaned. Clear it before each re-check.
  • Note: a slow-unit-* artifact (libFuzzer writes one when a unit runs ≥10 s but the alarm hasn't fired yet) is just as load-dependent and still goes straight to filing. It's outside this slice.

Evidence: dev-02 ~/.cache/codex-scratch/j274/ (hn_check.log, hn_check.rs, fuzz-host_name_parse.log, fuzz-run.log) and compute01 ~/.cache/codex-scratch/j274/mut/ (summary.txt, per-mutant logs). Build targets deleted. Nothing merged, pushed or closed.

J274 review of `fuzz-fixes-j264` @ d1450e1a0bc234a2acf8fc346c6cb521f06eebb9 (base 159edcedc). This covers #353, #370, #374 and #358. ## Verdict: REQUEST CHANGES The **parser commit 271913dc0 is approved as is**. The changes requested are only in the **runner commit d1450e1a0**: two P2s. I did not run `verify fast` or `verify quick` (reviewer rule). I read the author's quick output in 81353. --- ## 1. 271913dc0, the host-name parser (#353): approve **Against `inet_aton`.** I included the branch's `host_name.rs` byte for byte in a harness and compared it with glibc `inet_aton` through FFI. It enumerated all 5 229 071 strings up to 6 chars over `0189xXafg.-@:`, which covers decimal, octal, hex, 1 to 5 parts, empty parts, trailing dots, users, ports and IPv6-style colons. It also ran 28 hand cases (`0x100000000`, `1.2.3.256`, `089.1.2.3`, `1.2.3.4 junk`, `0xdeadbeef.lan`, `a-0x40.lan`, `deadbeef`, and so on). Three properties were checked on every string: | property | failures | |---|---| | self-mapping: `machine_name(s)=Some(n)` ⇒ `machine_name(n)=Some(n)` | 0 | | a returned name is never accepted by `inet_aton` on its own | 0 | | a host part that `inet_aton` accepts never gives a name | 0 | Log: dev-02 `~/.cache/codex-scratch/j274/hn_check.log`. The harness is `hn_check.rs` next to it. - **Is self-mapping now total?** Yes, by construction as well as by test. A returned label has no `@`, `:`, `[`, `%`, `.` or whitespace. On the second call it is therefore the whole host, and `is_ipv4_spelling(host)` and `is_ipv4_spelling(label)` are the same check the first call already passed. Every other check is idempotent on `[a-z0-9-]`. - **Can an IPv4 address still become a name?** No. Every part of an `inet_aton` address is numeric, so its first label is refused (third row above). - **Can a legal name now be refused?** The new rule refuses only first labels of the form `0x<hex digits>`. - Where glibc also reads that label as an address, the refusal is required. If `0x40` were returned, `ssh 0x40` would go to 0.0.0.64. - The harness found only two kinds of over-refusal compared with glibc. One is all-digit labels that glibc rejects (`08`, `0009`), which the old rule already refused. The other is hex labels too large for 32 bits (`0x100000000`), which is new and harmless. - Names such as `0xygen`, `0x1g`, `deadbeef` and `a-0x40` stay names. - **`0x40.lan` getting the D339 fallback is acceptable.** The function also takes addresses (`root@0x40` is an address), so the bare label must be refused, and then idempotence forces refusing it as a first label. `10.lan` already worked this way. - Strictly, D339 says only empty and `localhost` fall back. Refusing numeric first labels predates this branch, so I'm only mentioning it. - **The bare `0x` now being a name follows glibc, musl and BSD `inet_aton`.** The WHATWG URL parser is the exception: it reads `http://0x/` as 0.0.0.0. Callers (`remote_inventory.rs:173`, `mesh/src/store.rs:288`) use the name as a label, not as a bare URL host, so this is not a finding. - **Fresh-corpus fuzz run.** 600 s from an empty corpus on dev-02, with the nightly flags (`-timeout=10 -rss_limit_mb=2048 -max_len=1048580`): - Result: `Done 578382211 runs in 601 second(s)`, rc=0, 0 artifacts, peak RSS 29 MB, corpus 281 units / cov 136. - Build: the nightly toolchain lives under the fuzz service user, so I built the branch's exact `host_name.rs` and `fuzz_targets/host_name_parse.rs` body in a standalone crate (`libfuzzer-sys =0.4.13`). It used cargo-fuzz's `-s none` sancov flags plus debug assertions and overflow checks, without ASan. ASan adds nothing for this safe-Rust assertion oracle. - Logs: dev-02 `~/.cache/codex-scratch/j274/fuzz-host_name_parse.log` and `fuzz-run.log`. - The doc comments and the regression tests (exact fuzz bytes, plus `0x` and `0x40` cases) are good. ## 2. d1450e1a0, the nightly runner (#370, #374, #358): request changes **What's right:** - **Crash inputs are filed exactly as before.** `confirmed_inputs` touches only names starting with `timeout-`. The order (`sorted`) and `report_crash` are unchanged. - **A re-check that crashes is filed.** Its exit code is non-zero, so it counts as reproduced, and the timeout input is filed. - **A re-check that times out again is filed.** libFuzzer exits with code 70. - **A re-check that can't run is filed.** Any non-zero exit counts as reproduced, so a tool error never drops an input. - Fixing the runner and not the decoders is right. The #370, #374 and #358 replays are convincing. ### P2-1: the tests don't cover the code that can drop a real hang I mutated `nightly-fuzz.py` at d1450e1a0 and ran `test-nightly-fuzz.py` against each version (compute01 `~/.cache/codex-scratch/j274/mut/summary.txt`, one log per mutant). The one error that appears in every run, baseline included (`test_existing_targets_and_budget`, needs `fuzz/Cargo.toml`), comes from my scratch layout, not the branch. | mutant | caught? | |---|---| | M1: the re-check's rule inverted (`returncode == 0`), so every real hang is set aside | **no** | | M2: `-timeout` removed from the re-check command | **no** | | M3: `confirmed_inputs` not wired into `main()` (`crashes = before`) | **no** | | M4: crashes re-checked too | yes | The two new tests check `confirmed_inputs` with a stubbed `reproduces`. Their red was "the function doesn't exist", which proves little. The rule "non-zero counts as reproduced" is the part whose failure loses real hangs, and it sits in an untested closure inside `main()`. **Requested:** - Move `reproduces` to module level, taking the runner as a parameter (default `subprocess.run`). - Test it with a fake runner: exit 0 → not reproduced; 70, 77 and 1 → reproduced. The command must carry `-timeout={TIMEOUT_SECONDS}` and the input path, and must not carry `-max_total_time`. - The red has to come from wrong behaviour, not from a missing name: show M1 and M2 failing. ### P2-2: a slow but real input under the bar is set aside permanently The re-check uses the same 10 s bar and runs once. Take an input that runs in about 7–9 s alone, for example quadratic work on a near-1 MiB input. Under load it trips 10 s and gets written as `timeout-*`. On its own it passes, so it moves to `unconfirmed/` and is never re-checked. Before this commit it was filed. That's a real performance bug (CLAUDE.md priority 1), and its only trace is a log line. libFuzzer already prints `Executed <input> in N ms` for the re-check, into the same log. **Requested:** count the re-check as reproduced when it fails **or** when it ran for at least `TIMEOUT_SECONDS/2`. One option is to parse `Executed … in N ms`; another is to re-check with `-timeout={TIMEOUT_SECONDS//2}`. - This still sets aside the #370, #374 and #358 artifacts, which ran in 0–1 ms. - If the host is still loaded during the re-check, the input is filed. That errs on the safe side. **On intermittent hangs:** this runner's targets are deterministic for a given input. A timeout that needs state built up by earlier inputs in the same process can't be reproduced from the filed input anyway. Keeping it in `unconfirmed/` is a reasonable outcome, as long as P2-2 stops the slow-input case from going there. ### P3 (do now or in a follow-up) - **P3-1: set-aside timeouts are still reported as new findings, and the target's shortened run goes unnoticed.** `found` is computed before the re-check, so a set-aside timeout still counts in `new_inputs` and in `FUZZ_DONE new_inputs=`. A libFuzzer timeout also ends that target's run, so with #370's opening-call stall the target fuzzed for about 10 s instead of its budget, and the night now reports `failed=False` with no issue filed. Subtract the unconfirmed inputs from `found`, and flag `cut_short` (`seconds` well under the budget) in `last-run.json` so lost coverage shows up. - **P3-2: `state/recheck/<target>/` keeps growing.** It collects the re-check's own artifacts and is never cleaned. Clear it before each re-check. - **Note:** a `slow-unit-*` artifact (libFuzzer writes one when a unit runs ≥10 s but the alarm hasn't fired yet) is just as load-dependent and still goes straight to filing. It's outside this slice. --- Evidence: dev-02 `~/.cache/codex-scratch/j274/` (`hn_check.log`, `hn_check.rs`, `fuzz-host_name_parse.log`, `fuzz-run.log`) and compute01 `~/.cache/codex-scratch/j274/mut/` (`summary.txt`, per-mutant logs). Build targets deleted. Nothing merged, pushed or closed.
Author
Owner

J264b round 2 finish: J274's runner findings (81439) are addressed on the new branch fuzz-runner-recheck, head 0fc6934ede, based on origin/dev 431441d53. The parser fix is already merged as 431441d53. Nothing merged, deployed or closed.

Commits, one per item, each pushed as it was made:

  1. e57e2fb4f: d1450e1a0 carried over unchanged ("re-check timeouts in isolation before filing them").
  2. bab8e2f94 (P2-1): Fuzzer (injectable run), recheck_flags, reproduces and run_target are now module level, and main() calls run_target per harness. A FakeLibFuzzer plays cargo-fuzz in the tests. It covers:
    • the exit rule: 0 sets the input aside; 70, 77 and 1 file it;
    • the re-check command: it carries -timeout=TIMEOUT_SECONDS and the input, and no -max_total_time;
    • run_target end to end, in both directions.
  3. b124595aa (P2-2): reproduces reads libFuzzer's Executed <input> in N ms from its own part of the log. It files the input on a non-zero exit, on ≥ 5000 ms (TIMEOUT_SECONDS*1000//2), or when no time was reported. I did not use -timeout=5: libFuzzer's alarm only checks every timeout/2+1 s, so a unit just over 5 s could pass unflagged.
  4. 0fc6934ed (P3):
    • set-aside timeouts no longer count in new_inputs or FUZZ_DONE new_inputs=;
    • each target records cut_short_by_timeout in last-run.json, the per-target line says CUT_SHORT_BY_TIMEOUT after Ns, and FUZZ_DONE gains cut_short=N;
    • recheck/<target>/ is cleared before each re-check and removed after it.

J274's mutants (M1–M3), red on every commit from 2 onward (compute01 ~/.cache/codex-scratch/j264b/mut-c2, mut-c3, mut-c4; runner mutants.sh):

after 2 (bab8e2f94) after 3 (b124595aa) head (0fc6934ed)
baseline OK OK OK
M1 exit rule inverted FAILED (failures=3) FAILED (failures=4) FAILED (failures=4)
M2 re-check without -timeout FAILED (failures=1) FAILED (failures=1) FAILED (failures=1)
M3 re-check not wired (crashes = before) FAILED (failures=1) FAILED (failures=1) FAILED (failures=1)

Reds before each fix come from behaviour, not missing names:

  • Commit 3: its tests against commit 2's runner give FAILED (failures=3): False != True : 5000 and [] != ['timeout-slow'] (the 6 s input not filed).
  • Commit 4: its tests against commit 3's runner give FAILED (failures=2, errors=2):
    • 2 != 1 (new_inputs);
    • ['timeout-old'] != [] (stale recheck/);
    • a KeyError on the new cut_short_by_timeout field.

Real-environment proof (dev-02, one slot run, ~/.cache/codex-scratch/j264b/prove-r2.log):

  • Setup: the branch's own reproduces() with a real Fuzzer, i.e. real cargo fuzz run with the nightly toolchain and ASan.
  • Inputs: the four nightly timeout artifacts, plus a synthetic 6 s input. That input comes from a throwaway cargo-fuzz project whose only target sleeps 6 s on slow6.
RESULT release_channel/timeout-3f4bde2f…: reproduces=False -> set aside ['Executed … in 4 ms'] recheck_dir_left=False
RESULT remote_standby_decode/timeout-1c839c82…: reproduces=False -> set aside ['Executed … in 0 ms'] recheck_dir_left=False
RESULT remote_standby_decode/timeout-941a98f0…: reproduces=False -> set aside ['Executed … in 4 ms'] recheck_dir_left=False
RESULT ssh_auth_once_decode/timeout-da39a3ee…: reproduces=False -> set aside ['Executed … in 0 ms'] recheck_dir_left=False
RESULT slow/slow6: reproduces=True -> FILED ['Executed …/slow-proj/slow6 in 6001 ms'] recheck_dir_left=False

Python tests at head (compute01, py-r2-head.log): Ran 22 tests in 9.695s / OK.

Gate: scripts/verify quick through build-slot on compute01 at 0fc6934ed (verify-quick-r2.log), verbatim:

quick scope vs merge base with origin/dev: workspace=0 packages=[] tracer=0
+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| scope guard               | PASS         |
| migration names           | PASS         |
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
+---------------------------+--------------+

QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done.
VERIFY_EXIT=0

Only scripts changed, so quick scoped no Rust package; the Python suite above is the relevant test. As tasked, verify fast was not run.

Out of scope (J274's note): slow-unit-* artifacts are still filed without a re-check.

J264b round 2 finish: J274's runner findings (81439) are addressed on the new branch **`fuzz-runner-recheck`, head 0fc6934ede35b31f4313a497d377412ec47410b7**, based on origin/dev 431441d53. The parser fix is already merged as 431441d53. Nothing merged, deployed or closed. **Commits**, one per item, each pushed as it was made: 1. e57e2fb4f: d1450e1a0 carried over unchanged ("re-check timeouts in isolation before filing them"). 2. bab8e2f94 (**P2-1**): `Fuzzer` (injectable `run`), `recheck_flags`, `reproduces` and `run_target` are now module level, and `main()` calls `run_target` per harness. A `FakeLibFuzzer` plays cargo-fuzz in the tests. It covers: - the exit rule: 0 sets the input aside; 70, 77 and 1 file it; - the re-check command: it carries `-timeout=TIMEOUT_SECONDS` and the input, and no `-max_total_time`; - `run_target` end to end, in both directions. 3. b124595aa (**P2-2**): `reproduces` reads libFuzzer's `Executed <input> in N ms` from its own part of the log. It files the input on a non-zero exit, on **≥ 5000 ms** (`TIMEOUT_SECONDS*1000//2`), or when no time was reported. I did not use `-timeout=5`: libFuzzer's alarm only checks every timeout/2+1 s, so a unit just over 5 s could pass unflagged. 4. 0fc6934ed (**P3**): - set-aside timeouts no longer count in `new_inputs` or `FUZZ_DONE new_inputs=`; - each target records `cut_short_by_timeout` in last-run.json, the per-target line says `CUT_SHORT_BY_TIMEOUT after Ns`, and FUZZ_DONE gains `cut_short=N`; - `recheck/<target>/` is cleared before each re-check and removed after it. **J274's mutants (M1–M3), red on every commit from 2 onward** (compute01 `~/.cache/codex-scratch/j264b/mut-c2`, `mut-c3`, `mut-c4`; runner `mutants.sh`): | | after 2 (bab8e2f94) | after 3 (b124595aa) | head (0fc6934ed) | |---|---|---|---| | baseline | OK | OK | OK | | M1 exit rule inverted | FAILED (failures=3) | FAILED (failures=4) | FAILED (failures=4) | | M2 re-check without `-timeout` | FAILED (failures=1) | FAILED (failures=1) | FAILED (failures=1) | | M3 re-check not wired (`crashes = before`) | FAILED (failures=1) | FAILED (failures=1) | FAILED (failures=1) | **Reds before each fix come from behaviour, not missing names:** - **Commit 3:** its tests against commit 2's runner give `FAILED (failures=3)`: `False != True : 5000` and `[] != ['timeout-slow']` (the 6 s input not filed). - **Commit 4:** its tests against commit 3's runner give `FAILED (failures=2, errors=2)`: - `2 != 1` (new_inputs); - `['timeout-old'] != []` (stale recheck/); - a KeyError on the new `cut_short_by_timeout` field. **Real-environment proof** (dev-02, one slot run, `~/.cache/codex-scratch/j264b/prove-r2.log`): - **Setup:** the branch's own `reproduces()` with a real `Fuzzer`, i.e. real `cargo fuzz run` with the nightly toolchain and ASan. - **Inputs:** the four nightly timeout artifacts, plus a synthetic 6 s input. That input comes from a throwaway cargo-fuzz project whose only target sleeps 6 s on `slow6`. ``` RESULT release_channel/timeout-3f4bde2f…: reproduces=False -> set aside ['Executed … in 4 ms'] recheck_dir_left=False RESULT remote_standby_decode/timeout-1c839c82…: reproduces=False -> set aside ['Executed … in 0 ms'] recheck_dir_left=False RESULT remote_standby_decode/timeout-941a98f0…: reproduces=False -> set aside ['Executed … in 4 ms'] recheck_dir_left=False RESULT ssh_auth_once_decode/timeout-da39a3ee…: reproduces=False -> set aside ['Executed … in 0 ms'] recheck_dir_left=False RESULT slow/slow6: reproduces=True -> FILED ['Executed …/slow-proj/slow6 in 6001 ms'] recheck_dir_left=False ``` **Python tests at head** (compute01, `py-r2-head.log`): `Ran 22 tests in 9.695s` / `OK`. **Gate:** `scripts/verify quick` through build-slot on compute01 at 0fc6934ed (`verify-quick-r2.log`), verbatim: ``` quick scope vs merge base with origin/dev: workspace=0 packages=[] tracer=0 +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | scope guard | PASS | | migration names | PASS | | Nix Cargo vendoring | PASS | | fmt | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | +---------------------------+--------------+ QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done. VERIFY_EXIT=0 ``` Only scripts changed, so quick scoped no Rust package; the Python suite above is the relevant test. As tasked, `verify fast` was not run. **Out of scope (J274's note):** `slow-unit-*` artifacts are still filed without a re-check.
Author
Owner

J283 review of fuzz-runner-recheck @ 0fc6934ede (base dev 431441d53). Scope: the runner changes for #370, #374 and #358 that answer J274 (81439), as reported by J264b (81484).

Verdict: APPROVE

No P0, P1 or P2 findings. There are three P3 notes below; none blocks the merge. As the reviewer rule says, I did not run verify fast or verify quick. I read J264b's quick output in 81484.

1. Mutations: every mutant fails the suite

My own mutants, applied to a detached worktree at 0fc6934ed on compute01, each run with python3 scripts/test-nightly-fuzz.py. Evidence is in compute01 ~/.cache/codex-scratch/j283/mut/ (summary.txt, plus one .py and one .log per mutant).

mutant result tests that fail
baseline (head) OK none
M1 (J274): exit rule inverted, returncode == 0 FAILED (failures=4) exit_rule, files_a_slow_input_that_passes, run_target_files_a_timeout_that_recurs, run_target_sets_aside…
M2 (J274): no -timeout in the re-check FAILED (failures=1) recheck_runs_the_input_alone_under_the_time_limit
M3 (J274): not wired, crashes = before FAILED (failures=1) run_target_sets_aside_a_timeout_that_passes_alone
M5 (mine): threshold set to the full limit (10 000 ms) FAILED (failures=2) files_a_slow_input_that_passes, run_target_files_a_six_second_timeout
M6 (mine): parsed time ignored (returncode != 0 only) FAILED (failures=3) files_a_slow_input_that_passes, without_a_reported_time_files_the_input, run_target_files_a_six_second_timeout
M7 (mine): a missing time read as fast FAILED (failures=1) recheck_without_a_reported_time_files_the_input
M8 (mine): >= changed to > at 5000 ms FAILED (failures=1) recheck_files_a_slow_input_that_passes
M9 (mine): stale recheck/ not cleared first FAILED (failures=1) recheck_directory_starts_empty

The tests now fail on wrong behaviour, not on a missing name.

2. Can a real hang be dropped?

I called the branch's real reproduces() with a runner that writes chosen libFuzzer output and exit codes. Evidence: compute01 ~/.cache/codex-scratch/j283/edge-cases.log.

FILED      signal-killed child (rc -9), no line
FILED      signal-killed after Executed 1 ms (rc -9)
FILED      cargo-fuzz rc 1 after kill, line 0 ms
FILED      no Executed line, rc 0
FILED      malformed: no number
FILED      malformed: seconds unit
FILED      ASan text spliced into line, rc 0
FILED      ASan text before a clean 7000 ms line
FILED      Executed line for another path, 1 ms
SET ASIDE  loaded-host real 4.9 s input, quiet re-check
FILED      loaded re-check of same input reads 5 s+
  • Executed line missing or malformed: the input is filed, because executed_ms returns None. The regex needs this exact input path and an integer count of ms. Only the re-check's own part of the log is read, from the recorded offset, so a line from the fuzz run or from an earlier re-check can't match.
  • Re-check killed by a signal: the input is filed. Python's negative returncode, and cargo-fuzz's non-zero exit when its child dies, both give != 0. That holds even when the Executed line came first.
  • ASan output mixed into the time line: the input is filed. ASan only prints on an error, and an error exits non-zero. A line it breaks no longer matches, which gives None and files the input. The ms value must come right after in , so ASan text can't produce a false "fast" reading.
  • Loaded host, real 4.9 s input: this is set aside, and it's the only path that drops a slow input. For it to happen, the input must take under 5 s on its own but over 10 s during the fuzz run. The host would have to be more than twice as slow during the fuzz run as during a re-check seconds later. If the load lasts into the re-check, the input is filed.
    • I accept this risk. An input that runs in under 5 s alone is below the bar the runner defines. It is kept in unconfirmed/ and logged, and it shows in unconfirmed and cut_short_by_timeout in last-run.json. If fuzzing finds it again on a later loaded night, it is re-checked again, because its name is a content hash.
  • J264b's dev-02 proof (prove-r2.log) shows real cargo fuzz output in the form the parser expects: the four nightly artifacts ran in 0–4 ms and were set aside, and the 6001 ms input was filed. That is the real-binary half of the claim.

3. Crash inputs and slow-unit-*

  • report_crash and written_inputs are untouched in the diff. confirmed_inputs only acts on names that start with timeout-, and it keeps the sorted order. Crash inputs are therefore filed exactly as before. test_timeout_that_does_not_recur_is_not_filed asserts that crashes skip the re-check.
  • slow-unit-* doesn't match timeout-, so it is still filed directly, with no re-check. That is unchanged and out of scope, as J274 noted.

4. Python suite at head

compute01, ~/.cache/codex-scratch/j283/py-suite-head.log:

Ran 22 tests in 9.856s

OK
rc=0

P3 notes (none blocks the merge; a follow-up is fine)

  • P3-a: filed timeouts are re-checked every night. report_crash returns early on attached and leaves the artifact in artifacts/<target>/. Each filed real hang therefore costs a cargo-fuzz start plus about 10 s every night until someone removes it. That is bounded and small next to a 3600 s budget, but it grows with the number of open hangs. One fix is to skip the re-check when the report record already has attached.
  • P3-b: a half-filed timeout that passes its re-check never gets its attachment. If an earlier night's upload failed after the issue or comment was posted, and the input now runs fast, it moves to unconfirmed/ and the upload doesn't resume. Inputs up to 1 KiB are inline in the comment, so this only matters for larger ones. Rare.
  • P3-c: one line in the docstring is too broad. confirmed_inputs says "a re-check that cannot run at all counts as reproduced". That holds for a non-zero exit. An OSError raised by subprocess.run itself, for example a missing rustup, propagates and stops the night. The input stays in artifacts/, so nothing is dropped, but the sentence should say "exits non-zero".

Evidence: compute01 ~/.cache/codex-scratch/j283/ (mut/summary.txt, the per-mutant .py and .log files, edge-cases.log, py-suite-head.log). The VM worktree has been removed, and no build target was created. Nothing was merged, pushed or closed.

J283 review of **`fuzz-runner-recheck` @ 0fc6934ede35b31f4313a497d377412ec47410b7** (base dev 431441d53). Scope: the runner changes for #370, #374 and #358 that answer J274 (81439), as reported by J264b (81484). ## Verdict: APPROVE No P0, P1 or P2 findings. There are three P3 notes below; none blocks the merge. As the reviewer rule says, I did not run `verify fast` or `verify quick`. I read J264b's quick output in 81484. ## 1. Mutations: every mutant fails the suite My own mutants, applied to a detached worktree at 0fc6934ed on compute01, each run with `python3 scripts/test-nightly-fuzz.py`. Evidence is in compute01 `~/.cache/codex-scratch/j283/mut/` (`summary.txt`, plus one `.py` and one `.log` per mutant). | mutant | result | tests that fail | |---|---|---| | baseline (head) | OK | none | | M1 (J274): exit rule inverted, `returncode == 0` | FAILED (failures=4) | exit_rule, files_a_slow_input_that_passes, run_target_files_a_timeout_that_recurs, run_target_sets_aside… | | M2 (J274): no `-timeout` in the re-check | FAILED (failures=1) | recheck_runs_the_input_alone_under_the_time_limit | | M3 (J274): not wired, `crashes = before` | FAILED (failures=1) | run_target_sets_aside_a_timeout_that_passes_alone | | M5 (mine): threshold set to the full limit (10 000 ms) | FAILED (failures=2) | files_a_slow_input_that_passes, run_target_files_a_six_second_timeout | | M6 (mine): parsed time ignored (`returncode != 0` only) | FAILED (failures=3) | files_a_slow_input_that_passes, without_a_reported_time_files_the_input, run_target_files_a_six_second_timeout | | M7 (mine): a missing time read as fast | FAILED (failures=1) | recheck_without_a_reported_time_files_the_input | | M8 (mine): `>=` changed to `>` at 5000 ms | FAILED (failures=1) | recheck_files_a_slow_input_that_passes | | M9 (mine): stale `recheck/` not cleared first | FAILED (failures=1) | recheck_directory_starts_empty | The tests now fail on wrong behaviour, not on a missing name. ## 2. Can a real hang be dropped? I called the branch's real `reproduces()` with a runner that writes chosen libFuzzer output and exit codes. Evidence: compute01 `~/.cache/codex-scratch/j283/edge-cases.log`. ``` FILED signal-killed child (rc -9), no line FILED signal-killed after Executed 1 ms (rc -9) FILED cargo-fuzz rc 1 after kill, line 0 ms FILED no Executed line, rc 0 FILED malformed: no number FILED malformed: seconds unit FILED ASan text spliced into line, rc 0 FILED ASan text before a clean 7000 ms line FILED Executed line for another path, 1 ms SET ASIDE loaded-host real 4.9 s input, quiet re-check FILED loaded re-check of same input reads 5 s+ ``` - **`Executed` line missing or malformed:** the input is filed, because `executed_ms` returns None. The regex needs this exact input path and an integer count of ms. Only the re-check's own part of the log is read, from the recorded offset, so a line from the fuzz run or from an earlier re-check can't match. - **Re-check killed by a signal:** the input is filed. Python's negative returncode, and cargo-fuzz's non-zero exit when its child dies, both give `!= 0`. That holds even when the `Executed` line came first. - **ASan output mixed into the time line:** the input is filed. ASan only prints on an error, and an error exits non-zero. A line it breaks no longer matches, which gives None and files the input. The ms value must come right after `in `, so ASan text can't produce a false "fast" reading. - **Loaded host, real 4.9 s input:** this is set aside, and it's the only path that drops a slow input. For it to happen, the input must take under 5 s on its own but over 10 s during the fuzz run. The host would have to be more than twice as slow during the fuzz run as during a re-check seconds later. If the load lasts into the re-check, the input is filed. - **I accept this risk.** An input that runs in under 5 s alone is below the bar the runner defines. It is kept in `unconfirmed/` and logged, and it shows in `unconfirmed` and `cut_short_by_timeout` in last-run.json. If fuzzing finds it again on a later loaded night, it is re-checked again, because its name is a content hash. - J264b's dev-02 proof (`prove-r2.log`) shows real `cargo fuzz` output in the form the parser expects: the four nightly artifacts ran in 0–4 ms and were set aside, and the 6001 ms input was filed. That is the real-binary half of the claim. ## 3. Crash inputs and `slow-unit-*` - `report_crash` and `written_inputs` are untouched in the diff. `confirmed_inputs` only acts on names that start with `timeout-`, and it keeps the sorted order. Crash inputs are therefore filed exactly as before. `test_timeout_that_does_not_recur_is_not_filed` asserts that crashes skip the re-check. - `slow-unit-*` doesn't match `timeout-`, so it is still filed directly, with no re-check. That is unchanged and out of scope, as J274 noted. ## 4. Python suite at head compute01, `~/.cache/codex-scratch/j283/py-suite-head.log`: ``` Ran 22 tests in 9.856s OK rc=0 ``` ## P3 notes (none blocks the merge; a follow-up is fine) - **P3-a: filed timeouts are re-checked every night.** `report_crash` returns early on `attached` and leaves the artifact in `artifacts/<target>/`. Each filed real hang therefore costs a cargo-fuzz start plus about 10 s every night until someone removes it. That is bounded and small next to a 3600 s budget, but it grows with the number of open hangs. One fix is to skip the re-check when the report record already has `attached`. - **P3-b: a half-filed timeout that passes its re-check never gets its attachment.** If an earlier night's upload failed after the issue or comment was posted, and the input now runs fast, it moves to `unconfirmed/` and the upload doesn't resume. Inputs up to 1 KiB are inline in the comment, so this only matters for larger ones. Rare. - **P3-c: one line in the docstring is too broad.** `confirmed_inputs` says "a re-check that cannot run at all counts as reproduced". That holds for a non-zero exit. An `OSError` raised by `subprocess.run` itself, for example a missing `rustup`, propagates and stops the night. The input stays in `artifacts/`, so nothing is dropped, but the sentence should say "exits non-zero". --- Evidence: compute01 `~/.cache/codex-scratch/j283/` (`mut/summary.txt`, the per-mutant `.py` and `.log` files, `edge-cases.log`, `py-suite-head.log`). The VM worktree has been removed, and no build target was created. Nothing was merged, pushed or closed.
Author
Owner

Orchestrator: both #353 fixes are on dev.

#370, #374 and #358 stay open until the first nightly run on 864e66cd1 or later sets their four inputs aside. I'll close them then, citing the run.

J283's three P3s are not done yet:

  • an already-filed timeout is re-checked every night (about 10 s each);
  • an earlier failed attachment upload is never retried once the input runs fast;
  • the confirmed_inputs docstring overstates what happens on an OSError.
Orchestrator: both #353 fixes are on dev. - The parser fix 271913dc0 is merged as 431441d53. - The runner re-check (J264b, `fuzz-runner-recheck` 0fc6934ed) is merged as **864e66cd1**, after J283 approved it (81499). #370, #374 and #358 stay open until the first nightly run on 864e66cd1 or later sets their four inputs aside. I'll close them then, citing the run. J283's three P3s are not done yet: - an already-filed timeout is re-checked every night (about 10 s each); - an earlier failed attachment upload is never retried once the input runs fast; - the `confirmed_inputs` docstring overstates what happens on an OSError.
Author
Owner

Failing input for host_name_parse from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 346a490816e47e98ecf15a08234e806fbf057650c406096089ebde756fec4342.

Input (base64): MFguLi4uLi4uLi4uLi4uLi4uLgAAAAAwLTAwLTA=

Failing input for `host_name_parse` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `346a490816e47e98ecf15a08234e806fbf057650c406096089ebde756fec4342`. Input (base64): `MFguLi4uLi4uLi4uLi4uLi4uLgAAAAAwLTAwLTA=`
Author
Owner

J333 start: branch fuzz-host-name-353, base fba745051 (origin/dev). Plan: replay comment 82929 exact bytes with the real cargo-fuzz harness on current dev and 521cb8990 via compute01 build-slot; retain a unit regression and corpus seed; audit nightly source selection and fix stale selection separately; run quick and fast, push atomic commits. Estimate: 90 minutes including shared slot queues. No merge, deployment, or issue closure.

J333 start: branch `fuzz-host-name-353`, base `fba745051` (origin/dev). Plan: replay comment 82929 exact bytes with the real cargo-fuzz harness on current dev and 521cb8990 via compute01 build-slot; retain a unit regression and corpus seed; audit nightly source selection and fix stale selection separately; run quick and fast, push atomic commits. Estimate: 90 minutes including shared slot queues. No merge, deployment, or issue closure.
Author
Owner

J333b start, taking over from J333 (83059). J333 pushed nothing. Its worktree is clean at origin/dev, so I continue on the same branch.

  • Branch: fuzz-host-name-353, base fba745051 (origin/dev)
  • Input (82929): 29 bytes, 0X + 17×. + 4×NUL + 0-00-0. SHA-256 346a4908…4342 checked.

Plan (about 90 min):

  1. Replay the exact bytes with cargo fuzz run host_name_parse <file> on compute01 through build-slot, both on fba745051 and on 521cb8990.
  2. Expected result: it is the bare-0x class that 48d2388a7 fixed (merged as 431441d53). If so, add the input to a fuzz/corpus/host_name_parse seed and add a unit regression. If dev still fails, write a red test and then fix the parser.
  3. Fix the nightly runner's stale source in its own commit.
  4. verify quick, then verify fast, on compute01.

Finding 1, already confirmed: the nightly fuzzes a stale head. dev-02 /var/lib/infraternal-fuzz/checkout is at 521cb8990 (2026-11-05 10:45 UTC). The run that filed 82929 logged FUZZ_DONE targets=36 new_inputs=1 failed=False head=521cb8990… at 2026-11-06 21:31 UTC (journal). The cause: install-nightly-fuzz pins the checkout to --ref, and nothing moves it after install. So neither 431441d53 (parser) nor 864e66cd1 (runner re-check) has been fuzzed yet. #370, #374 and #358 can't be closed from a nightly run until this changes.

J333b start, taking over from J333 (83059). J333 pushed nothing. Its worktree is clean at origin/dev, so I continue on the same branch. - **Branch:** `fuzz-host-name-353`, **base** `fba745051` (origin/dev) - **Input (82929):** 29 bytes, `0X` + 17×`.` + 4×NUL + `0-00-0`. SHA-256 `346a4908…4342` checked. **Plan (about 90 min):** 1. Replay the exact bytes with `cargo fuzz run host_name_parse <file>` on compute01 through build-slot, both on fba745051 and on 521cb8990. 2. Expected result: it is the bare-`0x` class that 48d2388a7 fixed (merged as 431441d53). If so, add the input to a `fuzz/corpus/host_name_parse` seed and add a unit regression. If dev still fails, write a red test and then fix the parser. 3. Fix the nightly runner's stale source in its own commit. 4. `verify quick`, then `verify fast`, on compute01. **Finding 1, already confirmed: the nightly fuzzes a stale head.** dev-02 `/var/lib/infraternal-fuzz/checkout` is at `521cb8990` (2026-11-05 10:45 UTC). The run that filed 82929 logged `FUZZ_DONE targets=36 new_inputs=1 failed=False head=521cb8990…` at 2026-11-06 21:31 UTC (journal). The cause: `install-nightly-fuzz` pins the checkout to `--ref`, and nothing moves it after install. So neither 431441d53 (parser) nor 864e66cd1 (runner re-check) has been fuzzed yet. #370, #374 and #358 can't be closed from a nightly run until this changes.
Author
Owner

J333b, finding 1 fixed (stale nightly source): 661a4eb2d on fuzz-host-name-353, pushed.

Cause: install-nightly-fuzz detaches the fuzz account's checkout at --ref, and only a reinstall moves it (PROCESS §12). Nobody reinstalled after 431441d53 or 864e66cd1 merged, so dev-02 kept fuzzing 521cb8990.

Fix: scripts/nightly-fuzz-service, the root helper, takes its slot as before. Then it runs these steps as the fuzz account, through setpriv:

  1. git -C checkout fetch --quiet origin dev
  2. git -C checkout checkout --quiet --force --detach FETCH_HEAD
  3. the fuzz pass

If step 1 or 2 fails, the night exits 1 and fuzzes nothing, so no report can carry an old head. No repository code runs as root. The token still meets only reviewed code, because only the orchestrator merges into dev. The installer docstring and PROCESS §12 now say this.

Rejected:

  • Reinstalling on every merge. That's the process that failed.
  • Having nightly-fuzz.py update its own checkout. The running code would update itself.
  • Falling back to the old head when the fetch fails. That hides the staleness.

Red/green (compute01, plain python3 scripts/test-nightly-fuzz.py):

  • Red: the new tests against fba745051's helper (evidence/py-red.log).
    FAIL: test_a_failed_update_runs_no_fuzz_pass (failing_step=0)  AssertionError: 128 != 1
    FAIL: test_a_failed_update_runs_no_fuzz_pass (failing_step=1)  AssertionError: 0 != 1
    FAIL: test_every_night_fuzzes_the_current_dev_head             AssertionError: 1 != 3
    Ran 5 tests in 0.079s
    FAILED (failures=3)
    
  • Green: head 661a4eb2d (evidence/py-green.log).
    Ran 24 tests in 9.810s
    OK
    

Logs are under compute01 ~/.cache/codex-scratch/j333b/evidence/.

Needed after merge (orchestrator, not done by me): the root helper is a fixed copy, so this takes effect only after one reinstall on dev-02:

sudo scripts/install-nightly-fuzz --ref <merged dev head>

The token is already in place. After that, every night fuzzes the current dev head.

J333b, finding 1 fixed (stale nightly source): **661a4eb2d** on `fuzz-host-name-353`, pushed. **Cause:** `install-nightly-fuzz` detaches the fuzz account's checkout at `--ref`, and only a reinstall moves it (PROCESS §12). Nobody reinstalled after 431441d53 or 864e66cd1 merged, so dev-02 kept fuzzing 521cb8990. **Fix:** `scripts/nightly-fuzz-service`, the root helper, takes its slot as before. Then it runs these steps as the fuzz account, through setpriv: 1. `git -C checkout fetch --quiet origin dev` 2. `git -C checkout checkout --quiet --force --detach FETCH_HEAD` 3. the fuzz pass If step 1 or 2 fails, the night exits 1 and fuzzes nothing, so no report can carry an old head. No repository code runs as root. The token still meets only reviewed code, because only the orchestrator merges into dev. The installer docstring and PROCESS §12 now say this. **Rejected:** - Reinstalling on every merge. That's the process that failed. - Having `nightly-fuzz.py` update its own checkout. The running code would update itself. - Falling back to the old head when the fetch fails. That hides the staleness. **Red/green** (compute01, plain `python3 scripts/test-nightly-fuzz.py`): - Red: the new tests against fba745051's helper (`evidence/py-red.log`). ``` FAIL: test_a_failed_update_runs_no_fuzz_pass (failing_step=0) AssertionError: 128 != 1 FAIL: test_a_failed_update_runs_no_fuzz_pass (failing_step=1) AssertionError: 0 != 1 FAIL: test_every_night_fuzzes_the_current_dev_head AssertionError: 1 != 3 Ran 5 tests in 0.079s FAILED (failures=3) ``` - Green: head 661a4eb2d (`evidence/py-green.log`). ``` Ran 24 tests in 9.810s OK ``` Logs are under compute01 `~/.cache/codex-scratch/j333b/evidence/`. **Needed after merge (orchestrator, not done by me):** the root helper is a fixed copy, so this takes effect only after one reinstall on dev-02: ``` sudo scripts/install-nightly-fuzz --ref <merged dev head> ``` The token is already in place. After that, every night fuzzes the current dev head.
Author
Owner

J333b progress: both commits are pushed (head 661a4eb2d).

Two runs are waiting in compute01's build-slot queue: 5 slots busy, 20 runs queued.

  • the cargo-fuzz replay of 82929 on fba745051 and on 521cb8990: queue position 12, script ~/j333b/replay.sh, log ~/.cache/codex-scratch/j333b/evidence/replay.log;
  • the gate: unit red/green, then verify quick, then verify fast. Queue position 16, script ~/j333b/gate.sh, log evidence/gate.log.

I'm polling in the foreground. The results follow.

J333b progress: both commits are pushed (head 661a4eb2d). Two runs are waiting in compute01's build-slot queue: 5 slots busy, 20 runs queued. - the cargo-fuzz replay of 82929 on fba745051 and on 521cb8990: queue position 12, script `~/j333b/replay.sh`, log `~/.cache/codex-scratch/j333b/evidence/replay.log`; - the gate: unit red/green, then `verify quick`, then `verify fast`. Queue position 16, script `~/j333b/gate.sh`, log `evidence/gate.log`. I'm polling in the foreground. The results follow.
Author
Owner

J333b, finding 2: 82929 is already fixed on dev, by 48d2388a7 (merged as 431441d53). It was filed only because the nightly's checkout was stale (finding 1).

Replay with the real harness. compute01, one build-slot run (~/j333b/replay.sh). It uses the runner's own command shape:

  • rustup run nightly cargo fuzz run --fuzz-dir fuzz --features <union of required-features> host_name_parse <input> -- -timeout=10 -rss_limit_mb=2048;
  • ASan, CARGO_PROFILE_RELEASE_LTO=false, in the flake dev shell;
  • rustc 1.101.0-nightly (249c1e4e3 2026-11-05) and cargo-fuzz 0.13.2, the same cargo-fuzz version as the nightly.

The input file's SHA-256 is 346a4908…4342, as filed. Key lines from evidence/replay.log:

=== j333b-dev fba745051416c60a8523f0fe529c2aee6bbe0dd0
Running: /home/claude/.cache/codex-scratch/j333b/evidence/input-346a4908
Executed /home/claude/.cache/codex-scratch/j333b/evidence/input-346a4908 in 1 ms
REPLAY_RC 0 j333b-dev
=== j333b-521 521cb8990b720ce751466d2eb81e5881b496bfda
Running: /home/claude/.cache/codex-scratch/j333b/evidence/input-346a4908
thread '<unnamed>' (1807157) panicked at fuzz/fuzz_targets/host_name_parse.rs:24:5:
assertion `left == right` failed
  left: None
 right: Some("0x")
==1807157== ERROR: libFuzzer: deadly signal
SUMMARY: libFuzzer: deadly signal
REPLAY_RC 1 j333b-521

Cause on 521cb8990:

  • The host is 0X + 17×. + 4×NUL + 0-00-0.
  • It has more than 4 dot-separated parts, so it isn't an IPv4 spelling, and the first label 0x is returned.
  • On the second call, the old is_ipv4_spelling accepted a bare 0x as a hex number, so machine_name("0x") returned None.
  • This is the bare-0x class from J264b's 80602, cause 1. 48d2388a7 requires a hex part to have at least one digit.

Which commit fixed it:

  • git log 521cb8990..fba745051 -- host_name.rs fuzz_targets/host_name_parse.rs lists only 48d2388a7.
  • host_name.rs is unchanged from 48d2388a7 to dev.

Kept as a regression (7fea2187b):

  • the unit test fuzz_input_82929_from_353_is_idempotent, with the exact 29 bytes;
  • the seed fuzz/corpus/host_name_parse/issue-353-82929, which the nightly copies into its corpus.

Next: the unit red/green against the pre-48d2388a7 parser, then verify quick and verify fast. They're queued in one slot run (evidence/gate.log).

J333b, finding 2: **82929 is already fixed on dev, by 48d2388a7 (merged as 431441d53).** It was filed only because the nightly's checkout was stale (finding 1). **Replay with the real harness.** compute01, one build-slot run (`~/j333b/replay.sh`). It uses the runner's own command shape: - `rustup run nightly cargo fuzz run --fuzz-dir fuzz --features <union of required-features> host_name_parse <input> -- -timeout=10 -rss_limit_mb=2048`; - ASan, `CARGO_PROFILE_RELEASE_LTO=false`, in the flake dev shell; - rustc 1.101.0-nightly (249c1e4e3 2026-11-05) and cargo-fuzz 0.13.2, the same cargo-fuzz version as the nightly. The input file's SHA-256 is `346a4908…4342`, as filed. Key lines from `evidence/replay.log`: ``` === j333b-dev fba745051416c60a8523f0fe529c2aee6bbe0dd0 Running: /home/claude/.cache/codex-scratch/j333b/evidence/input-346a4908 Executed /home/claude/.cache/codex-scratch/j333b/evidence/input-346a4908 in 1 ms REPLAY_RC 0 j333b-dev === j333b-521 521cb8990b720ce751466d2eb81e5881b496bfda Running: /home/claude/.cache/codex-scratch/j333b/evidence/input-346a4908 thread '<unnamed>' (1807157) panicked at fuzz/fuzz_targets/host_name_parse.rs:24:5: assertion `left == right` failed left: None right: Some("0x") ==1807157== ERROR: libFuzzer: deadly signal SUMMARY: libFuzzer: deadly signal REPLAY_RC 1 j333b-521 ``` **Cause on 521cb8990:** - The host is `0X` + 17×`.` + 4×NUL + `0-00-0`. - It has more than 4 dot-separated parts, so it isn't an IPv4 spelling, and the first label `0x` is returned. - On the second call, the old `is_ipv4_spelling` accepted a bare `0x` as a hex number, so `machine_name("0x")` returned `None`. - This is the bare-`0x` class from J264b's 80602, cause 1. 48d2388a7 requires a hex part to have at least one digit. **Which commit fixed it:** - `git log 521cb8990..fba745051 -- host_name.rs fuzz_targets/host_name_parse.rs` lists only 48d2388a7. - `host_name.rs` is unchanged from 48d2388a7 to dev. **Kept as a regression** (7fea2187b): - the unit test `fuzz_input_82929_from_353_is_idempotent`, with the exact 29 bytes; - the seed `fuzz/corpus/host_name_parse/issue-353-82929`, which the nightly copies into its corpus. Next: the unit red/green against the pre-48d2388a7 parser, then `verify quick` and `verify fast`. They're queued in one slot run (`evidence/gate.log`).
Author
Owner

J333b, gate progress at head 661a4eb2d. Everything ran on compute01 in one build-slot run (~/j333b/gate.sh, logs in ~/.cache/codex-scratch/j333b/evidence/).

Unit red/green for the 82929 test:

  • Red (unit-red.log): this branch's tests against host_name.rs from 48d2388a7^, the parser before the bare-0x fix:
    test host_name::tests::fuzz_input_82929_from_353_is_idempotent ... FAILED
      left: None
     right: Some("0x")
    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 75 filtered out
    
  • Green (unit-green.log, cargo test -p infraternal-plugin-api --lib host_name): test result: ok. 10 passed; 0 failed.

verify quick (verify-quick.log): passed, all 10 rows PASS.

verify fast (verify-fast.log): exit 1, with two rows failing, verbatim:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| migration names           | PASS         |
| migration tooling tests   | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| Traefik component manifest | PASS         |
| release component publication | PASS         |
| test                      | FAIL (101)   |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| privileged integration    | PASS         |
| probe release exclusion   | FAIL (1)     |
| nightly fuzz reporter     | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

Neither failure involves this diff. The diff is one plugin-api unit test, a seed file, .gitattributes, the fuzz scripts and docs.

probe release exclusion: the shim's native build hit the disk limit:

fatal error: error in backend: IO failure on output stream: No space left on device

compute01 then had about 45G free under heavy shared load.

test: one test failed, agent_daemon::real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness; the other 10 in that file passed. terminal.login.revoke returned resource_unavailable (agent_fixture.py:702). No Rust code that test builds differs from origin/dev: the only Rust change is a #[cfg(test)] test in plugin-api.

Rerun queued (position 24, ~/j333b/rerun.sh, evidence/rerun.log), in the same target dir:

  • that test 3 times, then all of agent_daemon;
  • scripts/test-runtime-probe-gating.py.

Static builds PASS. No shipped code changed, so the delta against origin/dev is 0 by construction. The row's own figures are against the old D243 baseline:

  • x86_64: 27 787 280 B (+5 369 024)
  • aarch64: 25 000 776 B (+4 547 496)
J333b, gate progress at head **661a4eb2d**. Everything ran on compute01 in one build-slot run (`~/j333b/gate.sh`, logs in `~/.cache/codex-scratch/j333b/evidence/`). **Unit red/green for the 82929 test:** - **Red** (`unit-red.log`): this branch's tests against `host_name.rs` from `48d2388a7^`, the parser before the bare-`0x` fix: ``` test host_name::tests::fuzz_input_82929_from_353_is_idempotent ... FAILED left: None right: Some("0x") test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 75 filtered out ``` - **Green** (`unit-green.log`, `cargo test -p infraternal-plugin-api --lib host_name`): `test result: ok. 10 passed; 0 failed`. **`verify quick`** (`verify-quick.log`): passed, all 10 rows PASS. **`verify fast`** (`verify-fast.log`): exit 1, with two rows failing, verbatim: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | Nix Cargo vendoring | PASS | | fmt | PASS | | migration names | PASS | | migration tooling tests | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | Traefik component manifest | PASS | | release component publication | PASS | | test | FAIL (101) | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | privileged integration | PASS | | probe release exclusion | FAIL (1) | | nightly fuzz reporter | PASS | | static builds | PASS | +---------------------------+--------------+ ``` Neither failure involves this diff. The diff is one plugin-api unit test, a seed file, `.gitattributes`, the fuzz scripts and docs. **`probe release exclusion`:** the shim's native build hit the disk limit: ``` fatal error: error in backend: IO failure on output stream: No space left on device ``` compute01 then had about 45G free under heavy shared load. **`test`:** one test failed, `agent_daemon::real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness`; the other 10 in that file passed. `terminal.login.revoke` returned `resource_unavailable` (`agent_fixture.py:702`). No Rust code that test builds differs from origin/dev: the only Rust change is a `#[cfg(test)]` test in plugin-api. **Rerun queued** (position 24, `~/j333b/rerun.sh`, `evidence/rerun.log`), in the same target dir: - that test 3 times, then all of `agent_daemon`; - `scripts/test-runtime-probe-gating.py`. **Static builds PASS.** No shipped code changed, so the delta against origin/dev is 0 by construction. The row's own figures are against the old D243 baseline: - x86_64: 27 787 280 B (+5 369 024) - aarch64: 25 000 776 B (+4 547 496)
Author
Owner

J333b finish. Branch fuzz-host-name-353, head 661a4eb2d208299052dfac2abff98abd3816ea3f, base origin/dev fba745051, which hasn't moved since. I have not merged, deployed or closed anything.

Commits:

  • 7fea2187b test(plugin-api): keeps 82929 as a unit test and a seed in fuzz/corpus/host_name_parse/.
  • 661a4eb2d fix(fuzz): every night fuzzes the current dev head, not the install's pinned commit.

Outcome:

  • The parser needs no change. 82929 fails on 521cb8990 and passes on dev under the real cargo-fuzz harness (83216). It was fixed by 48d2388a7, merged as 431441d53.
  • The nightly fuzzed a stale head. The fix is 661a4eb2d (83087). It takes effect after one reinstall on dev-02, see below.

Gate, scripts/verify quick, compute01 through build-slot (evidence/verify-quick.log), verbatim:

quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0
verify quick scope: fuzz packages are excluded from the test run; PASS
+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| scope guard               | PASS         |
| migration names           | PASS         |
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| clippy (workspace)        | PASS         |
| unit tests (changed)      | PASS         |
+---------------------------+--------------+

QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done.

Gate, scripts/verify fast, compute01 through build-slot (evidence/verify-fast.log), verbatim:

+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| migration names           | PASS         |
| migration tooling tests   | PASS         |
| VT native fmt             | PASS         |
| CLI plugin compile-out    | PASS         |
| clippy                    | PASS         |
| SSH runtime test helper   | PASS         |
| real-binary fixture paths | PASS         |
| Traefik component manifest | PASS         |
| release component publication | PASS         |
| test                      | FAIL (101)   |
| SSH inode reuse fixture   | PASS         |
| shim process-per-user     | PASS         |
| cargo-deny                | PASS         |
| gate architecture         | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| privileged integration    | PASS         |
| probe release exclusion   | FAIL (1)     |
| nightly fuzz reporter     | PASS         |
| static builds             | PASS         |
+---------------------------+--------------+

The two FAIL rows don't come from this branch. I reran them in the same target dir, at the same head, one slot run (evidence/rerun.log), verbatim:

TEST_RC run1 0 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.59s
TEST_RC run2 0 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.43s
TEST_RC run3 0 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.31s
AGENT_DAEMON_ALL_RC 0 test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 34.12s
PROBE_RC 0 df:   36G
  • probe release exclusion: the first run hit No space left on device in the shim's native build. The rerun prints runtime_probe import rejected (E0432) and probe consumer compiled (rerun-probe.log).
  • test: the failure was a one-off resource_unavailable in real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness, while compute01 was heavily loaded. On rerun it passed 3 of 3, and all 11 agent_daemon tests passed. That binary contains no code from this branch.

Static size: no shipped code changed, so the binary is unchanged from origin/dev. The row's own report-only figures are against the old D243 baseline:

  • x86_64: 27 787 280 B (+5 369 024)
  • aarch64: 25 000 776 B (+4 547 496)

Real-environment evidence:

  • Replay: the actual cargo fuzz run host_name_parse with ASan and the runner's flags. On fba745051: Executed … in 1 ms, rc 0. On 521cb8990: the host_name_parse.rs:24 panic, left: None, right: Some("0x"), rc 1.
  • Live install read: I read the live install on dev-02 as root, read-only. It's at 521cb8990, and the 20:32 UTC night's journal logged head=521cb8990….
  • Not done: I did not run the new service helper on dev-02. Installing it is a root deploy step and is left to the orchestrator.

Evidence is on compute01 under ~/.cache/codex-scratch/j333b/evidence/:

  • replay: replay.log, input input-346a4908;
  • unit red/green: unit-red.log, unit-green.log;
  • runner red/green: py-red.log, py-green.log;
  • gates: verify-quick.log, verify-fast.log;
  • reruns: rerun*.log.

The target dirs and the scratch toolchain are deleted, and the VM worktrees are removed.

For the orchestrator after merge:

  1. Reinstall once on dev-02 so the new root helper is in place: sudo scripts/install-nightly-fuzz --ref <merged dev head> from the meshtest account. The token is already there. Until then, every night still fuzzes 521cb8990 and keeps refiling inputs that are already fixed.
  2. The first night after that also runs 864e66cd1's timeout re-check for the first time, so #370, #374 and #358 can then be judged from that night's run.

For kayg: 661a4eb2d changes PROCESS §12. Before, moving the checkout needed an explicit reinstall. Now it follows dev. The token still meets only reviewed code, because only the orchestrator merges into dev. It's reversible: revert the commit and reinstall.

J333b finish. **Branch `fuzz-host-name-353`, head `661a4eb2d208299052dfac2abff98abd3816ea3f`**, base origin/dev `fba745051`, which hasn't moved since. I have not merged, deployed or closed anything. **Commits:** - `7fea2187b` test(plugin-api): keeps 82929 as a unit test and a seed in `fuzz/corpus/host_name_parse/`. - `661a4eb2d` fix(fuzz): every night fuzzes the current dev head, not the install's pinned commit. **Outcome:** - **The parser needs no change.** 82929 fails on 521cb8990 and passes on dev under the real cargo-fuzz harness (83216). It was fixed by **48d2388a7, merged as 431441d53**. - **The nightly fuzzed a stale head.** The fix is 661a4eb2d (83087). It takes effect after one reinstall on dev-02, see below. **Gate, `scripts/verify quick`**, compute01 through build-slot (`evidence/verify-quick.log`), verbatim: ``` quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0 verify quick scope: fuzz packages are excluded from the test run; PASS +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | scope guard | PASS | | migration names | PASS | | Nix Cargo vendoring | PASS | | fmt | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | clippy (workspace) | PASS | | unit tests (changed) | PASS | +---------------------------+--------------+ QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done. ``` **Gate, `scripts/verify fast`**, compute01 through build-slot (`evidence/verify-fast.log`), verbatim: ``` +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | Nix Cargo vendoring | PASS | | fmt | PASS | | migration names | PASS | | migration tooling tests | PASS | | VT native fmt | PASS | | CLI plugin compile-out | PASS | | clippy | PASS | | SSH runtime test helper | PASS | | real-binary fixture paths | PASS | | Traefik component manifest | PASS | | release component publication | PASS | | test | FAIL (101) | | SSH inode reuse fixture | PASS | | shim process-per-user | PASS | | cargo-deny | PASS | | gate architecture | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | privileged integration | PASS | | probe release exclusion | FAIL (1) | | nightly fuzz reporter | PASS | | static builds | PASS | +---------------------------+--------------+ ``` **The two FAIL rows don't come from this branch.** I reran them in the same target dir, at the same head, one slot run (`evidence/rerun.log`), verbatim: ``` TEST_RC run1 0 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.59s TEST_RC run2 0 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.43s TEST_RC run3 0 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 10 filtered out; finished in 0.31s AGENT_DAEMON_ALL_RC 0 test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 34.12s PROBE_RC 0 df: 36G ``` - **`probe release exclusion`:** the first run hit `No space left on device` in the shim's native build. The rerun prints `runtime_probe import rejected (E0432)` and `probe consumer compiled` (`rerun-probe.log`). - **`test`:** the failure was a one-off `resource_unavailable` in `real_binary_rootless_cleanup_survives_sender_disconnect_before_readiness`, while compute01 was heavily loaded. On rerun it passed 3 of 3, and all 11 `agent_daemon` tests passed. That binary contains no code from this branch. **Static size:** no shipped code changed, so the binary is unchanged from origin/dev. The row's own report-only figures are against the old D243 baseline: - x86_64: 27 787 280 B (+5 369 024) - aarch64: 25 000 776 B (+4 547 496) **Real-environment evidence:** - **Replay:** the actual `cargo fuzz run host_name_parse` with ASan and the runner's flags. On fba745051: `Executed … in 1 ms`, rc 0. On 521cb8990: the `host_name_parse.rs:24` panic, `left: None, right: Some("0x")`, rc 1. - **Live install read:** I read the live install on dev-02 as root, read-only. It's at 521cb8990, and the 20:32 UTC night's journal logged `head=521cb8990…`. - **Not done:** I did not run the new service helper on dev-02. Installing it is a root deploy step and is left to the orchestrator. **Evidence** is on compute01 under `~/.cache/codex-scratch/j333b/evidence/`: - replay: `replay.log`, input `input-346a4908`; - unit red/green: `unit-red.log`, `unit-green.log`; - runner red/green: `py-red.log`, `py-green.log`; - gates: `verify-quick.log`, `verify-fast.log`; - reruns: `rerun*.log`. The target dirs and the scratch toolchain are deleted, and the VM worktrees are removed. **For the orchestrator after merge:** 1. Reinstall once on dev-02 so the new root helper is in place: `sudo scripts/install-nightly-fuzz --ref <merged dev head>` from the meshtest account. The token is already there. Until then, every night still fuzzes 521cb8990 and keeps refiling inputs that are already fixed. 2. The first night after that also runs 864e66cd1's timeout re-check for the first time, so #370, #374 and #358 can then be judged from that night's run. **For kayg:** 661a4eb2d changes PROCESS §12. Before, moving the checkout needed an explicit reinstall. Now it follows `dev`. The token still meets only reviewed code, because only the orchestrator merges into dev. It's reversible: revert the commit and reinstall.
Author
Owner

Failing input for host_name_parse from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 015944122b528c26764dea3d7064192557fdba221a381309c3afc49cf98c715e.

Input (base64): MFgwLgEOMC03Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3

Failing input for `host_name_parse` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `015944122b528c26764dea3d7064192557fdba221a381309c3afc49cf98c715e`. Input (base64): `MFgwLgEOMC03Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3`
Author
Owner

Failing input for host_name_parse from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 5f9c779661bee94681587b03132268287f3291a7baacc55280f2a045ca94a4e1.

Input (base64): MFguMVg=

Failing input for `host_name_parse` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `5f9c779661bee94681587b03132268287f3291a7baacc55280f2a045ca94a4e1`. Input (base64): `MFguMVg=`
Author
Owner

Failing input for host_name_parse from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 44753430920fe8f545d4814cabaab724f2add90547a0bf788b3d8758361b2114.

Input (base64): MFguWA==

Failing input for `host_name_parse` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `44753430920fe8f545d4814cabaab724f2add90547a0bf788b3d8758361b2114`. Input (base64): `MFguWA==`
Sign in to join this conversation.
No labels
launch-qa
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/infraternal#353
No description provided.