fuzz: release_channel found a failing input #374

Open
opened 2026-11-05 22:43:58 +00:00 by kayg · 5 comments
Owner

Nightly parser fuzzing (D351, #344).

Target: release_channel
Source: 521cb8990b720ce751466d2eb81e5881b496bfda
Input SHA-256: 33b121d4175378860540301124f495d0079a0d38efb8f5efbb4f827234d930a0

Input (base64): ICB7ImNoYW5uZWwiIAkKIDo0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NC4QIl0=

Reproduce with cargo +nightly fuzz run release_channel <input>, after unzipping the input attached to the report comment.

Nightly parser fuzzing (D351, #344). Target: `release_channel` Source: `521cb8990b720ce751466d2eb81e5881b496bfda` Input SHA-256: `33b121d4175378860540301124f495d0079a0d38efb8f5efbb4f827234d930a0` Input (base64): `ICB7ImNoYW5uZWwiIAkKIDo0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NC4QIl0=` Reproduce with `cargo +nightly fuzz run release_channel <input>`, after unzipping the input attached to the report comment. <!-- infraternal-fuzz-target: release_channel -->
Author
Owner

Failing input for release_channel from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 33b121d4175378860540301124f495d0079a0d38efb8f5efbb4f827234d930a0.

Input (base64): ICB7ImNoYW5uZWwiIAkKIDo0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NC4QIl0=

Failing input for `release_channel` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `33b121d4175378860540301124f495d0079a0d38efb8f5efbb4f827234d930a0`. Input (base64): `ICB7ImNoYW5uZWwiIAkKIDo0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NC4QIl0=`
Author
Owner

J264b triage (on fuzz-fixes-j264, together with #370): this is a timeout from host load, not a parser bug. No parser change is needed. The runner fix from #370 (d1450e1a0) covers it.

What the nightly saw (dev-02, /var/lib/infraternal-fuzz/.local/state/infraternal-fuzz/release_channel.log, run of 521cb8990):

INFO:     1906 files found in .../corpus/release_channel
ALARM: working on the last Unit for 21 seconds
MS: 0 ; base unit: 0000000000000000000000000000000000000000
... Test unit written to .../artifacts/release_channel/timeout-3f4bde2f461a127553d5d33afa822c77afc17d74
==2037859== ERROR: libFuzzer: timeout after 21 seconds

The failure type is a timeout. The alarm fired while libFuzzer was replaying the seed corpus (MS: 0, before INITED), which is the same pattern as #370.

Replay of the 41-byte input (SHA-256 33b121d4…) on the nightly's own release_channel binary on dev-02:

  • -timeout=10 <input>: Executed in 1 ms / 0 ms / 1 ms, rc=0 three times out of three (wall time 41–53 ms including process start)
  • 100 000 mutation runs seeded from it: Done 100000 runs in 1 second(s)
  • while the replay ran, dev-02 was at /proc/pressure/io some avg10=98%

Evidence: /tmp/j264b/374-358-repro.txt on the control host. A per-input run through the runner's own confirmed_inputs() re-check, with the exact cargo fuzz run command main() builds, and a 600 s release_channel run are queued in one slot run on dev-02 (~/.cache/codex-scratch/j264b/fuzz-run2.log).

J264b triage (on `fuzz-fixes-j264`, together with #370): **this is a timeout from host load, not a parser bug.** No parser change is needed. The runner fix from #370 (d1450e1a0) covers it. **What the nightly saw** (dev-02, `/var/lib/infraternal-fuzz/.local/state/infraternal-fuzz/release_channel.log`, run of 521cb8990): ``` INFO: 1906 files found in .../corpus/release_channel ALARM: working on the last Unit for 21 seconds MS: 0 ; base unit: 0000000000000000000000000000000000000000 ... Test unit written to .../artifacts/release_channel/timeout-3f4bde2f461a127553d5d33afa822c77afc17d74 ==2037859== ERROR: libFuzzer: timeout after 21 seconds ``` The failure type is a **timeout**. The alarm fired while libFuzzer was replaying the seed corpus (`MS: 0`, before `INITED`), which is the same pattern as #370. **Replay of the 41-byte input** (SHA-256 33b121d4…) on the nightly's own `release_channel` binary on dev-02: - `-timeout=10 <input>`: `Executed in 1 ms / 0 ms / 1 ms`, rc=0 three times out of three (wall time 41–53 ms including process start) - 100 000 mutation runs seeded from it: `Done 100000 runs in 1 second(s)` - while the replay ran, dev-02 was at `/proc/pressure/io some avg10=98%` Evidence: `/tmp/j264b/374-358-repro.txt` on the control host. A per-input run through the runner's own `confirmed_inputs()` re-check, with the exact `cargo fuzz run` command `main()` builds, and a 600 s `release_channel` run are queued in one slot run on dev-02 (`~/.cache/codex-scratch/j264b/fuzz-run2.log`).
Author
Owner

J264b finish.

Branch fuzz-fixes-j264, head d1450e1a0b. It has two commits on base 159edcedc:

  • 271913dc0 fix(plugin-api): machine_name refuses numeric first labels, accepts bare 0x (#353)
  • d1450e1a0 fix(fuzz): re-check timeouts in isolation before filing them (#370). This also covers #374 and #358.

I have not merged, deployed or closed anything.

Gate: scripts/verify quick through build-slot on compute01 at d1450e1a0 (~/.cache/codex-scratch/j264b/verify-quick2.log), verbatim:

quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0
+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| scope guard               | PASS         |
| migration names           | PASS         |
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| clippy (workspace)        | PASS         |
| unit tests (changed)      | PASS         |
+---------------------------+--------------+

QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done.
VERIFY_EXIT=0
  • An earlier quick run at the pre-amend head b1e8d164e failed only on fmt (one assert_eq! in the #353 test). I folded the fix into the #353 commit and force-with-lease pushed the branch, so each commit passes the gate.
  • quick scoped only infraternal-plugin-api. The runner's Python tests (the nightly fuzz reporter row, in fast) passed separately on compute01: Ran 13 tests ... OK (370-green.log). The scripts are byte-identical between b1e8d164e and d1450e1a0.
  • As tasked, I did not run scripts/verify fast.

Fuzz runs: one batched build on dev-02, with the nightly's own toolchain (rustc 1.101.0-nightly 708c94ef0, cargo-fuzz 0.13.2, ASan) and its flags (-timeout=10 -rss_limit_mb=2048 -max_len=1048580), seeded from copies of the nightly corpora. 600 s per target:

target executions rc artifacts
host_name_parse 102 384 284 in 601 s 0 none
ssh_auth_once_decode 13 828 284 in 601 s 0 none
release_channel 57 369 130 in 601 s 0 none
remote_standby_decode 10 143 408 in 601 s 0 none

Logs on dev-02: ~/.cache/codex-scratch/j264b/fuzz-run1.log, fuzz-run2.log and fuzz-<target>.log.

Runner re-check, per input (fuzz-run2.log). I called the runner's own confirmed_inputs() with the exact cargo fuzz run <target> <input> -- -timeout=10 -rss_limit_mb=2048 -artifact_prefix=... command that main() builds, on copies of the nightly's four timeout artifacts:

re-check release_channel/timeout-3f4bde2f…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check remote_standby_decode/timeout-1c839c82…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check remote_standby_decode/timeout-941a98f0…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check ssh_auth_once_decode/timeout-da39a3ee…: rc=0 Executed … in 0 ms  -> set aside (not filed)

On the fixed build, the four #353 inputs replay with rc=0 (Executed … in 31/5/1/5 ms).

Once this merges and the nightly checkout moves to it, its first run re-checks the four timeout artifacts still in the nightly's artifact directory and moves them to unconfirmed/. These issues can then be closed by the owner.

J264b finish. **Branch** `fuzz-fixes-j264`, **head d1450e1a0bc234a2acf8fc346c6cb521f06eebb9**. It has two commits on base 159edcedc: - 271913dc0 `fix(plugin-api): machine_name refuses numeric first labels, accepts bare 0x (#353)` - d1450e1a0 `fix(fuzz): re-check timeouts in isolation before filing them (#370)`. This also covers #374 and #358. I have not merged, deployed or closed anything. **Gate:** `scripts/verify quick` through build-slot on compute01 at d1450e1a0 (`~/.cache/codex-scratch/j264b/verify-quick2.log`), verbatim: ``` quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0 +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | scope guard | PASS | | migration names | PASS | | Nix Cargo vendoring | PASS | | fmt | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | clippy (workspace) | PASS | | unit tests (changed) | PASS | +---------------------------+--------------+ QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done. VERIFY_EXIT=0 ``` - An earlier quick run at the pre-amend head b1e8d164e failed only on `fmt` (one `assert_eq!` in the #353 test). I folded the fix into the #353 commit and force-with-lease pushed the branch, so each commit passes the gate. - `quick` scoped only `infraternal-plugin-api`. The runner's Python tests (the `nightly fuzz reporter` row, in `fast`) passed separately on compute01: `Ran 13 tests ... OK` (`370-green.log`). The scripts are byte-identical between b1e8d164e and d1450e1a0. - As tasked, I did **not** run `scripts/verify fast`. **Fuzz runs:** one batched build on dev-02, with the nightly's own toolchain (rustc 1.101.0-nightly 708c94ef0, cargo-fuzz 0.13.2, ASan) and its flags (`-timeout=10 -rss_limit_mb=2048 -max_len=1048580`), seeded from copies of the nightly corpora. 600 s per target: | target | executions | rc | artifacts | |---|---|---|---| | host_name_parse | 102 384 284 in 601 s | 0 | none | | ssh_auth_once_decode | 13 828 284 in 601 s | 0 | none | | release_channel | 57 369 130 in 601 s | 0 | none | | remote_standby_decode | 10 143 408 in 601 s | 0 | none | Logs on dev-02: `~/.cache/codex-scratch/j264b/fuzz-run1.log`, `fuzz-run2.log` and `fuzz-<target>.log`. **Runner re-check, per input** (`fuzz-run2.log`). I called the runner's own `confirmed_inputs()` with the exact `cargo fuzz run <target> <input> -- -timeout=10 -rss_limit_mb=2048 -artifact_prefix=...` command that `main()` builds, on copies of the nightly's four timeout artifacts: ``` re-check release_channel/timeout-3f4bde2f…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check remote_standby_decode/timeout-1c839c82…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check remote_standby_decode/timeout-941a98f0…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check ssh_auth_once_decode/timeout-da39a3ee…: rc=0 Executed … in 0 ms -> set aside (not filed) ``` On the fixed build, the four #353 inputs replay with rc=0 (`Executed … in 31/5/1/5 ms`). **Once this merges and the nightly checkout moves to it**, its first run re-checks the four timeout artifacts still in the nightly's artifact directory and moves them to `unconfirmed/`. These issues can then be closed by the owner.
Author
Owner

J274 review of fuzz-fixes-j264 @ d1450e1a0: REQUEST CHANGES on the runner commit (P2: the tests do not cover the re-check exit-code rule; P2: an input that is slow but under the 10 s bar gets set aside). The parser commit is approved. Full review: #353 (comment)

J274 review of `fuzz-fixes-j264` @ d1450e1a0: **REQUEST CHANGES** on the runner commit (P2: the tests do not cover the re-check exit-code rule; P2: an input that is slow but under the 10 s bar gets set aside). The parser commit is approved. Full review: https://git.kayg.org/kayg/infraternal/issues/353#issuecomment-81439
Author
Owner

J264b round 2: the runner fixes for this issue are on fuzz-runner-recheck, head 0fc6934ede. Details, mutation reds, the real-input proof (this issue's input set aside; a synthetic 6 s input filed) and the gate are in #353 (comment 81484): #353 (comment)

J264b round 2: the runner fixes for this issue are on `fuzz-runner-recheck`, head 0fc6934ede35b31f4313a497d377412ec47410b7. Details, mutation reds, the real-input proof (this issue's input set aside; a synthetic 6 s input filed) and the gate are in #353 (comment 81484): https://git.kayg.org/kayg/infraternal/issues/353#issuecomment-81484
Sign in to join this conversation.
No labels
launch-qa
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
kayg/infraternal#374
No description provided.