fuzz: remote_standby_decode found a failing input

kayg commented

2026-11-04 21:51:26 +00:00

Owner

Nightly parser fuzzing (D351, #344).

Target: remote_standby_decode
Source: 100bd91f6dd966b55409f0dd53dd9b3d468c359b
Input SHA-256: 9f092ab21fff82b0ead592c73bfa2e1a65812f528c6c3f9781bf41587dd562da

Input (base64): eyJwcmltZSI6eyL/913/////d///XG4uKf/////E/1xu

Reproduce with cargo +nightly fuzz run remote_standby_decode <input>, after unzipping the input attached to the report comment.

Nightly parser fuzzing (D351, #344). Target: `remote_standby_decode` Source: `100bd91f6dd966b55409f0dd53dd9b3d468c359b` Input SHA-256: `9f092ab21fff82b0ead592c73bfa2e1a65812f528c6c3f9781bf41587dd562da` Input (base64): `eyJwcmltZSI6eyL/913/////d///XG4uKf/////E/1xu` Reproduce with `cargo +nightly fuzz run remote_standby_decode <input>`, after unzipping the input attached to the report comment.

kayg commented

2026-11-04 21:51:26 +00:00

Author

Owner

Failing input for remote_standby_decode from source 100bd91f6dd966b55409f0dd53dd9b3d468c359b, SHA-256 9f092ab21fff82b0ead592c73bfa2e1a65812f528c6c3f9781bf41587dd562da.

Input (base64): eyJwcmltZSI6eyL/913/////d///XG4uKf/////E/1xu

Failing input for `remote_standby_decode` from source `100bd91f6dd966b55409f0dd53dd9b3d468c359b`, SHA-256 `9f092ab21fff82b0ead592c73bfa2e1a65812f528c6c3f9781bf41587dd562da`. Input (base64): `eyJwcmltZSI6eyL/913/////d///XG4uKf/////E/1xu`

remote_standby_decode-9f092ab21fff82b0ead592c73bfa2e1a65812f528c6c3f9781bf41587dd562da.zip

300 B

kayg referenced this issue

2026-11-05 06:03:02 +00:00

Engineering practices proposal #344

kayg commented

2026-11-05 22:50:06 +00:00

Author

Owner

Failing input for remote_standby_decode from source 521cb8990b720ce751466d2eb81e5881b496bfda, SHA-256 5ecc1bed894117ea1ec9b17f8467077f31d042a440a1f19c9abcea21d26ab210.

Input (base64): DQ0NDQ0NeyJwcmltZSIKCQkJCQkJDQ06DQ0NDQ0KeyIyXA0N/XIK//////8JCg==

Failing input for `remote_standby_decode` from source `521cb8990b720ce751466d2eb81e5881b496bfda`, SHA-256 `5ecc1bed894117ea1ec9b17f8467077f31d042a440a1f19c9abcea21d26ab210`. Input (base64): `DQ0NDQ0NeyJwcmltZSIKCQkJCQkJDQ06DQ0NDQ0KeyIyXA0N/XIK//////8JCg==`

remote_standby_decode-5ecc1bed894117ea1ec9b17f8467077f31d042a440a1f19c9abcea21d26ab210.zip

306 B

kayg commented

2026-11-05 23:40:57 +00:00

Author

Owner

J264b triage (on fuzz-fixes-j264, together with #370): both inputs are timeouts from host load, not a parser bug. No parser change is needed. The runner fix from #370 (d1450e1a0) covers them.

The failure type is a timeout for both inputs. The artifacts are timeout-1c839c82… (33 bytes, SHA-256 9f092ab2…, source 100bd91f6) and timeout-941a98f0… (46 bytes, SHA-256 5ecc1bed…, source 521cb8990).
The newer one was hit mid-fuzzing: ALARM: working on the last Unit for 36 seconds → ERROR: libFuzzer: timeout after 36 seconds (remote_standby_decode.log on dev-02). The nightly log of the older run was overwritten.
Why a real hang isn't plausible: wants_prime and activation (crates/infraternal-server/src/remote_standby_request.rs) refuse anything over 64 bytes before parsing, and the file is unchanged between 100bd91f6 and the branch head.
Replay on the nightly's own remote_standby_decode binary on dev-02:

input	`-timeout=10 <input>`, 3 runs	100 000 mutation runs seeded from it
9f092ab2…	`Executed in 1 / 0 / 0 ms`, rc=0 each time	`Done 100000 runs in 14 second(s)`
5ecc1bed…	`Executed in 0 / 0 / 0 ms`, rc=0 each time	`Done 100000 runs in 7 second(s)`

While the replay ran, dev-02 was at /proc/pressure/io some avg10=98%.

Evidence: /tmp/j264b/374-358-repro.txt on the control host. A per-input run through the runner's confirmed_inputs() re-check, with the exact cargo fuzz run command, and a 600 s remote_standby_decode run are queued on dev-02 (~/.cache/codex-scratch/j264b/fuzz-run2.log).

J264b triage (on `fuzz-fixes-j264`, together with #370): **both inputs are timeouts from host load, not a parser bug.** No parser change is needed. The runner fix from #370 (d1450e1a0) covers them. - The **failure type** is a **timeout** for both inputs. The artifacts are `timeout-1c839c82…` (33 bytes, SHA-256 9f092ab2…, source 100bd91f6) and `timeout-941a98f0…` (46 bytes, SHA-256 5ecc1bed…, source 521cb8990). - The newer one was hit mid-fuzzing: `ALARM: working on the last Unit for 36 seconds` → `ERROR: libFuzzer: timeout after 36 seconds` (`remote_standby_decode.log` on dev-02). The nightly log of the older run was overwritten. - **Why a real hang isn't plausible:** `wants_prime` and `activation` (`crates/infraternal-server/src/remote_standby_request.rs`) refuse anything over 64 bytes before parsing, and the file is unchanged between 100bd91f6 and the branch head. - **Replay** on the nightly's own `remote_standby_decode` binary on dev-02: | input | `-timeout=10 <input>`, 3 runs | 100 000 mutation runs seeded from it | |---|---|---| | 9f092ab2… | `Executed in 1 / 0 / 0 ms`, rc=0 each time | `Done 100000 runs in 14 second(s)` | | 5ecc1bed… | `Executed in 0 / 0 / 0 ms`, rc=0 each time | `Done 100000 runs in 7 second(s)` | While the replay ran, dev-02 was at `/proc/pressure/io some avg10=98%`. Evidence: `/tmp/j264b/374-358-repro.txt` on the control host. A per-input run through the runner's `confirmed_inputs()` re-check, with the exact `cargo fuzz run` command, and a 600 s `remote_standby_decode` run are queued on dev-02 (`~/.cache/codex-scratch/j264b/fuzz-run2.log`).

kayg referenced this issue

2026-11-06 00:27:36 +00:00

fuzz: host_name_parse found a failing input #353

kayg referenced this issue

2026-11-06 00:27:36 +00:00

fuzz: ssh_auth_once_decode found a failing input #370

kayg referenced this issue

2026-11-06 00:27:37 +00:00

fuzz: release_channel found a failing input #374

kayg commented

2026-11-06 00:27:37 +00:00

Author

Owner

J264b finish.

Branch fuzz-fixes-j264, head d1450e1a0b. It has two commits on base 159edcedc:

271913dc0 fix(plugin-api): machine_name refuses numeric first labels, accepts bare 0x (#353)
d1450e1a0 fix(fuzz): re-check timeouts in isolation before filing them (#370). This also covers #374 and #358.

I have not merged, deployed or closed anything.

Gate: scripts/verify quick through build-slot on compute01 at d1450e1a0 (~/.cache/codex-scratch/j264b/verify-quick2.log), verbatim:

quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0
+---------------------------+--------------+
| Rust gate                 | Result       |
+---------------------------+--------------+
| scope guard               | PASS         |
| migration names           | PASS         |
| Nix Cargo vendoring       | PASS         |
| fmt                       | PASS         |
| doc coverage ratchet      | PASS         |
| shape lints off (D351)    | PASS         |
| doc comments on diff      | PASS         |
| fixed test waits          | PASS         |
| clippy (workspace)        | PASS         |
| unit tests (changed)      | PASS         |
+---------------------------+--------------+

QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done.
VERIFY_EXIT=0

An earlier quick run at the pre-amend head b1e8d164e failed only on fmt (one assert_eq! in the #353 test). I folded the fix into the #353 commit and force-with-lease pushed the branch, so each commit passes the gate.
quick scoped only infraternal-plugin-api. The runner's Python tests (the nightly fuzz reporter row, in fast) passed separately on compute01: Ran 13 tests ... OK (370-green.log). The scripts are byte-identical between b1e8d164e and d1450e1a0.
As tasked, I did not run scripts/verify fast.

Fuzz runs: one batched build on dev-02, with the nightly's own toolchain (rustc 1.101.0-nightly 708c94ef0, cargo-fuzz 0.13.2, ASan) and its flags (-timeout=10 -rss_limit_mb=2048 -max_len=1048580), seeded from copies of the nightly corpora. 600 s per target:

target	executions	artifacts
host_name_parse	102 384 284 in 601 s	none
ssh_auth_once_decode	13 828 284 in 601 s	none
release_channel	57 369 130 in 601 s	none
remote_standby_decode	10 143 408 in 601 s	none

Logs on dev-02: ~/.cache/codex-scratch/j264b/fuzz-run1.log, fuzz-run2.log and fuzz-<target>.log.

Runner re-check, per input (fuzz-run2.log). I called the runner's own confirmed_inputs() with the exact cargo fuzz run <target> <input> -- -timeout=10 -rss_limit_mb=2048 -artifact_prefix=... command that main() builds, on copies of the nightly's four timeout artifacts:

re-check release_channel/timeout-3f4bde2f…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check remote_standby_decode/timeout-1c839c82…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check remote_standby_decode/timeout-941a98f0…: rc=0 Executed … in 0 ms  -> set aside (not filed)
re-check ssh_auth_once_decode/timeout-da39a3ee…: rc=0 Executed … in 0 ms  -> set aside (not filed)

On the fixed build, the four #353 inputs replay with rc=0 (Executed … in 31/5/1/5 ms).

Once this merges and the nightly checkout moves to it, its first run re-checks the four timeout artifacts still in the nightly's artifact directory and moves them to unconfirmed/. These issues can then be closed by the owner.

J264b finish. **Branch** `fuzz-fixes-j264`, **head d1450e1a0bc234a2acf8fc346c6cb521f06eebb9**. It has two commits on base 159edcedc: - 271913dc0 `fix(plugin-api): machine_name refuses numeric first labels, accepts bare 0x (#353)` - d1450e1a0 `fix(fuzz): re-check timeouts in isolation before filing them (#370)`. This also covers #374 and #358. I have not merged, deployed or closed anything. **Gate:** `scripts/verify quick` through build-slot on compute01 at d1450e1a0 (`~/.cache/codex-scratch/j264b/verify-quick2.log`), verbatim: ``` quick scope vs merge base with origin/dev: workspace=0 packages=[infraternal-plugin-api] tracer=0 +---------------------------+--------------+ | Rust gate | Result | +---------------------------+--------------+ | scope guard | PASS | | migration names | PASS | | Nix Cargo vendoring | PASS | | fmt | PASS | | doc coverage ratchet | PASS | | shape lints off (D351) | PASS | | doc comments on diff | PASS | | fixed test waits | PASS | | clippy (workspace) | PASS | | unit tests (changed) | PASS | +---------------------------+--------------+ QUICK pre-check passed. This is not the gate: run scripts/verify fast before reporting done. VERIFY_EXIT=0 ``` - An earlier quick run at the pre-amend head b1e8d164e failed only on `fmt` (one `assert_eq!` in the #353 test). I folded the fix into the #353 commit and force-with-lease pushed the branch, so each commit passes the gate. - `quick` scoped only `infraternal-plugin-api`. The runner's Python tests (the `nightly fuzz reporter` row, in `fast`) passed separately on compute01: `Ran 13 tests ... OK` (`370-green.log`). The scripts are byte-identical between b1e8d164e and d1450e1a0. - As tasked, I did **not** run `scripts/verify fast`. **Fuzz runs:** one batched build on dev-02, with the nightly's own toolchain (rustc 1.101.0-nightly 708c94ef0, cargo-fuzz 0.13.2, ASan) and its flags (`-timeout=10 -rss_limit_mb=2048 -max_len=1048580`), seeded from copies of the nightly corpora. 600 s per target: | target | executions | rc | artifacts | |---|---|---|---| | host_name_parse | 102 384 284 in 601 s | 0 | none | | ssh_auth_once_decode | 13 828 284 in 601 s | 0 | none | | release_channel | 57 369 130 in 601 s | 0 | none | | remote_standby_decode | 10 143 408 in 601 s | 0 | none | Logs on dev-02: `~/.cache/codex-scratch/j264b/fuzz-run1.log`, `fuzz-run2.log` and `fuzz-<target>.log`. **Runner re-check, per input** (`fuzz-run2.log`). I called the runner's own `confirmed_inputs()` with the exact `cargo fuzz run <target> <input> -- -timeout=10 -rss_limit_mb=2048 -artifact_prefix=...` command that `main()` builds, on copies of the nightly's four timeout artifacts: ``` re-check release_channel/timeout-3f4bde2f…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check remote_standby_decode/timeout-1c839c82…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check remote_standby_decode/timeout-941a98f0…: rc=0 Executed … in 0 ms -> set aside (not filed) re-check ssh_auth_once_decode/timeout-da39a3ee…: rc=0 Executed … in 0 ms -> set aside (not filed) ``` On the fixed build, the four #353 inputs replay with rc=0 (`Executed … in 31/5/1/5 ms`). **Once this merges and the nightly checkout moves to it**, its first run re-checks the four timeout artifacts still in the nightly's artifact directory and moves them to `unconfirmed/`. These issues can then be closed by the owner.

kayg referenced this issue

2026-11-06 01:05:07 +00:00

fuzz: host_name_parse found a failing input #353

kayg commented

2026-11-06 01:05:08 +00:00

Author

Owner

J274 review of fuzz-fixes-j264 @ d1450e1a0: REQUEST CHANGES on the runner commit (P2: the tests do not cover the re-check exit-code rule; P2: an input that is slow but under the 10 s bar gets set aside). The parser commit is approved. Full review: #353 (comment)

J274 review of `fuzz-fixes-j264` @ d1450e1a0: **REQUEST CHANGES** on the runner commit (P2: the tests do not cover the re-check exit-code rule; P2: an input that is slow but under the 10 s bar gets set aside). The parser commit is approved. Full review: https://git.kayg.org/kayg/infraternal/issues/353#issuecomment-81439

kayg commented

2026-11-06 01:36:14 +00:00

Author

Owner

J264b round 2: the runner fixes for this issue are on fuzz-runner-recheck, head 0fc6934ede. Details, mutation reds, the real-input proof (this issue's input set aside; a synthetic 6 s input filed) and the gate are in #353 (comment 81484): #353 (comment)

J264b round 2: the runner fixes for this issue are on `fuzz-runner-recheck`, head 0fc6934ede35b31f4313a497d377412ec47410b7. Details, mutation reds, the real-input proof (this issue's input set aside; a synthetic 6 s input filed) and the gate are in #353 (comment 81484): https://git.kayg.org/kayg/infraternal/issues/353#issuecomment-81484

kayg referenced this issue

2026-11-06 01:42:38 +00:00

fuzz: host_name_parse found a failing input #353

kayg referenced this issue

2026-11-06 01:44:49 +00:00

fuzz: host_name_parse found a failing input #353

kayg referenced this issue

2026-11-06 16:54:37 +00:00

Housekeeping 2026-11-06 #402

kayg referenced this issue

2026-11-06 21:38:59 +00:00

fuzz: host_name_parse found a failing input #353