docs(roadmap): add dogfood timeout retry evidence

This commit is contained in:
Yeachan-Heo
2026-05-19 16:31:00 +00:00
parent 0063c0d698
commit 78d334c4e2

View File

@@ -6451,3 +6451,6 @@ Original filing (2026-04-18): the session emitted `SessionStart hook (completed)
458. **Global output-format flag ordering is parser-hostile, and even `version --output-format json` can hang under clean env despite normal-env success** — dogfooded 2026-05-19 from the `#clawcode-building-in-public` 16:00 UTC nudge on `/home/bellman/Workspace/claw-code-pr2967` with normal-env `./rust/target/debug/claw version --output-format json` reporting `git_sha:"25d663d"`. Clean-environment flag-order probes showed `--output-format json --help`, `--output-format json help`, `--output-format json version`, `--version --output-format json`, and `--output-format json --version` each failed immediately with text-mode stderr only, e.g. `[error-kind: cli_parse] error: unknown option: --output-format json --help`, and `stdout=0`. The parser appears to treat the whole trailing string as one unknown option rather than recognizing a global output-format flag before the command. Separately, in the same clean `HOME`/`CLAW_CONFIG_HOME`, canonical `version --output-format json` timed out after 8s with `stdout=0`/`stderr=0`, even though the same command succeeds in the normal environment. **Required fix shape:** (a) make global flags such as `--output-format json` accepted before or after the subcommand, or return structured JSON `cli_parse` errors on stdout when JSON format is requested anywhere in argv; (b) parse flag values as separate tokens in error reporting instead of echoing combined strings like `--output-format json --help` as one option; (c) ensure `version` is fully local/static and cannot hang under clean env or missing config/auth; (d) add clean-env regression coverage for `version --output-format json`, `--output-format json version`, `--version --output-format json`, and JSON parse errors with stdout envelopes. **Why this matters:** claws often put global flags first for CLI uniformity and run in sanitized envs. If global JSON selection is order-sensitive and local version can hang only under clean env, startup probes become unreliable exactly in CI/sandbox contexts. Source: gaebal-gajae dogfood response to Clawhip message `1506325598245748828` on 2026-05-19.
459. **Dogfood timeout claims lack a required retry/evidence contract, so transient hangs can be recorded as durable product gaps without immediate reproducibility metadata** — dogfooded 2026-05-19 from the `#clawcode-building-in-public` 16:30 UTC nudge while narrowing #458 on `/home/bellman/Workspace/claw-code-pr2967` with `./rust/target/debug/claw version --output-format json` reporting `git_sha:"25d663d"`. The previous 16:00 pass saw clean-env `version --output-format json` time out with zero bytes, but a focused 16:30 retry matrix using isolated `HOME`/`CLAW_CONFIG_HOME` plus minimal env variants (`TERM`, `USER`/`LOGNAME`, `SHELL`, `LANG`/`LC_ALL`, and all combined) returned valid version JSON every time. That means the earlier timeout may have been transient harness load, process scheduling, or invocation interference, while the report format had no mandatory retry count, timing, command log artifact, or “reproduced N/M” field to distinguish flaky evidence from stable behavior. **Required fix shape:** (a) dogfood timeout reports must include retry count, per-attempt exit code/stdout/stderr byte counts, elapsed duration, env summary, binary provenance, and whether the failure reproduced after process isolation; (b) add a standard `timeout_evidence` block to report templates and ROADMAP entries before filing zero-byte hang claims; (c) classify un-reproduced hangs as `flaky_unconfirmed` with follow-up probes instead of stable product bugs; (d) provide a small harness command that runs bounded retries and emits machine-readable evidence JSON. **Why this matters:** zero-byte timeouts are high-severity but easy to misattribute. Without a retry/evidence contract, claws can pollute the backlog with transient scheduler artifacts or miss real nondeterministic hangs because the evidence shape is too thin. Source: gaebal-gajae dogfood response to Clawhip message `1506333141571211314` on 2026-05-19.