d0zingcat/claude-code
1
0
Fork 0
mirror of https://github.com/instructkr/claude-code.git synced 2026-05-20 20:56:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(roadmap): add sse parser quadratic buffering

Browse Source
This commit is contained in:
Yeachan-Heo
2026-05-20 18:00:47 +00:00
parent ad8a0b3deb
commit 7bc373f951
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ROADMAP.md
View File

@@ -6565,3 +6565,5 @@ Original filing (2026-04-18): the session emitted `SessionStart hook (completed)
504. **`AgentOutput.laneEvents` produced by real agent runs violates the G004 conformance contract because production `LaneEvent::new` emits `metadata.seq=0` for every event while the validator requires strictly increasing sequence numbers** — dogfooded 2026-05-20 from the `#clawcode-building-in-public` 17:00 UTC nudge on `/home/bellman/Workspace/claw-code-pr2967` with branch/origin `docs/roadmap-workdir-provenance@8e8dea5`, building on Jobdori's live #505 `LaneEvent::new seq=0` report. Code inspection: `AgentOutput` manifests are initialized with `LaneEvent::started(...)` and terminal persistence appends `LaneEvent::blocked/failed/finished/commit_created(...)`; all of those convenience constructors route through `LaneEvent::new(... metadata: LaneEventMetadata::new(0, EventProvenance::LiveLane))`. `write_agent_manifest` only dedupes commit events and does not restamp sequences. Meanwhile `runtime/src/g004_conformance.rs::validate_lane_events` explicitly requires `/metadata/seq` to be present and strictly increasing (`if seq <= previous { "sequence must be strictly increasing" }`). Therefore any successful agent manifest with `lane.started` + `lane.finished` or failed manifest with `lane.started` + `lane.blocked` + `lane.failed` is invalid under the repo's own G004 contract, even before external consumers sort by seq. **Required fix shape:** (a) restamp lane event metadata seqs before manifest write (`for (idx,event) in lane_events.iter_mut().enumerate() { event.metadata.seq = idx as u64 + 1; }`) as an immediate containment, or better stamp from a per-session event counter at creation; (b) run `validate_g004_contract_bundle` (or an AgentOutput-specific wrapper) in tests against real initialized/success/failed manifests; (c) add a regression that `write_agent_manifest` never persists duplicate/non-increasing seqs after terminal append/dedupe; (d) keep `reconcile_terminal_events` sorting semantics meaningful by ensuring production seqs are nonzero and monotonic. **Why this matters:** this is event/log opacity in the literal contract layer: the product advertises machine-checkable event ordering, but real persisted manifests fail that checker. Downstream clawhips/watchers either cannot trust the conformance helper or must special-case production data. Source: gaebal-gajae dogfood response to Clawhip message `1506703078492082197` on 2026-05-20.
505. **G004 report conformance expects `schemaVersion:"g004.report.v1"`, but the runtime's canonical report implementation emits `schemaVersion:"claw.report.v1"`, so first-party canonical reports cannot pass the repo's own G004 bundle validator without an undocumented schema rewrite** — dogfooded 2026-05-20 from the `#clawcode-building-in-public` 17:30 UTC nudge on `/home/bellman/Workspace/claw-code-pr2967` with branch/origin `docs/roadmap-workdir-provenance@19a1918`. Code inspection: `runtime/src/g004_conformance.rs` hardcodes `const REPORT_SCHEMA_VERSION: &str = "g004.report.v1"` and `validate_reports` requires every `/reports/*/schemaVersion` to equal that value. Separately, the actual report schema module defines `pub const REPORT_SCHEMA_V1: &str = "claw.report.v1"`; `canonicalize_report` overwrites reports with `report.schema_version = REPORT_SCHEMA_V1.to_string()`, and the report registry/capability projection also key off `claw.report.v1`. Grep shows no adapter mapping `claw.report.v1` to `g004.report.v1`; the G004 fixture is hand-authored with `g004.report.v1`. Result: a real `CanonicalReportV1` produced by runtime and inserted into a G004 contract bundle is rejected by `validate_g004_contract_bundle` solely on schema-version mismatch. **Required fix shape:** (a) decide whether G004 should validate the first-party `claw.report.v1` schema directly or introduce an explicit projection adapter from `CanonicalReportV1` to `g004.report.v1`; (b) do not hardcode a competing report schema string in the conformance helper without a conversion path; (c) add a regression that builds a canonical report via `canonicalize_report`, wraps it in a G004 bundle with valid lane events, and verifies either acceptance or a typed `unsupported_schema_version` with documented adapter guidance; (d) update fixtures to use the same path real producers use. **Why this matters:** conformance tests should protect interoperability, not validate an artificial fixture dialect that production cannot emit. Otherwise downstream report consumers see event/log opacity: the report looks valid to the runtime registry and invalid to the G004 bundle validator. Source: gaebal-gajae dogfood response to Clawhip message `1506710631254986793` on 2026-05-20.
506. **SSE stream parsers repeatedly rescan and drain from the front of a growing buffer, making large batched streams quadratic and adding avoidable latency to provider streaming/event handling** — dogfooded 2026-05-20 from the `#clawcode-building-in-public` 18:00 UTC nudge on `/home/bellman/Workspace/claw-code-pr2967` with branch/origin `docs/roadmap-workdir-provenance@ad8a0b3`, after Jobdori noted the unfiled parser shape. Code inspection: `api/src/sse.rs::SseParser::push` appends a chunk, then loops `while let Some(frame) = self.next_frame()`. `next_frame` searches `self.buffer.windows(2)` from byte 0 for `\n\n` (then separately scans `windows(4)` for `\r\n\r\n`), drains `..position+separator_len` into a new `Vec`, and converts to `String`. `api/src/providers/openai_compat.rs::next_sse_frame` duplicates the same algorithm. `runtime/src/sse.rs::IncrementalSseParser::push_chunk` does the string analogue with repeated `self.buffer.find('\n')` plus `drain(..=index)`. For a single network read or proxy flush containing thousands of small SSE frames/lines, each extracted frame/line rescans and moves the remaining buffer from the front; total work trends O(N²) in bytes/frames and allocates a fresh buffer per frame. **Required fix shape:** (a) replace front-drain parsing with an index/cursor-based parser (`scan_pos`, `consumed_until`) and compact the buffer only occasionally; (b) search for `\n\n`/`\r\n\r\n` from the previous scan position, not from 0 every loop; (c) share one bounded SSE framing helper between Anthropic and OpenAI-compatible providers; (d) add a micro-benchmark or regression that pushes one chunk containing 10k tiny frames and asserts linear-ish parse time/allocation behavior; (e) add a max pending-buffer size and emit a typed stream framing error when no separator arrives before the cap. **Why this matters:** streaming is the main event/log surface for prompt delivery, tool calls, and usage. A proxy, provider, or test harness that batches many small SSE frames into one chunk should not turn the parser into a CPU/allocation hotspot or make streaming look stalled before any model event is delivered. Source: gaebal-gajae dogfood response to Clawhip message `1506718176509952130` on 2026-05-20.