Bound model tool envelopes with ranged artifacts and process cursors #676

Merged

toasterson merged 12 commits from fix/anima-a3-model-envelopes into fix/anima-a1-provider-outcomes 2026-10-19 20:01:02 +00:00

Conversation 0 Commits 12 Files changed 16 +1286 -21

toasterson commented 2026-10-19 08:27:13 +00:00

Owner

Copy link

Bound the complete model-visible tool result while retaining complete broker evidence and scoped ranged reads; collect the new bulk storage after declared retention.

Targets #674 branch; supplies Anima S2. No PR merge, deployment, live recovery or canary.

Review fixes

Handles are bounded to2KiB including encoded keys/values, with explicit handles_elided rather than quarantining a job after its tool executed. Artifacts have16MiB per-result serialization and128MiB per-job caps. Process captures have24h retention independent of permanent effect receipts and are collected by idle lease maintenance; live cursor polls encode only their requested slice. Artifacts survive archival/prior-job reads but terminal inactive jobs expire after7days. GC removes chunks/manifests/counters/indexes. Codex text-only results intentionally use the same structured success/receipt transport as other ToolResults.

Exact consumer contract

Tool output v1 (A3 output; Anima S2)

The model-facing text path bounds the serialized ToolResult to 16 KiB,
including nested JSON and receipt wrapping. Oversized results retain exactly one
head/tail excerpt, the original success and receipt, and data:

{"kind":"tool_output_ref","schema_version":1,"job_id":"job","sha256":"<64 lowercase hex>","bytes":2334424,"path":"/v1/jobs/job/tool-output/<sha256>","tool":"akh_read_tool_output","handles":{"process_id":"process","status":"running"}}

Hash/size cover the complete original serialized ToolResult, before model elision.
The runtime commits all bytes as hashed chunks before returning an excerpt. Identical
artifacts deduplicate under a 128 MiB/job budget; exhausting storage refuses the
model callback rather than silently discarding evidence. Original broker/executor
receipts retain their authority. Large ToolCompleted event details already have
lossless job-scoped references; this extends the distinct provider-input boundary.
ToolCompleted details also include the model output reference as provenance.

GET /v1/jobs/{job}/tool-output/{sha256}?offset=0&limit=8192 requires existing
session-service authentication and an existing job owned by this runtime. Model
access is through the runtime-local read-only akh_read_tool_output tool:

{"job_id":"job","sha256":"<hash>","offset":0,"limit":8192}

job_id defaults to the current job. Prior jobs require the same exact session;
foreign-session references are denied. Offset is a UTF-8 byte cursor. Limit is
1..8192; the model tool may return fewer bytes to reserve JSON-envelope space.
The response contains {schema_version:1,sha256,bytes,offset,next_offset,done,text}.
Continue at next_offset. Mid-codepoint cursors are refused; range ends shorten
on UTF-8 boundaries; no replacement character conceals omitted bytes. Chunk hashes
are verified on read; each range reads only the needed chunks, not the whole artifact.

Codex emits one serialized result rather than dropping receipt metadata on its
text-only path. Codex and internal role/tool messages check the final escaped
provider-facing envelope against 64 KiB. Every runtime-host text result is bounded
before native adapter wrapping. Native Candle image reads retain the existing
image modality path and its tokenizer/context budget; arbitrary base64 text files
are not image exemptions. Prompt contents and credential-bearing URLs are never
metric labels.

Process cursors and retained capture

Existing #394 24 KiB per-stream head/tail clipping remains for legacy polls.
exec_poll optionally accepts stdout_offset, stderr_offset and
max_output_bytes (1..8192, default4096). With either offset supplied, response
contains cursor_mode:true, process ID/status, terminal success/receipt/exit code,
and stdout/stderr ranges {sha256,bytes,offset,next_offset,done,text,base64}.
UTF-8 text is present only when the complete captured stream is valid UTF-8;
base64 always preserves exact bytes. Process ownership/generation checks remain.
Each poll receipt freezes the observed range and replay returns the same observation.

Complete captured stdout/stderr are committed with the terminal receipt before
clipped text becomes visible. Cursor reads survive executor restart. The existing
1 MiB/stream capture cap stays explicit (capture_truncated); this is lossless for
captured bytes, not a claim to recover bytes beyond that pre-existing cap. Old or
interrupted processes without durable capture return explicit unavailable.
Controller/model tool schemas must allow the new optional cursor fields to use
this path; ordinary polls keep their previous meaning.

The internal HTTP endpoint optionally declares max_input_bytes in 1..8388608.
The final serialized body includes mandatory instructions, schemas and history.
An over-budget request is refused without dropping mandatory text. Absent setting
preserves the existing 8 MiB hard limit; it does not assert a token-window fit.
Broader per-call usage/model/context provenance is the separate A3 telemetry PR.
Deployment and authorized canary pending; historical producers are not inferred.

Review bounds: serialized handles map <=2KiB (including escaped keys/values), extra handles explicitly marked handles_elided and remain in the full artifact. Per-result serialization caps at16MiB before chunking; per-job budget128MiB remains. Codex serializes every ToolResult, including text-only values, to preserve receipt/success metadata; the final escaped64KiB envelope guard is separately tested.

Complete process captures expire after24h independently of permanent effect receipts; deleting a collected read receipt also deletes its capture/index. The existing executor sweep runs from lease maintenance as well as operations, so idle executors collect. Live cursor polls slice before base64 encoding. Runtime tool artifacts survive archival/prior-job reads but expire after7days for terminal inactive jobs, in up-to64-job sweeps. Unknown/active jobs retain their bounded evidence. GC removes manifests/chunks/budget/index atomically; post-expiry range reads return unavailable, not fabricated empty output.

Prior validation

Fable Round2 approved reviewed head 621027446d49. Previous follow-up head: dae86ef17b02dc41b2123842dc19c0595471de70.

The64 collection budget is applied after active/nonterminal/missing owner checks. As CoS explicitly requested, missing and Unknown owners retain evidence and consume no collection slots. The regression places64 Internal Unknown and64 missing jobs before65 terminal jobs: successive rounds collect64, then1, then0, while stale owners remain. This implements the operator request to skip missing owners rather than the optional review suggestion to collect orphan evidence.

Current-head collector regression:1 passed (r2-followup-output.log). Final integrated #678 suite115 passed/1 existing real-backend ignored. Full exact-head Solstice CI remains a merge gate; local results do not substitute for it. No merge, force-push, deployment or canary.

Finished task target was reclaimed after verifying no open files;37.08GiB free at10:16Z. Peer target preserved.

CI4360 follow-up

Current head: c8a1ea495fa299ebef79bdbe89686955d09da033.

Solstice exact-head evidence for #676 dae86ef17b identifies failed request e257c6a0-5f0a-4a9e-8b58-d372ffb7ad0f, category step:cargo-test-features-postgres-lib. The root suite had3961 passed,22 ignored and1 failure: agent::runtime::tests::codex_tools_use_typed_akh_registry_and_record_evidence, src/agent/runtime/tests.rs:117, expecting plain "7" while the fixture echoed the complete ToolResult JSON. This was a missed regression expectation from the reviewed complete-envelope contract, not a refused connection or runner fault. It reproduced locally before the fix. Reported Forgejo job: /toasterson/akh-medu/actions/runs/4360/jobs/7.

The root regression now checks all decoded fields {success:true,text:"7",data:null,receipt:null} and keeps its tool-evidence trace assertion. New Codex envelope coverage preserves both text-only and structured data/receipt results alongside the existing64KiB escaped-envelope refusal. The production envelope implementation hash is unchanged; no wire contract change. The identical stale root test and implementation existed on #677/#678, so normal stack integration carries this fix through both. No force-push or PR merge.

Current-head local receipts:

• ci4360-root-fixed.log: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 3984 filtered out; finished in 0.23s
• ci4360-envelope.log: test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 14 filtered out; finished in 0.01s
• ci4360-provider.log: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 13 filtered out; finished in 0.46s

Local workspace packages were cleaned between worktrees to exclude stale Cargo reuse. Final full Linux CI is still a separate gate. No blind rerun was requested; the normal follow-up commits publish the diagnosed fix. No deployment, Talos/Ptah change or live provider probe. Builds used31.5GiB stop/32GiB preflight guards; the finished task target is reclaimed on handoff.

Own finished target was verified without open files and deleted; 37.269GiB free at 2026-10-19T11:58:41.253438+00:00. Peer targets were untouched.

Bound the complete model-visible tool result while retaining complete broker evidence and scoped ranged reads; collect the new bulk storage after declared retention. Targets #674 branch; supplies Anima S2. No PR merge, deployment, live recovery or canary. ## Review fixes Handles are bounded to2KiB including encoded keys/values, with explicit handles_elided rather than quarantining a job after its tool executed. Artifacts have16MiB per-result serialization and128MiB per-job caps. Process captures have24h retention independent of permanent effect receipts and are collected by idle lease maintenance; live cursor polls encode only their requested slice. Artifacts survive archival/prior-job reads but terminal inactive jobs expire after7days. GC removes chunks/manifests/counters/indexes. Codex text-only results intentionally use the same structured success/receipt transport as other ToolResults. ## Exact consumer contract # Tool output v1 (A3 output; Anima S2) The model-facing text path bounds the **serialized ToolResult** to 16 KiB, including nested JSON and receipt wrapping. Oversized results retain exactly one head/tail excerpt, the original `success` and `receipt`, and `data`: ```json {"kind":"tool_output_ref","schema_version":1,"job_id":"job","sha256":"<64 lowercase hex>","bytes":2334424,"path":"/v1/jobs/job/tool-output/<sha256>","tool":"akh_read_tool_output","handles":{"process_id":"process","status":"running"}} ``` Hash/size cover the complete original serialized ToolResult, before model elision. The runtime commits all bytes as hashed chunks before returning an excerpt. Identical artifacts deduplicate under a 128 MiB/job budget; exhausting storage refuses the model callback rather than silently discarding evidence. Original broker/executor receipts retain their authority. Large ToolCompleted event details already have lossless job-scoped references; this extends the distinct provider-input boundary. ToolCompleted details also include the model output reference as provenance. `GET /v1/jobs/{job}/tool-output/{sha256}?offset=0&limit=8192` requires existing session-service authentication and an existing job owned by this runtime. Model access is through the runtime-local read-only `akh_read_tool_output` tool: ```json {"job_id":"job","sha256":"<hash>","offset":0,"limit":8192} ``` `job_id` defaults to the current job. Prior jobs require the same exact session; foreign-session references are denied. Offset is a UTF-8 byte cursor. Limit is 1..8192; the model tool may return fewer bytes to reserve JSON-envelope space. The response contains `{schema_version:1,sha256,bytes,offset,next_offset,done,text}`. Continue at `next_offset`. Mid-codepoint cursors are refused; range ends shorten on UTF-8 boundaries; no replacement character conceals omitted bytes. Chunk hashes are verified on read; each range reads only the needed chunks, not the whole artifact. Codex emits one serialized result rather than dropping receipt metadata on its text-only path. Codex and internal role/tool messages check the final escaped provider-facing envelope against 64 KiB. Every runtime-host text result is bounded before native adapter wrapping. Native Candle image reads retain the existing image modality path and its tokenizer/context budget; arbitrary base64 text files are not image exemptions. Prompt contents and credential-bearing URLs are never metric labels. ## Process cursors and retained capture Existing #394 24 KiB per-stream head/tail clipping remains for legacy polls. `exec_poll` optionally accepts `stdout_offset`, `stderr_offset` and `max_output_bytes` (1..8192, default4096). With either offset supplied, response contains `cursor_mode:true`, process ID/status, terminal success/receipt/exit code, and stdout/stderr ranges `{sha256,bytes,offset,next_offset,done,text,base64}`. UTF-8 text is present only when the complete captured stream is valid UTF-8; base64 always preserves exact bytes. Process ownership/generation checks remain. Each poll receipt freezes the observed range and replay returns the same observation. Complete captured stdout/stderr are committed with the terminal receipt before clipped text becomes visible. Cursor reads survive executor restart. The existing 1 MiB/stream capture cap stays explicit (`capture_truncated`); this is lossless for captured bytes, not a claim to recover bytes beyond that pre-existing cap. Old or interrupted processes without durable capture return explicit unavailable. Controller/model tool schemas must allow the new optional cursor fields to use this path; ordinary polls keep their previous meaning. The internal HTTP endpoint optionally declares `max_input_bytes` in 1..8388608. The final serialized body includes mandatory instructions, schemas and history. An over-budget request is refused without dropping mandatory text. Absent setting preserves the existing 8 MiB hard limit; it does not assert a token-window fit. Broader per-call usage/model/context provenance is the separate A3 telemetry PR. Deployment and authorized canary pending; historical producers are not inferred. Review bounds: serialized handles map <=2KiB (including escaped keys/values), extra handles explicitly marked handles_elided and remain in the full artifact. Per-result serialization caps at16MiB before chunking; per-job budget128MiB remains. Codex serializes every ToolResult, including text-only values, to preserve receipt/success metadata; the final escaped64KiB envelope guard is separately tested. Complete process captures expire after24h independently of permanent effect receipts; deleting a collected read receipt also deletes its capture/index. The existing executor sweep runs from lease maintenance as well as operations, so idle executors collect. Live cursor polls slice before base64 encoding. Runtime tool artifacts survive archival/prior-job reads but expire after7days for terminal inactive jobs, in up-to64-job sweeps. Unknown/active jobs retain their bounded evidence. GC removes manifests/chunks/budget/index atomically; post-expiry range reads return unavailable, not fabricated empty output. ## Prior validation Fable Round2 approved reviewed head `621027446d49`. Previous follow-up head: `dae86ef17b02dc41b2123842dc19c0595471de70`. The64 collection budget is applied after active/nonterminal/missing owner checks. As CoS explicitly requested, missing and Unknown owners retain evidence and consume no collection slots. The regression places64 Internal Unknown and64 missing jobs before65 terminal jobs: successive rounds collect64, then1, then0, while stale owners remain. This implements the operator request to skip missing owners rather than the optional review suggestion to collect orphan evidence. Current-head collector regression:1 passed (r2-followup-output.log). Final integrated #678 suite115 passed/1 existing real-backend ignored. Full exact-head Solstice CI remains a merge gate; local results do not substitute for it. No merge, force-push, deployment or canary. Finished task target was reclaimed after verifying no open files;37.08GiB free at10:16Z. Peer target preserved. ## CI4360 follow-up Current head: `c8a1ea495fa299ebef79bdbe89686955d09da033`. Solstice exact-head evidence for #676 dae86ef17b02dc41b2123842dc19c0595471de70 identifies failed request e257c6a0-5f0a-4a9e-8b58-d372ffb7ad0f, category `step:cargo-test-features-postgres-lib`. The root suite had3961 passed,22 ignored and1 failure: `agent::runtime::tests::codex_tools_use_typed_akh_registry_and_record_evidence`, src/agent/runtime/tests.rs:117, expecting plain "7" while the fixture echoed the complete ToolResult JSON. This was a missed regression expectation from the reviewed complete-envelope contract, not a refused connection or runner fault. It reproduced locally before the fix. Reported Forgejo job: /toasterson/akh-medu/actions/runs/4360/jobs/7. The root regression now checks all decoded fields `{success:true,text:"7",data:null,receipt:null}` and keeps its tool-evidence trace assertion. New Codex envelope coverage preserves both text-only and structured data/receipt results alongside the existing64KiB escaped-envelope refusal. The production envelope implementation hash is unchanged; no wire contract change. The identical stale root test and implementation existed on #677/#678, so normal stack integration carries this fix through both. No force-push or PR merge. Current-head local receipts: - `ci4360-root-fixed.log`: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 3984 filtered out; finished in 0.23s - `ci4360-envelope.log`: test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 14 filtered out; finished in 0.01s - `ci4360-provider.log`: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 13 filtered out; finished in 0.46s Local workspace packages were cleaned between worktrees to exclude stale Cargo reuse. Final full Linux CI is still a separate gate. No blind rerun was requested; the normal follow-up commits publish the diagnosed fix. No deployment, Talos/Ptah change or live provider probe. Builds used31.5GiB stop/32GiB preflight guards; the finished task target is reclaimed on handoff. Own finished target was verified without open files and deleted; 37.269GiB free at 2026-10-19T11:58:41.253438+00:00. Peer targets were untouched.

toasterson added 2 commits 2026-10-19 08:27:13 +00:00

Bound model tool envelopes with durable ranged artifacts and process cursors 261d159321

Merge branch 'fix/anima-a1-provider-outcomes' into fix/anima-a3-model-envelopes bc41ada1d4

toasterson added 1 commit 2026-10-19 08:27:41 +00:00

Assert text-only Codex result keeps success and receipt metadata 1ad4349f04

toasterson added 1 commit 2026-10-19 08:32:22 +00:00

Scope prior artifact reads and preserve non-model worker protocol results 0c2bd2dfab

toasterson referenced this pull request 2026-10-19 08:32:28 +00:00

Persist typed model usage identity and context provenance #677

toasterson added 1 commit 2026-10-19 08:45:50 +00:00

Integrate typed provider reset contract into output stack cdd82ee545

toasterson referenced this pull request 2026-10-19 08:46:04 +00:00

Emit per-turn usage and explicit context diagnostics #678

toasterson added 1 commit 2026-10-19 08:50:05 +00:00

Integrate explicit monthly limit classification c5606ed5fe

toasterson added 3 commits 2026-10-19 09:38:23 +00:00

Integrate reviewed A1 fixes 1d5f1be803

Bound output handles and collect expired process and tool artifacts 4fbc363f57

Integrate metadata reader compatibility floor 621027446d

toasterson added 2 commits 2026-10-19 10:14:31 +00:00

fix(runtime): budget artifact GC after owner eligibility 967f3a2de1

Integrate A1 reset deadline test follow-up dae86ef17b

toasterson added 1 commit 2026-10-19 11:55:30 +00:00

test(runtime): align embedded Codex result with complete envelope c8a1ea495f

toasterson referenced this pull request 2026-10-19 11:58:54 +00:00

Emit per-turn usage and explicit context diagnostics #678

toasterson merged commit c08c42901d into fix/anima-a1-provider-outcomes 2026-10-19 20:01:02 +00:00

toasterson referenced this pull request from a commit 2026-10-19 20:01:04 +00:00

Merge PR #676: Bound model tool envelopes with ranged artifacts and process cursors

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

follow-up

Documented operational follow-up

No labels

follow-up

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

toasterson/akh-medu!676

No description provided.