S2: bound final MCP responses with private lossless artifact receipts #802

Merged

toasterson merged 16 commits from fix/s2-bounded-tool-artifacts-20261019 into main 2026-10-19 20:58:12 +00:00

Conversation 0 Commits 16 Files changed 31 +2142 -36

toasterson commented 2026-10-19 10:36:19 +00:00

Owner

Copy link

Current exact head: c37f3cf4a5f3e7fb62cec3fe17f837883e26342a. Current main b083a89ad8224c844a3db37e38167e21184aa163 merged normally, with no history rewrite. Focused main repair #805 is a dependency, addressing only introduced Q1/Q2 fixture lifecycle calls and the Q4 denial expectation. Human order: #805 → #802 → #803 → #804.

Exact-head Solstice failure at55edd630 was one S2 test assertion assuming process_id was the first JSON object key. The fix compares exact decoded head/tail source bytes, preserves the complete hash/byte/handle assertions, and passes both default JSON ordering and explicit serde_json/preserve_order. No artifact-production behavior changed in this follow-up.

Local final-head validation: cargo check -p anima-server --lib PASS; artifact suite5 including real PostgreSQL PASS; explicit preserve_order regression1 PASS. The external wrapper was edited during its first execution, producing an error after all Rust commands completed; original exit2 receipt is retained, final clean-source/diff verification completed separately, and future wrappers are immutable copies. See pr-802-refresh-verification.json; no failed Rust test was relabelled green.

Local receipt log SHA256 f4be37418f8e33862b86aa7ceed68a7b5218bbd5fbb4920a04bc230ea9d24d1c. Exact-head remote CI/review remains a distinct operator gate. No merge/deployment/live recovery by this agent.

Model-session MCP results now fit Akh A3's transport budget without changing routine PAT/operator reads. Only authenticated non-local session kinds with a persisted Akh dispatch default to an 8192-byte final serialized JSON-RPC boundary (configurable4096..8192); PAT calls keep their existing behavior, with an optional separate64KiB..8MiB cap. Default PAT list pages stay direct, and Q4's PAT64KiB default/1MiB whole-archive contract remains reachable through the actual dispatcher.

Akh-dispatched non-local sessions receive real handler pages: direct WI description/authorized section bytes, stable thread message/content pages, and archive UTF8 or native after_seq transcript pages. Mutable-source hashes prevent concatenating changed versions; collection cursors survive timestamp ties. Default Akh list_work_items pages cap at10 rows; ACP/local sessions keep their existing list/full-read shapes. PAT detail retains its full-state default shape plus additive read_windows; optional page selection is explicit. Full query/memory pagination remains S4; the safe read prerequisite here still fetches/hashes source text.

An oversized fallback commits the complete exact original as hashed chunks before returning decoded tool-text head/tail excerpts and an exact-owner read_tool_output reference. Handles, error state and receipt/request-ID binding survive. Owner quota pressure evicts that owner's oldest artifacts under its advisory lock; surviving expiry is immutable. The bounded minute sweep also converges a configurable global soft cap and publishes aggregate byte/count/eviction metrics. Reads suggest narrower pages on persistence failure; mutation failures retain the warning against replaying a possibly completed operation. Default unbounded PAT calls allocate no artifacts.

Dependencies: Q4 #799 at76cbc9c55528c864a9542d9abeb60a2d3ccafd2a is integrated by an ordinary local merge (no history rewrite). Its transcript regression now tests both role paths through handle_request. Akh #676 provides the complementary final ToolResult/native bounds; this producer uses distinct anima_tool_output_ref and never invents a runtime job ID. D1 #798's exact-WI detail restriction is preserved for model tokens; watch for session-token denials after rollout. No scheduler feedback pagination was added: generic MCP feedback results cross this boundary, while mandatory dispatch briefs retain Q2's refusal behavior.

Storage: PostgreSQL preserves atomic owner/scope/bytes publication without a cross-store protocol. Per-owner cap128MiB, maximum seven-day availability subject to earlier quota/global eviction; global soft cap defaults8GiB. Dedup intentionally includes request ID so different JSON-RPC receipts stay distinct. Retrieval limits are upper bounds reduced for metadata/escaping; tiny limits advance a whole Unicode scalar. Contract and configuration: docs/ai/contracts/mcp-tool-artifact-v1.md, with the token refinement also in transcript-read-v1.md.

Validation: focused MCP suite152 passed, including full-schema disposable PostgreSQL artifact/owner/eviction/range/PAT tests, direct-page/source-pin/ACL tests and merged Q4 transcript tests. The previously broken Docker-gated mcp_work_item_pagination_test passes both tests on Colima, including205-row/tied-timestamp/concurrent-insert traversal. All-target anima-server Clippy with -D warnings, diff check, shell syntax and Solstice dry-run pass. Final full library/exact pushed-head receipts are recorded in patches-anima.md.

CI coverage correction: mcp_work_item_pagination_test was missing from the nightly build and job lists; both are added, plus the new direct-read PG job. Existing artifact/transcript jobs remain. Other touched central integration suites use explicit disposable PG rather than require_docker skipping and already have nightly jobs. Commit-mode CI does not execute the nightly DB pool; local explicit PG/Docker evidence is separate from Solstice exact-head CI. Independent re-review and genuine Solstice receipts remain pending. No merge, deployment or live recovery.

Producer evidence remains E05's sanitized measured nested-output shapes and the read-only source/digest census: merged Akh #394 was already present in the observed service/runtime sources, while historical producer provenance remains unknown. Tests contain no historical private prompts.

Round2 follow-up headee277438f40a0fcdef474f9d94c53c6dc9508175: auth carries server-loaded SessionKind plus EXISTS(akh_session_dispatches); both are required. A production-auth PostgreSQL fixture covers Akh, ACP runner and local pickup, proving full local get_work_item and50-row ACP/local lists. FOR SHARE holds a parent lease through each range; an eviction concurrency fixture proves the writer waits and reads complete. Migration164 adds owner/age index without changing161's checksum. Direct pages fit up to4096 source bytes against actual serialized escaping, model transcript tails honour tail_bytes (zero starts0; explicit offsets win), and every non-read mutation scope retains the no-replay warning. Focused154 MCP tests including3PG, full565lib/8ignored, real Docker pagination2 and all-target Clippy pass. Exact pushed-head PG/Docker rerun is active.

Constructor compatibility follow-up: D1#798 and S1#801 land before #802 in the agreed order. Their new MCP fixture constructors therefore now receive session_kind=None and akh_dispatched=false inside #802, through ordinary dependency merges93e4a000 and a normal follow-upccfb5b52. Sites: src/mcp/context_diagnostics.rs, src/central_cleanup.rs, tests/central_session_retirement.rs. These are test fixture changes; production S2 boundary/lease/pagination logic is unchanged from Fable-approved ee277438. Current head55edd630b97ef713e138dd90b43e66a1c0c9b3b8. Exact focused reruns at this head await safe disk headroom; prior ee277438 receipts remain separate evidence. The final merged-tree remote nightly branch includes these committed changes, rather than validation-only repairs. No force push/main merge/deployment.

Current final head3ec13eca079455da6c1a0b807cc751527183dbd8 carries the updated Akh#67580464684ecc2b9afb7c9978aab8841d14dc55971 provenance pin through ordinary #801→#802→#803 dependency merges. The final validation branch is validation/anima-qseries-nightly-v2-20261019 at exact #803headf8f860c6962936813a024533e386f4523f5f873b; every commit is reachable from a PR head. No new Anima wire/attestation logic changed for this pin. Remote nightly exact-head receipts and safe-headroom focused reruns remain pending.

Integration refresh (2026-10-19): normal main/dependency merge commits, no force push. Main is 8d3eb5855f. This head also includes refreshed #801 so the agreed merge order cannot repeat the same insertion conflict; its diff narrows as predecessors land. Shell/Rust syntax, diff and predecessor-ancestry checks pass. No fresh local cargo check: available disk headroom above the 31.25 GiB floor was smaller than the last measured owned compile target. Exact new-head CI is the gate; prior receipts remain prior-head evidence.

<!-- cos-final-train:start --> Current exact head: `c37f3cf4a5f3e7fb62cec3fe17f837883e26342a`. Current main `b083a89ad8224c844a3db37e38167e21184aa163` merged normally, with no history rewrite. Focused main repair #805 is a dependency, addressing only introduced Q1/Q2 fixture lifecycle calls and the Q4 denial expectation. Human order: #805 → #802 → #803 → #804. Exact-head Solstice failure at55edd630 was one S2 test assertion assuming process_id was the first JSON object key. The fix compares exact decoded head/tail source bytes, preserves the complete hash/byte/handle assertions, and passes both default JSON ordering and explicit serde_json/preserve_order. No artifact-production behavior changed in this follow-up. Local final-head validation: cargo check -p anima-server --lib PASS; artifact suite5 including real PostgreSQL PASS; explicit preserve_order regression1 PASS. The external wrapper was edited during its first execution, producing an error after all Rust commands completed; original exit2 receipt is retained, final clean-source/diff verification completed separately, and future wrappers are immutable copies. See pr-802-refresh-verification.json; no failed Rust test was relabelled green. Local receipt log SHA256 `f4be37418f8e33862b86aa7ceed68a7b5218bbd5fbb4920a04bc230ea9d24d1c`. Exact-head remote CI/review remains a distinct operator gate. No merge/deployment/live recovery by this agent. <!-- cos-final-train:end --> Model-session MCP results now fit Akh A3's transport budget without changing routine PAT/operator reads. Only authenticated non-local session kinds with a persisted Akh dispatch default to an 8192-byte final serialized JSON-RPC boundary (configurable4096..8192); PAT calls keep their existing behavior, with an optional separate64KiB..8MiB cap. Default PAT list pages stay direct, and Q4's PAT64KiB default/1MiB whole-archive contract remains reachable through the actual dispatcher. Akh-dispatched non-local sessions receive real handler pages: direct WI description/authorized section bytes, stable thread message/content pages, and archive UTF8 or native after_seq transcript pages. Mutable-source hashes prevent concatenating changed versions; collection cursors survive timestamp ties. Default Akh list_work_items pages cap at10 rows; ACP/local sessions keep their existing list/full-read shapes. PAT detail retains its full-state default shape plus additive read_windows; optional page selection is explicit. Full query/memory pagination remains S4; the safe read prerequisite here still fetches/hashes source text. An oversized fallback commits the complete exact original as hashed chunks before returning decoded tool-text head/tail excerpts and an exact-owner read_tool_output reference. Handles, error state and receipt/request-ID binding survive. Owner quota pressure evicts that owner's oldest artifacts under its advisory lock; surviving expiry is immutable. The bounded minute sweep also converges a configurable global soft cap and publishes aggregate byte/count/eviction metrics. Reads suggest narrower pages on persistence failure; mutation failures retain the warning against replaying a possibly completed operation. Default unbounded PAT calls allocate no artifacts. Dependencies: Q4 #799 at76cbc9c55528c864a9542d9abeb60a2d3ccafd2a is integrated by an ordinary local merge (no history rewrite). Its transcript regression now tests both role paths through handle_request. Akh #676 provides the complementary final ToolResult/native bounds; this producer uses distinct anima_tool_output_ref and never invents a runtime job ID. D1 #798's exact-WI detail restriction is preserved for model tokens; watch for session-token denials after rollout. No scheduler feedback pagination was added: generic MCP feedback results cross this boundary, while mandatory dispatch briefs retain Q2's refusal behavior. Storage: PostgreSQL preserves atomic owner/scope/bytes publication without a cross-store protocol. Per-owner cap128MiB, maximum seven-day availability subject to earlier quota/global eviction; global soft cap defaults8GiB. Dedup intentionally includes request ID so different JSON-RPC receipts stay distinct. Retrieval limits are upper bounds reduced for metadata/escaping; tiny limits advance a whole Unicode scalar. Contract and configuration: docs/ai/contracts/mcp-tool-artifact-v1.md, with the token refinement also in transcript-read-v1.md. Validation: focused MCP suite152 passed, including full-schema disposable PostgreSQL artifact/owner/eviction/range/PAT tests, direct-page/source-pin/ACL tests and merged Q4 transcript tests. The previously broken Docker-gated mcp_work_item_pagination_test passes both tests on Colima, including205-row/tied-timestamp/concurrent-insert traversal. All-target anima-server Clippy with -D warnings, diff check, shell syntax and Solstice dry-run pass. Final full library/exact pushed-head receipts are recorded in patches-anima.md. CI coverage correction: mcp_work_item_pagination_test was missing from the nightly build and job lists; both are added, plus the new direct-read PG job. Existing artifact/transcript jobs remain. Other touched central integration suites use explicit disposable PG rather than require_docker skipping and already have nightly jobs. Commit-mode CI does not execute the nightly DB pool; local explicit PG/Docker evidence is separate from Solstice exact-head CI. Independent re-review and genuine Solstice receipts remain pending. No merge, deployment or live recovery. Producer evidence remains E05's sanitized measured nested-output shapes and the read-only source/digest census: merged Akh #394 was already present in the observed service/runtime sources, while historical producer provenance remains unknown. Tests contain no historical private prompts. Round2 follow-up headee277438f40a0fcdef474f9d94c53c6dc9508175: auth carries server-loaded SessionKind plus EXISTS(akh_session_dispatches); both are required. A production-auth PostgreSQL fixture covers Akh, ACP runner and local pickup, proving full local get_work_item and50-row ACP/local lists. FOR SHARE holds a parent lease through each range; an eviction concurrency fixture proves the writer waits and reads complete. Migration164 adds owner/age index without changing161's checksum. Direct pages fit up to4096 source bytes against actual serialized escaping, model transcript tails honour tail_bytes (zero starts0; explicit offsets win), and every non-read mutation scope retains the no-replay warning. Focused154 MCP tests including3PG, full565lib/8ignored, real Docker pagination2 and all-target Clippy pass. Exact pushed-head PG/Docker rerun is active. Constructor compatibility follow-up: D1#798 and S1#801 land before #802 in the agreed order. Their new MCP fixture constructors therefore now receive session_kind=None and akh_dispatched=false inside #802, through ordinary dependency merges93e4a000 and a normal follow-upccfb5b52. Sites: src/mcp/context_diagnostics.rs, src/central_cleanup.rs, tests/central_session_retirement.rs. These are test fixture changes; production S2 boundary/lease/pagination logic is unchanged from Fable-approved ee277438. Current head55edd630b97ef713e138dd90b43e66a1c0c9b3b8. Exact focused reruns at this head await safe disk headroom; prior ee277438 receipts remain separate evidence. The final merged-tree remote nightly branch includes these committed changes, rather than validation-only repairs. No force push/main merge/deployment. Current final head3ec13eca079455da6c1a0b807cc751527183dbd8 carries the updated Akh#67580464684ecc2b9afb7c9978aab8841d14dc55971 provenance pin through ordinary #801→#802→#803 dependency merges. The final validation branch is validation/anima-qseries-nightly-v2-20261019 at exact #803headf8f860c6962936813a024533e386f4523f5f873b; every commit is reachable from a PR head. No new Anima wire/attestation logic changed for this pin. Remote nightly exact-head receipts and safe-headroom focused reruns remain pending. <!-- anima-main-refresh-20261019:begin --> Integration refresh (2026-10-19): normal main/dependency merge commits, no force push. Main is 8d3eb5855f74c616d176b788e2ca881d2f933e3f. This head also includes refreshed #801 so the agreed merge order cannot repeat the same insertion conflict; its diff narrows as predecessors land. Shell/Rust syntax, diff and predecessor-ancestry checks pass. No fresh local cargo check: available disk headroom above the 31.25 GiB floor was smaller than the last measured owned compile target. Exact new-head CI is the gate; prior receipts remain prior-head evidence. <!-- anima-main-refresh-20261019:end -->

toasterson added 1 commit 2026-10-19 10:36:20 +00:00

Bound final MCP responses with private lossless artifact receipts f13ccc3164

toasterson added 5 commits 2026-10-19 11:35:16 +00:00

Label transcript sources and enforce explicit byte-tail and native cursor semantics 85e43729fe

test: cover all seven artifact tools sharing exact work-item ACL 98f1cac6ff

Fix transcript boundary Clippy findings from exact-head CI 76cbc9c555

Merge branch 'fix/q4-transcript-source-20261019' into fix/s2-bounded-tool-artifacts-20261019 ... ed0cdf24d2

# Conflicts:
#	.solstice/job.sh

Scope model result bounds and provide direct read pages 839423cee5

toasterson referenced this pull request 2026-10-19 12:05:58 +00:00

Run database integration pool for migration and test PRs #804

toasterson added 1 commit 2026-10-19 12:10:04 +00:00

Restrict nested model bounds to authenticated Akh dispatches ee277438f4

toasterson added 10 commits 2026-10-19 12:38:56 +00:00

Persist typed turn context diagnostics and expose bounded scoped reads 72c8d0ef49

WIP S1 exact release proof and durable cleanup diagnostics; pause for review fixes 5529e799b5

fix: isolate invalid telemetry from journal and legacy dispatch progress bd369174b4

fix: bind final manager receipt and reject nested false cleanup proofs f93b015b2f

test: retain undelivered retirement Changes before workspace archive c28019b637

fix: accept bounded provider display labels without weakening identifiers 17030d469b

Recover attested cancelled workspaces and keep cleanup retries actionable 7b777d8659

Merge commit '17030d469b' into fix/s2-bounded-tool-artifacts-20261019 ... 93e4a00096

# Conflicts:
#	crates/anima-server/src/mcp/handlers.rs

Merge commit '7b777d8659' into fix/s2-bounded-tool-artifacts-20261019 25ce440ff2

Keep preceding cleanup fixtures compatible with model auth context ccfb5b52e8

toasterson referenced this pull request 2026-10-19 12:42:32 +00:00

Persist canonical central model usage and terminal provenance #803

toasterson added 2 commits 2026-10-19 12:45:40 +00:00

Pin cleanup proof contract to the validated fixture follow-up 65cb3f97e0

Merge commit '65cb3f97' into fix/s2-bounded-tool-artifacts-20261019 24aee59cee

toasterson added 2 commits 2026-10-19 12:46:29 +00:00

Record paired proof contract provenance at the consumer a0b2f2fa40

Merge commit 'a0b2f2fa40' into fix/s2-bounded-tool-artifacts-20261019 3ec13eca07

toasterson added 1 commit 2026-10-19 15:26:15 +00:00

Merge current main into S2, preserving all paged reads and CI jobs 25b36af54c

toasterson added 17 commits 2026-10-19 15:31:56 +00:00

Persist central provider cooldowns and gate fresh admission eeaa095cf3

Refuse oversized final dispatch briefs without losing mandatory input fd0c99bb8f

Queue usage-limited providers behind durable reset windows and one probe 256b2df4a5

fix: recover ended limit probes and audit bounded operator clearing 465f8fe9f8

fix: retrieve lessons before opening allocation transaction 522178f193

Merge Q1 dependency to validate provider and allocation lock scope ... 03ad2d0929

# Conflicts:
#	.solstice/job.sh

fix: settle cancelled provider probes through completion observation 68a739f929

Merge Q1 cancelled-probe followup into brief-budget dependency ae9218991c

Merge current main into Q4, preserving transcript and conflict CI jobs bce6f02766

Merge current main into Q1, retaining usage and conflict diagnostics 42a8e9498b

Merge current main into Q2 with validated diagnostic and CI resolutions c424888e2b

Merge current main into D1, preserving context and conflict detail fields 3e48011285

Integrate refreshed Q4 dependency so Q1 merges cleanly in order 7bd713fa2b

Integrate refreshed Q1 and Q4 dependencies before Q2 ebf87a5ceb

Integrate refreshed Q2 dependencies into D1 with validated prompt assembly 36d3cb084f

Integrate refreshed D1 and Q-series dependencies before S1 bcfb7ae2ae

Integrate refreshed S1 prefix into S2 and retain all CI targets 55edd630b9

toasterson added 4 commits 2026-10-19 20:25:13 +00:00

Merge remote-tracking branch 'origin/main' into fix/s2-bounded-tool-artifacts-20261019 426714e8d8

Fix Q1 and Q4 integration fixtures after the merge train 7e39db1076

Assert exact excerpt bytes independently of JSON object key order 611fa65191

Merge branch 'fix/qseries-merged-integration-fixtures-20261019' into fix/s2-bounded-tool-artifacts-20261019 d9adb6d305

toasterson added 4 commits 2026-10-19 20:32:06 +00:00

Bind the limit fixture to the canonical graph phase instance a022fdeb1d

Merge branch 'fix/qseries-merged-integration-fixtures-20261019' into fix/s2-bounded-tool-artifacts-20261019 ea1d666450

Use canonical lifecycle setup for the Q2 dispatch fixture too 378d9e154a

Merge branch 'fix/qseries-merged-integration-fixtures-20261019' into fix/s2-bounded-tool-artifacts-20261019 c37f3cf4a5

toasterson referenced this pull request 2026-10-19 20:32:35 +00:00

Fix merged Q1/Q2/Q4 integration fixtures exposed by nightly #805

toasterson referenced this pull request 2026-10-19 20:52:41 +00:00

Fix baseline operator fixtures and centralize the external-merge lifecycle writer #806

toasterson merged commit b20a6fec6c into main 2026-10-19 20:58:12 +00:00

toasterson referenced this pull request from a commit 2026-10-19 20:58:12 +00:00

Merge PR #802: S2: bound final MCP responses with private lossless artifact receipts

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

Compat/Breaking

Breaking change that won't be backward compatible

  

Kind/Bug

Something is not working

  

Kind/Documentation

Documentation changes

  

Kind/Enhancement

Improve existing functionality

  

Kind/Feature

New functionality

  

Kind/Security

This is security issue

  

Kind/Testing

Issue or pull request related to testing

  

Priority

Critical

The priority is critical

  

Priority

High

The priority is high

  

Priority

Low

The priority is low

  

Priority

Medium

The priority is medium

  

Reviewed

Confirmed

Issue has been confirmed

  

Reviewed

Duplicate

This issue or pull request already exists

  

Reviewed

Invalid

Invalid issue

  

Reviewed

Won't Fix

This issue won't be fixed

  

Status

Abandoned

Somebody has started to work on this but abandoned work

  

Status

Blocked

Something is blocking this issue or pull request

  

Status

Need More Info

Feedback is required to reproduce issue or to continue work

No labels

Compat/Breaking

Kind/Bug

Kind/Documentation

Kind/Enhancement

Kind/Feature

Kind/Security

Kind/Testing

Priority

Critical

Priority

High

Priority

Low

Priority

Medium

Reviewed

Confirmed

Reviewed

Duplicate

Reviewed

Invalid

Reviewed

Won't Fix

Status

Abandoned

Status

Blocked

Status

Need More Info

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

toasterson/Anima!802

No description provided.