feat(runtime): disposable profiles and durable receipt acceptance (PR 3/9) #654

Merged

toasterson merged 4 commits from feat/disposable-runtime-profiles into main 2026-10-16 03:15:11 +00:00

Conversation 0 Commits 4 Files changed 16 +1193 -28

toasterson commented 2026-10-16 00:25:12 +00:00

Owner

Copy link

Losing a runtime profile must not erase accepted request receipts or silently turn an old provider-thread continuation into a fresh execution. This is PR 3/9 of the merged platform relocation design (#651), based on main after #653.

• Add opt-in disposable profile preparation: bounded ESO auth seeding, private files, a local generation marker, and retained warm auth/cache/config. Unchanged managed-file indexes are no longer rewritten; no cache restoration or startup installation is introduced.
• Persist generation/continuation evidence through Store, reject stale history before admission, and retain unknown disposable-profile receipts until reconciliation. Exact request retries still return live/archived receipts before touching local profiles.
• Fix pre-admission cancellation to honor archived receipts and add the authorized native-inference endpoint. It never replaces an existing accepted/completed job.
• Close a transcript coverage gap: offsets previously conflated acknowledged and quarantined records. Add separate post-ack counters and a quiesced counts-only report with explicit pending/unknown/lost evidence. Compressed replay and legacy watermarks remain conservative unknowns; reports never authorize discard.

The runbook is scripts/platform-storage/DISPOSABLE-RUNTIMES.md. It records the Pi write budget, profile ownership/rendering requirements, provider refresh-token prerequisite, actual-image conformance gates, transcript evidence limitations and rollback boundaries.

Validation: 342 targeted tests passed, zero failures; one existing live OpenCode test ignored (requires an installed/authenticated provider). Command: cargo test --locked --no-default-features -p akh-session-types -p akh-runtime -p akh-session-services -p akh-transcripts --features akh-session-services/runtime --lib --test providers. The akh-harvest binary build and synthetic coverage CLI smoke check passed (quiescence required, valid counts-only JSON, no DB credential, no watermark writes, no payload output). Formatting and diff whitespace checks passed. CI now also runs the transcript library tests. Local builds used four jobs, debug info disabled and incremental compilation disabled in a task-specific local cache; the shared houses target was untouched. Deterministic provider fixtures and local redb stores test the runtime contract; they do not constitute real-provider/image conformance or new live Postgres acceptance. No Docker harness was changed or run. No image build, provider installation, deployment, data import, archive extraction, manifest/volume switch, Board use or new credential/store provisioning. Legacy configuration does not opt into disposable behavior. PRs 7–9 remain gated by the disposition ledger and separate operator GO. Please review and merge; I will not merge this PR.

Publication provenance: Git HTTP reference advertisements arrived completely but their responses did not finish, preventing Git push. Published the same files through Forgejo's batch file API. Remote bbf8c22e is exactly local signed 62c4c10f with its signature omitted by that API: matching tree b5d67125024592cbe8279d268ca317fa11606697, parent, author/committer, timestamps and message, verified by reconstructing and hashing the unsigned commit. Local tests cover that exact tree; the remote commit is unsigned.

Review fixes

Intentional compatibility tightening: the legacy POST /v1/jobs/cancel-unsubmitted route now uses the same authorization as submit, even without disposable profiles. Inference payloads require the inference role, a grant for the requested Akh, and successful inference.validate; a session-role-only principal now receives HTTP 403. Non-inference payloads still require session. Core clients continue using the legacy route and must meet these requirements for inference pre-cancellation. This alignment is intentional and is now explicit in the runbook.

New pre-admission cancellation receipts live under archived_job/ instead of job/: by-ID reads cover both, but scans of job/ alone no longer include these Cancelled rows.

The runbook also documents that cold preparation can materialize private auth/config and the generation marker before history/conformance refusal, without admitting a job or launching a provider. Archived jobs retain their generation side records; no garbage collection is introduced.

Review follow-up b754b167 changes documentation only. Wording was checked against the authorization, admission and archival code; git diff --check passed. The existing 342-test evidence above applies to unchanged source code; no local rebuild or Docker harness rerun was needed.

Publication follow-up: the operator identified the earlier HTTP stall as a crawler swarm on milestone pages; Talos #215 adds nginx limiting there. This follow-up was published with normal Git push, with no batch API fallback. The earlier unsigned API publication described above remains part of the branch history.

CI retry after review fixes

Run 9955 (UI run 4287, test-session-platform) failed at runtime::tests::sweep_closes_a_job_unknown_past_the_stall_threshold_and_only_that_one: actual ["fresh", "stale"], expected ["stale"] (runtime.rs:2273; 170 passed, 1 failed, 6 ignored). Expanded Forgejo UI logs also contained repeated/misassigned output, so the assertion was recovered from Solstice's original per-step log for request d24f9843-fe16-479b-8983-54ff0fc8499b, task 37054, lines 222–240.

This test and its 1,000 ms threshold are unchanged from base main. It records fresh with wall-clock time before committing and constructing the runtime; runner delay can exceed that window and legitimately expire both records. Runtime/provider suites passed, and the transcript-coverage step was not reached. Classified as an existing test timing flake, not a disposable-profile or coverage assertion failure.

Per operator direction, empty commit 6b3f821e retriggers CI without changing the reviewed tree. Whole-run validation remains pending; this retry does not authorize merge or deployment.

The first retry (run 9956 / UI 4288, 6b3f821e) passed test-session-platform: the broad service library suite reported 171 passed / 0 failed / 6 ignored, transcript coverage 204 passed / 0 failed, and the disposable Postgres/OpenBao contracts completed successfully. All three daemon Docker checks, Sokar, Seshat, the core compile check, and chart check also passed.

That run instead failed the unchanged thinking::tests::analogy_goal_recovers_structured_binding test in test-akh-medu: actual Some(SymbolId(16)), expected Some(SymbolId(14)), halt DiminishingReturns { marginal_value: -0.05 }; 3,943 passed / 1 failed / 21 ignored. Original evidence: Solstice request 8e5c211a-a39c-4cc3-8d2f-8b07451e7189, task 37072, lines 255–271. The test, thinking implementation and lockfile have no diff from base main. Its approximate search index uses unordered insertion; nondeterministic search is a plausible explanation, not a proven deterministic root cause. No unrelated thinking code was changed.

Second empty retry d41d5ed4 preserves the same reviewed tree and starts run 9958. Whole-run CI is still pending; the earlier platform success alone is not the requested final gate.

Final current-head CI evidence

All 14/14 jobs passed in run 9958 / UI 4289 at d41d5ed46563b97efa049bbc29aa355692d1da9c. Both retry commits are empty; the tree is identical to reviewed docs-fix head b754b167.

The broad CI-feature platform suite passed, including the formerly timing-sensitive sweep assertion; transcript coverage passed 204/204; disposable Postgres/OpenBao contracts passed. The core library suite also passed (3,944 passed, 0 failed, 21 ignored), including the unchanged analogy test that failed in the first retry. Core integration, all Docker checks, chart and all five platform-image jobs are green at this head. The two earlier failures and their original runner evidence remain documented above; no assertion was disabled or weakened.

No local Rust rebuild was performed for these empty retries. No merge, deployment, data import or volume switch was performed. Ready for operator review and merge.

Losing a runtime profile must not erase accepted request receipts or silently turn an old provider-thread continuation into a fresh execution. This is PR 3/9 of the merged platform relocation design (#651), based on main after #653. - Add opt-in disposable profile preparation: bounded ESO auth seeding, private files, a local generation marker, and retained warm auth/cache/config. Unchanged managed-file indexes are no longer rewritten; no cache restoration or startup installation is introduced. - Persist generation/continuation evidence through `Store`, reject stale history before admission, and retain unknown disposable-profile receipts until reconciliation. Exact request retries still return live/archived receipts before touching local profiles. - Fix pre-admission cancellation to honor archived receipts and add the authorized native-inference endpoint. It never replaces an existing accepted/completed job. - Close a transcript coverage gap: offsets previously conflated acknowledged and quarantined records. Add separate post-ack counters and a quiesced counts-only report with explicit pending/unknown/lost evidence. Compressed replay and legacy watermarks remain conservative unknowns; reports never authorize discard. The runbook is `scripts/platform-storage/DISPOSABLE-RUNTIMES.md`. It records the Pi write budget, profile ownership/rendering requirements, provider refresh-token prerequisite, actual-image conformance gates, transcript evidence limitations and rollback boundaries. Validation: **342 targeted tests passed**, zero failures; one existing live OpenCode test ignored (requires an installed/authenticated provider). Command: `cargo test --locked --no-default-features -p akh-session-types -p akh-runtime -p akh-session-services -p akh-transcripts --features akh-session-services/runtime --lib --test providers`. The `akh-harvest` binary build and synthetic coverage CLI smoke check passed (quiescence required, valid counts-only JSON, no DB credential, no watermark writes, no payload output). Formatting and diff whitespace checks passed. CI now also runs the transcript library tests. Local builds used four jobs, debug info disabled and incremental compilation disabled in a task-specific local cache; the shared houses target was untouched. Deterministic provider fixtures and local redb stores test the runtime contract; they do not constitute real-provider/image conformance or new live Postgres acceptance. No Docker harness was changed or run. No image build, provider installation, deployment, data import, archive extraction, manifest/volume switch, Board use or new credential/store provisioning. Legacy configuration does not opt into disposable behavior. PRs 7–9 remain gated by the disposition ledger and separate operator GO. Please review and merge; I will not merge this PR. Publication provenance: Git HTTP reference advertisements arrived completely but their responses did not finish, preventing Git push. Published the same files through Forgejo's batch file API. Remote `bbf8c22e` is exactly local signed `62c4c10f` with its signature omitted by that API: matching tree `b5d67125024592cbe8279d268ca317fa11606697`, parent, author/committer, timestamps and message, verified by reconstructing and hashing the unsigned commit. Local tests cover that exact tree; the remote commit is unsigned. ## Review fixes Intentional compatibility tightening: the legacy `POST /v1/jobs/cancel-unsubmitted` route now uses the same authorization as `submit`, even without disposable profiles. Inference payloads require the `inference` role, a grant for the requested Akh, and successful `inference.validate`; a session-role-only principal now receives HTTP 403. Non-inference payloads still require `session`. Core clients continue using the legacy route and must meet these requirements for inference pre-cancellation. This alignment is intentional and is now explicit in the runbook. New pre-admission cancellation receipts live under `archived_job/` instead of `job/`: by-ID reads cover both, but scans of `job/` alone no longer include these Cancelled rows. The runbook also documents that cold preparation can materialize private auth/config and the generation marker before history/conformance refusal, without admitting a job or launching a provider. Archived jobs retain their generation side records; no garbage collection is introduced. Review follow-up `b754b167` changes documentation only. Wording was checked against the authorization, admission and archival code; `git diff --check` passed. The existing 342-test evidence above applies to unchanged source code; no local rebuild or Docker harness rerun was needed. Publication follow-up: the operator identified the earlier HTTP stall as a crawler swarm on milestone pages; Talos #215 adds nginx limiting there. This follow-up was published with normal Git push, with no batch API fallback. The earlier unsigned API publication described above remains part of the branch history. ## CI retry after review fixes Run 9955 ([UI run 4287, test-session-platform](https://code.aopc.cloud/toasterson/akh-medu/actions/runs/4287/jobs/0)) failed at `runtime::tests::sweep_closes_a_job_unknown_past_the_stall_threshold_and_only_that_one`: actual `["fresh", "stale"]`, expected `["stale"]` (`runtime.rs:2273`; 170 passed, 1 failed, 6 ignored). Expanded Forgejo UI logs also contained repeated/misassigned output, so the assertion was recovered from Solstice's original per-step log for request `d24f9843-fe16-479b-8983-54ff0fc8499b`, task 37054, lines 222–240. This test and its 1,000 ms threshold are unchanged from base main. It records `fresh` with wall-clock time before committing and constructing the runtime; runner delay can exceed that window and legitimately expire both records. Runtime/provider suites passed, and the transcript-coverage step was not reached. Classified as an existing test timing flake, not a disposable-profile or coverage assertion failure. Per operator direction, empty commit `6b3f821e` retriggers CI without changing the reviewed tree. Whole-run validation remains pending; this retry does not authorize merge or deployment. The first retry (run 9956 / UI 4288, `6b3f821e`) passed `test-session-platform`: the broad service library suite reported 171 passed / 0 failed / 6 ignored, transcript coverage 204 passed / 0 failed, and the disposable Postgres/OpenBao contracts completed successfully. All three daemon Docker checks, Sokar, Seshat, the core compile check, and chart check also passed. That run instead failed the unchanged `thinking::tests::analogy_goal_recovers_structured_binding` test in `test-akh-medu`: actual `Some(SymbolId(16))`, expected `Some(SymbolId(14))`, halt `DiminishingReturns { marginal_value: -0.05 }`; 3,943 passed / 1 failed / 21 ignored. Original evidence: Solstice request `8e5c211a-a39c-4cc3-8d2f-8b07451e7189`, task 37072, lines 255–271. The test, thinking implementation and lockfile have no diff from base main. Its approximate search index uses unordered insertion; nondeterministic search is a plausible explanation, not a proven deterministic root cause. No unrelated thinking code was changed. Second empty retry `d41d5ed4` preserves the same reviewed tree and starts run 9958. Whole-run CI is still pending; the earlier platform success alone is not the requested final gate. ## Final current-head CI evidence All **14/14 jobs passed** in [run 9958 / UI 4289](https://code.aopc.cloud/toasterson/akh-medu/actions/runs/4289) at `d41d5ed46563b97efa049bbc29aa355692d1da9c`. Both retry commits are empty; the tree is identical to reviewed docs-fix head `b754b167`. The broad CI-feature platform suite passed, including the formerly timing-sensitive sweep assertion; transcript coverage passed 204/204; disposable Postgres/OpenBao contracts passed. The core library suite also passed (3,944 passed, 0 failed, 21 ignored), including the unchanged analogy test that failed in the first retry. Core integration, all Docker checks, chart and all five platform-image jobs are green at this head. The two earlier failures and their original runner evidence remain documented above; no assertion was disabled or weakened. No local Rust rebuild was performed for these empty retries. No merge, deployment, data import or volume switch was performed. Ready for operator review and merge.

toasterson added 1 commit 2026-10-16 00:25:20 +00:00

feat(runtime): prepare disposable profiles with durable receipt fencing bbf8c22ebe

toasterson added 1 commit 2026-10-16 00:40:30 +00:00

docs(runtime): clarify cancellation compatibility and profile preparation b754b16761

toasterson added 1 commit 2026-10-16 01:37:24 +00:00

ci: retry after legacy sweep timing flake 6b3f821ec5

toasterson added 1 commit 2026-10-16 02:32:39 +00:00

ci: retry unchanged analogy search test flake d41d5ed465

toasterson merged commit f0153cf636 into main 2026-10-16 03:15:11 +00:00

toasterson referenced this pull request from a commit 2026-10-16 03:15:14 +00:00

feat(runtime): disposable profiles and durable receipt acceptance (PR 3/9) (#654)

toasterson deleted branch feat/disposable-runtime-profiles 2026-10-16 03:15:16 +00:00

toasterson referenced this pull request 2026-10-16 08:21:57 +00:00

Make unknown-job sweep tests deterministic after platform-storage PR 6 (CI incident evidence) #662

toasterson referenced this pull request 2026-10-16 08:23:34 +00:00

feat(sharing): Postgres companion and verified link migration (PR 5/9) #661

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

follow-up

Documented operational follow-up

No labels

follow-up

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

toasterson/akh-medu!654

No description provided.