feat(lilith): episode tool-result budget for sibling teachers (dojo Broker) #551

Merged

toasterson merged 1 commit from feat/lilith-sibling-result-budget into main 2026-10-06 10:44:38 +00:00

Conversation 0 Commits 1 Files changed 4 +231 -12

toasterson commented 2026-10-06 08:52:24 +00:00

Owner

Copy link

Run 3 (RESULT: training/results/lilith-program-r3-20261005-RESULT.md, PR #550, hypothesis 1) lost 20 of 21 Green teacher siblings and 5 of 10 Brown ones as unfit: the sealed teachers read 29k–59k-character replay results (one get_work_item_detail, ten list_work_items pages) and the rendered records were 29k–118k tokens. Per-call max_result_chars bounds one result; nothing bounded the episode.

This adds an episode-level tool-result budget to the dojo Broker, enforced where the teacher's calls flow (the client-pushed HTTP MCP in acp_teacher, and the in-process runner), so a sibling record is exactly what the teacher saw:

• Broker(..., episode_result_chars): the straddling call is cut with the existing truncation encoding; later calls get a well-formed result_budget_exhausted marker (isError: false, receipt kept, the replay call still served and recorded).
• RunnerConfig.episode_result_chars (default None = today's behaviour); RunTrace.result_budget accounting (budget, used, calls, truncated, exhausted) lands in episodes.jsonl and on each sibling row as teacher.result_budget.
• dojo.siblings --max-result-chars (60000) and --episode-result-chars (None).

Run 4 (plan docs/ai/plans/2026-10-06-lilith-program-r4-siblings-in-context.md) runs the teachers with --max-result-chars 16000 --episode-result-chars 40000 against a 32k training context.

Tests: uv run --with pytest python -m pytest dojo/tests -q → 505 passed (10 new in test_result_budget.py).

Run 3 (RESULT: `training/results/lilith-program-r3-20261005-RESULT.md`, PR #550, hypothesis 1) lost 20 of 21 Green teacher siblings and 5 of 10 Brown ones as unfit: the sealed teachers read 29k–59k-character replay results (one `get_work_item_detail`, ten `list_work_items` pages) and the rendered records were 29k–118k tokens. Per-call `max_result_chars` bounds one result; nothing bounded the episode. This adds an episode-level tool-result budget to the dojo `Broker`, enforced where the teacher's calls flow (the client-pushed HTTP MCP in `acp_teacher`, and the in-process runner), so a sibling record is exactly what the teacher saw: - `Broker(..., episode_result_chars)`: the straddling call is cut with the existing truncation encoding; later calls get a well-formed `result_budget_exhausted` marker (`isError: false`, receipt kept, the replay call still served and recorded). - `RunnerConfig.episode_result_chars` (default None = today's behaviour); `RunTrace.result_budget` accounting (budget, used, calls, truncated, exhausted) lands in `episodes.jsonl` and on each sibling row as `teacher.result_budget`. - `dojo.siblings --max-result-chars` (60000) and `--episode-result-chars` (None). Run 4 (plan `docs/ai/plans/2026-10-06-lilith-program-r4-siblings-in-context.md`) runs the teachers with `--max-result-chars 16000 --episode-result-chars 40000` against a 32k training context. Tests: `uv run --with pytest python -m pytest dojo/tests -q` → 505 passed (10 new in `test_result_budget.py`).

toasterson added 1 commit 2026-10-06 08:52:25 +00:00

feat(lilith): episode tool-result budget for sibling teachers (dojo Broker) ... 58aeb7d52a

Run 3 lost most teacher-corrected siblings because replay tool results (59k-char details, 29k-char list pages read ten times) pushed the rendered record far past the training context, and only a per-call cap existed. The dojo Broker now enforces an optional episode-level result budget (cut the straddling call, then an exhaustion marker with receipts still served), so the record is exactly what the teacher saw; the accounting lands on the trace and on each sibling row.

toasterson referenced this pull request 2026-10-06 08:54:32 +00:00

feat(lilith): run 4 plan and run id (lilith-program-r4-20261006) #552

toasterson merged commit d22f42c255 into main 2026-10-06 10:44:38 +00:00

toasterson referenced this pull request from a commit 2026-10-06 10:44:39 +00:00

Merge pull request 'feat(lilith): episode tool-result budget for sibling teachers (dojo Broker)' (#551) from feat/lilith-sibling-result-budget into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

follow-up

Documented operational follow-up

No labels

follow-up

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

toasterson/akh-medu!551

No description provided.