docs(lilith): RESULT for program r4 (siblings in context) — NOT CERTIFIED #568
Loading…
Reference in a new issue
No description provided.
Delete branch "docs/lilith-program-r4-result"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Acceptance RESULT for
lilith-program-r4-20261006(plandocs/ai/plans/2026-10-06-lilith-program-r4-siblings-in-context.md, PR #552; treatment = run 3's hypotheses 1–3 with run 3 as the control).NOT CERTIFIED, 97 blockers. The treatment reached the trainer (15 teacher siblings and 9 first-divergence pairs trained, against 1 and 0 in run 3), and the candidate is worse than both run 3 and the frozen parent because it loops on tool reads: 126 of 218 sealed dojo episodes never answered (111 context overflows at 64k after repeated identical calls), 62 more hit the 20-round cap. Run 3's candidate on the same serving stack: 30/30 clean. Stage samples on the same 30 episodes: green 0.248, brown 0.499, black 0.059, candidate 0.000 (run 3 candidate 0.569) — the Brown-stage adapter is sound, the Black stage (Brown on-policy archive + Black base) collapses it.
Panel (κ verdict 0.67; grounding κ 0.31 after the new grounding rule, PR #554): candidate 19/51 vs parent 5/105 where it answered. Commitment on_time 0.275 (run 3 0.321, parent 0.551). Care 2/6 missed. Gap: measured for the first time with the thinking channel open (PR #559): recall 0.0 on 10 evidence-absent episodes; the trace fabricates before reading.
Incidents include the Ollama Cloud monthly limit (operator: capped until reset), the exam crash fixed by PR #563, ~$12 of idle pod spend after the crash, and a stranded pod host. Cost $28.70 (credit $201.49 → $172.79). Nine open hypotheses, the first being the Brown on-policy archive (length-asymmetric pairs).