feat(lilith): dojo --enable-thinking for the openai arm (vLLM chat_template_kwargs) #559

Merged

toasterson merged 1 commit from feat/lilith-dojo-enable-thinking into main 2026-10-06 14:00:46 +00:00

Conversation 0 Commits 1 Files changed 3 +20 -2

toasterson commented 2026-10-06 12:57:11 +00:00

Owner

Copy link

Run 4 finding (plan §4 item 5, hypothesis 6): the served Gemma template opens the thinking channel only when the request sends chat_template_kwargs.enable_thinking=true. The dojo's OpenAI adapter never did, so every dojo and exam run so far measured a candidate that was trained with its High Grug trace on that channel without ever opening it (0 reasoning turns in 602 in the run-4 Green round; with the kwarg the Brown-stage candidate thinks in High Grug). Gap recall is unmeasurable without it.

--enable-thinking on the dojo runner (openai arm only) sends the kwarg and records enable_thinking in the run provenance; omitted keeps the server default, so the run-3-comparable arms are unchanged. One adapter test added.

Tests: uv run --no-project --with pytest python -m pytest dojo/tests -q green.

Run 4 finding (plan §4 item 5, hypothesis 6): the served Gemma template opens the thinking channel only when the request sends `chat_template_kwargs.enable_thinking=true`. The dojo's OpenAI adapter never did, so every dojo and exam run so far measured a candidate that was trained with its High Grug trace on that channel without ever opening it (0 reasoning turns in 602 in the run-4 Green round; with the kwarg the Brown-stage candidate thinks in High Grug). Gap recall is unmeasurable without it. `--enable-thinking` on the dojo runner (openai arm only) sends the kwarg and records `enable_thinking` in the run provenance; omitted keeps the server default, so the run-3-comparable arms are unchanged. One adapter test added. Tests: `uv run --no-project --with pytest python -m pytest dojo/tests -q` green.

toasterson added 1 commit 2026-10-06 12:57:11 +00:00

feat(lilith): dojo --enable-thinking for the openai arm (vLLM chat_template_kwargs) ... 6b0dc58f34

The served Gemma template opens the thinking channel only on request, so every dojo and exam run so far measured a candidate trained with its trace on that channel without it: 0 reasoning turns in 602, gap never observable. The flag sends chat_template_kwargs.enable_thinking=true and records it in the run provenance; omitted keeps the server default used in runs 1-3.

toasterson force-pushed feat/lilith-dojo-enable-thinking from 6b0dc58f34 to c2f85b97b5 2026-10-06 12:59:20 +00:00 Compare

toasterson merged commit 4d45b65ca7 into main 2026-10-06 14:00:46 +00:00

toasterson referenced this pull request from a commit 2026-10-06 14:00:47 +00:00

Merge pull request 'feat(lilith): dojo --enable-thinking for the openai arm (vLLM chat_template_kwargs)' (#559) from feat/lilith-dojo-enable-thinking into main

toasterson referenced this pull request 2026-10-06 18:56:58 +00:00

fix(lilith): exam adapter args have no enable_thinking; read it tolerantly #563

toasterson referenced this pull request from a commit 2026-10-06 18:56:59 +00:00

fix(lilith): exam adapter args have no enable_thinking; read it tolerantly

toasterson referenced this pull request 2026-10-06 20:12:14 +00:00

fix(lilith): exam adapter args have no enable_thinking; read it tolerantly #563

toasterson referenced this pull request 2026-10-06 22:20:23 +00:00

docs(lilith): RESULT for program r4 (siblings in context) — NOT CERTIFIED #568

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

follow-up

Documented operational follow-up

No labels

follow-up

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

toasterson/akh-medu!559

No description provided.