Fix Gemma 4 tool handoff and native call parsing #558
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/electra-gemma-handoff"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Gemma 4 emits
<|tool_response>to yield to a tool, but Candle currently continues generation beyond it. A recorded GPU qualification selected read_file correctly, then fabricated a file result before the runtime rejected its markup.Stop generation at that handoff token and recognize Gemma native call envelopes and typed argument syntax. Validate the complete response before dispatch, preserve argument text, reject malformed calls, duplicate keys, excessive nesting and generated tool observations, and keep calls inside reasoning non-executable. Generic JSON and Qwen XML paths remain supported.
Added regression tests for the observed read_file output, native and JSON arguments, malformed second calls, invented observations, reasoning-only calls and argument strings containing reasoning markup. Targeted Vulkan-feature library tests are running on Hephaistos; no feature-branch daemon has been deployed.
Hephaistos validation on exact head
0fd81afeb019adf66a633de7c6f026dacd304c41:cargo test --locked --release -p akh-session-services --features vulkan --lib internal::nativepassed 22 tests, 0 failures, 1 existing ignored test. This includes both native Gemma argument encodings, rejection of generated observations and malformed subsequent calls, and non-execution of calls inside reasoning. Broader Candle backend tests are running separately. Shared PR CI has started. No feature-branch daemon was deployed; the currently served qualification artifacts still come from clean mainfcb8c6f7.