toasterson/akh-medu
Watch
1
0
Fork
You've already forked akh-medu
 0
Code Issues 12 Pull requests 37 Projects Releases 3 Packages 8 Wiki Activity Actions 1

feat(gemma4): CLM-R1 KV store — sliding ring, v̂-only global KV, strict window, chunked prefill, M1 parity harness (ADR 125) #556

Open
toasterson wants to merge 19 commits from clm/r1-gemma4-kv into docs/context-language-model
pull from: clm/r1-gemma4-kv
merge into: toasterson:docs/context-language-model
Conversation 0 Commits 19 Files changed 13 +4428 -392
toasterson commented 2026-10-06 10:56:45 +00:00
Owner
Copy link

Slice CLM-R1 of the Context Language Model plan (ADR 125 D13/D15; stacked on #547, base docs/context-language-model). Implemented, not yet certified: the real-model parity gates (M1 §3) are running on hephaistos (CPU, ≤8K rows first); this PR carries the code and the pre-registered harness.

src/nlu/vendored/quantized_gemma4.rs (+ parity harness in tests/ and examples/):

  • One typed KV store replacing three ad-hoc paths: SlidingRing (1,024-token window + prefill chunk, f16 where supported, bf16 fallback) and GlobalVHat (global layers store only v̂; K = R(p)·(w_k ⊙ v̂) derived at read; the duplicate K projection matmul is gone).
  • Position-indexed causal masks, strict window q − k < 1024 in both prefill and decode — fixes the pre-existing bug where single-token decode ignored the window (mask = None).
  • Chunked prefill (default 256 tokens): 49,152-token session KV = 822,083,584 bytes (M1a B(C)), under the 0.85 GB budget.
  • Pre-registered numerics from the M1/M1a/M1b math notes (Kimi K3): bf16 unit roundoff 2⁻⁸, f16 mixed bound, dtype-dependent ring shift caps f16 52 / bf16 6 (M1b: the f32 RoPE table's large-angle rounding is the model's own convention — HF and llama.cpp share it — and its phase gap is budgeted inside the 3.125e-2 envelope).
  • Parity harness: f32 references (a1 unmodified, a2 windowed), tier (a)/(b) gates with M1's numbers, report-only histograms, cross-device RoPE table diff, and AKH_GEMMA4_PARITY_MAX_LENGTH for RAM-limited hosts (manifest records the cap; gates unchanged).

Implemented by GPT-6.1 Sol from docs/ai/phases/clm/CLM-R1-brief.md; review round 1 in CLM-R1-review.md, all items fixed; the six previously failing tests outside Gemma 4 were shown to fail identically on the base commit.

Validation (local, synthetic fixtures only, no model download): default suite 3,787 passed; Metal suite 3,868 passed; focused Gemma 4 17 CPU / 21 Metal; HIP/Vulkan unmeasured.

Slice **CLM-R1** of the Context Language Model plan (ADR 125 D13/D15; stacked on #547, base `docs/context-language-model`). **Implemented, not yet certified**: the real-model parity gates (M1 §3) are running on hephaistos (CPU, ≤8K rows first); this PR carries the code and the pre-registered harness. `src/nlu/vendored/quantized_gemma4.rs` (+ parity harness in `tests/` and `examples/`): - One typed KV store replacing three ad-hoc paths: `SlidingRing` (1,024-token window + prefill chunk, f16 where supported, bf16 fallback) and `GlobalVHat` (global layers store only `v̂`; `K = R(p)·(w_k ⊙ v̂)` derived at read; the duplicate K projection matmul is gone). - Position-indexed causal masks, strict window `q − k < 1024` in **both** prefill and decode — fixes the pre-existing bug where single-token decode ignored the window (`mask = None`). - Chunked prefill (default 256 tokens): 49,152-token session KV = 822,083,584 bytes (M1a `B(C)`), under the 0.85 GB budget. - Pre-registered numerics from the M1/M1a/M1b math notes (Kimi K3): bf16 unit roundoff 2⁻⁸, f16 mixed bound, dtype-dependent ring shift caps **f16 52 / bf16 6** (M1b: the f32 RoPE table's large-angle rounding is the model's own convention — HF and llama.cpp share it — and its phase gap is budgeted inside the 3.125e-2 envelope). - Parity harness: f32 references (a1 unmodified, a2 windowed), tier (a)/(b) gates with M1's numbers, report-only histograms, cross-device RoPE table diff, and `AKH_GEMMA4_PARITY_MAX_LENGTH` for RAM-limited hosts (manifest records the cap; gates unchanged). Implemented by GPT-6.1 Sol from `docs/ai/phases/clm/CLM-R1-brief.md`; review round 1 in `CLM-R1-review.md`, all items fixed; the six previously failing tests outside Gemma 4 were shown to fail identically on the base commit. Validation (local, synthetic fixtures only, no model download): default suite 3,787 passed; Metal suite 3,868 passed; focused Gemma 4 17 CPU / 21 Metal; HIP/Vulkan unmeasured.
docs(clm): M1b brief — RoPE table precision at large positions, delta-rotation bounds under f32 table error
Some checks failed
CI / check-seshat (pull_request) Has been cancelled
Details
CI / check-akh-medu (pull_request) Has been cancelled
Details
CI / test-akh-medu (pull_request) Has been cancelled
Details
CI / docker-seshd (pull_request) Has been cancelled
Details
CI / docker-sokard (pull_request) Has been cancelled
Details
CI / docker-akh-transcripts (pull_request) Has been cancelled
Details
CI / publish-chart (pull_request) Has been cancelled
Details
CI / platform-images-home (pull_request) Has been cancelled
Details
CI / platform-images-core (pull_request) Has been cancelled
Details
CI / platform-images-daemons (pull_request) Has been cancelled
Details
CI / platform-images-fleet (pull_request) Has been cancelled
Details
CI / platform-images-collect (pull_request) Has been cancelled
Details
CI / test-sokar (pull_request) Has been cancelled
Details
CI / test-session-platform (pull_request) Has been cancelled
Details
e438fd11a8
 
Answers the five M1b-brief questions: the large-angle f32 error is the
model's own convention (HF transformers and llama.cpp carry the same
f32 angle + f32 trig), bounds the angle/trig error and explains the
measured 0.0113, re-derives the delta-rotation composed-vs-fresh gap
(<= 2u*theta*p, constant in m) and tightens the shift caps to f16 52 /
bf16 6, pre-registers the decision to keep the conventional f32 table
(no argument reduction), and confirms within-device parity with
extended report-only cross-device archival.

Documentation only; no Rust changes.
fix(clm-r1): ring shift caps 52 (f16) / 6 (bf16) per M1b (table phase gap in the envelope)
Some checks failed
CI / test-session-platform (pull_request) Has been cancelled
Details
CI / test-sokar (pull_request) Has been cancelled
Details
CI / check-seshat (pull_request) Has been cancelled
Details
CI / check-akh-medu (pull_request) Has been cancelled
Details
CI / test-akh-medu (pull_request) Has been cancelled
Details
CI / docker-seshd (pull_request) Has been cancelled
Details
CI / docker-sokard (pull_request) Has been cancelled
Details
CI / docker-akh-transcripts (pull_request) Has been cancelled
Details
CI / publish-chart (pull_request) Has been cancelled
Details
CI / platform-images-home (pull_request) Has been cancelled
Details
CI / platform-images-core (pull_request) Has been cancelled
Details
CI / platform-images-daemons (pull_request) Has been cancelled
Details
CI / platform-images-fleet (pull_request) Has been cancelled
Details
CI / platform-images-collect (pull_request) Has been cancelled
Details
75a45fd776
test(clm-r1): shift-envelope assertion uses M1b's exact table phase gap in f64
All checks were successful
CI / test-sokar (pull_request) Successful in 22m25s
Details
CI / test-session-platform (pull_request) Successful in 27m0s
Details
CI / docker-sokard (pull_request) Successful in 15m4s
Details
CI / check-akh-medu (pull_request) Successful in 27m16s
Details
CI / publish-chart (pull_request) Successful in 12m59s
Details
CI / check-seshat (pull_request) Successful in 33m56s
Details
CI / docker-akh-transcripts (pull_request) Successful in 23m43s
Details
CI / test-akh-medu (pull_request) Successful in 1h9m33s
Details
CI / docker-seshd (pull_request) Successful in 1h12m14s
Details
CI / platform-images-core (pull_request) Successful in 6m38s
Details
CI / platform-images-home (pull_request) Successful in 19m30s
Details
CI / platform-images-daemons (pull_request) Successful in 2h23m34s
Details
CI / platform-images-fleet (pull_request) Successful in 7m26s
Details
CI / platform-images-collect (pull_request) Successful in 31m56s
Details
de9fcbdae5
All checks were successful
CI / test-sokar (pull_request) Successful in 22m25s
Details
CI / test-session-platform (pull_request) Successful in 27m0s
Details
CI / docker-sokard (pull_request) Successful in 15m4s
Details
CI / check-akh-medu (pull_request) Successful in 27m16s
Details
CI / publish-chart (pull_request) Successful in 12m59s
Details
CI / check-seshat (pull_request) Successful in 33m56s
Details
CI / docker-akh-transcripts (pull_request) Successful in 23m43s
Details
CI / test-akh-medu (pull_request) Successful in 1h9m33s
Details
CI / docker-seshd (pull_request) Successful in 1h12m14s
Details
CI / platform-images-core (pull_request) Successful in 6m38s
Details
CI / platform-images-home (pull_request) Successful in 19m30s
Details
CI / platform-images-daemons (pull_request) Successful in 2h23m34s
Details
CI / platform-images-fleet (pull_request) Successful in 7m26s
Details
CI / platform-images-collect (pull_request) Successful in 31m56s
Details
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin clm/r1-gemma4-kv:clm/r1-gemma4-kv
git switch clm/r1-gemma4-kv

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch docs/context-language-model
git merge --no-ff clm/r1-gemma4-kv
git switch clm/r1-gemma4-kv
git rebase docs/context-language-model
git switch docs/context-language-model
git merge --ff-only clm/r1-gemma4-kv
git switch clm/r1-gemma4-kv
git rebase docs/context-language-model
git switch docs/context-language-model
git merge --no-ff clm/r1-gemma4-kv
git switch docs/context-language-model
git merge --squash clm/r1-gemma4-kv
git switch docs/context-language-model
git merge --ff-only clm/r1-gemma4-kv
git switch docs/context-language-model
git merge clm/r1-gemma4-kv
git push origin docs/context-language-model
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
follow-up
Documented operational follow-up

No labels
follow-up
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
toasterson/akh-medu!556
No description provided.