toasterson/akh-medu
Watch
1
0
Fork
You've already forked akh-medu
 0
Code Issues 12 Pull requests 37 Projects Releases 3 Packages 8 Wiki Activity Actions 2

WIP: Replace continuous_learning Wikipedia/Wikidata/ConceptNet ingest with Seshat-mediated RAG ingester #269

Draft
toasterson wants to merge 1 commit from claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi into main
pull from: claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
merge into: toasterson:main
Conversation 0 Commits 1 Files changed 1 +46 -40
toasterson commented 2026-07-27 21:49:39 +00:00
Owner
Copy link

Anima work item 019dc57a-81aa-7762-ae0d-8f5219237b51.

⚠️ REACTIVATED 2026-06-02 — NOT IMPLEMENTED / PARTIAL (false "done").
Previously merged as a HOLLOW SCAFFOLD in Forgejo (only the .anima/wip marker was committed — zero implementation code for the replacement). A code audit on 2026-06-02 confirmed the replacement was never built. Reset from done → agent-ready for real implementation.

Tracking issue: Forgejo #233 — #233
Audit (per-PR evidence): docs/ai/2026-06-02-scaffold-merge-audit.md
Repo: code.aopc.cloud/toasterson/akh-medu — branches develop + main, currently v0.7.0

Real status & scope (PARTIAL — what actually needs building)

Build an idle-task ingester that pulls knowledge through Seshat's Archive (pgvector RAG, crates/seshat) and wire it into BOTH the daemon and the TUI IdleScheduler. The old direct Wikipedia/Wikidata/ConceptNet ingest is already deactivated in src/agent/idle.rs (the only part that landed) — the Seshat-mediated replacement was never built. This is why the item is marked PARTIAL: deactivation done, replacement missing.

Everything below is the ORIGINAL specification, retained for reference. The "done" status it previously carried was false.

Why this exists

The continuous_learning idle task in src/agent/idle.rs ran every 2h, took curiosity targets from the KG, fanned them out via bootstrap/expand.rs (Wikipedia categories + Wikidata SPARQL + ConceptNet relations), and asserted the merged result. In practice the merge produced too much low-quality / contradictory / duplicate noise — every cycle made the KG worse. It is now deactivated by removing it from the IdleScheduler::tick overdue array (match arm + timing fields kept so re-enabling is a one-line diff).

Don't re-enable the old path. Build a replacement that runs background ingestion through Seshat's already-deployed pipeline.

Proposed shape

curiosity_target (concept, gap)
  └── seshat_search    → relevant existing chunks (if any)
       └── seshat_wish → microtheory (RAG over corpus, optional LLM synthesis)
            └── triple_extractor → candidate triples with provenance
                 └── ElicitValidator (Phase 32) → arity, contradiction,
                       neural-coherence, self-consistency
                      └── confidence-gated assertion to KG

Key differences from the old path:

  • Source-of-truth is the curated corpus, not the open web; the garbage boundary moves into Seshat where it can be policed.
  • Two-pass synthesis: search retrieves chunks, then synthesis produces structured triples — seshat_wish is designed for this but currently returns "synthesis_not_yet_implemented" (crates/seshat/src/mcp/mod.rs ~line 207). Wiring synthesis is part of this work.
  • No raw-API ingest in the loop: Wikipedia/Wikidata/ConceptNet become bulk corpus imports into Seshat (one-shot, vetted, versioned), not live per-tick fetches.
  • Provenance everywhere: each assertion records (curiosity target, retrieved chunks, synthesis confidence, validator scores) → bulk retraction possible.

Work to do

  • Wire seshat_wish synthesis (today returns "synthesis_not_yet_implemented").
  • Build the triple extractor: Seshat synthesis output → candidate triples + provenance.
  • Build (or stub) the ElicitValidator — syntactic + dedup + arity gates for v1; full multi-layer validation tracks Phase 32.
  • Hook curiosity-target generation into a new background idle task seshat_curiosity in IdleScheduler (replacing the old continuous_learning slot), driven by KG-detected gaps. Wire it into BOTH the daemon and the TUI IdleScheduler.
  • Re-enable in IdleScheduler::tick once the pipeline is solid.
  • Add a kg_health_check tool: contradiction count, orphan count, dedup rate before/after — verify the ingester improves the KG, not erodes it.

Followups

  • bootstrap/expand.rs still holds the Wikipedia/Wikidata/ConceptNet querying code. Decide: keep as a one-shot vetted bulk-import path, or remove.
  • The 1d-bootstrap initial-seeding flow used the same path and needs a parallel update.

References

  • Disabled in the commit removing continuous_learning from src/agent/idle.rs::tick.
  • Old expansion entry point: src/bootstrap/expand.rs::query_wikipedia etc.
  • Seshat synthesis stub: crates/seshat/src/mcp/mod.rs::seshat_wish ~line 207.
2026-06-02 15:50 UTC

Summary

Replace the deactivated direct-web Knowledge Graph ingestion with a background Seshat-mediated RAG pipeline. This requires implementing seshat_wish for synthesis, building a triple extractor and an initial ElicitValidator, and hooking the new ingestion pipeline into the daemon and UI idle schedulers.

Approach

Implement the RAG synthesis step by replacing the "synthesis_not_yet_implemented" stub in crates/seshat/src/mcp/mod.rs with logic that takes chunked context from seshat_search and uses the configured LLM client (e.g., via anima-ai) to generate a microtheory. Next, build a triple_extractor step to parse the microtheory into candidate triples with full provenance (curiosity target, chunks used, confidence score). Create the ElicitValidator to filter these triples by basic arity rules, perform deduplication against the existing KG, and filter obvious contradictions. Finally, register a new seshat_curiosity idle task in src/agent/idle.rs (and its UI equivalent) to poll missing concept gaps and trigger the pipeline. Add a kg_health_check diagnostic CLI tool to monitor graph stability.

Files likely to change

  • crates/seshat/src/mcp/mod.rs
  • crates/seshat/src/synthesis.rs (guess)
  • crates/seshat/src/triple_extractor.rs (guess)
  • crates/seshat/src/validator.rs (guess)
  • src/agent/idle.rs
  • src/bootstrap/expand.rs
  • crates/anima-agent/src/idle.rs (guess for workspace-aligned daemon logic)
  • crates/anima-desktop/ui/pages/dashboard.slint (guess for TUI/UI IdleScheduler)
  • crates/anima-desktop/src/views/dashboard.rs (guess for TUI/UI IdleScheduler map)

Open questions

  • Should the "TUI" IdleScheduler references map to the Slint desktop client (anima-desktop), or does the agent run a distinct console terminal interface that isn't highlighted in the context?
  • Will seshat_wish utilize local LLMs provided by anima-exec or offload to the Gemini/OpenRouter bindings configured in anima-ai?
  • Should the legacy web fetching logic in bootstrap/expand.rs be completely excised now, or retained/refactored strictly as an offline bulk-import CLI command?
  • What are the explicit initial rules for the ElicitValidator (e.g., how strict is the deduplication/arity check before Phase 32 multi-layered logic)?

Complexity

L: Integrating a multi-stage background LLM pipeline (retrieval, synthesis, extraction, validation) into a daemon's concurrency model, establishing robust async error handling + provenance, and synchronizing it with a UI makes this a substantially large task.

Anima work item `019dc57a-81aa-7762-ae0d-8f5219237b51`. > ⚠️ **REACTIVATED 2026-06-02 — NOT IMPLEMENTED / PARTIAL (false "done").** > Previously merged as a **HOLLOW SCAFFOLD** in Forgejo (only the `.anima/wip` marker was committed — **zero implementation code** for the replacement). A code audit on 2026-06-02 confirmed the replacement was never built. Reset from `done` → **agent-ready** for real implementation. > > **Tracking issue:** Forgejo #233 — https://code.aopc.cloud/toasterson/akh-medu/issues/233 > **Audit (per-PR evidence):** `docs/ai/2026-06-02-scaffold-merge-audit.md` > **Repo:** code.aopc.cloud/toasterson/akh-medu — branches `develop` + `main`, currently v0.7.0 > > ### Real status & scope (PARTIAL — what actually needs building) > Build an **idle-task ingester that pulls knowledge through Seshat's Archive** (pgvector RAG, `crates/seshat`) and wire it into **BOTH the daemon and the TUI `IdleScheduler`**. The old direct Wikipedia/Wikidata/ConceptNet ingest is **already deactivated** in `src/agent/idle.rs` (the only part that landed) — the **Seshat-mediated replacement was never built**. This is why the item is marked **PARTIAL**: deactivation done, replacement missing. > > _Everything below is the ORIGINAL specification, retained for reference. The "done" status it previously carried was false._ > > --- ## Why this exists The `continuous_learning` idle task in `src/agent/idle.rs` ran every 2h, took curiosity targets from the KG, fanned them out via `bootstrap/expand.rs` (Wikipedia categories + Wikidata SPARQL + ConceptNet relations), and asserted the merged result. In practice the merge produced too much low-quality / contradictory / duplicate noise — every cycle made the KG worse. It is now deactivated by removing it from the `IdleScheduler::tick` overdue array (match arm + timing fields kept so re-enabling is a one-line diff). Don't re-enable the old path. Build a replacement that runs background ingestion through Seshat's already-deployed pipeline. ## Proposed shape ``` curiosity_target (concept, gap) └── seshat_search → relevant existing chunks (if any) └── seshat_wish → microtheory (RAG over corpus, optional LLM synthesis) └── triple_extractor → candidate triples with provenance └── ElicitValidator (Phase 32) → arity, contradiction, neural-coherence, self-consistency └── confidence-gated assertion to KG ``` Key differences from the old path: - **Source-of-truth is the curated corpus**, not the open web; the garbage boundary moves into Seshat where it can be policed. - **Two-pass synthesis**: search retrieves chunks, then synthesis produces structured triples — `seshat_wish` is designed for this but currently returns `"synthesis_not_yet_implemented"` (`crates/seshat/src/mcp/mod.rs` ~line 207). Wiring synthesis is part of this work. - **No raw-API ingest in the loop**: Wikipedia/Wikidata/ConceptNet become bulk corpus *imports* into Seshat (one-shot, vetted, versioned), not live per-tick fetches. - **Provenance everywhere**: each assertion records (curiosity target, retrieved chunks, synthesis confidence, validator scores) → bulk retraction possible. ## Work to do - [ ] Wire `seshat_wish` synthesis (today returns `"synthesis_not_yet_implemented"`). - [ ] Build the triple extractor: Seshat synthesis output → candidate triples + provenance. - [ ] Build (or stub) the `ElicitValidator` — syntactic + dedup + arity gates for v1; full multi-layer validation tracks Phase 32. - [ ] Hook curiosity-target generation into a new background idle task `seshat_curiosity` in `IdleScheduler` (replacing the old `continuous_learning` slot), driven by KG-detected gaps. **Wire it into BOTH the daemon and the TUI IdleScheduler.** - [ ] Re-enable in `IdleScheduler::tick` once the pipeline is solid. - [ ] Add a `kg_health_check` tool: contradiction count, orphan count, dedup rate before/after — verify the ingester improves the KG, not erodes it. ## Followups - `bootstrap/expand.rs` still holds the Wikipedia/Wikidata/ConceptNet querying code. Decide: keep as a one-shot vetted bulk-import path, or remove. - The `1d-bootstrap` initial-seeding flow used the same path and needs a parallel update. ## References - Disabled in the commit removing `continuous_learning` from `src/agent/idle.rs::tick`. - Old expansion entry point: `src/bootstrap/expand.rs::query_wikipedia` etc. - Seshat synthesis stub: `crates/seshat/src/mcp/mod.rs::seshat_wish` ~line 207. <!-- ANIMA TRIAGE PLAN --> 2026-06-02 15:50 UTC ## Summary Replace the deactivated direct-web Knowledge Graph ingestion with a background Seshat-mediated RAG pipeline. This requires implementing `seshat_wish` for synthesis, building a triple extractor and an initial `ElicitValidator`, and hooking the new ingestion pipeline into the daemon and UI idle schedulers. ## Approach Implement the RAG synthesis step by replacing the `"synthesis_not_yet_implemented"` stub in `crates/seshat/src/mcp/mod.rs` with logic that takes chunked context from `seshat_search` and uses the configured LLM client (e.g., via `anima-ai`) to generate a microtheory. Next, build a `triple_extractor` step to parse the microtheory into candidate triples with full provenance (curiosity target, chunks used, confidence score). Create the `ElicitValidator` to filter these triples by basic arity rules, perform deduplication against the existing KG, and filter obvious contradictions. Finally, register a new `seshat_curiosity` idle task in `src/agent/idle.rs` (and its UI equivalent) to poll missing concept gaps and trigger the pipeline. Add a `kg_health_check` diagnostic CLI tool to monitor graph stability. ## Files likely to change * `crates/seshat/src/mcp/mod.rs` * `crates/seshat/src/synthesis.rs` (guess) * `crates/seshat/src/triple_extractor.rs` (guess) * `crates/seshat/src/validator.rs` (guess) * `src/agent/idle.rs` * `src/bootstrap/expand.rs` * `crates/anima-agent/src/idle.rs` (guess for workspace-aligned daemon logic) * `crates/anima-desktop/ui/pages/dashboard.slint` (guess for TUI/UI IdleScheduler) * `crates/anima-desktop/src/views/dashboard.rs` (guess for TUI/UI IdleScheduler map) ## Open questions * Should the "TUI" IdleScheduler references map to the Slint desktop client (`anima-desktop`), or does the agent run a distinct console terminal interface that isn't highlighted in the context? * Will `seshat_wish` utilize local LLMs provided by `anima-exec` or offload to the Gemini/OpenRouter bindings configured in `anima-ai`? * Should the legacy web fetching logic in `bootstrap/expand.rs` be completely excised now, or retained/refactored strictly as an offline bulk-import CLI command? * What are the explicit initial rules for the `ElicitValidator` (e.g., how strict is the deduplication/arity check before Phase 32 multi-layered logic)? ## Complexity L: Integrating a multi-stage background LLM pipeline (retrieval, synthesis, extraction, validation) into a daemon's concurrency model, establishing robust async error handling + provenance, and synchronizing it with a UI makes this a substantially large task.
wip: triage Replace continuous_learning Wikipedia/Wikidata/ConceptNet ingest with Seshat-mediated RAG ingester
All checks were successful
CI / publish-chart (pull_request) Successful in 9m58s
Details
CI / publish-chart (push) Successful in 10m18s
Details
CI / check-seshat (pull_request) Successful in 14m45s
Details
CI / check-seshat (push) Successful in 14m46s
Details
CI / docker-seshd (pull_request) Successful in 18m43s
Details
CI / docker-seshd (push) Successful in 18m44s
Details
96cc16d125
All checks were successful
CI / publish-chart (pull_request) Successful in 9m58s
Details
CI / publish-chart (push) Successful in 10m18s
Details
CI / check-seshat (pull_request) Successful in 14m45s
Details
CI / check-seshat (push) Successful in 14m46s
Details
CI / docker-seshd (pull_request) Successful in 18m43s
Details
CI / docker-seshd (push) Successful in 18m44s
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi:claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git switch claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git switch claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git rebase main
git switch main
git merge --ff-only claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git switch claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git rebase main
git switch main
git merge --no-ff claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git switch main
git merge --squash claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git switch main
git merge --ff-only claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git switch main
git merge claude/wi-019dc57a-replace-continuous-learning-wikipedia-wi
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
follow-up
Documented operational follow-up

No labels
follow-up
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
toasterson/akh-medu!269
No description provided.