Switch from :latest to SHA-pinned image tag #6

Open
toasterson wants to merge 8 commits from claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t into main
Owner

Draft PR for Anima work item Switch from :latest to SHA-pinned image tag (019e534c-4757-7de0-8752-267cf300f23a).

This PR is a scaffold opened by anima-triage. The implementation worker picks it up from here.

Close or mark ready manually once the implementation lands.

Draft PR for Anima work item **Switch from :latest to SHA-pinned image tag** (`019e534c-4757-7de0-8752-267cf300f23a`). This PR is a scaffold opened by `anima-triage`. The implementation worker picks it up from here. - Marker: [`.anima/wip/019e534c.md`](.anima/wip/019e534c.md) - Current phase: `triaged` Close or mark ready manually once the implementation lands.
wip: triage Switch from :latest to SHA-pinned image tag
Some checks failed
CI / test (pull_request) Failing after 2h0m1s
CI / test (push) Failing after 2h0m1s
CI / docker (pull_request) Has been skipped
CI / docker (push) Has been skipped
23fa0e511e
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 0e14133dda
Some checks failed
CI / test (push) Failing after 41s
CI / docker (push) Has been skipped
to 6bc6da1b74
Some checks failed
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
CI / test (push) Has been cancelled
CI / docker (push) Has been cancelled
2026-05-31 22:46:28 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 6bc6da1b74
Some checks failed
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
CI / test (push) Has been cancelled
CI / docker (push) Has been cancelled
to f66f599f45
Some checks are pending
CI / test (pull_request) Waiting to run
CI / test (push) Waiting to run
CI / docker (pull_request) Blocked by required conditions
CI / docker (push) Blocked by required conditions
2026-06-01 02:06:40 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from f66f599f45
Some checks are pending
CI / test (pull_request) Waiting to run
CI / test (push) Waiting to run
CI / docker (pull_request) Blocked by required conditions
CI / docker (push) Blocked by required conditions
to f392bb8db2
Some checks are pending
CI / docker (push) Blocked by required conditions
CI / test (push) Waiting to run
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 03:05:05 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from f392bb8db2
Some checks are pending
CI / docker (push) Blocked by required conditions
CI / test (push) Waiting to run
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to 00be33f5f4
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / docker (pull_request) Blocked by required conditions
CI / test (pull_request) Waiting to run
2026-06-01 03:46:10 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 00be33f5f4
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / docker (pull_request) Blocked by required conditions
CI / test (pull_request) Waiting to run
to 5c543b5430
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 04:35:09 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 96f2693610
Some checks are pending
CI / test (pull_request) Waiting to run
CI / test (push) Waiting to run
CI / docker (pull_request) Blocked by required conditions
CI / docker (push) Blocked by required conditions
to a158272729
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 06:35:07 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from a158272729
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to cf66985c73
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 09:02:45 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from cf66985c73
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to 9a8237366e
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 09:46:22 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 9a8237366e
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to 51715fc454
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 10:35:59 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 51715fc454
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to 89247c267d
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 13:06:49 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 89247c267d
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to 51a8d72359
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
2026-06-01 13:49:08 +00:00
Compare
toasterson force-pushed claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t from 51a8d72359
Some checks are pending
CI / test (push) Waiting to run
CI / docker (push) Blocked by required conditions
CI / test (pull_request) Waiting to run
CI / docker (pull_request) Blocked by required conditions
to f702b8dfcb
Some checks failed
CI / test (push) Failing after 2h0m1s
CI / test (pull_request) Failing after 2h0m2s
CI / docker (push) Has been skipped
CI / docker (pull_request) Has been skipped
2026-06-01 16:02:24 +00:00
Compare
Native CI: add .solstice/job.sh and share one check script with Forgejo
Some checks failed
CI / docker (push) Has been cancelled
CI / docker (pull_request) Has been cancelled
CI / test (push) Has been cancelled
CI / test (pull_request) Has been cancelled
e5ead22bbf
`get_project_ci_status` reported `resolvable: false` for Sia — no
`default_ci_steps` and no `.solstice/job.sh` — so every native dispatch
failed at config resolution before running anything.

That failure is not neutral. anima-server's solstice.rs routes it to
`handle_config_failure`, which parks the work item with
`ParkedReason::CiConfigUnresolvable`. The ABSENCE of this file reads as
"this work item's CI is broken" and parks real work. akh-medu learned this
the expensive way (8/8 native dispatches failed, the WI 1/2/3 series parked)
and its job.sh states the rule: a missing file must not read as "the code is
red". Sia had CI only on the Forgejo-Actions lane, so the native lane — the
one Anima's exact-head gate reads — had never produced a verdict at all.

One definition of the checks, two lanes
---------------------------------------
scripts/ci-checks.sh is now the only place the cargo commands are written.
Both .solstice/job.sh and .forgejo/workflows/ci.yml call it; neither re-lists
them. akh-medu's rule is that a native verdict must never be red for work the
Actions lane would call green, because the disagreement parks work items on
phantom failures. Two inline copies drift the first time someone edits one;
sharing the script makes that structurally impossible rather than a reviewer's
responsibility.

job.sh only provisions. It deliberately differs from the Anima/akh-medu
scripts in two places:

- It does not assume passwordless sudo. Those scripts call `sudo apt-get`
  unconditionally because their image grants it; a runner without sudo would
  fail with a bare "sudo: command not found" that classifies as a code
  failure and parks the item. Privilege is resolved once, only missing
  packages are installed, and the no-privilege case exits with a message
  naming the packages and the reason.
- It does not `rm -rf ../anima`. The workflow did, and on a persistent or
  shared runner that deletes a checkout the job does not own. An existing
  sibling is reused; a non-git path there is refused rather than removed.
  The resolved revision is always logged so an upstream anima-proto change
  is attributable. ANIMA_SIBLING_REV pins a run without freezing CI to a
  revision the shipped image does not use.

protoc keeps ci.yml's pinned 25.3 release, since Ubuntu's 3.12 rejects
anima-proto's proto3 `optional` fields, and installs into the checkout so it
needs no privileges. The build is debug: release codegen is what OOM-killed
rustc on the Solstice VMs for akh-medu, and an OOM reads as red on healthy
code. Sia's release build is still covered by the `docker` job on push to
main.

Validation
----------
scripts/test-ci-job.sh replaces every external command with a logging stub and
asserts on the control flow — the branches that only execute on a runner in an
unusual state, i.e. never until the day they matter. 10 scenarios, 36
assertions, all passing; it needs no toolchain and runs in seconds, so it is
wired into ci-checks.sh and guards the CI definition on both lanes.

It earned its keep immediately, catching three defects before any runner saw
the script: `"${profile_args[@]}"` on an empty array is an unbound-variable
error under `set -u` on bash < 4.4, so the script would have worked on the
runner but not locally, defeating the point of sharing it; the openssl probe
hardcoded /usr/include/openssl/ssl.h and reported a false "missing" on any
other layout, triggering a pointless privileged install; and the harness's own
no-sudo case was inheriting the host's cached sudo credentials, leaving the
no-privilege branch untested exactly where it was most likely to be wrong.

No application change: no Cargo.toml edit, no dependency version change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Native CI: fix four blockers found in review of e5ead22
Some checks failed
CI / test (pull_request) Failing after 5m45s
CI / docker (pull_request) Has been skipped
CI / test (push) Failing after 7m1s
CI / docker (push) Has been skipped
255b3fc3b2
All four were real. Reproduced each before fixing.

1. The install died as root. `$SUDO DEBIAN_FRONTEND=noninteractive apt-get`
   looks like an assignment prefix but is not one: bash decides that from the
   text BEFORE expansion, and the first word is `$SUDO`. Once that expands to
   nothing — i.e. running as root, the case that should always work — the
   assignment becomes the command name:

       $ SUDO=""; $SUDO DEMO_ONLY=1 true
       bash: DEMO_ONLY=1: command not found   (rc=127)

   Routed through `env`, which is a real command in both shapes. The harness
   now exercises root explicitly via an `id` stub rather than only covering
   sudo/non-root, so the escalation-free path has real coverage.

2. A git worktree's `.git` is a FILE holding a gitdir pointer, so `[ -d .git ]`
   rejected a perfectly valid shared worktree — and the next branch then
   refused to run at all. This checkout is itself a worktree, so the script
   would have rejected the very tree it was written in. Now asks git
   (`rev-parse --is-inside-work-tree`) and confirms the path is that
   worktree's ROOT by comparing realpaths, so a subdirectory of an unrelated
   repo above it is still refused — non-destructively.

3. ANIMA_SIBLING_REV claimed a pin it never enforced: it was only applied on
   the clone path, so with an existing checkout the requested revision was
   silently ignored and the verdict was attributed to a revision that was
   never built. Now validated against the existing checkout and REFUSED on
   mismatch, without touching a checkout this job does not own. An unknown rev
   is likewise refused rather than fetched into it.

   ANIMA_SIBLING_DIR is gone. It was misleading: cargo reads the override path
   from the workspace [patch] table, so pointing the variable elsewhere
   provisioned a directory cargo would never look at while the build still
   failed on the real path. The sibling location is now derived from
   Cargo.toml, which is the only thing that actually decides it, and a missing
   [patch] path fails loudly instead of guessing.

4. `clippy --workspace --all-targets -- -D warnings` was stricter than the
   ci.yml it replaced (libs/bins only). Pulling tests, benches and examples
   into a -D warnings gate for the first time risks an unrelated baseline
   failure landing as a red verdict on unrelated work — the phantom-failure
   park this work item exists to stop. Restored to the previously-verified
   scope. Widening is a separate change that needs a clean baseline run and
   then moves both lanes together.

Also fixes unbounded recursion introduced when the harness was wired into
ci-checks.sh: the harness runs job.sh, which execs ci-checks.sh, which ran the
harness. Guarded with CI_SKIP_JOB_HARNESS, which the harness sets for its
children, and the guard itself is now covered.

Harness: 18 scenarios, 62 assertions, all passing. New fixtures are real, not
mocked around — an actual shared worktree created with `git worktree add`, an
actual non-root checkout, actual commit SHAs for the pin cases, and a real
Cargo.toml with a [patch] table so the derivation runs the same code path the
runner takes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The deployed half of this work item already landed: the homelab HelmRelease
pins `image.tag` to `3012b550d3e573588a59ba261cfdcc22fbcb95e4@sha256:61949ee…`
with the note "Source: Sia PR #12; immutable release, never follow latest",
and sets `pullPolicy: Always`. The cluster is not following `:latest`. Two
things in THIS repo still worked against that.

The chart defaulted to `tag: latest`. Any render that does not override it —
a `helm install` from the chart directory, a second environment, a
troubleshooting `helm template` — silently produces a mutable-tag Deployment,
against a registry that has served a stale `:latest` before. That is the
original bug, still reachable through the chart's own default.

`image.tag` is now empty and `required`, so a missing tag fails at render
time with a message naming the accepted forms instead of quietly deploying
whatever `:latest` currently resolves to. Nothing in the cluster changes: the
HelmRelease already supplies the value. The `name:tag@digest` form it uses
renders correctly through the same expression.

Pinning also required a human to read the digest out of the registry by hand,
which is how a release gets pinned to the wrong one. CI and
`scripts/build-image.sh` now print the exact immutable reference after a
successful push, so updating the HelmRelease is a copy, not a lookup.

The `:latest` tag is still pushed. Stale `:latest` was a consumption problem,
and dropping the tag would break developer pulls for no gain now that nothing
rolls out from it.

Not touched: the homelab HelmRelease itself lives in the Talos GitOps repo
and is owned by another worker. It is already correct for this item; no
change is needed there for WI-2.

Verified: values.yaml parses; ci.yml parses and both jobs are intact;
build-image.sh passes `bash -n`; all five templates keep balanced
if/range/with vs end directives.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Puts this PR's original head into the branch's ancestry so the canonical
work-item branch can fast-forward without a force push and without losing
the PR's history.

The final tree is deliberately the reconciled one. The original branch is 44
commits behind main; much of its diff is .anima-exec/ runner telemetry that
was never part of the change, and the packaging surface it targeted has since
moved (the homelab HelmRelease now pins an immutable digest and disables
Keel). The work is reimplemented against main's current shape rather than
merged forward.

Net diff against current main is unchanged by this merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merge the native CI job script so this head can produce a CI verdict
Some checks failed
CI / test (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / test (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
9ab96b5270
Native dispatch reads .solstice/job.sh from the tested commit; without it
every dispatch against this head fails at config resolution and re-parks the
work item. Additive; no packaging change on this branch is modified.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
toasterson changed title from WIP: Switch from :latest to SHA-pinned image tag to Switch from :latest to SHA-pinned image tag 2026-09-22 11:17:53 +00:00
CI: make the job-script harness hermetic in command visibility and identity
All checks were successful
CI / test (push) Successful in 31m30s
CI / docker (push) Has been skipped
CI / test (pull_request) Successful in 29m37s
CI / docker (pull_request) Has been skipped
91070144c8
Native Linux CI on 255b3fc failed 56 passed / 6 failed
(request 982fb0cb-eb22-46d6-8dc1-2076e0e4e19c). The harness was not hermetic,
so it asserted against the host instead of the fixture, and the same suite
passed on macOS and failed on the CI container.

Two leaks, both real.

PATH fell through to /usr/bin:/bin. "Missing pkg-config" was therefore not
missing on any image that ships it, so scenarios 4, 5 and 11 silently
exercised the already-provisioned path rather than the install branches they
name. macOS has no /usr/bin/pkg-config, Debian does — hence the split.

Identity was inherited. job.sh branches on `id -u`, and an `id` stub was
installed only for the root scenario. On a container running as root every
"non-root" scenario took the root path, leaving the no-privilege branch
untested exactly where it was most likely to be wrong.

Fixes, in the harness only — job.sh's provisioning semantics are unchanged:

  - $BIN is now the ENTIRE PATH under test. Every legitimately-needed real
    tool is linked into it explicitly (bash sh env git sed awk grep cut head
    sort uniq mkdir rm cp ln chmod cat true false dirname basename touch wc),
    and nothing else is reachable, so absence is real absence.
  - `id` is stubbed for EVERY mode from the scenario's declared identity
    (root -> 0, otherwise 1000), never inherited from the host.

No assertion was dropped or weakened; the six that were failing now exercise
what their names claim. A new scenario 19 guards the leak itself — that PATH
under test is exactly the sandbox bin, that an unprovided command is absent,
that host pkg-config is unreachable, that identity is the declared one, and
that real tools are still reachable — so this cannot regress silently into
"passes on the developer's machine".

Proof, same container both ways (debian:bookworm-slim, pkg-config installed):

  pre-fix  (255b3fc harness), uid 0 : passed: 56   failed: 6
                                      exactly the six CI reported
  post-fix (this commit),     uid 0 : passed: 68   failed: 0
  post-fix (this commit),  uid 1000 : passed: 68   failed: 0
  post-fix, macOS host              : passed: 68   failed: 0

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merge the hermetic-harness CI fix
Some checks failed
CI / test (push) Failing after 34m5s
CI / docker (push) Has been skipped
CI / test (pull_request) Successful in 33m52s
CI / docker (pull_request) Has been skipped
7802121c6e
Native Linux CI on the CI branch failed 56/6 because the job-script harness
was not hermetic: PATH fell through to /usr/bin so "missing pkg-config" was
not missing on Debian, and `id` was inherited so non-root scenarios ran as
root. Both are fixed in the harness only; job.sh's provisioning semantics are
unchanged.

Additive: no application or packaging change on this branch is modified.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Some checks failed
CI / test (push) Failing after 34m5s
CI / docker (push) Has been skipped
CI / test (pull_request) Successful in 33m52s
CI / docker (pull_request) Has been skipped
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t:claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git switch claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main
git merge --no-ff claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git switch claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git rebase main
git switch main
git merge --ff-only claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git switch claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git rebase main
git switch main
git merge --no-ff claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git switch main
git merge --squash claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git switch main
git merge --ff-only claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git switch main
git merge claude/wi-019e534c-switch-from-latest-to-sha-pinned-image-t
git push origin main
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
toasterson/sia!6
No description provided.