OpenIndiana/ips
Watch
2
0
Fork
You've already forked ips
 0
Code Issues 29 Pull requests 2 Projects Releases 4 Packages Wiki Activity Actions

V1: validate OpenIndiana Hipster clients from 2020 onward #42

New issue
Open
opened 2026-09-22 12:11:29 +00:00 by forge · 7 comments
forge commented 2026-09-22 12:11:29 +00:00
Copy link

Part of #24

Preserved snapshot support promise — 2026-09-22

Maintainer decision: selected historical ISO snapshots must support maintaining the historical system, not merely booting/installing and upgrading to current Hipster. Users must be able to install additional packages and repair installed packages while retaining that snapshot's original version constraints. Selected snapshots remain protected indefinitely under the agreed retention policy.

Consequences for design and acceptance:

  • Inventorying only packages already installed in the ISO root filesystem is insufficient. Reconstruct and preserve a usable historical package set, including compatible additional packages, relevant incorporation metadata, dependencies, manifests and payloads. Record provenance/ISO hashes and report unavailable historical content explicitly rather than claiming complete reconstruction.
  • Packages required for the supported snapshot must remain discoverable to the legacy OpenIndiana pkg5 solver, not just retrievable by an exact-FMRI HTTP request. Design a usable snapshot catalog/view if needed; endpoint/layout mechanics are delegated to engineering and must be validated against local pkg5.
  • The earlier newest-only incorporation proposal is subordinate to this support promise. Do not remove or hide metadata needed for historical maintenance unless the snapshot view preserves the required semantics and native tests prove it works.
  • Import exclusions conflicting with the protected historical set still fail preflight and require explicit KDL keep exceptions. Retirement/purge must preserve the full protected snapshot set and shared payload references. Main-catalog hiding must not make a supported snapshot unusable.
  • Acceptance: from selected historical ISO-installed clients (Hipster approximately 2020 onward), install a package absent from the original image and repair installed content without forcing a move to current package constraints. Also validate the supported update path separately. Use the real local OpenIndiana pkg5 implementation and retain evidence.

This settles the product-level snapshot promise. Reconstruction, snapshot catalog design and validation remain engineering work, not completed functionality.

Acceptance scope

  • A separate depot imports the historical split OpenIndiana repositories supplied by the maintainer. Production replacement is not a v1 gate.
  • OpenIndiana Hipster from approximately 2020 onward is mandatory. Select an available 2020 ISO plus current and representative intermediate fixtures; record exact ISO hashes and client/source revisions.
  • Use local OpenIndiana/pkg5 source as the behavior authority. Test catalog refresh, search, manifests/files, real installation and updates from ISO-installed images, including retirement and atomic import behavior.
  • Prove old installed incorporations still support the promised operations with older repository incorporations hidden. An exact-FMRI HTTP success alone is insufficient.
  • Preserve existing hosts/checkouts and run native OS tests on isolated illumos test systems.
  • Coordinate datasets and measured budgets with #31/#5. OpenIndiana is the v1 gate; OmniOS verification is deferred to v1.2 and remains unverified until run.

Source: maintainer grilling, 2026-09-22. Requirements only; no implementation or validation claimed.

Part of #24 ## Preserved snapshot support promise — 2026-09-22 Maintainer decision: selected historical ISO snapshots must support maintaining the historical system, not merely booting/installing and upgrading to current Hipster. Users must be able to install additional packages and repair installed packages while retaining that snapshot's original version constraints. Selected snapshots remain protected indefinitely under the agreed retention policy. Consequences for design and acceptance: - Inventorying only packages already installed in the ISO root filesystem is insufficient. Reconstruct and preserve a usable historical package set, including compatible additional packages, relevant incorporation metadata, dependencies, manifests and payloads. Record provenance/ISO hashes and report unavailable historical content explicitly rather than claiming complete reconstruction. - Packages required for the supported snapshot must remain discoverable to the legacy OpenIndiana pkg5 solver, not just retrievable by an exact-FMRI HTTP request. Design a usable snapshot catalog/view if needed; endpoint/layout mechanics are delegated to engineering and must be validated against local pkg5. - The earlier newest-only incorporation proposal is subordinate to this support promise. Do not remove or hide metadata needed for historical maintenance unless the snapshot view preserves the required semantics and native tests prove it works. - Import exclusions conflicting with the protected historical set still fail preflight and require explicit KDL keep exceptions. Retirement/purge must preserve the full protected snapshot set and shared payload references. Main-catalog hiding must not make a supported snapshot unusable. - Acceptance: from selected historical ISO-installed clients (Hipster approximately 2020 onward), install a package absent from the original image and repair installed content without forcing a move to current package constraints. Also validate the supported update path separately. Use the real local OpenIndiana pkg5 implementation and retain evidence. This settles the product-level snapshot promise. Reconstruction, snapshot catalog design and validation remain engineering work, not completed functionality. ## Acceptance scope - A separate depot imports the historical split OpenIndiana repositories supplied by the maintainer. Production replacement is not a v1 gate. - OpenIndiana Hipster from approximately 2020 onward is mandatory. Select an available 2020 ISO plus current and representative intermediate fixtures; record exact ISO hashes and client/source revisions. - Use local OpenIndiana/pkg5 source as the behavior authority. Test catalog refresh, search, manifests/files, real installation and updates from ISO-installed images, including retirement and atomic import behavior. - Prove old installed incorporations still support the promised operations with older repository incorporations hidden. An exact-FMRI HTTP success alone is insufficient. - Preserve existing hosts/checkouts and run native OS tests on isolated illumos test systems. - Coordinate datasets and measured budgets with #31/#5. OpenIndiana is the v1 gate; OmniOS verification is deferred to v1.2 and remains unverified until run. Source: maintainer grilling, 2026-09-22. Requirements only; no implementation or validation claimed.
forge commented 2026-09-23 08:01:45 +00:00
Author
Copy link

Draft PR #45 keeps genuine pkg.obsolete manifests in the pkg5 dependency catalog, removes the overloaded obsolete-package verb, and preserves exact serving while rebuilding active search. It also validates stored gzip payload wire compatibility against manifest digests. Real pkg5-produced p5p import and retry pass locally. Full pkg5 client catalog refresh/install/update/repair against a native depot and historical ISO snapshots remain open.

Draft PR #45 keeps genuine pkg.obsolete manifests in the pkg5 dependency catalog, removes the overloaded obsolete-package verb, and preserves exact serving while rebuilding active search. It also validates stored gzip payload wire compatibility against manifest digests. Real pkg5-produced p5p import and retry pass locally. Full pkg5 client catalog refresh/install/update/repair against a native depot and historical ISO snapshots remain open.
forge commented 2026-09-23 10:23:21 +00:00
Author
Copy link

PR #51 is integrated into PR #46 at 48f8b64. The tagged OpenIndiana release job now starts the installed depot, imports a multi-package p5p, and uses native pkg(1) and pkgrecv to check catalog, manifest, exact payload transfer and scratch-image installation. Local script tests pass (324/0), but the live smoke has not run yet. This is the current-client gate; the requested Hipster 2020-onward historical-client matrix still needs the ISO/client artifacts and its own evidence. Keep this ticket open.

PR #51 is integrated into PR #46 at `48f8b64`. The tagged OpenIndiana release job now starts the installed depot, imports a multi-package p5p, and uses native pkg(1) and pkgrecv to check catalog, manifest, exact payload transfer and scratch-image installation. Local script tests pass (324/0), but the live smoke has not run yet. This is the current-client gate; the requested Hipster 2020-onward historical-client matrix still needs the ISO/client artifacts and its own evidence. Keep this ticket open.
forge commented 2026-09-23 12:04:57 +00:00
Author
Copy link

RC2 compatibility evidence on exact source ca42462105cb5b5c9c42145bd3ccf73502256d5e (OpenIndiana oibldofgate, global zone, scratch repo/image and private port 18082):

  • The native optimized build and pkg5 .p5p build passed. The archive is pkg6depotd-1.0.0-rc.2-illumos-x86_64.p5p (SHA-256 f9610ab5d04d5f3a8959853b3cb11e7e46ad165b3a0eec192c0d79c5f24e4b23). Its full FMRI is pkg://openindiana.org/package/pkg6/depotd@1.0.0,5.11-0.3.2:20260923T115343Z.
  • Imported that archive and a dummy package into pkg6. A repeat import was a no-op with an unchanged repository tree. The served catalog.attrs contains pkg5's required empty updates object; local OpenIndiana pkg5 Catalog.validate() accepted attrs and all catalog parts.
  • The installed OpenIndiana pkg5 client completed full and incremental refresh, catalog listing, remote manifest info, pkgrecv of both manifests and payloads, and dummy package install/verify/uninstall in a scratch IPS image. The received manifests were byte-identical and payload content matched. FAILS=0; the task depot was stopped and the port released.
  • Evidence: logs/oibldofgate/ips-rc2-smoke/ in the illumos workspace. Exact-head Forgejo CI is still running: https://code.aopc.cloud/OpenIndiana/ips/actions/runs/19.

This proves the current installed pkg5 client path, not the 2020 ISO matrix or historical snapshot maintenance acceptance in this ticket. The historical split repository access and an isolated 2020 ISO guest remain needed before this ticket can close. pkgrepo info against the depot reports missing status/0; pkg(1) and pkgrecv operations above do not require it, but it is a separate compatibility gap to track.

RC2 compatibility evidence on exact source `ca42462105cb5b5c9c42145bd3ccf73502256d5e` (OpenIndiana `oibldofgate`, global zone, scratch repo/image and private port 18082): - The native optimized build and pkg5 `.p5p` build passed. The archive is `pkg6depotd-1.0.0-rc.2-illumos-x86_64.p5p` (SHA-256 `f9610ab5d04d5f3a8959853b3cb11e7e46ad165b3a0eec192c0d79c5f24e4b23`). Its full FMRI is `pkg://openindiana.org/package/pkg6/depotd@1.0.0,5.11-0.3.2:20260923T115343Z`. - Imported that archive and a dummy package into pkg6. A repeat import was a no-op with an unchanged repository tree. The served `catalog.attrs` contains pkg5's required empty `updates` object; local OpenIndiana pkg5 `Catalog.validate()` accepted attrs and all catalog parts. - The installed OpenIndiana pkg5 client completed full and incremental refresh, catalog listing, remote manifest info, `pkgrecv` of both manifests and payloads, and dummy package install/verify/uninstall in a scratch IPS image. The received manifests were byte-identical and payload content matched. `FAILS=0`; the task depot was stopped and the port released. - Evidence: `logs/oibldofgate/ips-rc2-smoke/` in the illumos workspace. Exact-head Forgejo CI is still running: https://code.aopc.cloud/OpenIndiana/ips/actions/runs/19. This proves the current installed pkg5 client path, not the 2020 ISO matrix or historical snapshot maintenance acceptance in this ticket. The historical split repository access and an isolated 2020 ISO guest remain needed before this ticket can close. `pkgrepo info` against the depot reports missing `status/0`; `pkg(1)` and `pkgrecv` operations above do not require it, but it is a separate compatibility gap to track.
forge commented 2026-09-23 14:04:38 +00:00
Author
Copy link

Maintainer clarification (2026-09-23): historical OpenIndiana repositories and Hipster ISOs back to 2020 will be copied from an NFS share and imported locally. Every selected historical repository/archive must feed one final pkg6 repository, with its versions visible together, rather than separate serving repositories. The NFS mount path, source host/URL, and infrastructure access are deferred until the maintainer is at the computer; do not hard-code or assume them in the importer. Current work is code only: accept configurable local source paths and preserve archive/ISO provenance for the later inventory and import. This does not relax the previously recorded protected-snapshot install/repair acceptance.

Maintainer clarification (2026-09-23): historical OpenIndiana repositories and Hipster ISOs back to 2020 will be copied from an NFS share and imported locally. Every selected historical repository/archive must feed **one final pkg6 repository**, with its versions visible together, rather than separate serving repositories. The NFS mount path, source host/URL, and infrastructure access are deferred until the maintainer is at the computer; do not hard-code or assume them in the importer. Current work is code only: accept configurable local source paths and preserve archive/ISO provenance for the later inventory and import. This does not relax the previously recorded protected-snapshot install/repair acceptance.
forge commented 2026-09-23 14:06:42 +00:00
Author
Copy link

Testing venue clarification (2026-09-23): Hephaistos SSH is timing out, and this work must not depend on it. Use Solstice CI's disposable OpenIndiana VMs for client/package testing. If a persistent native build workspace is needed for code, use oibuild or oibldofgate under the workspace rules. NFS/ISO host access and infrastructure setup remain deferred; focus on code and CI now. The current Solstice release VM is Hipster 2026, so it does not yet satisfy the separate Hipster 2020 client acceptance in this issue.

Testing venue clarification (2026-09-23): Hephaistos SSH is timing out, and this work must not depend on it. Use Solstice CI's disposable OpenIndiana VMs for client/package testing. If a persistent native build workspace is needed for code, use `oibuild` or `oibldofgate` under the workspace rules. NFS/ISO host access and infrastructure setup remain deferred; focus on code and CI now. The current Solstice release VM is Hipster 2026, so it does **not** yet satisfy the separate Hipster 2020 client acceptance in this issue.
toasterson commented 2026-09-26 17:00:06 +00:00
Owner
Copy link

Targeted local pkg5 compatibility result (2026-09-26): on oibldofgate, the native OpenIndiana pkg5 client successfully listed entire and text/less from the isolated oirepo6 test endpoint. pkgrecv successfully retrieved the exact text/less@679,5.11-2025.0.0.0:20250704T104633Z manifest and seven payloads after those missing blobs were restored to the test clone from the OpenIndiana hipster.old7 archive. Before restoration, the manifest was fetchable but all seven payloads returned 404. This demonstrates why catalog success alone is insufficient.

This is not the 2020 ISO-installed client acceptance test, historical snapshot maintenance proof, or live repository completeness proof. The separate OpenSolaris legacy repository on the same NFS export is not a Hipster archive and was not used for this test.

Targeted local pkg5 compatibility result (2026-09-26): on oibldofgate, the native OpenIndiana pkg5 client successfully listed `entire` and `text/less` from the isolated oirepo6 test endpoint. `pkgrecv` successfully retrieved the exact `text/less@679,5.11-2025.0.0.0:20250704T104633Z` manifest and seven payloads after those missing blobs were restored to the test clone from the OpenIndiana `hipster.old7` archive. Before restoration, the manifest was fetchable but all seven payloads returned 404. This demonstrates why catalog success alone is insufficient. This is not the 2020 ISO-installed client acceptance test, historical snapshot maintenance proof, or live repository completeness proof. The separate OpenSolaris `legacy` repository on the same NFS export is not a Hipster archive and was not used for this test.
toasterson commented 2026-09-27 13:14:06 +00:00
Owner
Copy link

ADR-015 item 3 (2026-09-27) fixes the matrix: Hipster 2020.04 (oldest supported), 2023.10 (midpoint) and the current 2026 image. From each ISO-installed client: catalog refresh, manifest and payload retrieval, pkg install/verify/uninstall against the depot. The snapshot-maintenance proof (install a package absent from the image and repair installed content without moving to current constraints) gates v1.0 for 2020.04 only; the other snapshots move to v1.1. Solstice's Hipster 2026 VM covers only the current column; the two older guests still need a venue and NFS access.

ADR-015 item 3 (2026-09-27) fixes the matrix: Hipster **2020.04** (oldest supported), **2023.10** (midpoint) and the **current 2026** image. From each ISO-installed client: catalog refresh, manifest and payload retrieval, `pkg install`/`verify`/`uninstall` against the depot. The snapshot-maintenance proof (install a package absent from the image and repair installed content without moving to current constraints) gates v1.0 for **2020.04 only**; the other snapshots move to v1.1. Solstice's Hipster 2026 VM covers only the current column; the two older guests still need a venue and NFS access.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/separate-depots
v1-main-merge
feat/historical-import
v1-integration-20260923
v1-pkg5-smoke-20260923
v1-retention-20260923
v1-repo-cli-20260923
v1-cli-surface-20260923
v1-tls-20260923
v1-atomic-import-20260923
v1-otel-20260923
v1-release-pipeline-20260923
research/oidc-providers
research/illumos-crypto-stack
research/otlp-stack
research/pkg5-publication-model
claude/ecstatic-allen-0a5f4a
v1.0.0-rc.14
v1.0.0-rc.13
v1.0.0-rc.12
v1.0.0-rc.11
v1.0.0-rc.10
v1.0.0-rc.9
v1.0.0-rc.8
v1.0.0-rc.7
v1.0.0-rc.6
v1.0.0-rc.5
v1.0.0-rc.4
v1.0.0-rc.3
v1.0.0-rc.2
v1.0.0-rc.1
Labels
Clear labels   
Compat/Breaking
Breaking change that won't be backward compatible

  
Crazy Idea
Something that went through my mind which is not part of the core use case but fun

  
Kind/Bug
Something is not working

  
Kind/Documentation
Documentation changes

  
Kind/Enhancement
Improve existing functionality

  
Kind/Feature
New functionality

  
Kind/Security
This is security issue

  
Kind/Testing
Issue or pull request related to testing

  
Priority/Critical
The priority is critical

  
Priority/High
The priority is high

  
Priority/Low
The priority is low

  
Priority/Medium
The priority is medium

  
Reviewed/Confirmed
Issue has been confirmed

  
Reviewed/Duplicate
This issue or pull request already exists

  
Reviewed/Invalid
Invalid issue

  
Reviewed/Won't Fix
This issue won't be fixed

  
Status/Abandoned
Somebody has started to work on this but abandoned work

  
Status/Blocked
Something is blocking this issue or pull request

  
Status/Need More Info
Feedback is required to reproduce issue or to continue work

  
needs-info
Waiting on reporter for more information

  
needs-triage
Maintainer needs to evaluate this issue

  
ready-for-agent
Fully specified, ready for an AFK agent

  
ready-for-human
Requires human implementation

  
wayfinder:grilling
Wayfinder decision ticket: grilling

  
wayfinder:map
Wayfinder map issue (parent of decision tickets)

  
wayfinder:prototype
Wayfinder decision ticket: prototype

  
wayfinder:research
Wayfinder decision ticket: research

  
wayfinder:task
Wayfinder ticket: task that unblocks a decision

  
wontfix
Will not be actioned

No labels
Compat/Breaking
Crazy Idea
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority/Critical
Priority/High
Priority/Low
Priority/Medium
Reviewed/Confirmed
Reviewed/Duplicate
Reviewed/Invalid
Reviewed/Won't Fix
Status/Abandoned
Status/Blocked
Status/Need More Info
needs-info
needs-triage
ready-for-agent
ready-for-human
wayfinder:grilling
wayfinder:map
wayfinder:prototype
wayfinder:research
wayfinder:task
wontfix
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Depends on
#25 Define what 'v1.0 server release' must include and its acceptance criteria
OpenIndiana/ips
Reference
OpenIndiana/ips#42
No description provided.