Wayfinder map: Release 1 (server only) #24
Labels
No labels
Compat/Breaking
Crazy Idea
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority/Critical
Priority/High
Priority/Low
Priority/Medium
Reviewed/Confirmed
Reviewed/Duplicate
Reviewed/Invalid
Reviewed/Won't Fix
Status/Abandoned
Status/Blocked
Status/Need More Info
needs-info
needs-triage
ready-for-agent
ready-for-human
wayfinder:grilling
wayfinder:map
wayfinder:prototype
wayfinder:research
wayfinder:task
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
OpenIndiana/ips#24
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Destination
A tagged v1.0.0 pkg6depotd release on
main, qualified on the separate target depotoirepo6: it imports the historical OpenIndiana repositories locally and serves OpenIndiana Hipster pkg5 clients from 2020.04 onward. In scope: atomic local.p5pimport, staged import of large historical sources, two-stage retirement with protected snapshots, OTLP/HTTP telemetry, IPS/SMF packaging, both TLS modes plus ACME, mdBook plus illumos man pages. The exact gates are ADR-015. Production cutover, remote publication/OIDC/RBAC, automatic purge, S3 and OmniOS (v1.2) are not part of it.Notes
CLAUDE.md,CONTEXT.md(glossary),docs/agents/domain.mdand the ADRs indocs/ai/decisions/(001 to 015) before working a ticket. Until PR #54 merges, ADRs 004 to 015,CONTEXT.mdand the operator book live on branchv1-main-merge.doc/pkg5_docs/holds the pkg5 specifications; the local OpenIndiana pkg5 checkout is the compatibility authority, not Oracle Solaris.oirepo6, a zone next tooirepoon the same host behind the same reverse proxy, measured 4 GBzone.max-swap. RC13 is installed there on port 8080 via the packaged SMF service; the proxy still points at the old depot.cargo nextest; illumos is the deployment target, Linux the fast CI gate; release candidates are immutable tags and every release body lists the gates run and NOT RUN.docs/ai/plans/2026-04-09-publish-api-oidc.mdis background for the deferred remote-publish work only./grill-with-docsfor grilling tickets (terms intoCONTEXT.md, decisions into ADRs),/research,/to-tickets,/tdd+/implement.Tickets
import archiveandimport sourceDecisions so far
pkg6repo import archive|sourcebefore tag;status/0is a gate; tag frommainvia PR #54.docs/book/plus installed illumos-style man pages in the IPS package; contributor architecture stays in the repo.rename(2)at a stable path. Whole-batch validation, identical re-import a no-op, exact-FMRI conflict rejects the batch. ADR-007, ADR-013 (flat V3 sources, 10,000-manifest / 64 MiB budget), ADR-014 (staged resumable import, one final switch).legacyV3 source imports underopensolaris.orginto its own pkg6 repository root and SMF depot instance.oi-sparcandunleashedalso get separate roots and instances;solarm.organddesktop-2023are excluded. Hipster history stays together, including its separate encumbered publisher.install_default()(latent bug).Not yet specified
oirepois taken (request mix from real logs, duration, where the load generator runs) and the safe test budget on the shared host. Sharpens when #31 is grilled.legacyimport, proxy cutover with rollback, post-cutover sustained-load observation, BE reboot policy on oirepo6.Out of scope
oirepo6 pre-cutover evidence (2026-09-26): live inspection found a 4 GB
zone.max-swaplimit. This corrects the earlier unverified "no explicit resource limits" assumption in the map. The new packaged depot is running against an isolated clone; the old depot and live repository remain in place, with no reverse-proxy cutover.From the local OpenIndiana pkg5 client on oibldofgate, the clone endpoint now passes catalog listing for
entireandtext/less. An exactpkgrecvoftext/less@679,5.11-2025.0.0.0:20250704T104633Zinitially found seven missing payloads. Those seven blobs were restored only to the clone from the Hipsterhipster.old7source, after whichpkgrecvsucceeded. This is one end-to-end package retrieval, not proof that the live repository or all historical packages are complete. Native workspace release tests passed 420/420 (one skipped) for the untagged RC12 candidate; no tagged replacement or production switch is claimed.The NFS export also contains a distinct OpenSolaris
legacyflat V3 repository with publisheropensolaris.org. It is not one of the OpenIndiana Hipster archives. Do not count its contents as Hipster source coverage. The exact migration scope for that separate publisher is being clarified.RC12 is tagged at
v1.0.0-rc.12, commit690be9016d039b75dafe299597818b2ccb39b70e. The common release workflow was triggered by that tag; its result and tagged illumos asset upload are still pending.Pre-tag evidence on oibuild: release build passed with
-j2; release-script self-tests 543/543; release-profile nextest 438 passed, one documented opt-in network test skipped; IPS package creation and installation dry run against the pinned oldest supported image passed; the unprivileged native pkg5 smoke passed. The exact source candidate package was installed only in the isolated oirepo6 test service. The old live service remained online. On oibldofgate, native pkg5pkgrecvretrievedtext/lessand all seven payloads from that service.The separate OpenSolaris
legacyV3 source stays underopensolaris.org. A real one-package V3 sample was imported with byte-identical manifest and three payloads. A fresh test repository then served that sample alongside a Hipsteropenindiana.orgpackage; the native pkg5 client listed both publishers and retrieved the legacy sample. The 2,108,239-manifest full-source import is not complete. The existing serving repository still needs a planned adoption/cutover; direct import refused it withips::generation::adoption_requiredbefore mutation. Track the bounded, resumable importer and full-source validation in #53; RC12 does not claim to finish that migration.The RC12 release workflow passed, and
v1.0.0-rc.12is now a published prerelease with all eight expected Linux and illumos assets. The published illumos checksum file matches the verified oibuild artifacts. The release body records the native gate evidence and explicitly marks the disposable-VM lifecycle test NOT RUN.The exact tagged
.p5p(SHA-25688d2d534d7988e50ba66f63ae51453f63062a85b95b128fb037cc3d91f2a74fa) passed a real install dry run on oirepo6 with the package absent, then installed and passedpkg verify. Its tagged depot binary also passedldd -randconfig-test; the packaged isolated test service is online, alongside the unchanged live service. This verifies the tagged asset rather than assuming the earlier same-FMRI candidate had identical payloads. The full historicallegacyimport and live repository cutover remain tracked in #53.Published
v1.0.0-rc.13as a prerelease from exact commitf6691a0. Solstice release run 43 passed for Linux. The illumos assets were built from the same annotated tag onoibuild: 543 release-script tests passed; 457 native release-profile workspace tests passed (one opt-in test skipped); baseline IPS package build/dry-run and native pkg5 smoke passed. The release has all eight expected assets, and its published checksum file and.p5pmatch the verified local files. Official.p5pSHA-256:df3f6430a67c3fe6323498de05c04103a64db7459cc16d0c250464b043fbea46.The tagged archive passed the read-only target gate on oirepo6:
pkg install -nv --no-backup-be --deny-new-be -gplanned only an RC12-to-RC13 package update, with neither a new nor backup BE; all three tagged binaries passedldd -r, andpkg6depotd config-testaccepted the isolated service KDL. The installed package remains RC12; both existing 8080 and isolated 8083 services remained HTTP 200. Live-image install/upgrade/uninstall and SMF lifecycle testing was NOT RUN, stated in the release notes.#53 now has a bounded, resumable whole-source
import-historicalpath. It preserveslegacyas a distinctopensolaris.orgpublisher and makes staged batches visible together at one final generation switch. A real one-package V3 sample with three payloads passed native pkg5 listing/retrieval. The full 2,108,239-manifest NFS import, 4 GB memory measurement, 2020 snapshot package reconstruction, and historical-client install/repair acceptance remain open. Keep #24/#42/#53 open for those gates; no full historical import or live repository cutover occurred in this release.Deployment update: RC13 is installed and serving on oirepo6's local port 8080 through the packaged SMF service, replacing its running Rust 0.5 instance.
pkg verify, health/catalog HTTP checks, and native pkg5pkg listfrom oibldofgate passed. The public reverse proxy still points to the old repository server and was not changed. Full historical import, final repository switch, and boot-environment persistence remain open; the old 0.5 service is disabled but retained for rollback.import archiveandimport source#55Archived map body before the 2026-09-27 normalization (the prepended 2026-09-22 decision sections are now gists under Decisions so far, pointing at the ADRs and the tickets that carry them verbatim):
previous body
Preserved snapshot support promise — 2026-09-22
Maintainer decision: selected historical ISO snapshots must support maintaining the historical system, not merely booting/installing and upgrading to current Hipster. Users must be able to install additional packages and repair installed packages while retaining that snapshot's original version constraints. Selected snapshots remain protected indefinitely under the agreed retention policy.
Consequences for design and acceptance:
This settles the product-level snapshot promise. Reconstruction, snapshot catalog design and validation remain engineering work, not completed functionality.
Configurable retirement and KDL policy — 2026-09-22
Latest maintainer direction:
Engineering defaults/constraints to carry into the design (defaults selected here under that delegation, not independently quoted maintainer decisions): retirement is opt-in/disabled by default; disabling pauses policy-driven retirement without purging content or automatically restoring hidden versions. Continue serving hidden exact-FMRI content. Keep access tracking separately durable. Manual purge remains an explicit operator action with retention, snapshot and dependency checks; enabling retirement must never enable automatic purge. Document configuration effects and validate durations/rules before any mutation.
Still require evidence rather than assumptions: actual local pkg5 behavior, snapshot dependency closure, safe catalog replacement, and historical-client compatibility. No implementation or runtime validation is claimed by this decision update.
V1 administration boundary — 2026-09-22
Maintainer decision: the public HTTP interface is read-only in v1. Imports, retirement/purge and repository administration are available through local commands only. Remote publication and OIDC/RBAC remain deferred.
Apply this boundary to actual registered routes and any web UI: do not expose repository-mutating or administrative operations publicly in v1. Read-only package serving/search remain available. ACME HTTP-01 challenge serving remains supported and is not a repository administration endpoint. Document the local operator/CI script workflow and validate that public HTTP cannot mutate repository state. This records required behavior, not a claim that current routes already enforce it.
Load testing scope correction — 2026-09-22
The maintainer rejected the proposed mandatory 24-hour pre-cutover soak. Sustained-load testing/observation can take place after cutover and is not a v1 release gate. The workload is predominantly file downloads at low concurrency; search is the main query-computation concern highlighted by the maintainer. Do not infer that ingestion/catalog maintenance requires no server computation.
Keep pre-release correctness and targeted functional checks for real OpenIndiana pkg5 clients, search, atomic batch imports, and continuous reads across stable-path catalog rename. OTel failure isolation remains a required behavior. Do not add a mandatory soak duration, synthetic concurrency target, or numeric performance threshold without a new decision. Current oirepo remains the qualitative operational baseline. This supersedes earlier proposed soak/load release gates; it does not authorize production cutover or claim tests have run.
Deployment and load clarification — 2026-09-22
Maintainer-provided target:
oirepo6, a zone on the same existing host asoirepo, behind the same reverse proxy. Neither zone has explicit resource limits. This describes the intended deployment; no live host/resource inspection has yet been performed.The intended eventual role is the main repository host. V1 qualification still uses the separate depot; this does not authorize production cutover. It must serve the existing client population comfortably, with performance comparable to the current service. Traffic is predominantly a few requests at a time, accumulating substantial volume over time, rather than large simultaneous bursts.
For #31/#5, use the current service as the comparative baseline and representative low-concurrency sustained traffic as the workload. Actual request-rate/concurrency distribution and resource consumption still need measurement; no numerical latency/RSS/duration budget has been agreed. Include service behavior during imports/catalog maintenance. Absence of zone caps is not an allocation of the shared host's resources; establish a safe test budget before load generation.
Destination
A tagged v1.0 pkg6depotd release demonstrated on a separate depot importing historical OpenIndiana repositories, serving OpenIndiana Hipster pkg5 clients from approximately 2020 onward. Local atomic .p5p ingestion, two-stage retirement, OTLP/HTTP telemetry, IPS/SMF packaging, both TLS deployment modes and ACME, and mdBook plus illumos-style man pages are in scope. Production cutover, authenticated remote publication/OIDC/RBAC, automatic purge and S3 are deferred. OmniOS validation targets v1.2.
Notes
CLAUDE.md,docs/agents/domain.mdand the ADRs indocs/ai/decisions/before working a ticket.docs/ai/plans/2026-04-09-publish-api-oidc.mdis the current design for publish + OIDC + RBAC and background for deferred remote publication. Local p5p ingestion is the v1 path. The legacy pkg5 specifications live indoc/pkg5_docs/(publication protocol, repository format, depot protocol).pkgsendwire compatibility for publish;.p5pand.p6parchives plus manifest+file transactions are the upload shapes; no transaction timeout; errors use thiserror + miette; tests run undercargo nextest; illumos is the primary deployment target, Linux the fast CI gate..p6p), pkg6recv multithreaded download fix (#21), FTS duplicate entries (#22). OIDC JWT validation middleware (JWKS cache, scopes,rolesclaim) exists inpkg6depotd/src/http/middleware/auth.rsbut is only wired to/admin/auth/check./grill-with-docsfor grilling tickets (record resolved terms inCONTEXT.md, decisions as ADRs underdocs/ai/decisions/),/researchfor research tickets,/to-ticketswhen a decision graduates into build work,/tdd+/implementfor the build.Tickets
Decisions so far
legacyV3 source underopensolaris.org; it is not a Hipster archive. Its 2,108,239 manifests require bounded, resumable import (#53).Still open: exact ISO matrix, performance/concurrency/RSS budgets (#31), deployment resources, detailed retirement edge cases, ADR capture and build-ticket slicing. Keep this decision ticket open until the remaining release acceptance criteria are settled.
Newly tracked v1 work
Decision tickets remain open for unresolved details/ADR capture. Deferred tickets #3, #18, #19, #27, #28 and #36 have been removed from the v1 milestone, without closing them. #29 and #30 carry the agreed telemetry and documentation contracts.
Earlier open questions (apply the updated scope above)
.p5pthrough the existing importer vs.p6pthrough the archive backend, size limits, streaming to disk, partial-failure semantics. Sharpens with the publish protocol.pkg6repo publish --remoteverb? a smallpkg6send?), and how it obtains a token non-interactively.Out of scope
RC14 candidate source is committed at
5cbf36aon feat/separate-depots. Named pkg6 SMF instances now use direct pkg/inst_root, pkg/port, pkg/address, pkg/ssl_cert_file and pkg/ssl_key_file properties; incomplete pairs fail closed. KDL remains for ACME and telemetry. IPS actuators target all instances on upgrade and removal. Local pkg6depotd nextest passed 121/121, release-script checks 564/564, operator book and native svccfg manifest validation passed. The exact-source native build/package gate is running on oibuild; no RC14 tag, oirepo6 upgrade, public proxy change or historical source import has occurred yet.RC14 progress: PR #56 implements direct SMF properties (
pkg/inst_root,pkg/address,pkg/port,pkg/ssl_cert_file,pkg/ssl_key_file) with named instances. The exact-tag v1.0.0-rc.14 prerelease passed the native oibuild build, 460 release-profile tests, IPS baseline dry run, and pkg5 smoke. Its tagged IPS package is installed and verified on private oirepo6; default and three named instances are online. oibldofgate's pkg5 client listed and fetched one source-backedlegacypackage from the named instance. The public proxy remains on the old host.This does not complete the Release 1 acceptance gates: full bounded historical import, unified import CLI, Hipster 2020/2023/current snapshot client tests, load comparison, and automated live-image/SMF lifecycle testing remain open. Solstice's illumos PR job failed before compilation due to an
sccache rustc -vVserver-start timeout; the native oibuild gate passed.