Package transport Archive Format #4

New issue

Closed

opened 2026-02-07 14:06:24 +00:00 by Toasterson · 1 comment

Toasterson commented

2026-02-07 14:06:24 +00:00

(Migrated from codeberg.org)

Invent a Zipfile based format with which we can transport a set of packages to be published or installed. Replacement for the p5p file for Machines mostly using flash based storage and not spinning disk or tapes..

Toasterson commented

2026-04-09 20:12:03 +00:00

(Migrated from codeberg.org)

Tested the full round-trip against the production omnios repo: archived all 755 packages (66k files, 1.7GB), imported into a fresh repository, manifests match. The archive backend implements ReadableRepository so it plugs into the existing receiver pipeline without special casing.

Found and fixed a hash mismatch bug along the way, pkg5-imported repos store files by SHA-256 but legacy manifests reference SHA1. The fix was to prefer the JSON manifest everywhere, which also fixed local FileBackend receive that was broken for the same reason.

Implemented as .p6p — a zip container with zstd-compressed entries. The format stores manifests (both legacy text and JSON), file payloads, and a pkg6.archive.json index at the root. Created via pkg6repo archive, imported via pkg6repo import-archive or as a source for pkg6recv. Tested the full round-trip against the production omnios repo: archived all 755 packages (66k files, 1.7GB), imported into a fresh repository, manifests match. The archive backend implements ReadableRepository so it plugs into the existing receiver pipeline without special casing. Found and fixed a hash mismatch bug along the way, pkg5-imported repos store files by SHA-256 but legacy manifests reference SHA1. The fix was to prefer the JSON manifest everywhere, which also fixed local FileBackend receive that was broken for the same reason.