Stream executable OCI payloads and preserve native directory permissions

toasterson commented

2026-10-09 04:16:53 +00:00

Owner

Native image payload copying was limited to the 4 MiB metadata budget and lost executable permissions. Copying now streams regular files within a 4 GiB aggregate budget, preserves safe executable modes, validates pinned paths and identities, and atomically replaces the intended destination.

Image preparation preserves installed transient-directory metadata and applies mode 0755 only to newly created payload directories. The template reserves 8 GiB for IPS transactions, with 28 GiB pool headroom; output limits remain 4 GiB expanded and 2 GiB compressed. Destination or temporary ownership loss preserves typed uncertainty and diagnostics through outer teardown and journal finalization. Earlier transfer errors revalidate destination ownership before ordinary cleanup.

Native CI now creates a private disk-backed TMPDIR under /var/tmp or an explicitly qualified base, rejects unsafe ancestry and non-disk filesystems, and reports filesystem/available bytes. It retains failure evidence and the original exit status. Test thresholds, the sparse 4 GiB overflow case, the full native check sequence and the v1 helper expectation are unchanged.

Validation remains bound to its source revision:

Current head 3fcf48e: 14 focused local script checks, shell syntax and diff checks passed; independent exact-source review approved. Platform/filesystem observations and the first Cargo invocation were controlled test substitutes. Native CI has not run at this head.
Parent a405745: 49 local OCI tests and strict OCI Clippy passed; exact native OpenIndiana qualification passed 50 OCI tests with no failures or ignored tests. The 21-case ownership regression includes read/write failures and confirms destructive cleanup fails the negative control. These tests were not rerun as current-head native CI.
Original canonical CI request fe5f8cf8-da51-46a8-8c07-6718de0e7c61 at a405745 returned exit 101. Log persistence timed out; the actual failure output was not retained. Its cause remains unknown. The TMPDIR correction addresses an independently established native storage requirement, not a proven diagnosis of that result.
Historical candidate 02b717a passed 518 workspace tests, with 0 failed and 2 external tests ignored, plus strict workspace Clippy. Separate combined integration 7a3668e passed 443 native tests, with 0 failed and 1 external test ignored; that includes other runtime changes and is not exact PR-head CI.

Earlier component-equivalent source cf424648 completed native image build/verification and normal cleanup. API guest readback verified the original 92,220,304-byte Papercrab payload (SHA256 72be0cceefa975580619e24c16df0a71ae638a79956d46873e2b113a452500b3), mode 0755 and resolved libraries; papercrab backends scan --help and HTTP ran under observed UID/GID 10000. This remains prior happy-path evidence, not a Papercrab database migration or replacement-race proof.

WI-29 is merged. Current-head canonical review and native CI remain merge gates. Work item: 01a11896-8348-7052-8912-1fd5d334f1fe (WI-28).

Native image payload copying was limited to the 4 MiB metadata budget and lost executable permissions. Copying now streams regular files within a 4 GiB aggregate budget, preserves safe executable modes, validates pinned paths and identities, and atomically replaces the intended destination. Image preparation preserves installed transient-directory metadata and applies mode 0755 only to newly created payload directories. The template reserves 8 GiB for IPS transactions, with 28 GiB pool headroom; output limits remain 4 GiB expanded and 2 GiB compressed. Destination or temporary ownership loss preserves typed uncertainty and diagnostics through outer teardown and journal finalization. Earlier transfer errors revalidate destination ownership before ordinary cleanup. Native CI now creates a private disk-backed TMPDIR under `/var/tmp` or an explicitly qualified base, rejects unsafe ancestry and non-disk filesystems, and reports filesystem/available bytes. It retains failure evidence and the original exit status. Test thresholds, the sparse 4 GiB overflow case, the full native check sequence and the v1 helper expectation are unchanged. Validation remains bound to its source revision: - Current head `3fcf48e`: 14 focused local script checks, shell syntax and diff checks passed; independent exact-source review approved. Platform/filesystem observations and the first Cargo invocation were controlled test substitutes. Native CI has not run at this head. - Parent `a405745`: 49 local OCI tests and strict OCI Clippy passed; exact native OpenIndiana qualification passed 50 OCI tests with no failures or ignored tests. The 21-case ownership regression includes read/write failures and confirms destructive cleanup fails the negative control. These tests were not rerun as current-head native CI. - Original canonical CI request `fe5f8cf8-da51-46a8-8c07-6718de0e7c61` at `a405745` returned exit 101. Log persistence timed out; the actual failure output was not retained. Its cause remains unknown. The TMPDIR correction addresses an independently established native storage requirement, not a proven diagnosis of that result. - Historical candidate `02b717a` passed 518 workspace tests, with 0 failed and 2 external tests ignored, plus strict workspace Clippy. Separate combined integration `7a3668e` passed 443 native tests, with 0 failed and 1 external test ignored; that includes other runtime changes and is not exact PR-head CI. Earlier component-equivalent source `cf424648` completed native image build/verification and normal cleanup. API guest readback verified the original 92,220,304-byte Papercrab payload (SHA256 `72be0cceefa975580619e24c16df0a71ae638a79956d46873e2b113a452500b3`), mode 0755 and resolved libraries; `papercrab backends scan --help` and HTTP ran under observed UID/GID 10000. This remains prior happy-path evidence, not a Papercrab database migration or replacement-race proof. WI-29 is merged. Current-head canonical review and native CI remain merge gates. Work item: `01a11896-8348-7052-8912-1fd5d334f1fe` (WI-28).

toasterson added 4 commits

2026-10-09 04:16:55 +00:00

Stream native OCI payload copies with owned atomic output 739b6ec0ab

(cherry picked from commit eb985ac3d332d3b60389fa08421279ff0b0cf0c8)

Preserve installed transient directory metadata during native image sanitization 231756e4cf

(cherry picked from commit 78c5a2849518db247b6258b7d0fba8b9bcda93ab)

Set traversable modes on newly owned payload directories 02b717add9

(cherry picked from commit 7ababfdf84dbcdc2e0b92093d7e92ed71763da26)

Allow IPS transaction space in native template quota 5e7aeb5b91

(cherry picked from commit cf424648e378f2fe4e3f45225fb48258a73248c2)

toasterson added 1 commit

2026-10-09 04:42:17 +00:00

Preserve uncertain native payload cleanup 1c8775bc17

toasterson added 1 commit

2026-10-09 05:10:55 +00:00

Preserve native build resources after destination ownership loss a4057453e3

toasterson added 1 commit

2026-10-09 06:07:18 +00:00

Use private disk-backed temporary storage for native checks 3fcf48edef

toasterson merged commit c03410ba10 into main

2026-10-09 07:17:19 +00:00

toasterson referenced this pull request from a commit

2026-10-09 07:17:20 +00:00

Merge pull request 'Stream executable OCI payloads and preserve native directory permissions' (#16) from codex/node03-wi28-handback-20261009 into main

Rows
Columns

Stream executable OCI payloads and preserve native directory permissions #16