Fix akhomed startup ownership and side-effect-free CLI #667

Merged

toasterson merged 3 commits from fix/akhomed-single-instance-discovery into main 2026-10-18 01:58:55 +00:00

Conversation 0 Commits 3 Files changed 20 +5252 -213

toasterson commented 2026-10-18 00:24:56 +00:00

Owner

Copy link

A live akhomed --version probe on October 17 ignored the unsupported option and started a second daemon. That contender reached Postgres and wrote discovery before it was excluded by existing residents/listeners; its failed shutdown then removed the discovery file. This PR makes information probes side-effect-free and gives startup/discovery artifacts explicit instance ownership.

Behavior

• Parse the complete command line with Clap before creating a Tokio runtime, telemetry, directories, configuration, stores or listeners. --version, --help, unknown flags and missing values exit there; repeatable --auto-start and the offline store split-per-akh command remain supported.
• Acquire a permanent runtime-directory OS lock before config/store/workspace startup. Reserve enabled ACP Unix/TCP and core TCP listeners before any database initialization, also excluding legacy listeners and endpoint collisions across runtime directories. Configured listener conflicts now fail startup nonzero before stores open.
• Publish discovery atomically after startup succeeds. An open device/inode token binds cleanup to the actual published file; a failed contender never obtains that token and cannot remove the owner's file. A replacement record survives even if its PID/bytes match.
• Serialize Unix socket reclamation with a permanent endpoint lock and clean only the captured socket identity. Drop unserved reservations on ACP workspace failure; unwind fatal SQLite initialization; install SIGTERM/SIGINT streams before reservations so startup termination reaches the common drain.

Disposable validation and independent review

• Five ownership/artifact helper tests pass, zero ignored.
• Ten real-binary process regressions pass, zero ignored: CLI/help/version/errors with no XDG/store effects; duplicate running owner preserves bytes/inode/listener; legacy listener refusal; later TCP bind failure; SIGTERM cleanup/restart; replacement-file preservation; invalid SQLite cleanup; disabled ACP workspace cleanup; core port conflict before database access; late core-store failure drain; and termination during database startup. Some tests cover multiple assertions/paths.
• Process fixtures clear inherited environment and run only in disposable HOME/XDG/working directories with temporary files, generated fixture tokens and loopback canaries. CI explicitly executes both startup test targets with server features.
• Local builds use CARGO_BUILD_JOBS=2, dedicated /Users/toasty/.cache/mausbot-round6b-target, incremental/debug output disabled, and a monitored 20 GiB free-space floor on /System/Volumes/Data. No build crossed the floor. Rustfmt and diff whitespace checks pass.
• Fresh-context independent review found and closed two cleanup defects plus the startup-signal gap. Reports, source hashes, responses and validation receipts are retained in docs/mausbot/evidence/akhomed-startup-20261018-round6b. Exact-head CI remains pending; local tests are not CI or deployment acceptance.

Incident reconciliation

Read-only reconciliation used the enrolled native-house DB credentials, TLS verification, connection-level read-only defaults, explicit repeatable-read/read-only transactions and rollback. No DB repair is indicated:

• All 13 migrations in deployed source 45a1b9bedb0d43aaec7eafec6cd58dfeed045347 were applied before the incident. Timestamped public tables show zero rows in the precise and four-minute incident windows. All three native residents' ownership timestamps and advisory sessions predate it; the refusal branches fence identity/epoch, hydration, WAL/snapshot and executor work.
• Checked the core service store separately because CoreApp::new ran before the old occupied-port refusal. Its existing akhomed-core import marker value 0 is dated October 16 04:29:01Z, selecting the deployed rollback-before-insert branch. Its 22 records predate the incident, with no grant rows. Existing-schema/table checks therefore did not create new objects.
• Postgres logs corroborate seven transient incident-time sessions. They are not a statement audit. Commit timestamps and statement logging were disabled, and no complete pre-probe snapshot exists; the conclusion combines actual rows with exact deployed control flow and the three recorded lock refusals.

The result is no persistent database effect; no DB repair needed. The separate discovery-file loss and its authorized round-5 one-file repair remain recorded, with prior bytes/existence unknown. This does not relabel the original failed probe as read-only or claim zero transient effects. The requested scratch receipt is round6b/incident-reconciliation.json; a copy and query/log projections are in the evidence directory above.

No merge, live DB/pod write, cluster daemon/binary execution, restart, Board action or activation occurred in round 6b. Round 6a's native computer dispatch is separate work. Parent owns CI watching, merge and subsequent activation gates.

A live `akhomed --version` probe on October 17 ignored the unsupported option and started a second daemon. That contender reached Postgres and wrote discovery before it was excluded by existing residents/listeners; its failed shutdown then removed the discovery file. This PR makes information probes side-effect-free and gives startup/discovery artifacts explicit instance ownership. ### Behavior - Parse the complete command line with Clap before creating a Tokio runtime, telemetry, directories, configuration, stores or listeners. `--version`, `--help`, unknown flags and missing values exit there; repeatable `--auto-start` and the offline `store split-per-akh` command remain supported. - Acquire a permanent runtime-directory OS lock before config/store/workspace startup. Reserve enabled ACP Unix/TCP and core TCP listeners before any database initialization, also excluding legacy listeners and endpoint collisions across runtime directories. Configured listener conflicts now fail startup nonzero before stores open. - Publish discovery atomically after startup succeeds. An open device/inode token binds cleanup to the actual published file; a failed contender never obtains that token and cannot remove the owner's file. A replacement record survives even if its PID/bytes match. - Serialize Unix socket reclamation with a permanent endpoint lock and clean only the captured socket identity. Drop unserved reservations on ACP workspace failure; unwind fatal SQLite initialization; install SIGTERM/SIGINT streams before reservations so startup termination reaches the common drain. ### Disposable validation and independent review - Five ownership/artifact helper tests pass, zero ignored. - Ten real-binary process regressions pass, zero ignored: CLI/help/version/errors with no XDG/store effects; duplicate running owner preserves bytes/inode/listener; legacy listener refusal; later TCP bind failure; SIGTERM cleanup/restart; replacement-file preservation; invalid SQLite cleanup; disabled ACP workspace cleanup; core port conflict before database access; late core-store failure drain; and termination during database startup. Some tests cover multiple assertions/paths. - Process fixtures clear inherited environment and run only in disposable HOME/XDG/working directories with temporary files, generated fixture tokens and loopback canaries. CI explicitly executes both startup test targets with `server` features. - Local builds use `CARGO_BUILD_JOBS=2`, dedicated `/Users/toasty/.cache/mausbot-round6b-target`, incremental/debug output disabled, and a monitored 20 GiB free-space floor on `/System/Volumes/Data`. No build crossed the floor. Rustfmt and diff whitespace checks pass. - Fresh-context independent review found and closed two cleanup defects plus the startup-signal gap. Reports, source hashes, responses and validation receipts are retained in [`docs/mausbot/evidence/akhomed-startup-20261018-round6b`](docs/mausbot/evidence/akhomed-startup-20261018-round6b/README.md). Exact-head CI remains pending; local tests are not CI or deployment acceptance. ### Incident reconciliation Read-only reconciliation used the enrolled native-house DB credentials, TLS verification, connection-level read-only defaults, explicit repeatable-read/read-only transactions and rollback. No DB repair is indicated: - All 13 migrations in deployed source `45a1b9bedb0d43aaec7eafec6cd58dfeed045347` were applied before the incident. Timestamped public tables show zero rows in the precise and four-minute incident windows. All three native residents' ownership timestamps and advisory sessions predate it; the refusal branches fence identity/epoch, hydration, WAL/snapshot and executor work. - Checked the core service store separately because `CoreApp::new` ran before the old occupied-port refusal. Its existing `akhomed-core` import marker value `0` is dated October 16 04:29:01Z, selecting the deployed rollback-before-insert branch. Its 22 records predate the incident, with no grant rows. Existing-schema/table checks therefore did not create new objects. - Postgres logs corroborate seven transient incident-time sessions. They are not a statement audit. Commit timestamps and statement logging were disabled, and no complete pre-probe snapshot exists; the conclusion combines actual rows with exact deployed control flow and the three recorded lock refusals. The result is **no persistent database effect; no DB repair needed**. The separate discovery-file loss and its authorized round-5 one-file repair remain recorded, with prior bytes/existence unknown. This does not relabel the original failed probe as read-only or claim zero transient effects. The requested scratch receipt is `round6b/incident-reconciliation.json`; a copy and query/log projections are in the evidence directory above. No merge, live DB/pod write, cluster daemon/binary execution, restart, Board action or activation occurred in round 6b. Round 6a's native computer dispatch is separate work. Parent owns CI watching, merge and subsequent activation gates.

toasterson added 3 commits 2026-10-18 00:24:56 +00:00

fix(akhomed): claim startup ownership before discovery publication 070aa02f4d

docs(mausbot): reconcile failed startup and record round6b validation dc03a93269

docs(mausbot): retain exact validation logs as gzip receipts 5c471e75fa

toasterson merged commit 62bf61180c into main 2026-10-18 01:58:55 +00:00

toasterson referenced this pull request from a commit 2026-10-18 01:58:56 +00:00

Fix akhomed startup ownership and side-effect-free CLI (#667)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

follow-up

Documented operational follow-up

No labels

follow-up

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

toasterson/akh-medu!667

No description provided.