archy/.planning/phases/10-key-material-hardening/10-03-SUMMARY.md

phase, plan, subsystem, tags, requires, provides, affects, tech-stack, key-files, key-decisions, requirements-completed, coverage, duration, completed, status

phase	plan	subsystem	tags	requires	provides	affects	tech-stack	key-files	key-decisions	requirements-completed	coverage	duration	completed	status
10-key-material-hardening	03	iso-build	security iso first-boot systemd ssh-host-keys tls machine-id bash		Fail-closed first-boot per-device secret generation: the completion marker is written only when both TLS and SSH generation succeeded A single producer per secret — gen_tls()/gen_ssh() are the only code in the ISO build that create the TLS keypair and SSH host keys Retry-with-backoff inside a single boot, so a transient first-boot condition recovers without a reboot archipelago-first-boot-secrets.timer — unattended self-heal every 15 minutes until generation succeeds A build-time assertion that fails the ISO build if openssl or ssh-keygen is missing from the rootfs Parse-back validation (openssl pkey / openssl x509) before the staging swap, so no service ever reads a truncated artefact A durable failure record at /var/lib/archipelago/first-boot-secrets.failed plus console + journal + stderr on failure FIRST_BOOT_SECRETS_ROOT / FIRST_BOOT_SECRETS_BACKOFF test seams on the generated first-boot script An identity-free rootfs tar: no SSH host keys, no TLS keypair, machine-id truncated /opt/archipelago/rootfs-identity-stripped build-time provenance marker tests/first-boot-secrets/run-tests.sh — 6-case harness driving the shipped heredoc body against a temp root with stubbed generators	image-recipe first-boot sshd nginx-tls	added patterns Seam-for-testability over assertion-in-a-comment: FIRST_BOOT_SECRETS_ROOT prefixes every absolute path so the NEGATIVE property (on failure the marker is NOT created) can be forced and asserted. Same move the entropy fix in 8b51b7e2 made for the RNG. Test the shipped bytes, not a copy: the harness extracts the first-boot script from the builder heredoc between the SECRETSSCRIPT delimiters, so the test and the artefact cannot drift. Strip identity material in the last Dockerfile layer so fail-closed is structural (no key exists) rather than procedural (a script promises to replace it).	created modified tests/first-boot-secrets/run-tests.sh docs/security/KEY-02-ROOTFS-EVIDENCE.md image-recipe/_archived/build-auto-installer-iso.sh	UNIFY, DO NOT DELETE. The defect in F-03 was never that a second attempt to create a key existed — it was that failure was silent and the marker lied about it. A second attempt is only dangerous when it is an unaudited second PRODUCER with its own idea of success, its own absent retry policy and its own absent failure record. So both secondary producers were folded out (the Dockerfile bake and the installer fallback) leaving one generator per secret, rather than 'keep the fallback' (leaves a silent second source) or 'delete the fallback and accept a dead node' (a false trade between security and UX). Fail-closed governs SERVING; self-heal governs RECOVERING. These are separate properties and both must hold. Nothing serves on a key we did not generate; nothing dead-ends waiting for a human at a console. Self-heal uses a systemd timer, not a sleep loop in the script. A loop would hold a Type=oneshot open for hours and hide the failure from systemctl; the timer plus the service's existing ConditionPathExists=! costs a healthy node nothing and needs no teardown. The timer's enable uses a hand-written symlink fallback. Every other `chroot systemctl enable` here ends in `2>/dev/null || true`, which would silently drop the self-heal path — the one thing whose absence is invisible until a node is already broken. Consumers in `failed` state are explicitly restarted on success. try-reload-or-restart is a no-op on a failed unit, so without this a self-healed node would have valid keys on disk and nginx still down — recovery that isn't. Retry semantics: attempt count equals the number of FIRST_BOOT_SECRETS_BACKOFF entries, and the wait after the final attempt is skipped (a failed last attempt is terminal). With the default `2 8 20` that is 3 attempts at t=0s/2s/10s per generator; the trailing 20 is the ceiling if the list is lengthened. Documented in the script rather than left as a puzzle. `After=systemd-random-seed.service` added as its own unit line rather than appended to the existing After=local-fs.target, both because systemd accumulates After= lines and because the plan's acceptance criterion greps for exactly that string. /var/lib/dbus/machine-id is removed only when it is a real file, not when it is the symlink to /etc/machine-id that Debian normally ships. Deleting a live symlink risks a boot-time surprise for no gain; a real copy would be genuine shared state.		id description requirement verification human_judgment D1 A first-boot secret regeneration that fails does NOT set the completion marker, so the oneshot retries on the next boot (D-05) KEY-02 kind ref status unit tests/first-boot-secrets/run-tests.sh#openssl fails every attempt -> exit non-zero, NO marker, failure record names TLS pass kind ref status other Negative control: moving `touch "$MARKER"` back outside the success branch makes that case fail with MARKER-SET-ON-FAILURE (transcript below) pass false id description requirement verification human_judgment D2 Each generator is retried with backoff within a single boot before the boot is declared failed (D-05) KEY-02 kind ref status unit tests/first-boot-secrets/run-tests.sh#ssh-keygen fails twice then succeeds -> backoff recovers within one boot (3 attempts) pass false id description requirement verification human_judgment D3 A terminal failure is loud: console + durable on-disk record + journal, not only a log file nobody reads (D-05) KEY-02 kind ref status unit tests/first-boot-secrets/run-tests.sh#tls-fail case asserts first-boot-secrets.failed exists, names TLS, and stderr carries a FAILED line pass kind ref status other logger + tee -a $ROOT/dev/console emitted by shout(); the console leg cannot be exercised in a temp root and is UNVERIFIED on hardware partial false id description requirement verification human_judgment D4 The shipped rootfs tar contains no SSH host keys, no TLS private key and no populated machine-id KEY-04 kind ref status other docs/security/KEY-02-ROOTFS-EVIDENCE.md — requires an ISO build host; commands recorded, not yet run blocked true id description requirement verification human_judgment D5 The regeneration script is exercised by an automated test that fails when the marker is set on a failed run KEY-02 kind ref status unit tests/first-boot-secrets/run-tests.sh — 6/6 PASS; negative control reproduced pass false id description requirement verification human_judgment D6 Exactly one producer per secret: no second code path anywhere in the ISO build can mint a TLS key or SSH host key with its own accounting KEY-02 kind ref status unit tests/first-boot-secrets/run-tests.sh#single-producer invariant: every key-creating invocation is inside gen_tls/gen_ssh pass kind ref status other Negative control: reintroducing the installer's chroot openssl req block turns case 6 red naming the line, and nothing else pass false id description requirement verification human_judgment D7 A failure self-heals unattended — it never dead-ends a node whose only exit is physical access KEY-02 kind ref status unit tests/first-boot-secrets/run-tests.sh#self-heal: failed run then a later successful run -> key present, marker set, failed units restarted pass kind ref status other archipelago-first-boot-secrets.timer installed + enabled (symlink fallback) — UNVERIFIED on hardware; the harness proves the script half, not systemd's scheduling partial false id description requirement verification human_judgment D8 The deterministic total-failure cause (missing generator binary) fails the BUILD, not the fleet KEY-04 kind ref status other Rootfs RUN assertion on /usr/bin/openssl and /usr/bin/ssh-keygen; fires during the container build. UNVERIFIED until a build host runs it — see KEY-02-ROOTFS-EVIDENCE.md step 5b blocked true	1h	2026-08-02	complete

Phase 10 Plan 03: Fail-closed first-boot secrets + identity-free rootfs — Summary

First-boot per-device secret regeneration now retries with backoff and then fails closed, and the rootfs tar it repairs no longer carries the fleet-shared SSH host keys, TLS keypair or machine-id it was silently papering over. Closes the build side of audit finding F-03.

Task 3 (C-4 build-host evidence) is a blocking checkpoint and is NOT done. It needs an ISO build host. docs/security/KEY-02-ROOTFS-EVIDENCE.md carries the exact command sequence and is marked UNVERIFIED. Nothing in this plan claims the tar listing was observed.

What was wrong

first-boot-secrets.sh (a heredoc inside image-recipe/_archived/build-auto-installer-iso.sh, which is live — image-recipe/build-debian-iso.sh execs it) had two fail-open branches that logged WARNING: ... keeping baked key and continued, and touch "$MARKER" ran unconditionally outside both if blocks. Combined with the unit's ConditionPathExists=!/var/lib/archipelago/.secrets-regenerated and the script's own [ -f "$MARKER" ] && exit 0, one transient failure at first boot left that node on the image-wide shared SSH host key and TLS private key permanently and silently — and the ISO is a published artefact, so every downloader holds those keys.

Commits

Commit	Task	What
21043096	1	Fail-closed, retried regeneration + tests/first-boot-secrets/run-tests.sh
408b328c	2	Rootfs identity-strip layer + two comment corrections that follow from it
201ef474	3 (prep)	docs/security/KEY-02-ROOTFS-EVIDENCE.md, marked UNVERIFIED
2efab5f2	follow-up	Single-producer unification, build-time generator assertion, self-heal timer, 3 new test cases
d9b3a7d5	follow-up	Quote the Dockerfile.rootfs heredoc so comments cannot execute (closes deferred D1)
40b77e39	follow-up	Refuse to bless a cert minted under an untrustworthy clock; backdate notBefore

Nothing was pushed, tagged, built or deployed, per the execution brief.

Task 1 — fail-closed regeneration

• ROOT="${FIRST_BOOT_SECRETS_ROOT:-}" prefixes every absolute path. Unset in production the expansion is empty and behaviour is byte-identical; set, it is what makes the negative property assertable at all.
• retry() runs each generator up to N times with waits from FIRST_BOOT_SECRETS_BACKOFF (default 2 8 20). Staging-then-swap is preserved for both generators, with .new files removed on failure so no half-keypair is left behind.
• touch "$MARKER" now lives inside a TLS_OK == 1 && SSH_OK == 1 branch. Any other outcome writes /var/lib/archipelago/first-boot-secrets.failed (timestamp, which generator failed, both flags), shouts to console + logger + stderr, and exit 1 so the unit lands in failed rather than active. A later successful boot deletes the record so a recovered node does not carry a stale alarm.
• After=systemd-random-seed.service added to the unit. A no-op today (no seed file is baked, which the audit verified) and correct if one is ever introduced.
• The script header states the operational trade in plain words, including that recovery from a terminal failure needs the physical console.

Harness results (final, all six cases)

extracted 236 lines from the builder; bash -n clean
PASS: both generators succeed -> exit 0, marker set, keys swapped in
PASS: openssl fails every attempt -> exit non-zero, NO marker, failure record names TLS
PASS: ssh-keygen fails twice then succeeds -> backoff recovers within one boot (3 attempts)
PASS: TLS fails every attempt on a stripped root -> NO key, NO marker, non-zero exit, record names TLS
PASS: self-heal: failed run then a later successful run -> key present, marker set, failed units restarted
PASS: single-producer invariant: every key-creating invocation is inside gen_tls/gen_ssh

──────── first-boot-secrets summary ────────
passed: 6   failed: 0

Negative control (required by the plan's acceptance criteria)

touch "$MARKER" moved back outside the success branch, harness re-run, then reverted:

SCRATCH APPLIED: marker touch moved back outside the success branch
--- harness against the fail-open variant ---
extracted 175 lines from the builder; bash -n clean
PASS: both generators succeed -> exit 0, marker set, keys swapped in
FAIL: openssl fails every attempt -> MARKER-SET-ON-FAILURE
    exit=1 root=/tmp/tmp.Ta1YFhHWdi/root-tls-fail
    stderr: ARCHIPELAGO FIRST BOOT FAILED: could not generate this device's TLS key material. ...
PASS: ssh-keygen fails twice then succeeds -> backoff recovers within one boot (3 attempts)

──────── first-boot-secrets summary ────────
passed: 2   failed: 1
EXIT=1

The test fails on exactly the regression it exists to pin, and only that case.

Task 2 — identity-free rootfs

Final RUN layer added to Dockerfile.rootfs, after every package install and after the openssl req layer, so nothing regenerates the material afterwards:

• rm -f /etc/ssh/ssh_host_* (private keys and .pub alike)
• rm -f the archipelago TLS key and crt, keeping the /etc/archipelago/ssl directory
• : > /etc/machine-id (systemd's documented regenerate-on-next-boot state)
• /var/lib/dbus/machine-id removed only if it is a real file, not the usual symlink
• writes /opt/archipelago/rootfs-identity-stripped listing what it removed, with no timestamp so RECIPE_HASH reproducibility is unaffected

The openssl req layer is deliberately unmodified.

RECIPE_HASH changed. The strip layer is inside the hashed region (sed -n '/^# STEP 1.../,/^# STEP 2.../p' | grep -c rootfs-identity-stripped → 1), so the next build is forced to rebuild the rootfs tar. Task 3's evidence would be meaningless against a cached tar, and --rebuild is specified as well.

grep -c 'ssh_host' on the builder went 3 → 6.

Deviations from Plan

1. [Rule 1 — Bug] Backticks in my own strip-layer comment would have hung every ISO build

• Found during: Task 2, self-check of the added block.
• Issue: Dockerfile.rootfs is written with an unquoted heredoc (<<DOCKERFILE), so backticks in its body are command substitution evaluated by the build shell. Two comment lines I wrote contained `openssl req`. Reproduced in isolation: the heredoc hung for the full 2-minute timeout as openssl req waited on stdin. bash -n is clean on this — syntax checking cannot catch it.
• Fix: replaced with double quotes, and added an explicit NOTE: in the block warning that the heredoc is unquoted and backticks must never appear there.
• Commit: 408b328c

2. [Rule 1 — Correctness] Script header claim about TLS, twice corrected

• Found during: Task 2, after discovering the installer's TLS fallback.
• Issue: the Task 1 header claimed "the nginx TLS listener will not start". With the installer fallback in place that was false — the web UI would still come up. Shipping a confident false statement in a security-critical script is worse than shipping none.
• First fix (408b328c): narrowed the claim to SSH only, and described the TLS fallback honestly as per-install, never image-wide.
• Second fix (2efab5f2): the fallback is gone, so the original claim is true again for both. Restored, with the reasoning attached rather than left implicit. No comment anywhere in the builder now implies a TLS fallback exists.

3. [Rule 2 — Threat coverage] /var/lib/dbus/machine-id

• Issue: T-10-26 is machine-id correlation across nodes. The plan named /etc/machine-id only. If dbus ships a real copy rather than the usual symlink, truncating /etc/machine-id alone leaves correlated state.
• Fix: guarded removal — symlinks are left alone, real files are removed.
• Commit: 408b328c

Follow-up: unify to a single producer (2efab5f2)

The installer's TLS fallback prompted a decision cycle worth recording, because the reasoning matters more than the outcome.

The false trade. The question was framed as "keep the fallback (a second source of keys) or delete it (a first-boot failure costs the user the web UI, recoverable only at the console)". Both options were wrong, and the framing was wrong. The defect in F-03 was never that a second attempt to create a key existed. It was that failure was silent and the completion marker lied about it. A second attempt is only dangerous when it is an unaudited second producer — carrying its own idea of success, its own absent retry policy, its own absent failure record. So the fix is to unify, not to delete and accept a dead node.

What shipped:

1. One producer per secret. gen_tls() and gen_ssh() are the only code in the ISO build that create the TLS keypair and the SSH host keys. Two secondary producers were folded out: the Dockerfile's openssl req layer (which baked a keypair the strip layer deleted moments later in the same build) and the installer's "ensure SSL cert exists" block. The invariant is checked mechanically, not asserted in prose — case 6 of the harness fails if any executable openssl req / ssh-keygen -A invocation appears outside the generator heredoc.

2. The deterministic failure is caught at build time. The one realistic way generation fails on every retry forever is a missing generator binary, and that is deterministic — no retry or reboot fixes it. A rootfs RUN layer now fails the build if /usr/bin/openssl or /usr/bin/ssh-keygen is missing or non-executable. The build already guaranteed these (openssl and openssh-server are both in the package list, and openssh-server hard-depends openssh-client, which ships ssh-keygen), so this is cheap insurance rather than a fix. It earns its place the first time someone edits that package list.

3. Failure self-heals; it never dead-ends. archipelago-first-boot-secrets.timer (OnBootSec=5min, OnUnitActiveSec=15min) re-runs the service until it succeeds. The service's existing ConditionPathExists=! makes every trigger a no-op once the marker exists, so a healthy node pays nothing and no teardown is needed. Two details that would have made this theatre if missed:

   • chroot systemctl enable can fail silently, and every other enable in this file ends in || true. The timer's enable has a hand-written symlink fallback, because the absence of self-heal is invisible until a node is already broken.
   • try-reload-or-restart is a no-op on a failed unit. Without special handling, a self-healed node would have valid keys on disk and nginx still down. Consumers found in failed are now explicitly restarted (--no-block, to avoid a boot-transaction deadlock at first boot, where we are ordered Before= them).

4. Never serve a bogus key. gen_tls now parses both halves back (openssl pkey, openssl x509) before the staging swap, so a truncated or half-written artefact is never what nginx reads. Fail-closed governs serving; retry-and-self-heal governs recovering. They are different properties and both hold.

Negative controls for the three new cases

Each defect was reintroduced, the suite run, and the defect reverted. Each lights up exactly one case — a test that goes red for several reasons at once is not pinning any of them.

Control A — reintroduce a fallback-style key creation on the failure path (the deleted installer block's behaviour, moved into the script):

FAIL: TLS fails every attempt on a stripped root -> TLS-KEY-EXISTS-AFTER-FAILURE TLS-CRT-EXISTS-AFTER-FAILURE
passed: 5   failed: 1

(First run of this control also reddened case 5, because case 5's run-1 block redundantly re-asserted case 4's property. That assertion was removed — case 5 now tests recovery only — and the control re-run to confirm it is isolated. The transcript above is the re-run.)

Control B — dead-end a node that has already failed once (exit 0 early if the failure record exists, a plausible "don't retry a known-bad node" optimisation):

FAIL: self-heal -> run2-marker-missing run2-key-missing run2-crt-missing run2-stale-failure-record run2-did-not-restart-failed-nginx
passed: 5   failed: 1

Control C — reintroduce the installer's chroot ... openssl req block verbatim:

FAIL: single-producer invariant -> SECOND-PRODUCER-at-line-3586
    generator heredoc spans lines 1713-1950 of image-recipe/_archived/build-auto-installer-iso.sh
passed: 5   failed: 1

All three reverted; suite back to 6/6.

Residual operational risk — stated plainly

A machine on which secret generation can never succeed ends up with no SSH host key and no TLS key. sshd will not start, nginx will not serve the web UI, and that node needs physical console access. That is the honest worst case and it is not softened anywhere in the code comments either.

What shrinks it to genuinely-broken-hardware:

• The deterministic cause is gone before shipping. A missing openssl or ssh-keygen fails the ISO build, so it cannot reach a node.
• Transient causes are absorbed. Three attempts with backoff inside the boot (proven by harness case 3, which shows a generator failing twice and succeeding on the third), then every 15 minutes on the timer, then again on every boot — indefinitely, because the marker is never written on failure.
• Recovery completes itself. On a later success the script restarts the units that refused to start, so the node comes back without a reboot and without a human (harness case 5).

What is left is a machine where openssl or ssh-keygen is present but cannot ever produce a key — a disk that is permanently full, or failing hardware. On that machine the node refuses to serve rather than serving on a key nobody generated, which is the trade this phase exists to make. It says so on the console, in the journal, and in /var/lib/archipelago/first-boot-secrets.failed.

Follow-up: quote the Dockerfile heredoc (d9b3a7d5) — closes deferred D1

cat > "$WORK_DIR/Dockerfile.rootfs" <<DOCKERFILE was unquoted, so the build shell performed command substitution on the Dockerfile body: a backtick inside a Dockerfile comment was executed on the build host and its output spliced into the generated file. Six comments did this, and one of them ran systemctl start archipelago-fips.service against the build machine on every ISO build.

Boundary checked before editing. Only lines inside the heredoc body are at risk. The other backticked comments in this file (:264, :809, :1188, :1289, :1506, :1605, :3597, :3651) are ordinary shell comments outside any unquoted heredoc, plus one inside the quoted SECRETSSCRIPT heredoc — none were ever evaluated, and none were touched.

Fixed the class, not the instances. The body needs exactly four build-time values, all package names (LINUX_IMAGE_PKG, GRUB_EFI_PKG, GRUB_EFI_SIGNED_PKG, GRUB_PC_PKG), on four consecutive lines — so quoting was entirely practical. The heredoc is split into DOCKERFILE_HEAD and DOCKERFILE_TAIL, both quoted, with one explicit printf interpolating those four names between them. Escapes that existed only because the heredoc was unquoted were undone in the same pass: six trailing \\ → \ (Docker line continuations) and four \$ → $ (RUN arguments reach the shell verbatim; Docker does not substitute variables in RUN).

Substance verified by rendering, not by inspection. The generated Dockerfile was rendered before and after with identical inputs and diffed normalised (continuations joined, whitespace collapsed). Both are 190 normalised lines and the only differences are the six comments regaining their text — every instruction byte-identical:

< # the archipelago backend calls
> # the archipelago backend calls `systemctl start archipelago-fips.service`
< # fips-gateway is gated behind the Cargo feature (depends on
> # fips-gateway is gated behind the `gateway` Cargo feature (depends on

Case 7 asserts every heredoc writing Dockerfile.rootfs has a quoted delimiter, and when one does not, reports which body lines would execute. The assertion is on the delimiter, not on backticks — with quoting a backticked comment is legal, and six of them are back in the body on purpose; flagging backticks would flag a non-bug and fail on the very comments this restored.

Controls:

Control D — unquote the delimiter (the real regression):
FAIL: Dockerfile heredoc quoting -> UNQUOTED-DELIMITER-at-line-287 would-execute-at-lines:317,318
passed: 6   failed: 1

Control E — add a backticked comment, delimiter still quoted:
PASS: Dockerfile heredoc delimiters are quoted — a backticked comment cannot execute
passed: 7   failed: 0
  and it renders intact:
  175:# Control E: a backticked `systemctl start archipelago-fips.service` comment

Control E is the more informative of the two: the backtick that used to be a build-host RCE is now inert and renders as written. That is what "fixed the class" means, and it is why a bare backtick reintroduction correctly reddens nothing.

deferred-items.md held D1 as its only entry and has been deleted — nothing was left that is genuinely out of scope.

Follow-up: untrustworthy clock at cert-minting time (40b77e39)

The failure fail-closed cannot catch, because generation succeeds. This unit runs before time has synced; openssl req -x509 stamps notBefore from whatever the clock says. Dead RTC or flat CMOS battery → clock ahead gives "not yet valid" (harder to diagnose than a self-signed warning), clock behind gives an already-expired cert once time syncs. The marker was then set and never revisited: a node permanently serving a cert nothing accepts.

Finding, established rather than assumed: this image does not use systemd-timesyncd. It installs and enables chrony (:388, :575), and chrony-wait.service — the unit that is Before=time-sync.target — is not enabled. So time-sync.target is inert here and ordering After= it would buy nothing. Enabling chrony-wait to make it meaningful would stall boot behind NTP on a node with no network, and these nodes are routinely offline at first boot.

Decision: no ordering change. Not deadlocking boot outranks cert-date elegance (constraint 3). Fixed locally instead:

1. Backdate notBefore by 24h so ordinary node/client skew cannot invalidate a fresh cert. This does not weaken a self-signed cert — notBefore is not a security control here. -not_before/-not_after arrived in OpenSSL 3.5 and the rootfs is debian:trixie which ships it, but the capability is probed, not assumed: guessing wrong would fail every attempt and brick the node, the exact outcome all of this exists to prevent. Without the flags we do not backdate, and rule 2 still covers the dangerous case.
2. Refuse to bless a cert dated outside a plausible window (2026-01-01 … 2056-01-01). The material stays installed — the node is usable, sshd comes up — but the dates are recorded as failed=cert-dates and the cert is regenerated automatically once time syncs.

Generation is now driven by need rather than "is the marker absent", and ConditionPathExists=! was removed from the unit so a node that already completed can still be re-examined. Skipping the unit is precisely how such a node would stay broken forever. On a healthy node the script exits in milliseconds.

Anti-spin is one condition: a date-driven regeneration happens only when the clock is currently plausible. A node whose clock is still wrong re-checks and mints nothing.

Regression I introduced and caught

Driving generation purely by content made needs_ssh() false whenever any host key existed — which would have left an image-baked, fleet-shared host key in place forever. That is F-03 reopened. The marker check is back in both needs_ functions. Case 1 — which prestages a baked key and asserts it was replaced — is what caught it.

Controls

Control F — bless the cert regardless of clock (the pre-fix behaviour):
FAIL: wrong clock -> run1-BAD-DATES-NOT-RECORDED
passed: 7   failed: 1

Control G — remove the anti-spin guard:
FAIL: wrong clock -> SPINNING-reminted-while-clock-still-wrong(1->2)
passed: 7   failed: 1

Control G first passed against a deliberately broken guard, which was a flaw in my test, not in the fix: the assertion compared certificate dates, and a re-mint under a frozen fake clock produces a byte-identical notBefore. Dates cannot distinguish "left alone" from "regenerated again". The assertion now counts openssl req invocations, which can — and only then did the control redden. Worth recording as the second time in this plan that a first-draft assertion looked green for the wrong reason.

Not covered here

Nodes already deployed from earlier ISOs never receive this script — it is installed by the installer, not shipped by OTA. Fleet remediation for those nodes is 10-04/OTA work in core/**, which is held by other executors, so per the standing constraint it is reported rather than attempted.

Known Stubs

None. No placeholder values, no TODOs, no unwired code paths.

Threat Flags

None. No new network endpoint, auth path, file-access pattern or schema change at a trust boundary. The plan installs no packages (T-10-SC: accept); none were added.

UNVERIFIED — needs hardware

Task 3's C-4 checkpoint is now more important, not less: with the rootfs stripped and no install-time fallback, the tar listing is the only pre-hardware evidence that the shipped image is identity-free.

Item	Audit ref	What it needs	Command
Rootfs tar is identity-free after a forced rebuild	C-4	ISO build host with podman/docker and disk for a full rootfs rebuild	UNBUNDLED=1 bash image-recipe/build-debian-iso.sh --rebuild, then the tar listings in docs/security/KEY-02-ROOTFS-EVIDENCE.md steps 2/4/5/5b/6
The build-time generator assertion actually fires	C-4	same build host	grep 'first-boot secret generators present' <build log> — evidence doc step 5b
The self-heal timer ships and is enabled on the target	—	same build host, then a node	evidence doc step 5 (timer present on installer media); systemctl status archipelago-first-boot-secrets.timer on a node
Two nodes flashed from one ISO get different keys	C-3	two physical machines	audit §779; both SSH and TLS fingerprints are now equally sharp signals — see the C-3 section of the evidence doc
The console leg of the failure shout reaches a real screen	—	a real node, or a VM console	force a first-boot failure and observe /dev/console

The harness proves the script half of self-heal (a failed run followed by a successful run recovers the node and restarts the failed units). It does not and cannot prove systemd's scheduling — that the timer is enabled and actually fires at 5min/15min. That is hardware verification.

Self-Check

• image-recipe/_archived/build-auto-installer-iso.sh — FOUND, bash -n clean
• tests/first-boot-secrets/run-tests.sh — FOUND, mode 755, exits 0 with 8 PASS
• docs/security/KEY-02-ROOTFS-EVIDENCE.md — FOUND, contains C-4
• deferred-items.md — DELETED; its only entry (D1) is fixed, not filed
• Commits 21043096, 408b328c, 201ef474, 2efab5f2, d9b3a7d5, 40b77e39 — all FOUND
• Generated Dockerfile.rootfs rendered before/after the heredoc change and diffed normalised: 190 lines each, only the six comment restorations differ
• Single-producer grep: the only executable key-creating invocations in the builder are openssl req and ssh-keygen -A inside the generator heredoc; every other match is a comment
• git status --porcelain image-recipe/ — clean; _archived/ not moved or renamed
• No file authored by a concurrent agent (core/archipelago/src/**, neode-ui/**, .planning/STATE.md) was staged in any commit

Self-Check: PASSED