feat(host): crash/hardware-error capture, delivered by a new host-fixup OTA channel (#144)

kdump + rasdaemon on every node, per docs/kdump-rasdaemon-design.md with
the approved decisions: hang capture ON (a wedged kiosk dumps and reboots
itself instead of sitting dead), crashkernel=256M, backfill ships with
this release, phase-2 UI surfacing deferred.

Host fixups (docs/system-level-ota-design.md) are the general answer to
'deliver system-level updates OTA': curated OS packages, sysctl drop-ins,
service enablement and the GRUB crashkernel line, carried by the signed
binary and applied idempotently at startup — non-fatal by construction
(offline/locked-dpkg nodes converge on a later boot), skipped on dev
boxes and non-Debian hosts. This formalizes the polkit/audio repair
precedents into a channel with a stated policy: pinned packages and
parameter intent only, never dist-upgrade automation; the ISO bakes the
identical end state into fresh installs (next commit).

The one runtime limitation is honest: crashkernel memory can only be
reserved at boot, so the fixup writes GRUB, runs update-grub, and logs
that it takes effect on the next reboot.

tests/lifecycle/os-audit.sh gains section D — a graded baseline check:
FAIL if capture never landed, WARN if written but awaiting reboot, PASS
when reserved, policy live and rasdaemon recording. Section D runs
independently of RPC health: a wedged backend must not mask that the
node also stopped capturing evidence.

Verification: host_fixups unit tests 4/4; cargo fmt clean; full suite
runs in the release gate (create-release) and the archi-dev-box
lifecycle gate before the tag.
This commit is contained in:
archipelago
2026-08-31 07:23:44 -04:00
parent 9df580bf2b
commit cbd463e980
6 changed files with 481 additions and 1 deletions
+7
View File
@@ -55,6 +55,7 @@ mod entropy;
mod federation;
mod fips;
mod health_monitor;
mod host_fixups;
mod host_ip;
mod identity;
mod identity_manager;
@@ -435,6 +436,12 @@ async fn main() -> Result<()> {
// iframe on kiosk nodes (docs/tv-input-iframe-apps.md).
tokio::spawn(bootstrap::ensure_gamepad_keys());
// Host-level fixups (#144 + docs/system-level-ota-design.md): kdump +
// rasdaemon — crash/hardware-error capture delivered to already-deployed
// nodes over the signed binary OTA. Idempotent, non-fatal, background;
// the crashkernel= GRUB edit lands on the next reboot.
tokio::spawn(host_fixups::ensure_host_fixups());
// Mesh access: mirror IPv4-published app ports onto [::] so direct-port
// app URLs (http://[<fips0 ULA>]:<port>) work from the companion.
tokio::spawn(mesh_ports::run_mesh_port_mirror());