Fixes from real fresh-install feedback (Framework node .81) + its log bundle:
Backend:
- websocket: subscribe before initial snapshot — broadcasts in the gap were
silently lost, stranding clients on stale state until a hard refresh
(the "everything needs ctrl-r" bug: My Apps stuck Loading, App Store
stuck Checking, containers-scanned never arriving)
- crash recovery: check the crash marker BEFORE writing our own PID —
recovery had never run on any node (always saw its own PID and skipped);
PID-reuse guard via /proc cmdline
- boot status: pending-boot-starts registry (recovery, stack recovery,
reconciler, adoption) — scanner overlays queued-but-down apps as
Restarting instead of Stopped after a reboot; scanner-authored
Restarting resolves immediately on a settled scan (no transitional wedge)
- install deps: bounded wait (36x5s) when a dependency is installed but
still starting ("Waiting for Bitcoin to start…") instead of instant
rejection; dependency-gate rejections remove the optimistic entry (no
phantom Stopped tile) and surface as a notification
- seed backup: auth.setup persists the onboarding mnemonic as the
encrypted seed backup (reveal previously failed on EVERY node — nothing
ever wrote master_seed.enc); seed.restore stashes too; error sanitizer
lets seed/2FA errors through instead of "Check server logs"
- lnd: bitcoind.rpchost resolved from the running Bitcoin variant
(hardcoded bitcoin-knots broke Core nodes); manifest uses derived_env
- bitcoin status: clean human message for connection-reset/startup; raw
URLs + os-error chains no longer reach the app card
- fedimint-clientd: chown /var/lib/archipelago/fmcd to 1000:1000 (root-
created dir crash-looped the rootless container, EACCES) — first-boot
script + pre-start self-heal
- log volume (>1GB/day on a day-old node): journald caps drop-in (ISO +
bootstrap self-heal), bitcoind -printtoconsole=0 everywhere (90% of the
journal was IBD UpdateTip spam), tracing default debug→info
Frontend:
- Login: Enter advances to confirm field then submits; submit always
clickable with inline errors (was silently disabled on mismatch);
Restart Onboarding needs a confirming second click (the mismatch →
"onboarding restarted" trap)
- sync store: 30s state reconciliation + refetch on re-entrant connect;
20s containers-scanned escape hatch so Checking can never show forever;
fresh empty node reaches the real "no apps yet" state
- intro video: CRF20 re-encode (SSIM 0.988) + faststart — moov was at EOF
so playback needed the full 15MB first (the intro lag)
- backgrounds: 10 heaviest JPEGs → WebP q90 (9.4MB→6.6MB); 7 stayed JPEG
(WebP larger on noisy sources)
- Web5ConnectedNodes: drop unused template ref that failed vue-tsc -b
ISO/kiosk:
- nginx: /assets/ 404s no longer cached immutable for a year; HTTPS block
gained the missing /assets/ location (served index.html as images)
- kiosk: launcher/service spliced from configs/ at ISO build (stale
heredoc force-disabled GPU); MemoryHigh/Max 1200/1500→2200/2800M (kiosk
rode the reclaim throttle = the lag); firmware-intel-graphics +
firmware-amd-graphics (trixie split DMC blobs out of misc-nonfree)
Verified: cargo test 898/898 green, npm run build green with dist
contents confirmed (webp refs, lnd.png, faststart video, new strings).
Handover for ISO build + deploy: docs/HANDOVER-2026-07-02-iso-feedback.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- reticulum.rs: send_text_msg was lossy-UTF8-mangling binary CBOR control
envelopes (ReadReceipt etc.) before sending as LXMF text; base64-encode
with a marker instead, decoded losslessly on receive.
- typed_messages.rs: mesh.send-read-receipt fired automatically on every
chat view with no is_archy_peer gate, so viewing a message from a stock
(non-archy) LXMF peer auto-sent it an undecodable control envelope,
surfacing as garbage text right after whatever it just sent. Now a no-op
for non-archy peers.
- mesh/listener/mod.rs: RX_STALL_TIMEOUT was 300s and forced a full
auto-detect reconnect on any otherwise-healthy but quiet mesh link
(visible as "Connecting..." flapping); this also wiped Reticulum's
in-memory peer-address table every cycle, breaking messaging with peers
who hadn't re-announced in the window. Bumped to 1800s.
- reticulum.rs: persist the peer prefix/dest-hash/display-name table to
disk so a restart doesn't force every peer back to "Anonymous Peer"
until they re-announce.
- decode.rs/frames.rs: Meshcore was discarding the SNR its wire format
carries; wire it onto the peer record. Mesh.vue's signalBars() now falls
back to SNR-based bars when RSSI is unavailable (always true for
Meshcore); Reticulum has neither and correctly stays at 0/"no data".
- system/handlers.rs, dispatcher.rs: new system.get-hostname RPC + cert
regeneration (with a proper SAN) whenever server.set-name changes the
hostname, so HTTPS doesn't add a mismatch warning on top of the
self-signed one after a rename.
- AccountInfoSection.vue: surface the mDNS hostname + http/https links in
Settings (HTTPS needed for mic/camera secure-context features) — never
forced, both keep working.
- build-auto-installer-iso.sh: ship avahi-daemon so .local names actually
resolve on the LAN, and give the self-signed cert a real SAN instead of
a bare CN, both at image-build and install-time-fallback.
- Mesh.vue/MediaLightbox.vue/mesh-styles.css: mic/attach-stack no longer
closes on a plain hover-past; mesh images open in the shared lightbox
and have a real download button; lightbox close button moves to
bottom-center on mobile instead of under the status bar; mesh device
panel gets the same height/padding as its sibling tabs.
Verified: 108/108 mesh unit tests, deployed + confirmed healthy on
.116/.198/.228 (matching binary hash across all three), live Reticulum
messaging confirmed working end-to-end post-deploy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
configure_display picked whichever mode was already "active" on the HDMI
output, so if X ever booted cloned to the laptop panel's resolution it
would keep re-confirming that wrong mode forever instead of self-healing
to the display's native mode.
PyInstaller-based daemons (e.g. the Reticulum mesh daemon) self-extract
to /tmp on every launch and leak their extraction dir on abnormal exit;
combined with release-build staging dirs this exhausted the default cap
on .116 and silently broke Reticulum ("no space left on device" during
self-extraction, masquerading as a connect failure). Ship a tmp.mount.d
override (75% of RAM) in the installer image so fresh installs don't
inherit the same ceiling.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The kiosk chromium pinned ~92% of a core (software-compositing spin from
--enable-gpu-rasterization on a GPU-less/headless node), saturating the machine
and starving the backend + container builds — it caused the .198 receive timeout
and the deploy storms.
- archipelago-kiosk.service: CPUQuota=75% + MemoryMax/High + Delegate, so a
runaway kiosk can never take the whole node down.
- archipelago-kiosk-launcher.sh: detect /dev/dri — use GPU rasterization only
when a GPU exists, else --disable-gpu (avoids the headless spin).
- bootstrap::ensure_kiosk_hardened: OTA self-heal that installs the updated
unit+launcher on already-deployed nodes, daemon-reloads, and only try-restarts
a *running* kiosk (never re-enables an operator-disabled one).
cargo check clean; launcher bash -n clean; unit syntax valid.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On production nodes /var/lib/archipelago (the app data dir AND podman's
graphroot=/var/lib/archipelago/containers/storage) is a separate
device-mapper volume. archipelago.service ordered only After=network-online
.target, so on cold boots it (and its ExecStartPre) could start BEFORE
var-lib-archipelago.mount, write to the bare mountpoint on rootfs, fail every
podman call, exit, and be restarted every 5s until the volume mounted — the
"~20x [FAILED] Failed to start over ~5min" boot flap. Proven live on .198:
"var-lib-archipelago.mount: Directory /var/lib/archipelago to mount over is
not empty, mounting anyway" — the service had written there pre-mount.
Fix: RequiresMountsFor=/var/lib/archipelago (adds Requires= + After= on the
mount unit).
- image-recipe/configs/archipelago.service: ships the directive on fresh ISOs.
- bootstrap::ensure_archipelago_mount_ordering(): self-heals already-deployed
nodes' installed unit + daemon-reload (boot-ordering only, effective next
reboot; never restarts the running service). Idempotent; harmless on rootfs
installs (maps to the always-mounted root).
Verified on .198: after applying, systemctl shows After=var-lib-archipelago
.mount and systemd-analyze verify is clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fedimint UI HTML/CSS reference absolute /assets/* paths; under /app/fedimint/
those hit the main SPA, not the fedimint container, so the UI renders
unstyled. Add the proven sub_filter asset-rewrite pattern (as indeedhub/
botfights use) to the /app/fedimint/ block in the nginx template + https
snippet (also rewrites url(...) for the CSS background image). Bootstrap
self-heal for already-deployed nodes is the documented resume point.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hotfix: archipelago.service ExecStartPre now mkdirs /run/containers and
/var/lib/containers before the unit's mount-namespace setup tries to bind
them. Without this, fresh nodes that don't have /run/containers (e.g.
nodes provisioned without a prior podman session) fail at the namespace
step with:
Failed to set up mount namespacing: /run/containers: No such file or directory
Failed at step NAMESPACE spawning /bin/bash: No such file or directory
Existing nodes don't pick up systemd unit changes via OTA — they need a
one-time `systemctl edit archipelago` adding the same mkdir. ISO installs
from this version forward have the fix baked in.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resilience-validated release. Three full sweeps of the new resilience
harness against .228 confirm no shipstoppers.
Big user-visible:
- Bitcoin RPC auth durably correct via host-rendered nginx.conf bind-mount,
replaces fragile post-start exec that failed under restricted-cap rootless
podman ("crun: write cgroup.procs: Permission denied")
- Multi-container stack installs (indeedhub, immich, btcpay, mempool) now
emit phase events at every boundary so the progress bar advances
- Apps no longer vanish from the dashboard mid-install (absent-scanner skips
packages in transitional states)
- Indeedhub fresh installs work end-to-end (was 8500+ restart loop): five
missing env vars (DATABASE_PORT, QUEUE_HOST, QUEUE_PORT,
S3_PRIVATE_BUCKET_NAME, AES_MASTER_SECRET) added to install code
- Tailscale install fixed: --entrypoint string was being passed as a single
shell-line arg; switched to custom_args array
- Catalog cleaned of broken entries (dwn, endurain, ollama removed; nextcloud
restored on docker.io)
- Bitcoin Core update path uses correct image (was looking for nonexistent
lfg2025/bitcoin:28.4)
- ISO installs now allocate swap on the encrypted data partition
Infra:
- New resilience harness (scripts/resilience/) — black-box state-machine
tester, every app × every transition. Run before each release.
Sweep #3 final: PASS 107 / FAIL 12 / SKIP 14. The 12 fails are 1 cosmetic
(homeassistant trusted_hosts), 8 harness/timing false-positives, and 3
non-shipstopper tracked items. Down from 23 in baseline sweep #1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Releases no longer ship as bootable ISOs. Archipelago updates are
distributed as the backend binary plus a frontend tarball referenced by
releases/manifest.json. Nodes OTA-update via scripts/self-update.sh.
Filebrowser and AIUI remain bundled inside the frontend tarball and
deployed atomically, verified present in v1.7.43-alpha release artifact
(189 AIUI files, filebrowser-client bundle).
Archived under image-recipe/_archived/ (resurrectable if ISO distribution
is reintroduced):
- build-auto-installer-iso.sh
- build-unbundled-iso.sh
- test-iso-qemu.sh
- scripts/convert-iso-to-disk.sh
- BUILD-ISO-STATUS.md, ISO-BUILD-CHECKLIST.md
- branding/isohdpfx.bin
- .gitea/workflows/build-iso-dev.yml
Updated release process docs to drop ISO references:
- scripts/create-release.sh (next-steps text)
- docs/BETA-RELEASE-CHECKLIST.md
- docs/hotfix-process.md
- README.md
Every OTA self-update and every ISO capture was implicitly relying on
/opt/archipelago/web-ui/aiui/ already being present on disk. Any node that
had its web-ui directory atomically swapped (for example by a manual
deployment shipping only neode-ui dist output) lost aiui entirely and the
AI Assistant tab fell through to the "needs to be enabled" placeholder.
self-update.sh: drop the rsync --exclude aiui preservation trick and
instead stage demo/aiui into the freshly-built dist tree before rsync.
demo/aiui in the repo is now the source of truth; every update overwrites
the on-disk copy with a matching version rather than carrying forward
whatever stale bundle happened to survive.
build-auto-installer-iso.sh: prepend demo/aiui to the AIUI search list so
ISO builds from a fresh repo clone pick it up automatically, without
requiring a side-checkout of the AIUI project or a live dev server.
This matches create-release-manifest.sh which already bakes demo/aiui
into the release tarball (lines 86-89).
The backend runs as `archipelago` and calls `install_log()` to append
audit lines to the install log on every install / update / remove /
start / stop / restart. Target path was /var/log/archipelago-container-installs.log,
which does not exist and cannot be created by the service because
/var/log/ is root-owned. OpenOptions errors were silently swallowed,
so the log was never written on any node.
Ship a tmpfiles.d rule that pre-creates /var/log/archipelago/ and
container-installs.log with archipelago:archipelago ownership. Move
the const path to match, keeping logs inside the directory logrotate
already rotates (image-recipe/configs/logrotate.conf). Install the
rule from both the ISO build and self-update, and apply it
immediately on self-update so existing nodes get a working log
without needing a reboot.
Verified on .228: file created, backend user can write, backend
binary rebuilt with new const.
The Hetzner VPS at 23.182.128.160 was decommissioned. Replace it
everywhere with the OVH VPS at 146.59.87.168, which was previously
the tertiary mirror.
- update.rs: drop DEFAULT_TERTIARY_MIRROR_URL, promote .168 into
the secondary slot as "Server 1 (OVH)"; tx1138 becomes Server 2.
Default mirror list shrinks from 3 to 2.
- container/registry.rs: default RegistryConfig drops .23, promotes
.168 to Server 1 / priority 0, tx1138 stays Server 2 / priority 10.
- api/rpc/package/config.rs: trusted-registry allowlist swaps .23
for .168.
- api/handler/mod.rs: app-catalog fallback URL uses .168.
- neode-ui/views/marketplace/marketplaceData.ts: REGISTRY uses .168.
- scripts/image-versions.sh: ARCHY_REGISTRY_FALLBACK uses .168.
- image-recipe/build-auto-installer-iso.sh: installer ISO registries
use .168 (both podman registries.conf and backend registries.json).
Tests updated to assert on the new 2-entry default lists (registry +
mirror). URL-parser fixture tests in update.rs retain .23 strings —
they exercise string-parsing logic, not mirror policy.
Git remotes: dropped `gitea-vps` and the .23 push URL on the `origin`
multi-push alias (not part of this commit — pure working-copy change).
Adds Delegate=memory pids cpu io to the archipelago.service unit.
Context: the service runs as User=archipelago under system.slice with
rootless podman. When podman creates transient libpod-*.scope units for
containers under user.slice, systemd needs the caller to hold
CAP_SYS_ADMIN on the target cgroup subtree \u2014 which happens iff
Delegate= lists the controllers we want to set. Without Delegate, any
future code path that goes through the podman CLI (runtime.rs) instead
of the libpod HTTP API (podman_client.rs) would hit MemoryMax
rejections that have exactly the same symptom as the bug I just fixed
in parse_memory_limit but with a completely different root cause.
Belt-and-braces: current production path uses PodmanClient and was
fixed in the preceding commit. But the DockerRuntime CLI path in
runtime.rs:262-268 (cmd.arg("--memory")) is still reachable via
AutoRuntime fallback on hosts without podman, and future rust
orchestrator code may legitimately need cgroup delegation. This
directive is no-op harmful on hosts that already delegate upstream
(systemd gracefully handles duplicate/nested delegation).
BootReconciler (in-process, 30s interval, spawned from main.rs as of
Step 6 commit 48f08aa3) fully replaces the timer-driven bash
reconciliation path. Delete the systemd unit + timer and their
ISO-builder touchpoints.
Removed:
- image-recipe/configs/archipelago-reconcile.service
- image-recipe/configs/archipelago-reconcile.timer
- image-recipe/build-auto-installer-iso.sh L412-413 (COPY unit+timer)
- image-recipe/build-auto-installer-iso.sh L449 (systemctl enable)
- image-recipe/build-auto-installer-iso.sh L542-543 (cp to WORK_DIR)
Kept (intentionally):
- scripts/reconcile-containers.sh
- scripts/container-specs.sh
Reason: core/archipelago/src/api/rpc/package/update.rs still invokes
reconcile-containers.sh at two sites (OTA update + rollback paths).
Porting those call sites to ContainerOrchestrator::upgrade() requires
manifests for every container update.rs might touch — that scope
belongs in Step 8b. Until then the script stays on disk, just no
longer runs on a periodic timer.
No Rust code changes. cargo check -p archipelago clean, 6 pre-existing
warnings. Skipped full ISO rebuild validation per user decision —
edits are 5 textual deletions with zero behavioral ambiguity; Step 9
live hot-swap on .228 will catch any regression.
Install flow
- api/rpc/package/install.rs: always append the literal image URL as a
last-resort pull candidate in do_pull_image, so images not carried by
any configured mirror (docker.io/bitcoin/bitcoin:28.4) still install
instead of masquerading as a generic pull failure across every mirror.
- api/rpc/package/install.rs: write_bitcoin_conf now skips on any stat
error, not just "file exists". Once bitcoin-knots' first-boot chowns
/var/lib/archipelago/bitcoin into the container's user namespace (700
perms, UID 100100/100101), the archipelago daemon can't even traverse
in — try_exists returns Err which unwrap_or(false) treated as "not
present" and drove a doomed write. Now errors out of the directory
traversal are treated as "conf already owned by container user" and
the write is skipped. Mirrors the lnd.conf pattern.
- api/rpc/package/install.rs: drop the hardcoded `prune=550` from the
conf default. Operators with multi-TB drives shouldn't be silently
pruned; users who want a pruned node can set it in bitcoin.conf
themselves. Full archive is the only honest default.
- api/rpc/package/config.rs: bitcoin-core now passes explicit
-server/-rpcbind/-rpcallowip/-rpcport/-printtoconsole/-datadir CLI
args. Vanilla bitcoin/bitcoin:28.4 has no entrypoint wrapper and
reads conf + argv only; without these the RPC listens on 127.0.0.1
inside the container and rootlessport can't reach it, so the
bitcoin-ui companion gets 502 on every /bitcoin-rpc/ call.
Bitcoin Knots keeps its own entrypoint-driven defaults.
- container/docker_packages.rs: split bitcoin-core out of the shared
AppMetadata arm. bitcoin-core now surfaces as "Bitcoin Core" with
bitcoin-core.svg and a Reference-implementation description; the
bitcoin + bitcoin-knots ids keep the Knots branding. Fixes the home
card showing "Bitcoin Knots" for a Core install.
Bitcoin node UI (docker/bitcoin-ui)
- index.html: impl name/tagline/logo now dynamic. applyImplBranding()
reads subversion from getnetworkinfo — /Satoshi:X/Knots:Y/ resolves
to Bitcoin Knots, plain /Satoshi:X/ resolves to Bitcoin Core. Both
get their own icon and subtitle. Settings modal replaced its
hardcoded Regtest/txindex=1/port-18443 placeholders with live values
from getblockchaininfo + getindexinfo + getzmqnotifications.
- index.html: new Storage info card (Full Archive · X GB /
Pruned · X GB from blockchainInfo.pruned + size_on_disk) visible on
the main dashboard, same level as Network. Settings modal mirrors it
with the prune height when applicable.
- Dockerfile + assets/: bitcoin-core.svg, bitcoin-knots.webp, and the
bg-network.jpg used by the dashboard are now COPY'd into the image
under /usr/share/nginx/html/assets. Previously the <img src> pointed
at paths that 404'd into the SPA fallback and the onerror handler
hid the broken logo silently.
Frontend
- appSession/appSessionConfig.ts: add bitcoin-core to APP_PORTS (8334),
HTTPS_PROXY_PATHS (/app/bitcoin-ui/), and APP_TITLES (Bitcoin Core).
Without these the AppSessionFrame showed "No URL found for
bitcoin-core" and the home/app-list title fell through to the raw id.
- settings/AccountInfoSection.vue: backfill What's New entries for
v1.7.31 through v1.7.37 that had been missed in earlier cuts.
Release plumbing
- releases/v1.7.37-alpha/: binary + frontend tarball.
- releases/manifest.json: v1.7.37-alpha, sha256/size refreshed.
- Cargo.toml / package.json: version bumps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. FIPS auto-activate at server startup only fires if fips_key already
exists on disk, which on a fresh install is never true until AFTER
onboarding. By the time the user completes seed-generate/restore,
archipelago has been running for minutes and the startup task has
long since exited. User still had to hit Activate.
Fix: call spawn_post_onboarding_fips_activate() from the tail of
handle_seed_generate and handle_seed_restore — the moment the
fips_key materialises, a detached task runs `fips::config::install`
+ `archipelago-fips.service activate`. Logged only, never blocks
the onboarding RPC.
2. Kiosk health-poll window was 30 × 2s (configs/ copy was 60 × 2s
but unused — the heredoc in build-auto-installer-iso.sh is what
actually lands on disk). On .198's slower hardware archipelago
/health wasn't ready within 60s, so Chromium launched against a
not-yet-running backend → blank window until manual reboot. Bumped
to 150 × 2s (5 min) + TimeoutStartSec=360. .253 was already well
within the window; this protects the slower box too. Standalone
configs/archipelago-kiosk.service updated in lockstep so the two
copies don't drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Problems addressed (all observed on .198):
* fips_key was written as raw 32 bytes; upstream fips daemon reads it
with read_to_string() and bailed with "stream did not contain valid
UTF-8", crashlooping indefinitely.
* Activate button racy: user had to hit it, and it would keep failing
silently because the daemon couldn't parse its own config.
* FIPS schema drift (already fixed in 7d8a5864) put the config write
path behind the same broken "Activate" flow, so the fix alone
didn't help existing nodes.
* Journal was on tmpfs — every reboot wiped install/onboarding history,
making post-hoc debugging impossible.
Changes:
* identity.rs: write fips_key as bech32 nsec + newline. load_fips_keys
now auto-migrates legacy 32-byte files to bech32 the first time it
reads them, so OTA updates from v1.5.0-alpha self-heal without user
action.
* server.rs: post-onboarding auto-activate task runs on every
archipelago startup. If fips_key exists it ensures /etc/fips/fips.yaml
is schema-current and starts archipelago-fips.service. Pre-onboarding
nodes stay quiet (guarded on fips_key_exists).
* ISO build: un-mask archipelago-fips + archipelago-wg + wg-address —
all use ConditionPathExists on their key files, so systemd silently
skips them pre-onboarding (no MOTD [FAILED]). Only nostr-vpn stays
masked (legacy service, superseded by upstream fips).
* Journald made persistent via /var/log/journal + 500M cap, so
install and first-boot logs survive reboots for diagnosis.
After this, a fresh install + onboarding should bring FIPS up automatically
with no user interaction. The UI "Activate" button can stay as an escape
hatch (the RPC is still there) but is no longer on the critical path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Anchored regex was too strict — `strings` concatenates adjacent printable
bytes so the version never sits on its own line. The 1.5.0-alpha binary
DOES contain the version but as part of `1.5.0-alpharpcNot Found`. Fixed
by switching to `grep -qF $VERSION`: substring match is safe because the
version string is specific enough that accidental collisions are
vanishingly unlikely.
Caught mid-build today: check rejected the correct local binary, fell
through to container source-build — ISO still produced correctly but
wasted ~10 min on an unnecessary rebuild.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. FIPS daemon config schema drifted: upstream jmcorgan/fips now takes
`node.identity.persistent: true` (keys read from config-dir/fips.key)
and `transports.udp.bind_addr: "0.0.0.0:PORT"` instead of
`identity.key_file/pub_file` + `transports.udp.enabled/port`. The
`tor:` transport was dropped entirely; archipelago handles Tor
fallback itself. fips.yaml generated by archipelago::fips::config
now matches the upstream schema, and archipelago-fips.service stops
crashlooping on Activate. Observed on .198: 52 restarts with
"data did not match any variant of untagged enum TransportInstances
at line 7 column 3".
2. ISO backend-binary capture didn't verify that the captured binary
matched the checked-out Cargo.toml version. Today's 14:40 ISO
shipped a stale 1.4.0 binary because `core/target/release/archipelago`
pre-dated the 1.5.0-alpha bump — the build grabbed it via the
first-priority "local release build" path without looking at it.
All four capture sources now go through verify_backend_version()
which greps the binary for the expected version string; mismatches
are skipped so the build falls through to the source-build path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On this host (and potentially others with a particular podman/overlay
state), passing the multi-hundred-line stage-2 script via
`debian:trixie bash -c '...'` caused debootstrap to fail at
"Extracting apt... tar failed" on the very first package — no matter
what patch, storage cleanup, or env-reset we tried.
Running the exact same script body via a bind-mounted file
(`bash /installer-env.sh`) succeeds. So: write the body to a temp
file in WORK_DIR, bind-mount it read-only, and have the container
bash execute it from the file. Same behavior, different invocation,
works.
Was blocking every ISO rebuild since ~10:57 local. First successful
build since: 14:40, sha256 41fad2ff…, 2.3GB.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Debian Trixie apt 3.0.3's data.tar has duplicate entries for the same
path (regular file + symlink at e.g. libapt-private.so.0.0), and tar
bails on the second entry with "Cannot create symlink: File exists",
failing debootstrap on the very first package. Patch debootstrap's
tar invocation to use --skip-old-files so the duplicate is ignored.
Was blocking every unbundled ISO rebuild since the Trixie apt bump.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous build's STEP 47/50 log showed:
RUN for svc in nostr-vpn archipelago-wg archipelago-wg-address; do
rm -f /etc/systemd/system/.service
ln -sf /dev/null /etc/systemd/system/.service
done
The Dockerfile is generated via <<DOCKERFILE heredoc in the build
script, so unescaped $svc resolved in the outer bash BEFORE Docker
ever saw it, leaving nostr-vpn/wg masks as a hidden `.service` file
with no effect. nostr-vpn still tries to start on boot → [FAILED].
Fixed with \$svc so the literal lands in the Dockerfile for Docker's
shell to expand per iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fresh install of .198 reported "FIPS has an npub but says inactive".
The debian package writes /etc/fips/fips.pub during install (whence
the npub) but leaves the upstream fips.service disabled. Result:
FipsStatus.service_active = false, dashboard shows "inactive" until
the user hits Activate. Explicit `systemctl enable fips.service`
in the Dockerfile so first boot brings the daemon up immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. nostr-vpn still failing despite last mask attempt — confirmed in
the 6th ISO's rootfs.tar: the .service file was present but
not in multi-user.target.wants. Previous `systemctl mask` silently
no-oped because the real file was already there. Fixed properly
with explicit `rm -f` + `ln -sf /dev/null` for nostr-vpn,
archipelago-wg, and archipelago-wg-address — same /dev/null
symlink state that `mask` would produce on a clean install.
2. Kiosk didn't come up on first boot, only on reboot. Extended the
ExecStartPre health-poll from 30s → 120s (unbundled ISO takes
longer to settle on first boot: archipelago initializes state,
pulls FileBrowser, frontend settles), raised TimeoutStartSec to
180s, and added After=systemd-user-sessions.service +
After=network-online.target so X / Chromium aren't racing.
3. /init: line 29: can't create /root/etc/network/interfaces error
on installer boot — debootstrap --variant=minbase omits ifupdown
so the target has no /etc/network/ directory, and live-boot's
init tries to seed it. Non-fatal but noisy. Added ifupdown +
isc-dhcp-client to the debootstrap --include list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5th ISO attempt died in rustables's build.rs (which uses bindgen to
wrap libnftnl) with "couldn't find any valid shared libraries
matching: libclang". bindgen requires libclang.so at build time
to parse C headers. rustables also needs libnftnl-dev + libmnl-dev
for the actual wrappers.
Added to the fips-builder stage apt install line.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Third time's the charm. The upstream fips Cargo.toml puts fips-gateway
behind features.gateway = ["dep:rustables"], so the previous two
attempts (--bins, --workspace --bins) never produced the binary —
only the default feature set was compiled. cargo deb --no-build then
panics looking for the missing binary.
Inspected /tmp/fips-investigate (fresh clone of upstream main on
2026-04-19) to confirm — the feature flag is the gate, not a
workspace layout issue.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
