CORE_RPC_HOST was hardcoded to bitcoin-knots in three env-render paths, so on a
bitcoin-core node (container named bitcoin-core) mempool-api could not reach
Bitcoin RPC. Both node variants are reachable on archy-net by container name —
only the name differs.
- Legacy direct-podman (stacks.rs) and config.rs::get_app_config now use a new
dependencies::detect_bitcoin_rpc_host() (pure, unit-tested pick_bitcoin_host).
- Quadlet/manifest path (the modern fleet default): add a {{BITCOIN_HOST}}
derived-env placeholder — HostFacts.bitcoin_host + resolve_derived_env render
it; prod_orchestrator detects Knots/Core via podman ps, resolved on demand
only for manifests that use the placeholder. mempool-api manifest moves
CORE_RPC_HOST from static env to derived_env: {{BITCOIN_HOST}}.
Tests: pick_bitcoin_host (5 cases incl. substring safety), container-crate
resolve_derived_env, and orchestrator mempool_core_rpc_host_follows_bitcoin_node
(core->bitcoin-core, knots->bitcoin-knots). No-regression confirmed: picker
returns bitcoin-knots live on .198. Live bitcoin-core validation pending (no
core node available). Sibling hardcodes (lnd/btcpay/electrumx/fedimint) tracked
as B12b.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The whole fleet was silently never reaching the FIPS mesh: the default
public anchor was configured as fips.v0l.io:8668/udp, but the anchor only
answers on TCP/8443. Fix the default to 185.18.221.160:8443/tcp (IPv4
literal — the hostname resolves IPv6-first and the daemon binds v4-only,
which fails the handshake with EAFNOSUPPORT), and auto-seed it in
anchors::load() so every node dials it without operator action (removal
still persists). Proven live on .116: cold start → anchor_connected in
~400ms, anchor became mesh parent.
Wire fips::update::apply() against upstream GitHub releases (stable
channel only): resolve /releases/latest → SHA256-verify the .deb against
checksums-linux.txt → install → restart. dpkg runs via `systemd-run` to
escape archipelago's ProtectSystem=strict sandbox (else /var/lib/dpkg is
read-only), with --force-confold (archipelago manages /etc/fips conffiles)
and --force-downgrade (dev builds sort newer than the stable tag).
Validated live: .116 upgraded 0.3.0-dev -> stable v0.3.0.
Also: standalone fips-ui dashboard app (apps/fips-ui + docker/fips-ui,
static nginx proxying /rpc/v1 same-origin, copiable own-anchor address);
reserve UI port 8336; register fips/fips-ui as platform-managed. Includes
the Lightning wallet cross-origin (CORS) + LND proxy auth + nginx
self-healer fix so the wallet screen connects instead of "failed to fetch".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes three Bitcoin/wallet failures observed across the fleet on v1.7.90-alpha
(all nodes were already on the latest build — these were live bugs, not stale
builds), plus the missing ElectrumX tile, and adds automated coverage so each
can't regress silently.
Receive address (".116 receive fails", ".228 false 'wallet is locked'"):
- LND publishes its REST API on a host port that can drift from the manifest
(a container created when the mapping was 8080 kept publishing 8080 after the
manifest moved to 18080). The in-process client connects to the manifest port,
gets connection-refused, and wallet init fails forever while the container
looks "Up". Add published-port drift detection to the reconciler
(container_ports_drifted / host_port_bindings_drifted) that recreates a
drifted backend even for restart-sensitive apps — a drifted container is
already broken, so leaving it "untouched" only perpetuates the failure.
- Receive errors now carry a stable [CODE] token (REST_UNREACHABLE, WALLET_LOCKED,
WALLET_UNINITIALIZED, SYNCING) and always start with "Bitcoin address" so they
survive the RPC error sanitizer instead of collapsing to the generic
"Operation failed". The UI maps the code instead of guessing wallet state from
substrings — so an unreachable REST endpoint is no longer mislabelled "locked".
Bitcoin install (".198 bitcoin gone / reinstall just stops"):
- bitcoin-knots requires the secret bitcoin-rpc-txrelay-rpcauth, which was only
generated by the tx-relay flow. Nodes that never used tx-relay lacked it, so
secret resolution hard-failed and the whole Bitcoin stack cascaded. Generate
it idempotently before bitcoin starts (ensure_app_secrets, reusing
ensure_txrelay_credentials), and name the missing secret in the error so a
genuine gap is actionable instead of a bare "IO error".
ElectrumX app tile missing on every node with it installed:
- The catalog generator dropped electrumx because the manifest had no
interfaces.main block, so the tile had no launch URL and was hidden. Declare
the companion UI port (50002) in the manifest, regenerate the catalog, and let
an app with a known launch URL stay launchable while its backend is still
"starting" (ElectrumX indexes for 10m+).
Test harness:
- New lifecycle bats suites: bitcoin-receive, port-drift, secret-completeness
(validated live; port-drift catches the real .116 drift).
- Rust unit tests for drift detection, the receive reason-code classifier, and
the named-missing-secret error; vitest for the UI code mapping.
- create-release.sh now runs tests/release/run.sh and aborts the release on
failure — previously it ran no tests at all.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Snapshots the in-flight hardening work so subsequent reconcile/Quadlet
phases land on a clean before/after diff.
Changes:
- core/container/src/podman_client.rs: image_uses_insecure_registry()
whitelist for the OVH (146.59.87.168:3000) and legacy Hetzner
(23.182.128.160:3000) HTTP mirrors; podman_network_settings() lifts
custom networks into the Networks map so containers can join them.
- core/archipelago/src/container/prod_orchestrator.rs:
ensure_container_network() creates per-manifest networks on demand;
apply_data_uid() now goes through host_sudo for mkdir -p + chown so
bind-mount roots get created and chowned without password prompts.
- core/archipelago/src/api/rpc/package/{install,update,stacks}.rs:
podman pull adds --tls-verify=false only for whitelisted registries.
- core/archipelago/src/bootstrap.rs: removes stale dev-mode systemd
override on startup (live nodes carried it from old installers).
- core/archipelago/src/config.rs: ignore ARCHIPELAGO_DEV_MODE in prod
binaries — it had been silently rerouting volumes to /tmp.
- apps/bitcoin-{core,knots}/manifest.yml: locate bitcoind at runtime
so image-layout differences don't break entrypoint.
- scripts/app-catalog-image-smoke-test.py: production catalog/image
smoke test that probes a target node before users click Install.
- .gitignore: cover .codex, .pnpm-store, __pycache__, *.bak.
Removes filebrowser.rs.bak and two stale catalog.json.bak files
(verified identical to live counterparts).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sync-perf tuning for bitcoin/bitcoin-core/bitcoin-knots/electrumx.
- Drop the --cpus=2 cap on bitcoin/electrumx variants. Script verification
is parallelizable; the cap halved IBD speed on 4-8 core machines.
- Bump bitcoin --memory 4g→8g so dbcache=4096 has headroom for mempool +
connection buffers + I/O. 4g was OOM-prone during heavy IBD.
- Bump electrumx --memory 1g→2g + add CACHE_MB=2048 + MAX_SEND=10MB.
- bitcoin-core CLI args gain -dbcache=4096 -par=0 -maxconnections=125.
- bitcoin-knots manifest matched (1024MB pruned / 4096MB full + par=0).
Future v2: host-RAM-aware dbcache scaling.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resilience-validated release. Three full sweeps of the new resilience
harness against .228 confirm no shipstoppers.
Big user-visible:
- Bitcoin RPC auth durably correct via host-rendered nginx.conf bind-mount,
replaces fragile post-start exec that failed under restricted-cap rootless
podman ("crun: write cgroup.procs: Permission denied")
- Multi-container stack installs (indeedhub, immich, btcpay, mempool) now
emit phase events at every boundary so the progress bar advances
- Apps no longer vanish from the dashboard mid-install (absent-scanner skips
packages in transitional states)
- Indeedhub fresh installs work end-to-end (was 8500+ restart loop): five
missing env vars (DATABASE_PORT, QUEUE_HOST, QUEUE_PORT,
S3_PRIVATE_BUCKET_NAME, AES_MASTER_SECRET) added to install code
- Tailscale install fixed: --entrypoint string was being passed as a single
shell-line arg; switched to custom_args array
- Catalog cleaned of broken entries (dwn, endurain, ollama removed; nextcloud
restored on docker.io)
- Bitcoin Core update path uses correct image (was looking for nonexistent
lfg2025/bitcoin:28.4)
- ISO installs now allocate swap on the encrypted data partition
Infra:
- New resilience harness (scripts/resilience/) — black-box state-machine
tester, every app × every transition. Run before each release.
Sweep #3 final: PASS 107 / FAIL 12 / SKIP 14. The 12 fails are 1 cosmetic
(homeassistant trusted_hosts), 8 harness/timing false-positives, and 3
non-shipstopper tracked items. Down from 23 in baseline sweep #1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The LND UI container was unreachable on .228 after the v1.7.43-alpha
deploy because three sources of truth disagreed on which port nginx
listens on inside the container:
- docker/lnd-ui/nginx.conf listen 8081
- docker/lnd-ui/Dockerfile EXPOSE 8080
- apps/lnd-ui/manifest.yml host networking, ports: []
- scripts/first-boot-containers.sh -p 8081:8080
- scripts/deploy-to-target.sh -p 8081:80 (de-facto)
- scripts/deploy-tailscale.sh -p 8081:80
- scripts/container-specs.sh SPEC_PORTS=8081:80
Result: podman published host 8081 to container port 80, but no one was
listening on 80 inside, so connections were reset. Canonicalize on
container:80 with host:8081 publish, matching the three deploy paths
already in agreement.
Changes:
- docker/lnd-ui/nginx.conf: listen 8081 -> listen 80
- docker/lnd-ui/Dockerfile: EXPOSE 8080 -> EXPOSE 80
- apps/lnd-ui/manifest.yml: replace host-network (never true) with
bridge networking and explicit 8081:80 port mapping, correcting a
documentation-vs-reality mismatch
- scripts/first-boot-containers.sh: -p 8081:8080 -> -p 8081:80, and
fix the internal-port comment
Verified on .228 after rebuild: curl http://127.0.0.1:8081/ returns HTTP
200 and the /app/lnd/ host-nginx proxy resolves cleanly.
Replaces the first-boot-containers.sh sed/envsubst approach with a
Rust-native render step bound into the ContainerOrchestrator lifecycle.
- New container::bitcoin_ui module: embeds the nginx.conf template via
include_str!, reads the plaintext RPC password from
/var/lib/archipelago/secrets/bitcoin-rpc-password, substitutes
{{BITCOIN_RPC_AUTH}} with base64(archipelago:<password>), and atomic-
writes (tmp + rename) to /var/lib/archipelago/bitcoin-ui/nginx.conf.
Idempotent: byte-compares before writing so unchanged input is a
no-op (no inode churn, no restart cascade).
- ProdContainerOrchestrator gains run_pre_start_hooks(app_id) returning
HookOutcome::{Rewritten, Unchanged}. Fires in install_fresh before
create_container, and in ensure_running: on Running + Rewritten
triggers a restart; on Stopped re-renders then starts.
- bitcoin-ui Dockerfile no longer COPYs a default.conf; the file now
arrives via runtime bind-mount of the rendered config. If the bind-
mount is ever missing, nginx starts with no site configured and
returns 404 everywhere — safe failure vs. serving upstream RPC with
a stale Authorization header.
- apps/{bitcoin,electrs,lnd}-ui/manifest.yml land as first-class
manifests. bitcoin-ui declares the bind-mount target and a dependency
on bitcoin-core; electrs-ui and lnd-ui declare their own deps and
health checks.
- 8 new unit tests on the render fn (idempotency, rotation, trimming,
missing/empty secret, template invariants) plus an integration test
asserting install(bitcoin-ui) actually lands a substituted nginx.conf
on disk via the hook. 39/39 container:: tests pass
(test_parse_image_versions pre-existing failure unchanged, out of
scope).
- auth.rs now infers onboarding-complete from setup_complete + password_hash so
nodes stop bouncing users through the intro wizard after browser clear / update
/ reboot; the flag self-heals to disk on next check
- frontend: "backend uncertain" no longer defaults to /onboarding/intro —
useOnboarding returns null + callers poll / retry instead of flashing the wizard
- login sounds (synthwave, welcome voice, pop, whoosh, oomph) gated by
isFirstInstallPhase(); typing sounds unaffected
- removed FIPS app, Nostr Relay, Nostr VPN, Routstr, Penpot from catalog,
frontend config, Rust AppMetadata + install dispatch + install_penpot_stack;
docker/fips-ui + docker/nostr-vpn-ui + apps/penpot dirs and 5 icons deleted;
15 image versions deleted from tx1138, .168, gitea-local registries (.160
Gitea was 502 at release time — follow-up)
- AIUI baked into frontend release tarball via demo/aiui/; deploy-to-target
falls back to demo/aiui/ when the AIUI sibling checkout is missing
- prebuild hook syncs app-catalog/catalog.json → public/catalog.json so the
two copies can no longer drift (was the source of the "apps still visible"
bug — public/ had stale data)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- core/archipelago/src/bootstrap.rs (NEW): embed scripts/container-doctor.sh
and image-recipe/configs/archipelago-doctor.{service,timer} via
include_str! and sync to disk + enable the timer on every archipelago
startup. Idempotent (content-hash compare), dev-box symlink guard keeps
the git checkout untouched, best-effort (warn-only on failure) so
bootstrap never blocks server readiness. Wired in main.rs as a
background tokio task.
- scripts/container-doctor.sh: add fix_rootless_netns_egress(). Detects
when the rootless-netns has lost its pasta tap (container-to-container
still works but outbound DNS/TCP fails) via an nsenter probe into
aardvark-dns; with a two-probe 10s debounce to rule out transients and
a host-precheck that bails out if the host itself is offline. When the
rootless-netns is truly broken, does a graceful podman stop --all /
start --all so pasta + aardvark-dns rebuild the netns from scratch.
Bitcoin-knots and every other outbound container recover in one cycle.
- core/archipelago/src/update.rs: host_sudo → pub(crate) so bootstrap.rs
can reuse the existing systemd-run escape hatch.
- apps/bitcoin-core/manifest.yml: bump app version 24.0.0 → 28.4.0 and
image bitcoin/bitcoin:24.0 → bitcoin/bitcoin:28.4. Resources aligned
with the real container-specs.sh large-disk tune (4 GiB memory cap,
cpu_limit: 0 so bitcoind can run -par=auto across every core).
- neode-ui/src/views/apps/AppCard.vue + Apps.vue: add an Update button
+ Updating spinner to every app card that has available-update set.
Wires through serverStore.updatePackage(id) — the same RPC the detail
view already calls. common.update / common.updating i18n keys added in
en.json and es.json.
- core/archipelago/src/identity_manager.rs: add create_from_signing_key()
that mirrors an existing Ed25519 key as a manager-level identity with
a deterministic id (`node-<pubkey16>`). Idempotent across restarts,
gets the hex-SVG master avatar.
- core/archipelago/src/server.rs: the auto-create path on first boot now
mirrors the node's own signing_key (seed-derived on onboarded installs)
as a "Node" identity instead of generating a random "Default" keypair.
Once this ships, the DID on the Web5 DID Status card (via node.did
RPC), the Node entry on the Identities page (via identity.list), and
the DID used for peer-to-peer connects (via server_info.pubkey) all
resolve to the same seed-derived pubkey.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
App catalog served from Gitea repos (app-catalog) with 35 apps.
Nodes fetch catalog dynamically — new apps appear without frontend
rebuild. Test app added and removed to verify pipeline.
Gitea manifest updated with internal_port/nginx_proxy for iframe.
Updated catalog.json, nginx configs, app session configs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Convert botfights from external link to real container app on port 9100.
Add manifest, update marketplace/discover/kiosk/session configs, switch
registry URLs to git.tx1138.com.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend metadata and manifest now match the actual running config
and the frontend port mapping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch IndeedHub to staging API, add _next asset caching in nginx,
simplify NostrIdentityPicker component, and update Apps/Web5/Marketplace views.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added no_new_privileges: true, user: 1000, and seccomp_profile: default
to all app manifests. Created community app review checklist.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Updated Indeehub references throughout the codebase, changing the name from "IndeedHub" to "Indeehub" for consistency.
- Implemented a virtual app structure for Indeehub, allowing it to open an external URL without requiring a container.
- Enhanced deployment scripts and documentation to clarify SSH access and password management for Indeehub.
- Improved error handling and retry logic in various components to ensure better user experience during onboarding and app interactions.
- Updated CSS for visual enhancements and added new buttons for improved navigation in the AppLauncherOverlay.
- Upgraded Fedimint version to v0.10.0 in docker-compose.yml and manifest.yml, adding support for the built-in Guardian UI.
- Modified .gitignore to exclude deploy-config.sh script.
- Enhanced onboarding process in AuthManager to persist onboarding state and validate password strength during user setup.
- Updated API to handle onboarding completion and password change requests, ensuring a smoother user experience.
- Improved configuration management to support Nostr discovery and Tor proxy settings, enhancing node identity features.
- Updated BUILD-GUIDE.md to streamline instructions for building the Archipelago Auto-Installer ISO, including prerequisites and post-installation steps.
- Added detailed sections on capturing the live server state and building from source.
- Enhanced Docker and Podman integration in build scripts for improved backend and web UI capture.
- Introduced new app metadata for "IndeedHub" in the Docker package scanner and updated UI components for better installation progress tracking.
- Improved styling and functionality in the Bitcoin UI for a more cohesive user experience.
