Compare commits

...

113 Commits

Author SHA1 Message Date
archipelago
9b1c090e21 Merge feat/mesh-archy-command: lightning channel controls + open-race fix
Channel open fee control, channels tab in wallet settings, mempool tx
link, and the channel-open async peer-connect race fix. Conflict in
LightningChannels.vue resolved to the branch's extracted
LightningChannelsPanel component, with main's Teleport-to-body modal
fixes and glass-button-warning class (8256fde1) ported into the panel
so they aren't lost. lnd tests 16/16, LightningChannels vitest green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:42:54 -04:00
archipelago
755cd4f235 fix(orchestrator): reconcile recreates of bitcoin cascade-restart lnd
24fd97ed cascades on package.start/restart, but a reconcile pass that
recreates or starts a bitcoin backend (desired-state recovery, repair
recreate, boot InstallMissing) also moves the RPC address behind a
running lnd's back — §C 'restart lnd after ANY bitcoin recreate'.
After the pass, dependents from the shared address_caching_dependents
table (moved to app_ops as the single source of truth) that sat
untouched (NoOp) and aren't user-stopped are restarted under their op
lock; a dependent the pass itself (re)created already resolved the
fresh address and is left alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:27:36 -04:00
archipelago
854925c598 fix(orchestrator): user stop aborts host-port readiness/repair waits
wait_for_manifest_host_ports polls up to 420s while the caller holds
the per-app lock, so a user's stop request queued behind it and the
scanner painted the app Restarting for the whole wait (the dd3afbba
transitional-state stick; gate take-2 iter 5 vaultwarden). The wait
now selects against a user-stop-marker watcher (2s poll on
user-stopped.json, by app id or container name) and aborts with a
recognizable error, and the host-port repair path skips its restart
when a stop was requested while it settled — restarting there would
fight the queued stop worker.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:13:33 -04:00
archipelago
c07f38a903 fix(apps): repoint image refs from retired git.tx1138.com to vps2
Six manifests still pulled through git.tx1138.com/lfg2025, which is
dead — every boot reconcile of archy-btcpay-db/archy-nbxplorer failed
at the pull step (live-testing report 2026-07-10; the orchestrator
now also falls back to local storage, 3a5c5db1). All six tags
verified present on 146.59.87.168:3000/lfg2025 via the registry API.
Ships fleet-wide at the next catalog ceremony (catalog manifests
override disk ones).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 09:51:38 -04:00
archipelago
b7be296953 fix(quadlet): HealthCmd falls back to bash /dev/tcp before exit 0
The generated HealthCmd reported healthy (exit 0) whenever the image
shipped neither wget nor curl — btcpay-server sat 'healthy' in podman
ps while completely non-functional (live-testing report 2026-07-10).
Try a bash /dev/tcp connect to the health URL's host:port before
giving up; only images with none of wget/curl/bash still skip Podman
health (dropping exit 0 entirely would restart-loop those). Podman's
health timeout bounds the connect attempt.

Note: the rendered unit text changes, so every http-healthcheck app
gets a one-time unit rewrite + restart on first reconcile after this
binary lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 09:45:06 -04:00
archipelago
3a5c5db187 fix(orchestrator): install_fresh skips pull when the image is local
Reconcile recreates route through install_fresh, which pulled
unconditionally: with git.tx1138.com dead, every boot reconcile of
archy-btcpay-db/archy-nbxplorer hard-failed on 'pulling ...' even
though both image:tags sat in local storage the whole time
(live-testing report 2026-07-10). Check image_exists first and only
pull when the ref is truly absent — the same semantics as
ensure_resolved_source_available and the generated quadlets'
Pull=never.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 08:48:36 -04:00
archipelago
cfdf2da9a5 fix(health): monitor skips apps with a lifecycle op in flight
The health monitor restarted containers that were legitimately down
between a package.start/stop/restart worker's stop and start halves —
the same interleave class as the reconciler's mempool repair-recreate
(e275494a). Before restarting, probe the app_ops op-lock registry via
the container name, the derived app id, and a '_'->'-' normalization
(legacy stack containers are underscore-named while lock keys use
hyphenated app ids) and skip the cycle when a worker holds the lock.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 08:48:36 -04:00
archipelago
6fe623556c fix(apps): strfry healthcheck probes 127.0.0.1:7777, not localhost:8090
The healthcheck runs INSIDE the container, so it must target the
container port (7777), not the host mapping (8090) — and 127.0.0.1
explicitly, because `localhost` resolves to ::1 in the image while
strfry binds IPv4 0.0.0.0 only. Both bugs verified empirically on .228
(container was Up but "unhealthy"; 127.0.0.1:7777/health returns 200).

Ships at the next catalog regen/re-sign (catalog overlay supremacy);
with the runtime manifest-reload now in place it will apply fleet-wide
on the next catalog refresh, no service restarts needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 19:42:06 -04:00
archipelago
7e866c7c94 fix(catalog): compare/cache raw catalog bytes, not a re-serialization
Live verification on .228 showed catalog_changed:true on every
check-updates call: AppCatalog.apps is a HashMap, so the re-serialized
cache bytes flap with key order and the changed-detection never reports
"unchanged" — which would have reloaded the manifest overlay every
hourly poll. Cache the raw fetched body instead and compare against
that; as a bonus the cache now holds the exact signed preimage, so a
present release-root signature stays verifiable from disk.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 19:11:25 -04:00
archipelago
ceb319c292 feat(catalog): reload manifest overlay when a refreshed catalog changes
load_manifests() only ran at startup, so a manifest published via the
signed catalog sat dormant until the next service restart (found while
shipping the strfry manifest fixes: package.check-updates refreshed the
cache but the orchestrator kept rendering the old overlay).

- refresh_catalog now reports whether the cached bytes actually changed
  (write_cache skips identical rewrites), so unchanged hourly polls
  don't churn the manifest map.
- New ContainerOrchestrator::reload_manifests (default no-op); prod
  delegates to load_manifests — the rebuild is atomic under the state
  write lock, and the reconciler's durable user-stopped/uninstalled
  marker filters make a mid-session reload equivalent to the restart
  path that already runs on every boot.
- package.check-updates reloads on change and reports catalog_changed +
  manifests_reloaded; the hourly update-scheduler tick (and its startup
  refresh) do the same.

Tests: app_catalog 5/5 (new write_cache changed-detection test),
reconcile 16/16, knows_app 1/1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 18:54:00 -04:00
archipelago
2c46da387c fix(tests): lnd.newaddress settles across ALL transient post-restart codes
Run B failed the same probe with LND_ERROR (gRPC 'waiting to start' maps
there) — the 7b77462d window only matched WALLET_LOCKED, one of four
phases a cascade-restarted lnd passes through. Retry every code except
LND_WALLET_UNINITIALIZED (no wallet — never self-heals) within a bounded
180s window; persistent failures still fail after it. lnd verified
healthy minutes after both failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:30:49 -04:00
archipelago
7b77462ddf fix(tests): lnd.newaddress settle window for the post-cascade unlock race
Gate test 24 probed lnd.newaddress moments after the bitcoin bounce
cascade-restarted lnd; the auto-unlocker was still retrying against a
starting gRPC, so the probe saw LND_WALLET_LOCKED on a node that unlocks
itself seconds later (verified on .228 — same request succeeds post-run).
Treat WALLET_LOCKED as settling with a bounded 120s retry, mirroring the
e21f3baf settle window for tests 123/124; every other error still fails
fast, and a genuinely stuck-locked wallet still fails after the window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 17:05:42 -04:00
archipelago
11db822597 release(catalog): re-sign with working strfry manifest (7777 + declarative strfry.conf)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 16:27:20 -04:00
archipelago
55c20e0d6e feat(install): route any manifest-known app through the orchestrator
package.install routed to the manifest-driven orchestrator only for a
hardcoded per-app allowlist; every other app fell through to the legacy
flow, which ignores manifests entirely and creates a bare container with
no ports or volumes (strfry crash-looped this way on .228, 2026-07-09).

New ContainerOrchestrator::knows_app(app_id) (default false; prod checks
its loaded-manifest map, disk + signed-catalog overlay). The install gate
is now allowlist OR knows_app — no per-app Rust for manifest-driven apps,
matching the packaging invariant. Apps without a manifest keep the legacy
flow including the container-exists adopt block; the unknown-app_id →
legacy fallback inside the orchestrator branch is unchanged.

Tests: knows_app_reflects_loaded_manifests + install suite 37/37.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 14:46:50 -04:00
archipelago
707c912606 fix(apps): strfry container port 7777 + declarative strfry.conf via files:
The dockurr/strfry image listens on 7777 (not 8080) and its default config
demands a 1M NOFILES rlimit — above the rootless user-manager hard cap
(524288), so the relay exited at startup even with the mount fixed. Ship
the config declaratively (manifest files: + ro bind at /etc/strfry.conf)
with nofiles = 0; this also skips the entrypoint's copy into /etc, which a
readonly_root container cannot perform. Verified end-to-end on .228 in a
quadlet-equivalent systemd-run environment: active, GET / and /health 200.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 14:26:20 -04:00
archipelago
3835bfa1fc release(catalog): re-sign with strfry 8090+/app/strfry-db fix + drop 10.89.0.1 gateway binds
Catalog regenerated from fixed manifests (cfb8d953 strfry, loopback-only
bitcoin-core/knots binds) and re-signed by the release root. Semantic diff
vs the previous signed catalog is exactly those three manifests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 13:45:12 -04:00
archipelago
dce541dc49 fix(state): pending-boot Restarting overlay must not mask a user stop
Formal gate take-2 went 4/5: iteration 5's vaultwarden stop-wait saw
'restarting' for the full 120s window. A reconcile pass had queued every
app into pending_boot_starts moments before the stop marker landed, and
the scanner's overlay painted the deliberately-stopped app Restarting
until the slow sequential pass reached it (legacy container, so no unit
state to contradict it).

Two filters: the scanner overlay skips user-stopped ids (the marker is
written before the stop runs, so it's reliably present), and the
reconciler no longer queues apps whose lifecycle op is in flight into the
pending overlay (it skips them via the same probe anyway).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 09:45:06 -04:00
ai
6109074748 Merge pull request 'feat(mesh): !archy command — node status over the mesh, no AI' (#72) from feat/mesh-archy-command into main 2026-07-09 12:48:28 +00:00
archipelago
cfb8d953ae fix(apps): strfry host port 8082→8090 + correct LMDB mount point
Two defects kept strfry crash-looping since install (62,868 systemd
restarts on .228): its host port 8082 collides with vaultwarden's, and the
volume mounted at /strfry which the dockurr image never reads — its LMDB
lives at /app/strfry-db (entrypoint default), so mdb_env_open got ENOENT
even once the port was freed. 8090 is unclaimed across all manifests.
Uninstalled from .228 (db was empty — it never ran); reinstall after the
next catalog regen ships this manifest.

NOTE: repo-wide host-port collision audit also flags did-wallet/filebrowser
(8083), morphos-server/netbird-server (8086), lightning-stack/netbird
(8087), fedimint-gateway/lightning-stack (9737) — latent, tracked in the
hardening plan; a cross-app collision check belongs in the port allocator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 07:47:33 -04:00
archipelago
e275494a68 fix(lifecycle): reconciler skips apps with an in-flight lifecycle op
The reconciler doesn't take the RPC layer's per-app FIFO op lock (known
limit of 891cbba4): between a restart worker's stop and start halves it saw
the mempool frontend "missing", repair-recreated it behind systemd's back,
killed the worker's fresh container 11s after start, and left the unit down
for ~3.5 min until the next heal — gate test 123 measured exactly that
window (.228 iteration 3, 2026-07-09).

New crate::app_ops module owns the op-lock registry + stack member table
(runtime.rs and dependencies.rs now delegate) so the reconciler can probe
lifecycle_op_in_flight(app_id) — covering both the app's own key and its
owning stack package — and skip that app for the cycle. The ownership-sweep
podman restart gets the same guard. Health monitor is a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 06:49:52 -04:00
archipelago
e21f3baf22 fix(tests): settle window for quadlet active-state asserts (123/124)
mempool-api exits at startup while electrumx is catching up to the daemon
(7-day initial-sync ETA on .228), flapping through 'activating' for ~2 min
after each lifecycle cycle before systemd heals it. The instant asserts
probed inside that window both iterations. Poll up to 180s for
converges-to-active — a genuine crash-loop still fails.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 05:54:01 -04:00
archipelago
24fd97ed98 fix(lifecycle): cascade-restart lnd after bitcoin backend start/restart
lnd resolves the bitcoin RPC address once at startup and never re-resolves;
when bitcoin-core/knots restarts (new container IP) lnd spins on "dial tcp
<old-ip>:8332: no route to host" and its own RPC wedges until lnd restarts
(gate lnd getinfo test 75 failed every iteration on .228, 2026-07-09;
hardening plan §C cascade item). electrumx is already restarted by the
version-switch path — lnd was the only dependent left stranded.

After a successful package.start/restart of a bitcoin backend, restart
running, not-user-stopped address-caching dependents under their own app-op
lock (lock order backend→dependent only, so no deadlock).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 05:53:53 -04:00
archipelago
4fbe3d4ba0 fix(tests): exclude mid-integration wyoming-* units from quadlet backend asserts
wyoming-piper/wyoming-whisper are being integrated and are not yet part of
the platform contract; their hand-staged units (companion-style
Restart=always) failed gate test 121 on .228. Exclude wyoming-* from
backend_quadlet_units() until their packaging lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 05:16:42 -04:00
archipelago
6d60082997 fix(lifecycle): route stack stop/restart via member app ids, not container names
package.stop/restart on orchestrator-managed stacks (immich, indeedhub,
btcpay, netbird, mempool) addressed live CONTAINER names; unknown app ids
fell through to raw podman stop/start, racing systemd's --rm cleanup of the
quadlet unit — the container vanished on stop and the start half found
nothing (immich RESTART FAIL + indeedhub stuck restarting, gate 2026-07-09).

Now stack ops resolve member APP ids (reverse start order for stop) so the
orchestrator drives the quadlet .service; user-stop markers are written for
member app ids too (covers absent containers), and the orchestrator stop
fan-out treats an already-missing container as stopped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 04:18:55 -04:00
archipelago
75cb9f9aff docs(hardening): tick transitional-state stick + install_log + gate 123/124 fixes
All three landed 2026-07-09 (dd3afbba, c3f0a306, 2683ad4f0). Files the
repair-wait user-stop preemption as the open follow-up of the stick fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 03:11:45 -04:00
archipelago
c3f0a306b4 fix(observability): mirror install_log() to tracing — file append is sandbox-dead
/var/log/archipelago/container-installs.log has been 0 bytes since April:
the service sandbox (ProtectSystem=strict class) leaves /var/log
read-only for the unit, the open() fails, and install_log() drops every
line by design (fire-and-forget). These START/STOP/RESTART/FAIL
breadcrumbs are the primary forensic trail for gate failures, so mirror
every line to tracing (journald) unconditionally and keep the file
append as best-effort for hosts where the path is writable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 03:09:08 -04:00
archipelago
2683ad4f0e fix(tests): exempt user-stopped apps from quadlet unit active-state asserts
backend_quadlet_units() enumerates every *.container file on disk and the
active-state tests asserted each unit active + container running. A
user-stopped app keeps its unit file (e.g. the inactive half of the
bitcoin-core/bitcoin-knots multi-version pair after tonight's crash-loop
wave left core's unit on disk), so its inactive service / absent container
is the CORRECT state — gate run E false-failed on it (tests 123/124,
.228 2026-07-09). Honour the orchestrator's user-stopped.json intent
marker and skip those units in both tests.

Verified on .228: bitcoin-core's lingering unit now exempted; suite parses
(bats --count = 6) and the two fixed tests pass user-stopped filtering.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 03:02:22 -04:00
archipelago
dd3afbbac2 fix(lifecycle): let the scanner resolve visibly-completed stop/start transitions
One legacy app per gate run stuck at 'stopping'/'starting' past the test
window while its container was already settled (vaultwarden:stop run C,
jellyfin:stop run D, uptime-kuma:start run E — .228, 2026-07-09). The
package scanner already sees the truth every 60s but
merge_preserving_transitional refused to report it until the RPC worker
wrote the final state, and the workers can legitimately trail the
container by minutes:

- stop workers queue behind the orchestrator per-app lock, which the
  reconcile host-port repair path holds through multi-minute stability
  waits (repair_manifest_host_ports_after_stability: 5s + 5s probe +
  restart + 60-420s port wait + 15-90s stable-running);
- start workers hold Starting through the full readiness wait —
  host_port_wait_timeout_secs is 420s for uptime-kuma (HTTP probe),
  longer than any UI/test patience — and the Starting stuck-timeout is
  the 20-minute INSTALLING one.

New merge rules, both truth-driven by podman's live view:
- (Stopping, Stopped) + user-stop marker → Stopped: the scanner only
  normalizes exited→Stopped for user-stopped apps, so this is exactly
  "the user asked for a stop and the container has exited".
- (Starting, Running) → Running: the start visibly succeeded; live
  health readings are merged separately and keep reporting readiness.

Restarting is deliberately NOT resolved by a Running scan — mid-restart
readings are the pre-stop container.

Tests: merge_tests 13/13 (4 new).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 02:59:19 -04:00
archipelago
891cbba469 fix(lifecycle): serialize per-app start/stop/restart workers (FIFO app-op lock)
package.start/stop/restart reply immediately and run their multi-container
sequences in spawned tasks with no coordination. Back-to-back RPCs on the
same app interleave those sequences: gate run D (.228, 2026-07-09 04:28)
fired stop→start→restart on mempool; the start's member bring-up raced the
still-running stop's member shutdown — archy-mempool-db was stopped 2s
after it started and the stack finished with ZERO containers (tests 95/123/
124). A user double-clicking restart in the UI can do the same.

Workers now take a per-app tokio Mutex (fair/FIFO, keyed by the normalized
orchestrator app id) as their first await, so queued ops run in RPC arrival
order and the final state matches the last request. The RPC reply stays
immediate.

Known limit: container lists are still computed at RPC time (pre-lock), so
a stop landing mid-start can still return 'No containers found' — it can no
longer interleave/corrupt, only error. Reconciler/health-monitor restarts
don't take the lock yet (separate paths, same candidate follow-up).

Also from run D, filed separately: jellyfin/vaultwarden (legacy containers
on a quadlet node) stop fine but the package state sticks at 'stopping' —
'Failed to stop jellyfin.service: Unit not loaded' error path suspected.

Tests: runtime 9/9; archipelago builds clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:10:47 -04:00
archipelago
eb6ec71a56 fix(security): drop unbindable publish binds instead of crash-looping the app
The dd61a204 bind hardening published Bitcoin RPC on the archy-net gateway
10.89.0.1 — but rootlessport binds in the HOST netns, where that address
does not exist. First real deploy (.228, 2026-07-09) crash-looped
bitcoin-knots AND bitcoin-core the moment the gate's stop→start regenerated
the unit from the re-signed catalog: 'rootlessport listen tcp
10.89.0.1:8332: bind: cannot assign requested address', restart counter 132.
Hand-edits to the unit don't survive — the orchestrator regenerates it from
the signed catalog manifest within seconds.

- New archipelago_container::manifest::host_can_bind_publish_ip(): empty/
  wildcard/loopback accepted without probing, anything else ephemeral-bind
  probed. Applied at all three publish paths — quadlet from_manifest
  (PublishPort), podman API create (host_ip), and the legacy -p string
  table loop — each dropping the publish with a warn instead of taking the
  container down. This neutralizes the bad binds already in the SIGNED
  catalog, so nodes recover on binary deploy alone (no ceremony needed).
- Remove the gateway publishes from bitcoin-knots/-core manifests and the
  legacy config.rs tables: verified on .228 that every in-node consumer
  (lnd, btcpay/nbxplorer, fedimint, mempool-api) dials the container's
  archy-net alias directly (bitcoin-knots:8332) and lnd uses RPC polling
  (no ZMQ) — the gateway publish had zero consumers. Loopback-only RPC/ZMQ
  (the approved LAN lockdown) stands; P2P 8333 stays public.

Catalog still ships the gateway binds until the next signing ceremony
regenerates it from these manifests; the guard makes that non-urgent.

Tests: container crate 65/65 (2 new guard tests), quadlet 40/40.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:41:39 -04:00
archipelago
922b79bf95 fix(lifecycle): resurrect fully-stopped stacks member-by-member (quadlet)
Second instance of the stop→start destruction class from tonight's gate
runs (.228): package.stop of a multi-container stack marks every member
user-stopped and quadlet-removes their containers; package.start's fallback
then resurrected only the bare package id, leaving the other members absent
AND user-stopped — indeedhub lost all 7 containers this way (mempool needed
its umbrella alias for the same reason).

- New stack_member_app_ids map (real manifest app ids, dependency order) —
  used by the start fallback when a stack has no live containers, and by
  package.restart instead of hard-failing "No containers found".
- orchestrator_uninstall_app_ids grows the missing indeedhub arm (same
  reconciler-ghost rationale as the immich arm).
- do_orchestrator_package_start no longer aborts the whole sequence when one
  member's unknown-app-id fallback fails (the #73/#74 wish) — collects and
  continues.

NOTE: extends the known §G per-app-map debt (tracker updated) — the proper
home for stack membership is a manifest field; parked post-tag by design.

Tests: package:: 48/48 incl. new fallback + indeedhub-uninstall coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 21:22:40 -04:00
archipelago
e719c55fea release(catalog): re-sign with Bitcoin RPC loopback+gateway binds + accumulated manifest fixes
Regenerated from disk manifests and signed by the release root (verified
against the pinned anchor). Carries: bitcoin-knots/-core RPC/ZMQ bind
hardening (dd61a204), immich ownership-fix capabilities, lnd templated
BITCOIN_HOST, bitcoin:archival dependency tags for electrumx/mempool stack.
Nodes pick this up on their next catalog refresh; running bitcoin containers
converge on their next restart (restart-sensitive guard) — restart lnd after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 21:03:03 -04:00
archipelago
ffc1ebd0f3 docs(hardening): record the four user decisions of 2026-07-08
Bitcoin RPC loopback+gateway bind DONE dd61a204; version = 1.8.0-alpha;
bitcoin multi-version rides the next OTA; 3ccc RF validation dropped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 20:29:56 -04:00
archipelago
dd61a20413 feat(security): bind Bitcoin RPC/ZMQ publishes to loopback + archy-net gateway (§C)
USER DECISION 2026-07-08: accept breaking external wallets pointed at
nodeIP:8332. The 0.0.0.0 publish exposed auth-only RPC (and unauthenticated
ZMQ 28332/28333 on the legacy path) to the whole LAN.

- New `bind` field on manifest port mappings (validated as an IP; the same
  host port may repeat with distinct binds). Rendered as
  PublishPort=<ip>:<host>:<container> in quadlet units and host_ip in the
  podman API create — unbound ports render byte-identical to before, so no
  fleet-wide false-drift wave.
- bitcoin-knots/-core manifests publish 8332 on 127.0.0.1 + 10.89.0.1 only.
  In-node consumers (lnd, fedimint-gateway, btcpay/nbxplorer) are unaffected:
  they dial host.archipelago / host.containers.internal, which the
  orchestrator pins to the archy-net gateway 10.89.0.1. P2P 8333 stays public.
- Legacy config.rs port strings get the same treatment incl. ZMQ.

Tests: new quadlet bind-render + manifest bind-validation tests;
container:: suite 167/167, archipelago-container 63/63.

DEPLOY NOTE: PublishPort strings change for bitcoin containers → one
planned recreate per node; restart lnd afterwards (it caches the backend
IP — see the backend-recreate cascade tracker item). Catalog manifests for
bitcoin apps must be regenerated + re-signed for catalog-covered nodes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 20:29:02 -04:00
archipelago
81743874a1 fix(tests): bound the lnd getinfo probe + track the backend-recreate cascade gap
lnd.bats retry loop had no per-exec timeout — a wedged lnd RPC (chain-blind
after bitcoin-knots was recreated under it) hung one podman exec, and the
whole gate, indefinitely (.228, 30+ min at test 75). timeout 10 per attempt
keeps the intended ~4min bound. Tracker: new §C item for the root cause —
apps cache their backend container's IP; a backend recreate must cascade a
dependent restart/alert or lnd goes chain-blind silently.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 19:48:30 -04:00
archipelago
e706d446e7 docs(hardening): tick §B/§C/§H items landed 2026-07-08 + two new findings
Secrets→podman-secrets verified on .228; release-path guards e77ccff0; test
creds/timeouts 380f4f19; mempool umbrella lifecycle fix 161a6e4d. New open
items: manifests hardcoding secrets in plain environment, orphan umbrella
container cleanup follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 19:36:27 -04:00
archipelago
161a6e4dbd fix(orchestrator): make legacy mempool id start/stop/restart the split stack
Found by the .228 lifecycle gate (2026-07-08, first quadlet-mode run):
stop→start on the legacy `mempool` umbrella app id destroyed the mempool
deployment. Under quadlet, package.stop removes the containers; package.start
then failed with `unknown app_id: mempool` because load_manifests drops the
umbrella manifest whenever the three split-stack members are loaded — leaving
nothing to recreate from and the stack down.

Two fixes:
- prod_orchestrator start/stop/restart now resolve the historical
  `mempool`/`mempool-web` id to the split members (archy-mempool-db,
  mempool-api, archy-mempool-web) whenever the umbrella manifest was dropped —
  the same alias install already implements ("installing mempool assembles the
  split stack"). Restart falls back to start for members whose container/unit
  is gone. Legacy umbrella-only nodes are unaffected (alias inactive while the
  umbrella manifest is live).
- is_missing_container_error (3 sites) now recognizes podman 5.x's
  `no such object` inspect phrasing, so a genuinely absent container is
  classified as missing instead of surfacing as a hard inspect error.

Tests: 2 new alias tests + 2 classifier tests; prod_orchestrator suite 61/61
green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 19:34:46 -04:00
archipelago
380f4f1947 fix(tests): stop committing node passwords + bound multinode RPC calls (§H)
- multinode suites no longer carry real node passwords as defaults: *_PW
  vars are required, auto-loaded from git-ignored tests/multinode/.env
  (lib sources it; .env.example documents the shape). Suites fail fast
  with a clear message when unset.
- node_login/node_rpc get --connect-timeout 10 --max-time 120 (override:
  MULTINODE_RPC_TIMEOUT) so one hung server-side RPC can't stall the whole
  suite. Verified live: login+rpc against .116 OK; unroutable node fails
  in 10s instead of hanging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 19:08:55 -04:00
archipelago
e77ccff0e1 fix(release): close two publish-path verification gaps (§B hardening)
- create-release.sh: assert the freshly built frontend dist actually embeds
  the bumped version before packaging — the ui-dist-version guard in
  tests/release/run.sh is behind --with-build, which the release path never
  passes, so a silently no-opped npm build could ship a stale dist with a
  valid sha256. Verified against the real dist (passes on current version,
  trips on a missing one).
- publish-release-assets.sh: verify published assets by downloading and
  comparing sha256 against the manifest, not just Content-Length — a
  size-correct/content-wrong mirror asset now fails the publish gate.
  Verified live against the v1.7.99-alpha assets on the vps2 mirror.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 19:04:14 -04:00
archipelago
1316b0c0d4 docs(tracker): fix SESSION-1.8.0-OTA-PROGRESS links to archive/ location
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:50:52 -04:00
archipelago
1394994e0c docs(ops): fix runbook/hotfix/troubleshooting against reality + redact committed sudo password
- operations-runbook: redact the plaintext sudo password (rotate it — it
  stays in git history); replace the nonexistent test-cross-node.sh /
  test-reboot-survival.sh with the real entry points (lifecycle gate,
  multinode smoke, e2e/post-install scripts)
- hotfix-process: de-hardcode the v1.0.x framing, point releases at
  create-release.sh + primary Gitea, correct the rollback backup paths
  (/opt/archipelago/rollback + updater backup dir, not /usr/local/bin)
- troubleshooting: connectivity probe start9.com -> debian.org
- INSTALL.sh: drop removed endurain app, point at developer-guide.md
  (development-setup.md never existed)
- tests/lifecycle/TESTING.md: replace the stale 2026-06-21 mid-session
  resume block with a resolved note; release-gate item 8 reflects the
  2026-07-08 version decision (1.8.0-alpha)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 18:46:51 -04:00
archipelago
9ce8a0df05 fix(orchestrator): normalize YAML-folded newlines in command drift comparison
Second false-drift source found live on .228 right after the entrypoint
split fix: a YAML `>-` script with more-indented continuation lines
keeps literal newlines in custom_args, but shell_join flattens them to
spaces when writing the quadlet Exec= line, so Config.Cmd stores spaces
where the manifest has newlines. bitcoin-knots and fedimint-gateway
read as permanently command-drifted over line breaks alone (only the
restart-sensitive guard kept them from recreate-looping like electrumx).

Normalize both sides the way shell_join does before comparing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:27:50 -04:00
archipelago
6e2d128c51 docs(app-platform): sync the platform docs with the shipped code
- app-manifest-spec.md: full rewrite from the real schema in
  core/container/src/manifest.rs — it was 5 months stale and missing the
  entire modern feature set (build, network/network_aliases, derived_env,
  secret_env, generated_secrets/certs, data_uid, files, interfaces, hooks,
  extensions flatten) and documented wrong network_policy values.
- app-developer-guide.md: add generated_secrets/generated_certs/
  network_aliases/hooks to the field table.
- APP-PACKAGING-MIGRATION-PLAN.md: phase status stamped (1-3 done, 5 mostly,
  4+6 open); deleted meshtastic app removed from regression-proof lists.
- registry-manifest-design.md: status design → implemented (phases 1-3),
  stale manifest_dir:Option line fixed.
- marketplace-protocol.md: reframed proposal → as-built (marketplace.rs +
  RPCs + UI shipped; create-invoice noted; schema disambiguated).
- manifest-hooks-design.md: phase 3 (indeedhub) done, phase 4 resolved via
  orchestrator+generated_secrets instead of hooks.
- README/architecture: restore the NIP-07 signer-bridge claim — it is real
  (neode-ui/public/nostr-provider.js), the earlier audit only grepped Rust.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:24:22 -04:00
archipelago
9897af1236 docs: fix developer-guide route registration + CONTRIBUTING dead links
developer-guide: RPC dispatch moved from mod.rs handle_rpc_call to
dispatcher.rs dispatch() long ago — the registration instructions now point
at the real file; module tree updated (package/, federation/, mesh/ are
directories; mesh is tri-protocol); roadmap pointer loop/plan.md → docs/ROADMAP.md.
CONTRIBUTING: docs/development-setup.md never existed → developer-guide.md;
GitHub-specific fork URL generalized to the Gitea instance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:10:48 -04:00
archipelago
8b16999a6e docs(architecture): rewrite against code ground truth
Fixes: phantom parmanode crate removed and crate table now matches the real
workspace (5 members + 4 standalone), Debian 12 box → 13, LOC/file counts
were understated ~2x (49k→117k Rust, 47k→69k TS/Vue), 30→51 apps, dispatcher
is dispatcher.rs with ~380 methods (not mod.rs '100+'), NIP-07 claim replaced
with the NIPs actually implemented (33/44/04), Meshcore-only mesh → tri-protocol
with Reticulum daemon, dead MASTER_PLAN.md/BETA-PROGRESS.md links replaced.
Adds as-built sections for the app platform (manifest → Quadlet → signed
catalog overlay → marketplace) and drops the environment-specific node
IP inventory from a public doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:09:24 -04:00
archipelago
fb46c9fa39 docs(tracker): ceremony done 2026-07-02, version decided 1.8.0-alpha, note quadlet false-drift fix
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:08:41 -04:00
archipelago
77c5f5641a fix(orchestrator): stop quadlet entrypoint split from reading as permanent command drift
container_command_drifted compared the manifest's entrypoint and
custom_args against Config.Entrypoint and Config.Cmd separately, but
quadlet's Entrypoint= takes a single value, so a manifest entrypoint of
[sh, -lc] is written as Entrypoint=sh + Exec=-lc ... and podman reports
entry=[sh], cmd=[-lc, script]. Same argv, different split — every
quadlet-created app with a multi-element entrypoint read as drifted
forever. On .228 this recreated electrumx on every reconcile pass (114
times in 6h); bitcoin-knots and fedimint-gateway showed the same false
drift and were only spared by the restart-sensitive guard.

Compare the concatenated argv (entrypoint ++ args vs Entrypoint ++ Cmd)
instead — that is what actually runs. When the manifest declares no
custom_args, only the entrypoint prefix must match, since the trailing
Cmd may be the image's baked-in CMD.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:08:41 -04:00
archipelago
fcd5a065dc docs: rewrite README against code ground truth + add public ROADMAP
README fixes: version badge 1.3.1→1.8.0-alpha, real LOC (117k Rust/69k TS-Vue
vs the stale 49k/47k), 51 apps not 29, tri-protocol mesh
(Meshtastic/MeshCore/Reticulum) not Meshcore-only, Nostr NIP-33/44/04 (NIP-07
was never implemented), Ed25519 signed catalog + pinned release-root anchor,
OTA host de-hardcoded (tx1138 retired), dead docs/MASTER_PLAN.md link removed,
crate map corrected (openwrt added, phantom crates dropped), new Philosophy and
Roadmap sections reflecting the manifest-driven/signed-registry north star.

docs/ROADMAP.md is the public-facing shipped/in-progress/planned summary,
synthesized from the master plan, unified tracker, and 1.8.0 hardening plan —
including the completed signing ceremony and the 1.8.0-alpha version decision.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:07:57 -04:00
archipelago
9221d1bfaf docs: move 10 finished/historical docs into docs/archive/ with an index
Session logs, handovers, point-in-time snapshots, the v0.1.0 security audit,
stale generated HTML guides, and shipped design docs (three-mode UI, install
screens) move out of the living docs/ tree. docs/archive/README.md explains
what each was and why it's archived; all in-repo links updated.
(docs/UNIFIED-TASK-TRACKER.md links intentionally untouched — it has another
agent's uncommitted work in flight.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 17:04:35 -04:00
archipelago
9c02fbf164 fix(reticulum): stop leaking 48M _MEI dirs into /tmp on every daemon restart
The packaged reticulum-daemon is a PyInstaller one-file binary: the
bootloader extracts a ~48M _MEI* dir into TMPDIR and only deletes it on
clean exit. ReticulumLink's shutdown SIGKILLed the process group
immediately (kill_on_drop + instant SIGTERM+SIGKILL in Drop), so the
cleanup never ran and every reconnect/restart stranded another 48M in
/tmp — filling .116's 12G tmpfs in ~4h.

Two-pronged fix:
- terminate_group(): group-wide SIGTERM now, SIGKILL from a detached
  backstop thread 5s later, used on every shutdown path (Drop and all
  spawn-failure paths, which previously used bare start_kill and could
  orphan the forked Python child too).
- Per-interface private TMPDIR under the runtime dir, wiped on every
  (re)spawn — even a hard-killed or power-lost daemon can never
  accumulate stale extraction dirs, and they land on the data disk
  instead of the small tmpfs /tmp.

Verified: unit tests + the ignored mesh_service_connects_over_
reticulum_tcp_client integration test (spawns two real daemons over
loopback TCP through the changed spawn/terminate path), clean teardown
with no stray processes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 14:55:19 -04:00
ai
c6b9e5d0c6 Merge pull request 'Aurora Reticulum TCP interop + mesh/RNode reliability fixes' (#70) from worktree-reticulum-tcp-interop into main 2026-07-06 12:07:11 +00:00
ai
833527078c Merge branch 'main' into worktree-reticulum-tcp-interop 2026-07-06 11:54:53 +00:00
ai
e64e5615cb Merge pull request 'Ship archy-rnodeconf as an OS-level tool on every node' (#71) from feat/reticulum-daemon-packaging into worktree-reticulum-tcp-interop 2026-07-06 06:13:49 +00:00
archipelago
804874a78b docs(tracker): record the failed-unit self-healing gap observed live on .228
fedimint sat 'failed' for 7+ hours (exit 255 at 21:21, pre-gate) — the
reconciler repairs missing/drifted containers but never reset-fails +
starts a failed quadlet .service, and the health monitor can't see an
app whose container is gone. Concrete lever added to the Tier 2
container-flapping item.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 01:44:21 -04:00
archipelago
11a4f2910a fix(immich): declare the caps its root process needs over the subuid-owned data tree
capabilities:[] was latent — the long-lived legacy container predated
strict manifest enforcement, so nothing noticed that a recreate against
this manifest produces a root process without DAC_OVERRIDE that
EACCESes on upload/encoded-video and crash-loops (49 systemd restarts
on .228 when the 2026-07-05 secret-env migration finally recreated
it). Any reinstall or reboot-repair would have tripped the same wire.

Cap set mirrors immich-postgres minus SETUID/SETGID.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 16:33:30 -04:00
archipelago
4665e497d7 feat(security): move secret env out of podman inspect and Quadlet unit files
Secret env used to merge into manifest.app.environment, landing in
'podman inspect' Config.Env on the API backend and — worse — as
plaintext Environment= lines in Quadlet unit files on disk. Now:

- expand_and_partition_env (container crate, pure + tested) expands
  ${KEY} placeholders and splits env into plain entries and
  secret-bearing pairs. Plain entries that interpolate a secret
  (btcpay's Password=${BTCPAY_DB_PASS} connection strings) are
  tainted and travel as secrets too. Secret values themselves are
  never expanded (a generated value containing '${' passes verbatim).
- values register as podman secrets: stdin (never argv/tempfile),
  --replace, content-hash label to skip no-op rewrites; a per-app hash
  cache in the orchestrator makes steady-state reconciles free of
  podman secret calls. Registration goes through the runtime trait
  (default no-op keeps mocks/docker inert).
- containers reference secrets by name: secret_env map in the libpod
  create spec, Secret=<name>,type=env,target=<KEY> in Quadlet units.
  Verified empirically on fleet podman 5.4.2: value absent from
  inspect Config.Env, runtime injection works rootless.
- rotation detection: io.archipelago.secret-env-hash container label
  (API) / the changed unit bytes (Quadlet). Pre-upgrade containers
  lack the label, so every secret-bearing app recreates ONCE on the
  first reconcile after deploy — deliberate, it scrubs the plaintext
  secrets out of existing container configs. Data dirs untouched.
- docker dev fallback keeps plain -e injection (no secret store);
  podman secrets persist across uninstall, matching the
  preserve-credentials invariant (reinstall re-registers by hash).

In-container /proc/<pid>/environ is unchanged — env remains the
app-compat contract; the closed leaks are inspect output and unit
files on disk.

Tests: archipelago-container 61/61 (3 new: taint partition, verbatim
secrets, hash order-independence), archipelago container:: 160/160
(fedimint install test now asserts the secret arrives as a ref, not
env; quadlet render test asserts Secret=/Label= lines). NEEDS the
on-node gate re-run before the item counts as verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 13:55:15 -04:00
archipelago
eed830e1ee feat(security): enforce declared cosign image signatures at the pull sites
New container::image_verify gates PodmanClient::pull_image and the
dev-only DockerRuntime::pull_image. Signature claims classify three
ways: absent/empty (pull unverified, logged), the literal
'cosign://...' placeholder every fleet manifest carries today (same —
enforcement stays dormant until the signing ceremony ships real
values), or a declared signature, which must verify via
'cosign verify --key /etc/archipelago/cosign.pub
--insecure-ignore-tlog=true' (plus --allow-insecure-registry
--allow-http-registry for the HTTP mirror; flags checked against
cosign's own docs) before anything is fetched. Missing key, missing
cosign binary, timeout, or verification failure all hard-fail the
pull — a declared signature cannot be skipped on either runtime. Key
path overridable via ARCHIPELAGO_COSIGN_PUBKEY for tests/staging.

Deletes security::ImageVerifier: zero callers, blocking
std::process::Command on would-be async paths, and a fantasy
'cosign verify --signature' invocation (that flag belongs to
verify-blob).

Activation ships with the Workstream B ceremony, in order: pin
cosign.pub on nodes + install cosign, then publish real
image_signature values in the catalog.

Tests: archipelago-container 58/58 (5 new), archipelago container::
159/159, security check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 18:11:32 -04:00
archipelago
2c8c99fd28 fix(security): bind seq into mesh signatures (v2 preimage), guard DID slice, cfg-gate dev password
- mesh: verify_signature accepts a v2 preimage (t,v,ts,seq) alongside
  legacy v1 (t,v,ts); signed_with_seq() is the v2 sender path, not yet
  wired — senders stay v1 until the fleet verifies v2 (receivers
  hard-drop bad sigs, so flipping send-side first would break
  mixed-fleet alerts). Tests: v2 verify, v2 seq-tamper rejection,
  v1 sign-then-set-seq compat.
- mesh listener: malformed radio-supplied DID shorter than the
  'did🔑' prefix can no longer panic advert_name (slice -> .get()).
- auth: the pre-setup password123 dev login and the constant itself are
  now #[cfg(debug_assertions)] — no release binary carries the bypass,
  whatever its runtime config says.
- orchestrator: canned host-facts under #[cfg(test)] — awaiting real
  subprocesses under tokio's paused test clock deadlocks against
  auto-advanced timers (the old blocking detection only worked by never
  yielding).
- drop two now-unused std::process::Command imports left by 4c75bb3d.

Tests: mesh 110/110 (incl. 2 new), api 68/68, container 159/159,
archipelago-container check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:49:52 -04:00
archipelago
291f2d7186 docs(tracker): add explicit container-flapping/reconciler-churn workstream to Tier 2
Was implicit across the Phase-3 Quadlet flip and Workstream F; now one
consolidated pre-tag item with the lever list, an observability proposal
(per-app restart counter + flap log line), and an already-landed list so
nothing gets re-done.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:41:27 -04:00
archipelago
9020b8526c fix(security): stop trusting client-supplied forwarded headers in rate limiting
extract_client_ip took X-Real-IP/X-Forwarded-For from any request, so
a client talking to the backend directly (the FIPS peer listener, or
any non-proxy path) could rotate a fake IP per request and never trip
the login rate limiter. The accept loop now records the TCP peer
address in request extensions, and forwarded headers are honored only
when the connection itself is from loopback — where nginx overwrites
X-Real-IP with the real client address. Direct connections bucket
under their socket IP.

§C of the 1.8.0 hardening plan; 3 new unit tests cover the
loopback/direct/no-header matrix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 15:48:07 -04:00
archipelago
bd7edb4376 feat(update): deepen post-OTA verification beyond a frontend 200
verify_pending_update previously cleared the rollback marker on any
2xx/3xx from GET / — a release with a dead RPC API or broken podman
access passed and never rolled back. Verification now requires, in the
same attempt: the frontend via nginx, backend RPC liveness (an
unauthenticated POST /rpc/v1 — 401 proves the stack is up, 5xx/404/
refused fails it), and rootless podman reachability. A pre-loop check
also asserts the running binary's version matches what the marker says
was applied, catching a silent or half swap deterministically.

Per-app container assertions are deliberately excluded: the
pre-Quadlet service restart legitimately takes containers down and the
boot reconciler can need minutes for heavy apps — that would
false-rollback healthy updates. Revisit after the Phase-3 flip.

§B of the 1.8.0 hardening plan; update suite 38/38 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 13:50:00 -04:00
c692a8b52f fix(mesh): fix archy-rnodeconf exiting 1 on success (frozen exit() gotcha)
Confirmed on the just-packaged binary: `archy-rnodeconf --info` printed
everything correctly, then crashed with NameError: name 'exit' is not
defined and returned exit code 1. rnodeconf.py's own graceful_exit() calls
the bare exit() builtin, which is only ever defined by site.py for
interactive Python — a frozen PyInstaller app skips that init, so any
bundled script relying on it hits this the moment it tries to quit cleanly,
after the real work already succeeded. Classic, well-documented PyInstaller
gotcha; the standard fix is a runtime hook pre-defining exit/quit as
sys.exit before the bundled script's own code runs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 14:52:28 +00:00
3925843455 feat(mesh): ship archy-rnodeconf on every node, wire daemon tools into deploy
RNS's own rnodeconf utility (frequency/bandwidth/spreading-factor/coding-rate
read+write, firmware signature verification, board bootstrap) has been the
one tool that reliably diagnoses real RNode hardware — it's what finally
proved two live nodes were silently running at different spreading factors
(SF5 vs SF10, invisible from any of our own probe/logging code, and the
actual reason two correctly-flashed radios could detect each other's RF but
never decode a packet). Every node should have it, not just whichever one an
agent happened to build a throwaway venv on to debug a specific incident.

- reticulum-daemon/build.sh: also PyInstaller-package archy-rnodeconf
  alongside the existing archy-reticulum-daemon, same --collect-submodules/
  -d noarchive flags (same RNS.Interfaces __all__-glob requirement applies).
- scripts/deploy-to-target.sh: actually wire both packaged binaries into the
  live deploy path (neither was wired in before — a pre-existing gap noted
  in docs/RETICULUM-TRANSPORT-PROGRESS.md; this is why reticulum-daemon
  previously only worked via manual dev-venv setup on rsync-deployed nodes,
  not the ISO-imaged ones). Non-fatal on build/deploy failure — archipelago
  already falls back to its dev-venv path if the packaged binary is absent.
  Also installs the missing `python3.<minor>-venv` package when needed
  (same "ensurepip not available" gap hit manually twice this session).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 14:44:36 +00:00
archipelago
4b4a1f88fb feat(security): enforce trusted-registry image policy at the orchestrator pull sites
Catalog- and manifest-supplied image refs reached pull_image without
ever passing the RPC boundary's validator — a malicious catalog entry
or manifest could pull from an arbitrary registry. The allowlist now
lives in container::image_policy (the RPC check delegates to it) and
both orchestrator pull sites (install_fresh and
ensure_resolved_source_available) refuse refs that fail it.

The shared policy accepts trusted-registry refs and registry-less
Docker Hub shorthand (grafana/grafana etc., used by 8 shipped
manifests — a registry-less ref cannot name an attacker host), and
rejects explicit non-allowlisted hosts, shell metacharacters, and
malformed refs. §A of the 1.8.0 hardening plan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:52:31 -04:00
archipelago
2f20ba8148 fix(ui): sanitize WireGuard QR SVG, guard mesh poll interval, log catalog fetch failures
Server.vue rendered the backend-generated WireGuard peer QR with raw
v-html while the analogous TOTP QR was DOMPurify-sanitized — both now
use the same svg-profile sanitizer. Mesh.vue's 5s poll interval gets
the same start-guard as the arch poll (no leak on double-mount) and is
nulled on unmount. curatedApps.ts catalog fetches no longer fail
silently: each failed source logs a console.warn, including the final
all-sources-failed fallback to the hardcoded list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:02:00 -04:00
archipelago
4c75bb3d38 perf(async): remove blocking std::process::Command from async paths
Every production process spawn reachable from a tokio worker now uses
tokio::process: the install path's podman-port probe, the dependencies
disk check, factory-reset restart, config host-IP detection, the
orchestrator's host-facts helpers (resolve_dynamic_env and its call
sites made async to carry it through), and AutoRuntime's podman/docker
probes.

The FIPS transport probe is the special case: is_available() is a sync
trait method called from async route(), so instead of blocking ~50ms
on systemctl per stale-cache hit it now serves the cached value and
refreshes on a background thread (stale-while-revalidate) — bounded
staleness, zero stalled workers.

§C of the 1.8.0 hardening plan; container/transport/config/package
suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:00:50 -04:00
791618f96f fix(mesh): widen RNode KISS-detect read window from 800ms to 2.5s
Confirmed on real hardware: a genuine Heltec V4 RNode (firmware 1.86,
verified via rnodeconf --info against the same port) prints extra
boot/status chatter over the same serial line before answering KISS
commands. Replaying the exact probe bytes and timing budget our Rust
probe_rnode() uses showed DETECT_RESP arriving ~1.05s after the write —
past the old 800ms deadline, so a real, correctly-flashed, correctly-
responding RNode was being timed out and misclassified as "not an RNode".

2.5s leaves comfortable margin. Non-RNode devices (Meshcore/Meshtastic)
already budget ~5s each, so this doesn't meaningfully slow down detection
when the port turns out to be something else.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 08:01:13 +00:00
458b9fbbb7 fix(mesh): bind Reticulum peer identity directly, not via name-matching
Two MeshPeer rows were being created for one physical Reticulum node: a
radio twin (keyed by the RNS dest_hash, arch_pubkey_hex always None) and a
pseudo-federation twin (keyed by the archy ed25519 pubkey, created via the
generic identity-broadcast path meant for Meshcore/Meshtastic). The generic
path relies on bind_federation_twins matching both twins' advert_name, but
the Reticulum radio twin's display_name is deliberately never the identity
text — so the two rows could never merge, and the generic send path (keyed
off whichever twin the caller resolves) ended up looking up the archy
pubkey's prefix in ReticulumLink's `prefix_to_hash` map, which is only ever
populated with RNS dest_hash prefixes. Every send failed with "Unknown
Reticulum prefix ... peer hasn't announced yet", confirmed live between two
real nodes (archy-x250-exp / archy-x250-pa) that could see each other's
adverts but never exchange a message.

Unlike Meshcore/Meshtastic, Reticulum's ARCHY identity blob arrives in the
same announce event as the destination hash, so there's no ambiguity about
which peer it belongs to — bind it directly onto the RNS-hash-keyed radio
peer instead of relying on name-matching. Threaded through a new
`ParsedContact::arch_pubkey_hex` (None for Meshcore/Meshtastic, unchanged
behavior there) so `refresh_contacts` can set it on the correct peer row
without touching `bind_federation_twins`/`group_peer_twins`, which already
know how to collapse twins once they share an arch_pubkey_hex.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 06:18:08 +00:00
893508b700 fix(mesh): probe Reticulum/RNode before Meshcore/Meshtastic during auto-detect
Confirmed on real hardware (Heltec V4, RNode firmware): the board answers
the exact KISS detect probe correctly and instantly on a fresh port open,
but auto_detect_and_open (and open_preferred_path's unpinned branch) tried
Meshcore (~5s timeout) then Meshtastic (~5s timeout) first, leaving the
RNode firmware unresponsive by the time Reticulum's turn came ~10.6s later.

ReticulumLink::open() already gates its expensive daemon-spawn behind a
cheap ~1s probe_rnode check, so trying it first only costs ~1s extra when
the device turns out to be Meshcore/Meshtastic instead — the previous
comment's "most expensive, so goes last" reasoning only applies to a
successful match, not a failed one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 02:16:07 +00:00
9bbbb046a8 fix(mesh): deassert DTR/RTS before serial probes to avoid ESP32-S3 native-USB resets
Heltec V3/V4-class boards use the ESP32-S3's native USB-Serial-JTAG
peripheral (no discrete USB-UART bridge chip), which resets the chip on a
DTR/RTS transition — the same mechanism esptool uses to force bootloader
entry. The Meshcore/Meshtastic/RNode serial probes all open the port with
library defaults (DTR/RTS asserted), so each probe attempt was likely
rebooting a real, correctly RNode-flashed Heltec board mid-handshake,
surfacing as an endless "No supported mesh radio found" retry loop.

Deassert both lines immediately after open and let the board settle before
writing, in all three probe paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 01:28:23 +00:00
7f657ab099 feat(mesh): optional plain-TCP Reticulum interface (radio-less Aurora interop)
Adds an additive, loopback-only TCP server/client interface to
reticulum-daemon and the Rust mesh wiring, alongside the unchanged serial
RNode path. Aurora (~/aurora) already speaks standard RNS/LXMF over plain
TCP by default, the same way Sideband already proved interop over LoRa
(docs/RETICULUM-TRANSPORT-PROGRESS.md gates #2/#3) — this closes the gap so
that interop is provable without scarce LoRa hardware, and gives archy a
path to eventually be dialed by an Aurora client.

Verified: daemon TCP transport round-trip, bidirectional LXMF DM against a
scripted RNS/LXMF stand-in for Aurora's Dart stack (content + dest-hash
match both directions), cargo check/test -p archipelago green (108 mesh
tests, 0 regressions), and a real MeshService::start() end-to-end test
spawning the daemon in TCP client mode with no serial probe.

TCP server mode is hard-gated to loopback in both Python and Rust — WAN/LAN
exposure is a deliberate future decision, not part of this change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 18:08:35 +00:00
archipelago
01cbec27ed fix(robustness): surface swallowed persistence-write failures + federation tombstone durability
§C of the 1.8.0 hardening plan: persistence writes whose Results were
silently dropped now log a warn/error with context (mesh contact
blocklist, scheduler state, content catalog, container registry,
update state, bitcoin relay, package install markers, server shutdown
state). §I: federation tombstones are now flushed durably in
storage/sync so cleared peers can't resurrect after a crash.

Tracker updated with shas in docs/1.8.0-RELEASE-HARDENING-PLAN.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:02:54 -04:00
archipelago
206d5fe8cf fix(security): origin-check the NIP-07 bridge + share-to-mesh, gate all identity methods behind consent
The nostr bridge derived the caller from the launcher's own URL and
never checked event.origin, so any co-resident iframe could pull the
node's nostr pubkey or use nip04/nip44 decrypt as an oracle while an
app was open. The bridge now rejects senders whose real origin doesn't
match the open app's origin, and every identity-sensitive method
(getPublicKey, signEvent, encrypt/decrypt) requires user consent or a
remembered per-origin approval — previously only signEvent did.

share-to-mesh in App.vue likewise accepted messages from any sender
and force-navigated to /mesh with an attacker-staged CID; it now
requires same-origin, matching Chat.vue's existing handler.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 12:53:41 -04:00
archipelago
51647b21cd feat(trust): verify release-root signature on the OTA manifest
check_for_updates now fetches the manifest as raw JSON and runs
trust::verify_detached before parsing: a tampered or wrong-signer
signature rejects the mirror outright, and unsigned manifests are
offered for MANUAL apply only — the 3 AM auto-apply scheduler refuses
them, closing the unattended remote-root hole (§A of the 1.8.0
hardening plan). UpdateState gains manifest_signed so the UI can
surface authenticity.

Publisher side: create-release.sh signs the manifest during the
release (ceremony, mnemonic via TTY/env only), publish-release-assets
hard-refuses to ship an unsigned manifest (grep + new 'ceremony
verify' cryptographic gate), and scripts/sign-manifest.sh covers
re-signing outside a release run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 12:33:01 -04:00
archipelago
1977bdefb5 feat(trust): pin release-root anchor + ship signed app-catalog
Pin RELEASE_ROOT_PUBKEY_HEX from the 2026-07-02 release-root signing ceremony
(signer did🔑z6MkkidEnEpo6qHMCNSZoNKWtvQvxq3whnaME9wGgEFhq7ur) so nodes verify
the publisher identity of the app-catalog. Sign releases/app-catalog.json in place.

Fix two floats that made the catalog unsignable: archy-btcpay-db manifest version
-> string, fedimint-clientd cpu_limit 0.25 -> 1 (u32). Add scripts/sign-catalog.sh
helper, the 1.8.0 release-hardening plan/tracker, and the commit-and-push project
rule in CLAUDE.md.

Backward-compatible: old binaries still accept the signed catalog; the pinned-anchor
binary ships in the next build/OTA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 09:15:43 -04:00
archipelago
8b6485078a docs(handover): pushed-to-main state + pre-existing trust test failures caveat
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 08:36:12 -04:00
archipelago
f5d2479605 Merge branch 'iso-feedback-fixes-2026-07-02' into merge-iso-feedback
# Conflicts:
#	core/archipelago/src/api/rpc/middleware.rs
2026-07-02 08:03:25 -04:00
archipelago
c375ecc441 fix: fresh-ISO feedback bug-bash — onboarding, status truthfulness, recovery, kiosk, logs
Fixes from real fresh-install feedback (Framework node .81) + its log bundle:

Backend:
- websocket: subscribe before initial snapshot — broadcasts in the gap were
  silently lost, stranding clients on stale state until a hard refresh
  (the "everything needs ctrl-r" bug: My Apps stuck Loading, App Store
  stuck Checking, containers-scanned never arriving)
- crash recovery: check the crash marker BEFORE writing our own PID —
  recovery had never run on any node (always saw its own PID and skipped);
  PID-reuse guard via /proc cmdline
- boot status: pending-boot-starts registry (recovery, stack recovery,
  reconciler, adoption) — scanner overlays queued-but-down apps as
  Restarting instead of Stopped after a reboot; scanner-authored
  Restarting resolves immediately on a settled scan (no transitional wedge)
- install deps: bounded wait (36x5s) when a dependency is installed but
  still starting ("Waiting for Bitcoin to start…") instead of instant
  rejection; dependency-gate rejections remove the optimistic entry (no
  phantom Stopped tile) and surface as a notification
- seed backup: auth.setup persists the onboarding mnemonic as the
  encrypted seed backup (reveal previously failed on EVERY node — nothing
  ever wrote master_seed.enc); seed.restore stashes too; error sanitizer
  lets seed/2FA errors through instead of "Check server logs"
- lnd: bitcoind.rpchost resolved from the running Bitcoin variant
  (hardcoded bitcoin-knots broke Core nodes); manifest uses derived_env
- bitcoin status: clean human message for connection-reset/startup; raw
  URLs + os-error chains no longer reach the app card
- fedimint-clientd: chown /var/lib/archipelago/fmcd to 1000:1000 (root-
  created dir crash-looped the rootless container, EACCES) — first-boot
  script + pre-start self-heal
- log volume (>1GB/day on a day-old node): journald caps drop-in (ISO +
  bootstrap self-heal), bitcoind -printtoconsole=0 everywhere (90% of the
  journal was IBD UpdateTip spam), tracing default debug→info

Frontend:
- Login: Enter advances to confirm field then submits; submit always
  clickable with inline errors (was silently disabled on mismatch);
  Restart Onboarding needs a confirming second click (the mismatch →
  "onboarding restarted" trap)
- sync store: 30s state reconciliation + refetch on re-entrant connect;
  20s containers-scanned escape hatch so Checking can never show forever;
  fresh empty node reaches the real "no apps yet" state
- intro video: CRF20 re-encode (SSIM 0.988) + faststart — moov was at EOF
  so playback needed the full 15MB first (the intro lag)
- backgrounds: 10 heaviest JPEGs → WebP q90 (9.4MB→6.6MB); 7 stayed JPEG
  (WebP larger on noisy sources)
- Web5ConnectedNodes: drop unused template ref that failed vue-tsc -b

ISO/kiosk:
- nginx: /assets/ 404s no longer cached immutable for a year; HTTPS block
  gained the missing /assets/ location (served index.html as images)
- kiosk: launcher/service spliced from configs/ at ISO build (stale
  heredoc force-disabled GPU); MemoryHigh/Max 1200/1500→2200/2800M (kiosk
  rode the reclaim throttle = the lag); firmware-intel-graphics +
  firmware-amd-graphics (trixie split DMC blobs out of misc-nonfree)

Verified: cargo test 898/898 green, npm run build green with dist
contents confirmed (webp refs, lnd.png, faststart video, new strings).
Handover for ISO build + deploy: docs/HANDOVER-2026-07-02-iso-feedback.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 08:00:39 -04:00
archipelago
b9e4fbe9f7 docs: PR#67 + back-button fix merged/pushed but NOT deployed — resume note
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-02 03:33:52 -04:00
archipelago
7d7ba5734a fix(ui): wire up OpenWrtGateway's back button
BackButton is presentational-only (emits click, parent wires navigation)
per its own doc comment, but OpenWrtGateway.vue rendered it with no
@click handler at all -- clicking it did nothing. Added useRouter +
goBack() (-> the 'server' route, matching the page's location under
views/server/), same pattern as PeerFiles.vue/CloudFolder.vue.

Router-detection (openwrt.scan) spot-checked live: RPC plumbing works
end-to-end and returns a valid response, but no physical OpenWrt device
was on hand to confirm a true-positive detection. Also noted:
detect::scan_subnet does blocking TCP/SSH calls inside an async fn with
no .await points -- not proven to cause a real issue yet, but worth
hardening (spawn_blocking or async I/O) before a large subnet scan is
exercised for real.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-02 03:24:37 -04:00
archipelago
7a7fec21d4 Merge remote-tracking branch 'gitea-ai/fix/reticulum-daemon-process-group' 2026-07-02 02:56:40 -04:00
archipelago
61bfde3200 docs: consolidated deploy done — all 5 fleet nodes verified + unbundled ISO built
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 19:54:07 -04:00
archipelago
9f52e81471 fix(ui): remove vestigial ref, fix stale MeshMap test mock
Web5ConnectedNodes.vue declared nodesContainerRef but never consumed it
(the controller-nav system scans [data-controller-container] globally,
no other view uses a per-component ref for it) — broke the vue-tsc build.
MeshMap.test.ts's mocked mesh store predated federatedPositions (added
earlier this session for the Mesh Map federated-node feature) and crashed
on mount. Found live merging PR#67 (reticulum) + UI/UX work +
archy-openwrt into main for a combined fleet deploy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 18:20:04 -04:00
archipelago
27093e682f Merge remote-tracking branch 'gitea-ai/archy-openwrt' 2026-07-01 18:09:41 -04:00
archipelago
0da73a8ce1 Merge remote-tracking branch 'gitea-ai/fix/reticulum-daemon-pdeathsig' 2026-07-01 18:09:31 -04:00
archipelago
8256fde1a6 fix(ui): mesh/web5/apps layout, modal, and search UX fixes
- Mesh: fix 920-1280px bottom margin (phantom mobile-nav reservation
  leaking into the desktop-sidebar range), let the mesh view scale to
  full width on wide screens instead of capping at 1600px, and make the
  Device panel collapsible on desktop (previously mobile-only)
- Search/controller-nav: a global gamepad/keyboard-nav feature was
  auto-clicking "the next button in the DOM" on Enter in any text input,
  which cleared the mesh peer search and popped the sideload modal from
  the App Store/My Apps search boxes. Opt out via data-controller-no-submit
  on all filter inputs; bump the mesh clear button's touch target
- Modals: several (sideload, credential, Lightning channel open, identity
  create) used ad-hoc blue buttons and non-fullscreen backdrops that only
  covered the main content area, not the sidebar. Teleport them to body,
  unify backdrop/button theming to the dark+orange convention, fix the
  sideload modal's square bottom corners on desktop, and standardize
  close buttons to the ghost-icon style
- Web5: remove the redundant/dead "Messages" tab from Connected Nodes
  (its deep-link was unreachable dead code); fix the "view message" toast
  to actually open the Archipelago channel instead of silently failing to
  match a LoRa peer; make identity rows responsive via a container query
  (viewport-based breakpoints don't work in the page's 2-column grid) and
  right-justify their action icons; collapse DID/DHT/Wallet/Nostr/Connected
  Nodes by default on mobile
- Apps/App Store: match the search bar and sideload button's height,
  padding, and background to the mode-switcher tabs beside them
- Mesh chat: keep the compose input focused after sending

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 18:04:31 -04:00
archipelago
936b4cca29 fix(orchestrator): self-heal ANY installed app, not just baseline ones
The boot reconciler only self-healed a fully-absent container for one of
8 hardcoded "required baseline" apps (bitcoin-knots, electrumx, lnd,
mempool*, filebrowser, fedimint-clientd) — every other genuinely-installed
app whose container went missing (crash, lost record, wedged teardown)
was left as Left("absent") forever, with no path back short of an
explicit manual reinstall.

Surfaced live: indeedhub's backend containers (minio/postgres/relay) went
absent on .116 and never recovered despite indeedhub still being
installed. By the time this code path runs, the app is already confirmed
NOT user-stopped and NOT user-uninstalled (both checked earlier in the
same function, backed by durable markers correctly cleared on
reinstall/start) — so gating self-heal further behind a hardcoded app-id
list was an unnecessary restriction, not a safety measure. An app the
user installed and never removed should come back on its own, same as
baseline services always have.

Deleted the now-dead is_required_baseline_app(); updated the test that
had locked in the old (wrong) behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 17:27:16 -04:00
archipelago
2c1d2a2572 docs: multinode gate finished + boot-reconciler self-heal bug found+fixed
.5's 5x gate done: 5/5 iterations, all technically FAIL per run-gate.sh's
tally but only from .5's permanent pruned-bitcoin ceiling (accepted going
in); down to 2 failures/iteration by the end. Found + fixed a real hang
(lnd cached a dead bitcoin-knots IP after a restart) live mid-run.

Separately found a real boot-reconciler bug via indeedhub going stuck on
.116: any genuinely-installed-but-fully-absent app was left stuck forever
unless it was one of 8 hardcoded "baseline" apps. Fix tracked, code change
in the shared working tree pending test confirmation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 17:24:42 -04:00
archipelago
27e6747c2a feat(security): pin the release-root trust anchor (Workstream B)
Pins RELEASE_ROOT_PUBKEY_HEX from the signing ceremony
(did🔑z6MkkidEnEpo6qHMCNSZoNKWtvQvxq3whnaME9wGgEFhq7ur). The
corresponding mnemonic is held offline by the publisher, never committed
or stored on any node/build host. Nodes built with this binary now verify
the app catalog's signature against this anchor instead of accepting any
signer; unsigned catalogs are still accepted during the migration window
per docs/workstream-b-signing-runbook.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 16:59:48 -04:00
archipelago
469b0203b7 fix(reticulum-daemon): die with parent to stop RNode-jamming pile-ups
The daemon ships as a PyInstaller one-file binary; its direct parent is the
bootloader, which the Rust supervisor (mesh/reticulum.rs Drop) stops via
start_kill() == SIGKILL. SIGKILL can't be forwarded, so the Python child was
orphaned on every link recreation and kept holding the RNode serial port.
These stale daemons piled up (9 seen on one node), all clutching /dev/ttyUSB0
and garbling the RNode so it silently stopped transmitting (txb frozen,
interface status False).

Set PR_SET_PDEATHSIG(SIGTERM) at daemon startup so the kernel signals us when
the parent exits; our existing SIGTERM handler then shuts down cleanly and
frees the port. Linux-only, best-effort, no-op elsewhere.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 16:14:55 -04:00
2a6e624189 toggle for wifi and switch-router on openwrt page 2026-07-01 14:06:02 +00:00
e497f8fed1 feat(home): surface TollGate status on the Network tile
Add a TollGate row (Enabled/Disabled/Not installed) to the Home
dashboard's Network tile, polling the existing openwrt.get-status RPC
on the same cadence as the other network rows. Only rendered once an
OpenWrt router is actually configured, so nodes without one aren't
cluttered with an always-"Not configured" row.

Also fixes the underlying reason this could never have worked: nothing
in the OpenWrt Gateway flow ever persisted the router's host/credentials
server-side — the "connect" form only kept them in local component
state, so any no-args openwrt.get-status call (this new tile, and even
the Gateway page's own reload) always failed with "No router
configured" despite a fully working, provisioned router. Now
handle_openwrt_get_status saves the connection to router_config.json
whenever a host is explicitly passed in and the connection succeeds.
2026-07-01 13:25:43 +00:00
d6c1feca97 fix(openwrt): fix TollGate provisioning pipeline, add reconfigure UI
Several compounding bugs were blocking end-to-end TollGate provisioning
on OpenWrt 25.x (apk-native) routers:

- install_ipk's non-ar fallback assumed a flat tarball, but some .ipks are
  a gzip tar of the three classic ipk members one level deep; it was
  dumping debian-binary/data.tar.gz/control.tar.gz straight into / instead
  of unpacking the real payload.
- Manually-extracted packages never ran their pending /etc/uci-defaults/*
  scripts (that only happens through opkg/apk's own postinst bookkeeping),
  so nothing ever created /etc/config/tollgate.
- uci_apply() never ensured the target config file existed first — `uci
  set` fails outright on a config namespace nothing has created yet, which
  is true for a package-defined one like "tollgate" (unlike wireless/
  network/dhcp, which ship by default).
- The installed-check and restart_services looked for a binary/init script
  named after the opkg package ("tollgate-module-basic-go"/"tollgate"),
  but the real on-disk names are tollgate-wrt — so status always reported
  "not installed" and service restarts silently no-op'd.
- provision_ssid used `uci add`, creating a new wifi-iface section (and
  therefore a new duplicate broadcast SSID) on every provision call instead
  of updating one in place.

Also adds a TollGateConfig.enabled field so the enable/disable state is
actually applied to the running service and the SSID's own broadcast
(stop + disable at boot, or start + enable), not just written to UCI.

On the frontend, the OpenWrt Gateway page's TollGate panel was read-only
once installed — add an edit form (price, step size, min steps, mint URL,
enabled toggle) that reuses the same idempotent provision-tollgate call.
2026-07-01 11:59:43 +00:00
1866c40edf fix(openwrt): detect radios and scan networks on vendor MediaTek drivers
Routers running MediaTek's proprietary mt_wifi SDK driver (e.g. GL.iNet)
never register with cfg80211/mac80211, so they have no `iw dev` entry and
no /sys/class/ieee80211 phy even though the radio is real and working —
find_wireless_iface was bailing with "No wireless radio found" on these.
Fall back to iwinfo's device listing, which abstracts over vendor backends
too, and to the vendor's iwpriv site-survey ioctl for scanning when iwinfo
itself can't trigger a scan on the interface.
2026-07-01 11:59:28 +00:00
6299e91544 fix(kiosk): stop HDMI mode detection from perpetuating a bad clone state
configure_display picked whichever mode was already "active" on the HDMI
output, so if X ever booted cloned to the laptop panel's resolution it
would keep re-confirming that wrong mode forever instead of self-healing
to the display's native mode.
2026-07-01 11:59:21 +00:00
f3cbeb2834 first commit of openwrt-tollgate integration 2026-06-30 20:30:26 +00:00
edbad30501 fix(openwrt): TollGate apk-native install for OpenWrt 25.x
- WISP wizard: step-by-step flow for WiFi, DHCP, masquerade config
- WAN status: expose lan_ip, dhcp_start/limit, masq, sta_state, wifi_log
- wifi_scan: detect CCMP as WPA2 (psk2) so association succeeds
- opkg: PkgManager enum — detect apk-native mode when opkg not in repos
- tollgate: apk-native install path using manual ipk extraction
- arch detection: read DISTRIB_ARCH from /etc/openwrt_release; normalise
  bare mipsel/mips from uname -m to mipsel_24kc/mips_24kc
- install_ipk: install binutils via apk when ar not in BusyBox
- install_ipk: wget --no-check-certificate for routers without CA bundle
- install_ipk: ar fallback to tar -xzf for non-standard ipk formats
- install_ipk: 5MB overlay space check with clear user-facing error
- middleware: allow "Not enough flash/space" errors through sanitizer

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
a862877189 feat(openwrt): add WAN diagnostics to get-status and UI
get_wan_status now returns: radio0_disabled, sta_iface (from iw dev),
sta_state (operstate), assoc_ssid (actually associated SSID vs
configured), and recent wifi_log lines from logread. The WAN panel
shows a diagnostic grid when configured but not connected so the user
can see exactly what's wrong without digging into server logs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
33b96f4acf fix(openwrt): enable radio0 when configuring WISP
configure_wisp was setting up wireless.wwan but leaving
radio0.disabled=1, so wifi reload did nothing and the sta
interface never appeared. Explicitly set radio0.disabled=0
before committing the wireless UCI config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
5ab569f150 fix(openwrt): use iw phy interface add for scan when no UCI wifi-iface exists
wifi up does nothing without a wifi-iface section in UCI (common on fresh
flash). Instead, create a temporary managed interface directly on phy0
via nl80211 (iw phy phy0 interface add scan0 type managed), scan on it,
then delete it. No netifd/UCI involvement needed for scanning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
9dc2343b60 fix(openwrt): enable radio0 and run wifi up before scanning
On a freshly-flashed OpenWrt router, radio0 is disabled by default so
iw dev returns empty. Detect the PHY via /sys/class/ieee80211/, enable
radio0, run `wifi up`, then poll up to 8s for netifd to create the
virtual interface before handing it to iwinfo scan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
ddc839400a fix(openwrt): use iw dev for wireless interface detection
wlan0 doesn't exist on OpenWrt 25.x with mt76 drivers (Cudy TR1200);
interfaces are named phy0-ap0 etc. `iw dev` handles all mac80211
naming styles. The old while-read loop also exited with code 1 when
no match was found, causing run_ok to fail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
9a782fb551 feat(openwrt): WAN/WISP setup from the UI with WiFi network scan
New RPC methods:
- openwrt.scan-wifi: triggers iwinfo scan on the router radio,
  returns networks sorted by signal strength
- openwrt.configure-wan: creates UCI wireless.wwan (sta mode) +
  network.wwan (DHCP) + adds wwan to firewall WAN zone, then
  calls `wifi reload`

get-status now includes a `wan` object with configured/ssid/ip/
internet fields so the UI can show current uplink state.

Frontend WAN panel: scan → pick SSID (signal bars) → enter password
→ apply. Shows "Configure WAN first" hint above TollGate install
button when internet is not available.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
dd3a3dfbac fix(openwrt): capture apk stderr and run apk update before apk add opkg
apk errors were being silently dropped (stdout only). Run apk update
first and fail with a clear "router may have no internet" message if
it fails, rather than a cryptic exit-1 from apk add.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
5d82e6ff8d fix(openwrt): bootstrap opkg via apk on OpenWrt 25.x routers
OpenWrt 25.x switched from opkg to apk as the default package manager,
so devices like the Cudy TR1200 on 25.12.4 don't have /usr/bin/opkg.
When opkg is missing but apk is present, install opkg through apk first
so the rest of the provisioning flow can proceed unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
58266dea66 fix(openwrt): allow opkg-not-found error through RPC sanitizer
"opkg not found at /usr/bin/opkg" was being swallowed by the error
sanitizer and shown as generic "Operation failed". Also fix bare
`opkg list-installed` call in get-status handler to use full path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
bc1ec9aa3e fix(openwrt): use full opkg path and pre-check availability
`channel.exec()` doesn't source the shell profile, so PATH may not
include /usr/bin on some routers. Using /usr/bin/opkg explicitly
avoids exit-127 surprises. Added opkg_check() to give a clear error
("firmware may not support package management") before attempting
opkg_update, rather than a confusing "command not found" exit code.
Also split the BusyBox-hostile `grep -v 'all\|noarch'` into two
separate greps for the arch-detection fallback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
4c56e1bb96 fix(openwrt): detect opkg silent failure and show disk space error
BusyBox opkg exits 0 even when 'Cannot install' due to insufficient space,
causing the fallback to silently report success. Now captures stderr and
checks for the failure string explicitly.

Adds user-visible error for the common case where the router flash is too
small for the TollGate package (~19 MB needed vs ~9 MB available on typical
budget routers). Adds error prefixes to the RPC sanitizer allowlist so the
message reaches the UI instead of showing 'Check server logs'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
f69fac627a style(openwrt): adopt glass-card design system for contrast
Replace bg-white/5 card containers with glass-card (rgba(0,0,0,0.65) +
backdrop-blur), match input styling to Login.vue, and use glass-button
variants for actions. Fixes low contrast against the background image.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
f054766a58 feat(openwrt): add TollGate provision button and direct-download fallback
- OpenWrtGateway.vue: add "Install TollGate" button when not installed;
  tracks connected credentials for reuse in the provision call
- install.rs: fall back to wget download from GitHub releases when the
  package is not in any opkg feed (mips_24kc and other arches supported)
- openwrt.rs: provision-tollgate now falls back to saved router_config
  for credentials, matching the behaviour of get-status

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
6c534715ec fix(openwrt): allow No router/OpenWrt errors through RPC sanitizer
Without these prefixes in the allowlist, sanitize_error_message swallowed
the "No router configured" error and returned a generic "Operation failed",
so the frontend could never detect the unconfigured state and show the
connect form.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
d71f36370d feat(openwrt): add OpenWrt gateway status view and get-status RPC
Backend: new `openwrt.get-status` RPC endpoint SSHes into the saved (or
provided) OpenWrt router and returns system info, TollGate config, and WiFi
AP interfaces via UCI.

Frontend: new OpenWrtGateway.vue view at /dashboard/server/openwrt shows
system hostname, OpenWrt version, uptime, TollGate install/enable state with
pricing and mint URL, and all AP-mode WiFi interfaces. Linked from the Local
Network section of the Server view.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
e0cc00be0f feat(openwrt): add archipelago-openwrt crate with TollGate provisioning
New `archipelago-openwrt` workspace crate provides SSH/UCI-based management
of OpenWrt routers, including automated TollGate installation and configuration
of a pay-as-you-go "archipelago" SSID backed by the local Cashu mint.

Exposes two RPC endpoints:
- `openwrt.scan` — discover OpenWrt routers on the LAN
- `openwrt.provision-tollgate` — install tollgate-module-basic-go, write UCI
  config (TIP-01/TIP-02), and create isolated WiFi SSID + firewall zone

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 17:12:57 +00:00
206 changed files with 12483 additions and 3786 deletions

View File

@ -23,9 +23,31 @@ Detailed sub-plans (all linked from the master):
- App platform / packaging phases + security model → `docs/APP-PACKAGING-MIGRATION-PLAN.md`
- Registry-distributed manifests (in progress) → `docs/registry-manifest-design.md`
- External/decentralized marketplace for devs → `docs/marketplace-protocol.md`
- Current per-app state → `docs/app-registry-status-2026-06-21.md`
- Current per-app state → `docs/archive/app-registry-status-2026-06-21.md`
- Production test gate (exit criterion) → `tests/lifecycle/TESTING.md`
## Commit & push every unit of work (never violate)
**The #1 process rule: work is not "done" until it is committed AND pushed.** This
exists because finished work has been lost/clobbered by sitting uncommitted in the
shared tree across agents and sessions. To prevent that:
- **Commit each feature/fix the moment it works** — one focused, self-contained
commit per logical change (it compiles and its targeted tests pass). Do not let
unrelated changes accumulate uncommitted.
- **Push immediately after committing** so nothing lives only on one machine. `main`
is protected → push via `git push gitea-ai main` (account `ai`, see the memory
note); feature branches push to their own remote.
- **Never leave a stack of finished work uncommitted** overnight or when handing off
between agents — if you must pause mid-change, commit a clearly-labelled WIP
checkpoint rather than leaving it dirty.
- **Stage explicitly by path** (`git add <paths>`) when another agent's uncommitted
work shares the tree — never `git add -A` / `git commit -a`, which clobbers or
entangles their changes.
- **Never commit or push secrets** (mnemonics, private keys, API tokens). Signing is
done offline; artifacts (catalog/manifest) are signed, not the keys.
- Commit messages end with the `Co-Authored-By: Claude …` trailer.
## Invariants (never violate)
- **Rootless Podman only.** No rootful, no Docker-socket mounts, no privileged

View File

@ -8,9 +8,9 @@ Be respectful. We follow the [Contributor Covenant](https://www.contributor-cove
## Getting Started
1. Fork the repository
2. Clone your fork: `git clone https://github.com/YOUR_USERNAME/archy.git`
3. Set up the dev environment (see `docs/development-setup.md`)
1. Fork the repository on the project's Gitea instance
2. Clone your fork: `git clone <your-fork-url>/archy.git`
3. Set up the dev environment (see `docs/developer-guide.md`)
4. Create a feature branch: `git checkout -b feature/your-feature`
## Development Setup

View File

@ -122,7 +122,7 @@ echo ""
# Install custom app dependencies
echo "Installing custom app dependencies..."
for app in did-wallet endurain morphos-server router; do
for app in did-wallet morphos-server router; do
if [ -d "apps/$app" ]; then
echo " - Installing $app dependencies..."
cd "apps/$app"
@ -161,6 +161,6 @@ echo " http://localhost:8100"
echo ""
echo "For more information, see:"
echo " - README.md"
echo " - docs/development-setup.md"
echo " - docs/developer-guide.md"
echo " - apps/QUICKSTART.md"
echo ""

151
README.md
View File

@ -2,71 +2,99 @@
> Self-Sovereign Bitcoin Node OS
**Archipelago** is a bootable personal server OS. Flash it to a USB drive, install on any x86_64 or ARM64 machine, and manage Bitcoin infrastructure, self-hosted apps, and decentralized identity through a glassmorphism web UI.
**Archipelago** is a bootable personal server OS. Flash it to a USB drive, install on any x86_64 or ARM64 machine, and manage Bitcoin infrastructure, self-hosted apps, mesh communication, and decentralized identity through a glassmorphism web UI.
[![Debian 13](https://img.shields.io/badge/Debian-13%20Trixie-a80030)](https://www.debian.org/)
[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-stable-orange)](https://www.rust-lang.org/)
[![Vue.js](https://img.shields.io/badge/vue.js-3.5-brightgreen)](https://vuejs.org/)
[![Version](https://img.shields.io/badge/version-1.3.1--beta-blue)]()
[![Version](https://img.shields.io/badge/version-1.8.0--alpha-blue)]()
## Philosophy
Archipelago is being built as a **developer-ready app platform**, not a fixed appliance:
- **Manifest-driven apps.** Every app is declared in a single `manifest.yml` — image, ports, volumes, secrets, health checks, security policy. The orchestrator owns the entire lifecycle; there is no per-app installer code and no host-level provisioning.
- **Signed distribution.** App manifests ship inside an Ed25519-signed catalog verified against a pinned release-root key, not as loose files on disk. OTA release manifests are signed the same way.
- **Decentralized marketplace.** Third-party developers publish apps via Nostr-based discovery (NIP-78) with DID-signed manifests and federation-weighted trust scoring — no gatekept central store.
- **Rootless and secure by default.** Rootless Podman only. Read-only root, no-new-privileges, capability allow-list, secrets materialised 0600 and never logged. Never rootful, never a Docker socket mount.
- **100%-uptime-capable.** Every container is a systemd Quadlet unit under `user.slice` that survives backend restarts; a level-triggered reconciler self-heals drift every 30 seconds; migrations never destroy data.
## Features
### Bitcoin Infrastructure
- **Bitcoin Knots** full node with pruning support
- **LND** Lightning Network daemon with channel management
- **Bitcoin Core and Bitcoin Knots** full nodes with per-app version pinning and bulletproof version switching, automatic prune/full mode based on disk size
- **LND** and **Core Lightning** with channel management
- **ElectrumX** Electrum server for wallet connectivity
- **BTCPay Server** for accepting Bitcoin payments
- **Mempool** block explorer and fee estimator
- **Fedimint** federation guardian and gateway
- **Fedimint** federation guardian, gateway, and client — plus Cashu ecash wallet support
### Self-Hosted Apps (29)
Bitcoin, Storage (FileBrowser, Immich, Nextcloud), Productivity (Penpot, Vaultwarden), Media (Jellyfin, PhotoPrism), Search (SearXNG), AI (Ollama), Network (Tailscale, Nginx Proxy Manager), Home (Home Assistant), Nostr (nostr-rs-relay, Nostrudel), Dev (Grafana, Portainer), and more.
### Self-Hosted Apps (50+)
Storage (FileBrowser, Immich, Nextcloud), Productivity (Vaultwarden), Media (Jellyfin, PhotoPrism, IndeeHub), Search (SearXNG), Network (NetBird, Tailscale), Home (Home Assistant), Nostr (nostr-rs-relay, strfry), Dev/Ops (Gitea, Grafana, Portainer, Uptime Kuma), and more — 27 curated in the store UI, 50+ packaged as manifests.
### Mesh Networking (tri-protocol)
- **Meshtastic**, **MeshCore**, and **Reticulum (RNS/LXMF)** LoRa transports behind one mesh chat UI
- End-to-end encryption with X3DH key agreement + double-ratchet
- RNode radio support with an OS-level `archy-rnodeconf` tool; interop verified against Sideband
- Image/voice attachments, mesh AI assistant (`!ai`), Bitcoin balance relay over mesh
### Decentralized Identity
- Ed25519 node identity with DID Documents (did:key)
- Multi-identity management (Personal/Business/Anonymous)
- W3C Verifiable Credentials issuance and verification
- Decentralized Web Node (DWN) with bidirectional sync over Tor
- Nostr relay integration and NIP-07 signing for iframe apps
- Nostr integration: NIP-33 node discovery, NIP-44/NIP-04 encryption, NIP-07 signer bridge for iframe apps, relay hosting
- Decentralized Web Node (DWN) record sync between federated nodes over Tor
### Multi-Node Federation
- Invite-based node joining over Tor hidden services
- Trust levels (Trusted/Verified/Untrusted) with DID-based auth
- Bidirectional DWN state sync between federated nodes
- File sharing with access controls (free/peers-only/paid)
### Mesh Networking
- LoRa radio communication via Meshcore protocol
- Device discovery and mesh routing
- Off-grid Bitcoin balance checks (planned)
- State sync and app deployment across federated nodes
- File sharing with access controls (free/peers-only/paid via Lightning, on-chain, or ecash)
### System Updates
- OTA updates from self-hosted Gitea (git.tx1138.com) with SHA256 verification
- Three update modes: Manual, Daily Check, Auto Apply (3 AM window)
- Rollback support with automatic backup before applying
- Full UI for update management in Settings
- OTA updates from a self-hosted Gitea release server, Ed25519-signature-verified against a pinned release-root key
- Resumable downloads, automatic pre-update backup, rollback with a post-update self-verify window
- Manual, scheduled-check, and auto-apply modes (auto-apply refuses unsigned manifests)
### Security
- ChaCha20-Poly1305 encrypted secrets at rest, Argon2id password hashing
- Rootless Podman: read-only root, cap-drop ALL, non-root user, no-new-privileges
- TOTP two-factor authentication
- Per-endpoint rate limiting, CSRF protection, input validation
- AppArmor profiles for container confinement
- Tor hidden services for all inter-node communication
- All crypto and container dependencies pinned to exact versions
- Full penetration test completed (33 findings, all remediated)
- Argon2id password hashing (transparent upgrade from legacy hashes), ChaCha20-Poly1305 encrypted secrets at rest
- Rootless Podman: read-only root, cap-drop ALL with a reviewed allow-list, no-new-privileges
- Signed release manifests and signed app catalog (Ed25519, pinned trust anchor)
- TOTP two-factor authentication, per-endpoint rate limiting, CSRF protection
- AppArmor profiles for container confinement; Tor hidden services for inter-node traffic
- Independent security audit of an early version archived in [`docs/archive/`](docs/archive/security-code-audit-2026-03.md); top findings since remediated
## Roadmap
**Done**
- Single-node production gate **green** — install / stop / start / restart / reinstall / reboot-survive / uninstall, 5 consecutive full runs with zero failures on real hardware
- Quadlet migration validated (all backends as `user.slice` services on the canary node)
- Release signing ceremony completed — release-root key pinned, catalog and OTA manifests signed
- Reticulum third mesh transport (real-RF LoRa gates passed), Bitcoin Core/Knots multi-version switching, decentralized marketplace backend, public demo
**In progress**
- Multinode pass: the same production gate across the whole test fleet ([`docs/multinode-testing-plan.md`](docs/multinode-testing-plan.md))
- Quadlet default flip fleet-wide + container-flapping elimination
- 1.8.0 release hardening tail ([`docs/1.8.0-RELEASE-HARDENING-PLAN.md`](docs/1.8.0-RELEASE-HARDENING-PLAN.md)): OTA upgrade soak on real hardware, ISO/image hardening (per-device keys, no default creds, signed ISO)
**Planned**
- Developer CLI (`archy app validate/render/install/test`) to open third-party app publishing
- External marketplace trust UX + publishing tooling ([`docs/marketplace-protocol.md`](docs/marketplace-protocol.md))
- DHT/P2P distribution of releases and app images ([`docs/dht-distribution-design.md`](docs/dht-distribution-design.md))
- P2P encrypted voice/video over Tor, dual-ecash (Fedimint + Cashu) phases, paid streaming, hardware signer support
The live, priority-ordered task list is [`docs/UNIFIED-TASK-TRACKER.md`](docs/UNIFIED-TASK-TRACKER.md); the full narrative plan is [`docs/PRODUCTION-MASTER-PLAN.md`](docs/PRODUCTION-MASTER-PLAN.md).
## Quick Start
### Install from ISO
1. Download the ISO for your architecture (x86_64 or ARM64)
1. Build or download the ISO for your architecture (x86_64 or ARM64) — see [`image-recipe/`](image-recipe/)
2. Flash to USB drive with Balena Etcher or `dd`
3. Boot from USB on target hardware
4. Follow the automated installer
5. Access the web UI at `http://<device-ip>`
6. Set your password and start the onboarding wizard
3. Boot from USB on target hardware and follow the automated installer
4. Access the web UI at `http://<device-ip>`
5. Set your password and complete the onboarding wizard (seed backup, DID identity)
### Supported Hardware
@ -75,7 +103,7 @@ Bitcoin, Storage (FileBrowser, Immich, Nextcloud), Productivity (Penpot, Vaultwa
| **x86_64** | Intel NUC, mini PCs, any 64-bit PC | 4GB RAM, 32GB storage |
| **ARM64** | Raspberry Pi 5, ARM64 SBCs | 4GB RAM, 32GB storage |
**Recommended**: 8GB+ RAM, 1TB+ NVMe SSD (for full Bitcoin node)
**Recommended**: 8GB+ RAM, 1TB+ NVMe SSD (for a full Bitcoin node). Optional: an RNode-compatible LoRa radio for mesh networking.
## Development
@ -94,7 +122,15 @@ npm run type-check # TypeScript validation
npm run build # Production build → web/dist/neode-ui/
```
### Deploy to Server
### Backend Development
```bash
cd core # Rust workspace root (no Cargo.toml at repo root)
cargo build
cargo test
```
### Deploy to a Test Node
```bash
./scripts/deploy-to-target.sh --live # Deploy to primary dev server
@ -104,54 +140,57 @@ npm run build # Production build → web/dist/neode-ui/
### Release (tarball-only)
Releases ship as a backend binary and a frontend tarball referenced by
`releases/manifest.json`. Nodes OTA-update via `scripts/self-update.sh`.
`releases/manifest.json`, published to the self-hosted Gitea release server.
```bash
./scripts/create-release.sh 1.2.3
git push gitea-local main --tags
git push gitea-vps2 main --tags
git push origin main --tags
```
ISO builds are archived under `image-recipe/_archived/` and not part of the
release deliverable.
## Architecture
```
Debian 13 (Trixie)
├── Rootless Podman (30 containers, archy-net DNS)
├── Rootless Podman — every app a systemd Quadlet unit under user.slice
├── Nginx (reverse proxy, security headers, rate limiting)
├── Rust Backend (JSON-RPC API on 127.0.0.1:5678)
│ ├── core/archipelago/ — RPC endpoints, auth, identity, federation, mesh
│ ├── core/container/ — PodmanClient (REST API socket), manifests, health
│ ├── core/security/ — AppArmor, secrets, Cosign image verification
│ └── 6 more crates — models, helpers, js-engine, performance, etc.
├── Vue 3 Frontend (Composition API + TypeScript strict + Pinia + Tailwind)
├── Rust Backend (JSON-RPC API on 127.0.0.1:5678, ~380 RPC methods)
│ ├── core/archipelago/ — API, orchestrator + reconciler, mesh, identity,
│ │ federation, wallet, updates, marketplace
│ ├── core/container/ — Podman client, manifest schema, Quadlet compiler,
│ │ health monitor, signed app catalog
│ ├── core/security/ — AppArmor/seccomp policy, secrets manager
│ ├── core/openwrt/ — TollGate gateway provisioning (SSH/UCI)
│ └── core/performance/ — resource limits
├── Vue 3 Frontend (Composition API + TypeScript strict + Pinia + Tailwind, PWA)
│ └── Three UI modes (Pro/Easy/Chat) + gamepad navigation + i18n
├── Reticulum daemon (supervised Python/PyInstaller, one per LoRa radio)
└── System Tor (hidden services, SOCKS5 proxy)
```
~49,000 lines of Rust | ~47,000 lines of TypeScript/Vue | 78 shell scripts | 30 container apps
~117,000 lines of Rust | ~69,000 lines of TypeScript/Vue | 51 packaged apps | Android companion app
## Documentation
| Doc | Purpose |
|-----|---------|
| [Architecture](docs/architecture.md) | System design, codebase stats, data paths |
| [Architecture Review (HTML)](docs/architecture-review.html) | Interactive guide with diagrams and learning path |
| [Architecture](docs/architecture.md) | System design, crate map, data paths |
| [Developer Guide](docs/developer-guide.md) | Dev setup, workflow, code conventions |
| [API Reference](docs/api-reference.md) | Complete RPC endpoint reference |
| [API Reference](docs/api-reference.md) | RPC endpoint reference |
| [App Developer Guide](docs/app-developer-guide.md) | Building and publishing apps |
| [App Manifest Spec](docs/app-manifest-spec.md) | The `manifest.yml` schema |
| [User Walkthrough](docs/user-walkthrough.md) | End-user installation and usage guide |
| [Troubleshooting](docs/troubleshooting.md) | Diagnostic scenarios and solutions |
| [Operations Runbook](docs/operations-runbook.md) | Ops commands and emergency recovery |
| [Security Audit](docs/security-code-audit-2026-03.md) | Penetration test findings |
| [Master Plan](docs/MASTER_PLAN.md) | Phased roadmap and task tracking |
| [Production Master Plan](docs/PRODUCTION-MASTER-PLAN.md) | North star and workstream narrative |
| [Unified Task Tracker](docs/UNIFIED-TASK-TRACKER.md) | Live, priority-ordered open items |
| [Test Gate](tests/lifecycle/TESTING.md) | Production lifecycle test gate (definition of done) |
| [Archive](docs/archive/) | Historical audits, session logs, shipped designs |
## Contributing
1. Fork the repository
2. Create a feature branch (`feature/description`)
3. Follow the coding standards in [CLAUDE.md](CLAUDE.md)
3. Follow the coding standards in [CONTRIBUTING.md](CONTRIBUTING.md) and [CLAUDE.md](CLAUDE.md)
4. Submit a pull request
## License
@ -160,4 +199,4 @@ Debian 13 (Trixie)
## Acknowledgments
Built with: [Rust](https://www.rust-lang.org/), [Vue.js](https://vuejs.org/), [Podman](https://podman.io/), [Bitcoin Core](https://bitcoin.org/), [LND](https://lightning.engineering/), [Debian](https://www.debian.org/)
Built with: [Rust](https://www.rust-lang.org/), [Vue.js](https://vuejs.org/), [Podman](https://podman.io/), [Bitcoin Core](https://bitcoin.org/), [LND](https://lightning.engineering/), [Reticulum](https://reticulum.network/), [Debian](https://www.debian.org/)

View File

@ -1,11 +1,11 @@
app:
id: archy-btcpay-db
name: BTCPay Postgres
version: 15.17
version: "15.17"
description: Postgres backend for BTCPay and NBXplorer.
container:
image: git.tx1138.com/lfg2025/postgres:15.17
image: 146.59.87.168:3000/lfg2025/postgres:15.17
pull_policy: if-not-present
network: archy-net
data_uid: "100998:100998"

View File

@ -5,7 +5,7 @@ app:
description: MariaDB backend for the mempool explorer stack.
container:
image: git.tx1138.com/lfg2025/mariadb:11.4.10
image: 146.59.87.168:3000/lfg2025/mariadb:11.4.10
pull_policy: if-not-present
network: archy-net
data_uid: "100998:100998"

View File

@ -5,7 +5,7 @@ app:
description: BTCPay blockchain indexer service.
container:
image: git.tx1138.com/lfg2025/nbxplorer:2.6.0
image: 146.59.87.168:3000/lfg2025/nbxplorer:2.6.0
pull_policy: if-not-present
network: archy-net
secret_env:

View File

@ -17,6 +17,13 @@ app:
# the IBD sweet spot - 4GB on full nodes, 1GB on pruned. Container
# --memory=8g (config.rs::get_memory_limit) leaves headroom for
# mempool + connections.
#
# -printtoconsole=0: foreground bitcoind defaults console logging ON,
# which pushed every IBD "UpdateTip" line through conmon into journald
# (>1 GB/day on a fresh node). bitcoind still writes debug.log in the
# datadir (/var/lib/archipelago/bitcoin/debug.log, self-shrunk on
# restart) — use that for deep debugging; podman logs only carries
# entrypoint/startup errors.
- >-
BITCOIND="$(command -v bitcoind || true)";
if [ -z "$BITCOIND" ]; then
@ -36,9 +43,9 @@ app:
RPC_TXRELAY_FLAGS="$RPC_TXRELAY_FLAGS -rpcauth=$RPC_TXRELAY_AUTH -rpcwhitelist=txrelay:sendrawtransaction,submitpackage,testmempoolaccept,getmempoolinfo,getrawmempool,getmempoolentry,getnetworkinfo,getblockchaininfo,getblockcount,getblockhash,getblock,getblockheader,getrawtransaction,gettxout,gettxspendingprevout,decoderawtransaction,decodescript,estimatesmartfee,uptime,ping,getconnectioncount,getpeerinfo,getindexinfo,getdeploymentinfo,getchaintips";
fi;
if [ "${DISK_GB_VALUE:-0}" -lt 1000 ]; then
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=1024 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -printtoconsole=0 -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=1024 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
else
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -printtoconsole=0 -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
fi
derived_env:
- key: DISK_GB
@ -64,9 +71,17 @@ app:
network_policy: isolated
ports:
# RPC is auth-only: publish host-local ONLY - the LAN cannot reach
# nodeIP:8332. In-node consumers (lnd, fedimint, btcpay, mempool-api)
# dial the container's archy-net alias directly (bitcoin-core:8332),
# which needs no publish at all. Do NOT bind the archy-net gateway
# (10.89.0.1): rootlessport binds in the HOST netns where that address
# does not exist, and the whole unit crash-loops (2026-07-09, .228).
# P2P 8333 stays public.
- host: 8332
container: 8332
protocol: tcp
bind: 127.0.0.1
- host: 8333
container: 8333
protocol: tcp

View File

@ -17,6 +17,13 @@ app:
# the IBD sweet spot - 4GB on full nodes, 1GB on pruned. Container
# --memory=8g (config.rs::get_memory_limit) leaves headroom for
# mempool + connections.
#
# -printtoconsole=0: foreground bitcoind defaults console logging ON,
# which pushed every IBD "UpdateTip" line through conmon into journald
# (>1 GB/day on a fresh node). bitcoind still writes debug.log in the
# datadir (/var/lib/archipelago/bitcoin/debug.log, self-shrunk on
# restart) — use that for deep debugging; podman logs only carries
# entrypoint/startup errors.
- >-
BITCOIND="$(command -v bitcoind || true)";
if [ -z "$BITCOIND" ]; then
@ -36,9 +43,9 @@ app:
RPC_TXRELAY_FLAGS="$RPC_TXRELAY_FLAGS -rpcauth=$RPC_TXRELAY_AUTH -rpcwhitelist=txrelay:sendrawtransaction,submitpackage,testmempoolaccept,getmempoolinfo,getrawmempool,getmempoolentry,getnetworkinfo,getblockchaininfo,getblockcount,getblockhash,getblock,getblockheader,getrawtransaction,gettxout,gettxspendingprevout,decoderawtransaction,decodescript,estimatesmartfee,uptime,ping,getconnectioncount,getpeerinfo,getindexinfo,getdeploymentinfo,getchaintips";
fi;
if [ "${DISK_GB_VALUE:-0}" -lt 1000 ]; then
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=2048 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -printtoconsole=0 -server=1 -prune=550 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=2048 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
else
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
exec "$BITCOIND" -datadir=/home/bitcoin/.bitcoin -noconf -printtoconsole=0 -server=1 -txindex=1 -rpcallowip=0.0.0.0/0 -rpcbind=0.0.0.0:8332 -listen=1 -bind=0.0.0.0:8333 -dbcache=4096 -par=0 -maxconnections=125 $RPC_HEADROOM $RPC_TXRELAY_FLAGS -rpcuser="$RPC_USER" -rpcpassword="$RPC_PASS";
fi
derived_env:
- key: DISK_GB
@ -64,9 +71,17 @@ app:
network_policy: isolated
ports:
# RPC is auth-only: publish host-local ONLY - the LAN cannot reach
# nodeIP:8332. In-node consumers (lnd, fedimint, btcpay, mempool-api)
# dial the container's archy-net alias directly (bitcoin-knots:8332),
# which needs no publish at all. Do NOT bind the archy-net gateway
# (10.89.0.1): rootlessport binds in the HOST netns where that address
# does not exist, and the whole unit crash-loops (2026-07-09, .228).
# P2P 8333 stays public.
- host: 8332
container: 8332
protocol: tcp
bind: 127.0.0.1
- host: 8333
container: 8333
protocol: tcp

View File

@ -5,7 +5,7 @@ app:
description: Fedimint gateway service with automatic LND-or-LDK backend selection.
container:
image: git.tx1138.com/lfg2025/gatewayd:v0.10.0
image: 146.59.87.168:3000/lfg2025/gatewayd:v0.10.0
pull_policy: if-not-present
network: archy-net
entrypoint: ["sh", "-lc"]

View File

@ -5,7 +5,7 @@ app:
description: Baseline Archipelago file manager service.
container:
image: git.tx1138.com/lfg2025/filebrowser:v2.27.0
image: 146.59.87.168:3000/lfg2025/filebrowser:v2.27.0
pull_policy: if-not-present
network: archy-net
custom_args: ["--config", "/data/.filebrowser.json"]

View File

@ -30,7 +30,13 @@ app:
disk_limit: 200Gi
security:
capabilities: []
# Runs as container root over a data tree the legacy installer chowned
# to the subuid range (host 100000 = container uid 1). Without
# DAC_OVERRIDE the server EACCESes writing upload/encoded-video the
# moment the container is recreated against this manifest (latent until
# the 2026-07-05 secret-env migration recreated it). Same cap set as
# immich-postgres minus the setuid pair it doesn't use.
capabilities: [CHOWN, DAC_OVERRIDE, FOWNER]
readonly_root: false
network_policy: isolated

View File

@ -8,6 +8,13 @@ app:
image: 146.59.87.168:3000/lfg2025/lnd:v0.18.4-beta
pull_policy: if-not-present
network: archy-net
# BITCOIND_HOST must follow the node's actual Bitcoin container — Knots or
# Core — resolved at apply time from host facts. Hardcoding either breaks
# LND's chain backend connection on the other (lnd.conf is likewise
# resolved in lnd::ensure_config).
derived_env:
- key: BITCOIND_HOST
template: "{{BITCOIN_HOST}}"
secret_env:
- key: BITCOIND_RPCPASS
secret_file: bitcoin-rpc-password
@ -45,7 +52,6 @@ app:
options: [rw]
environment:
- BITCOIND_HOST=bitcoin-knots
- BITCOIND_RPCUSER=archipelago
- NETWORK=mainnet

View File

@ -5,7 +5,7 @@ app:
description: Backend API for mempool explorer.
container:
image: git.tx1138.com/lfg2025/mempool-backend:v3.0.0
image: 146.59.87.168:3000/lfg2025/mempool-backend:v3.0.0
pull_policy: if-not-present
network: archy-net
# CORE_RPC_HOST must follow the node's actual Bitcoin container — Knots or

View File

@ -3,51 +3,211 @@ app:
name: Strfry Nostr Relay
version: 0.9.0
description: Lightweight Nostr relay written in C++. Alternative to nostr-rs-relay with lower resource usage.
container:
image: dockurr/strfry:1.0.4
image_signature: cosign://...
pull_policy: verify-signature
dependencies:
- storage: 5Gi
resources:
cpu_limit: 1
memory_limit: 512Mi
disk_limit: 5Gi
security:
capabilities: []
readonly_root: true
no_new_privileges: true
user: 1000
seccomp_profile: default
network_policy: isolated
apparmor_profile: nostr-relay
ports:
- host: 8082
container: 8080
protocol: tcp # HTTP/WebSocket
- host: 8090
container: 7777
protocol: tcp # HTTP/WebSocket (strfry listens on 7777)
volumes:
- type: bind
source: /var/lib/archipelago/strfry
target: /strfry
target: /app/strfry-db
options: [rw]
environment:
- RELAY_NAME=Archipelago Strfry Relay
# Image default config demands a 1M NOFILES rlimit, above the rootless
# user-manager hard cap (524288) — ship the config with nofiles = 0.
# Mounting it also skips the entrypoint's copy into /etc, which a
# readonly_root container cannot do.
- type: bind
source: /var/lib/archipelago/strfry-config/strfry.conf
target: /etc/strfry.conf
options: [ro]
files:
- path: /var/lib/archipelago/strfry-config/strfry.conf
overwrite: true
content: |
##
## Default strfry config
##
# Directory that contains the strfry LMDB database (restart required)
db = "./strfry-db/"
dbParams {
# Maximum number of threads/processes that can simultaneously have LMDB transactions open (restart required)
maxreaders = 256
# Size of mmap() to use when loading LMDB (default is 10TB, does *not* correspond to disk-space used) (restart required)
mapsize = 10995116277760
# Disables read-ahead when accessing the LMDB mapping. Reduces IO activity when DB size is larger than RAM. (restart required)
noReadAhead = false
}
events {
# Maximum size of normalised JSON, in bytes
maxEventSize = 65536
# Events newer than this will be rejected
rejectEventsNewerThanSeconds = 900
# Events older than this will be rejected
rejectEventsOlderThanSeconds = 94608000
# Ephemeral events older than this will be rejected
rejectEphemeralEventsOlderThanSeconds = 60
# Ephemeral events will be deleted from the DB when older than this
ephemeralEventsLifetimeSeconds = 300
# Maximum number of tags allowed
maxNumTags = 2000
# Maximum size for tag values, in bytes
maxTagValSize = 1024
}
relay {
# Interface to listen on. Use 0.0.0.0 to listen on all interfaces (restart required)
bind = "0.0.0.0"
# Port to open for the nostr websocket protocol (restart required)
port = 7777
# Set OS-limit on maximum number of open files/sockets (if 0, don't attempt to set) (restart required)
nofiles = 0
# HTTP header that contains the client's real IP, before reverse proxying (ie x-real-ip) (MUST be all lower-case)
realIpHeader = ""
info {
# NIP-11: Name of this server. Short/descriptive (< 30 characters)
name = "Archipelago Strfry Relay"
# NIP-11: Detailed information about relay, free-form
description = "Self-hosted strfry Nostr relay on Archipelago."
# NIP-11: Administrative nostr pubkey, for contact purposes
pubkey = ""
# NIP-11: Alternative administrative contact (email, website, etc)
contact = ""
# NIP-11: URL pointing to an image to be used as an icon for the relay
icon = ""
# List of supported lists as JSON array, or empty string to use default. Example: "[1,2]"
nips = ""
}
# Maximum accepted incoming websocket frame size (should be larger than max event) (restart required)
maxWebsocketPayloadSize = 131072
# Maximum number of filters allowed in a REQ
maxReqFilterSize = 200
# Websocket-level PING message frequency (should be less than any reverse proxy idle timeouts) (restart required)
autoPingSeconds = 55
# If TCP keep-alive should be enabled (detect dropped connections to upstream reverse proxy)
enableTcpKeepalive = false
# How much uninterrupted CPU time a REQ query should get during its DB scan
queryTimesliceBudgetMicroseconds = 10000
# Maximum records that can be returned per filter
maxFilterLimit = 500
# Maximum number of subscriptions (concurrent REQs) a connection can have open at any time
maxSubsPerConnection = 20
writePolicy {
# If non-empty, path to an executable script that implements the writePolicy plugin logic
plugin = "/app/write-policy.py"
}
compression {
# Use permessage-deflate compression if supported by client. Reduces bandwidth, but slight increase in CPU (restart required)
enabled = true
# Maintain a sliding window buffer for each connection. Improves compression, but uses more memory (restart required)
slidingWindow = true
}
logging {
# Dump all incoming messages
dumpInAll = false
# Dump all incoming EVENT messages
dumpInEvents = false
# Dump all incoming REQ/CLOSE messages
dumpInReqs = false
# Log performance metrics for initial REQ database scans
dbScanPerf = false
# Log reason for invalid event rejection? Can be disabled to silence excessive logging
invalidEvents = true
}
numThreads {
# Ingester threads: route incoming requests, validate events/sigs (restart required)
ingester = 3
# reqWorker threads: Handle initial DB scan for events (restart required)
reqWorker = 3
# reqMonitor threads: Handle filtering of new events (restart required)
reqMonitor = 3
# negentropy threads: Handle negentropy protocol messages (restart required)
negentropy = 2
}
negentropy {
# Support negentropy protocol messages
enabled = true
# Maximum records that sync will process before returning an error
maxSyncEvents = 1000000
}
}
health_check:
type: http
endpoint: http://localhost:8082
# In-container probe: must target the CONTAINER port (7777), not the host
# mapping (8090), and 127.0.0.1 explicitly — `localhost` resolves to ::1
# inside the image while strfry binds IPv4 0.0.0.0 only (verified on .228:
# localhost:7777 refused, 127.0.0.1:7777/health = 200).
endpoint: http://127.0.0.1:7777
path: /health
interval: 30s
timeout: 5s
retries: 3
nostr_integration:
relay_type: public
monetization_enabled: true

81
core/Cargo.lock generated
View File

@ -99,6 +99,7 @@ version = "1.7.99-alpha"
dependencies = [
"anyhow",
"archipelago-container",
"archipelago-openwrt",
"archipelago-performance",
"archipelago-security",
"argon2",
@ -168,6 +169,7 @@ dependencies = [
"async-trait",
"chrono",
"futures",
"hex",
"hyper 0.14.32",
"indexmap",
"log",
@ -175,12 +177,29 @@ dependencies = [
"serde",
"serde_json",
"serde_yaml",
"sha2 0.10.9",
"thiserror 1.0.69",
"tokio",
"tracing",
"uuid",
]
[[package]]
name = "archipelago-openwrt"
version = "0.1.0"
dependencies = [
"anyhow",
"async-trait",
"reqwest 0.11.27",
"serde",
"serde_json",
"ssh2",
"thiserror 1.0.69",
"tokio",
"tokio-test",
"tracing",
]
[[package]]
name = "archipelago-performance"
version = "0.1.0"
@ -2840,6 +2859,32 @@ dependencies = [
"redox_syscall 0.7.3",
]
[[package]]
name = "libssh2-sys"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "220e4f05ad4a218192533b300327f5150e809b54c4ec83b5a1d91833601811b9"
dependencies = [
"cc",
"libc",
"libz-sys",
"openssl-sys",
"pkg-config",
"vcpkg",
]
[[package]]
name = "libz-sys"
version = "1.1.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85bc9657773828b90eeb625adff10eeac83cc21bbfd8e23a03eaa8a33c9e28d9"
dependencies = [
"cc",
"libc",
"pkg-config",
"vcpkg",
]
[[package]]
name = "linux-raw-sys"
version = "0.11.0"
@ -3581,6 +3626,18 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe"
[[package]]
name = "openssl-sys"
version = "0.9.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b47e7e6bb2c38cd930d25a23b40fa52e068c10e85f3e03a7f5ba5aaca5713695"
dependencies = [
"cc",
"libc",
"pkg-config",
"vcpkg",
]
[[package]]
name = "papaya"
version = "0.2.4"
@ -3759,6 +3816,12 @@ dependencies = [
"spki 0.8.0",
]
[[package]]
name = "pkg-config"
version = "0.3.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
[[package]]
name = "plain"
version = "0.2.3"
@ -4989,6 +5052,18 @@ dependencies = [
"der 0.8.0",
]
[[package]]
name = "ssh2"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f84d13b3b8a0d4e91a2629911e951db1bb8671512f5c09d7d4ba34500ba68c8"
dependencies = [
"bitflags 2.13.0",
"libc",
"libssh2-sys",
"parking_lot 0.12.5",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.1"
@ -5776,6 +5851,12 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "vcpkg"
version = "0.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
[[package]]
name = "vergen"
version = "9.1.0"

View File

@ -4,6 +4,7 @@ resolver = "2"
members = [
"archipelago",
"container",
"openwrt",
"performance",
"security",
]

View File

@ -43,6 +43,7 @@ futures-util = "0.3"
# Our modules
archipelago-container = { path = "../container" }
archipelago-openwrt = { path = "../openwrt" }
archipelago-security = { path = "../security" }
archipelago-performance = { path = "../performance" }

View File

@ -39,6 +39,17 @@ impl ApiHandler {
let (mut tx, mut rx) = ws_stream.split();
// Subscribe BEFORE taking the initial snapshot. Messages are full
// data dumps keyed by a monotonic revision, so a broadcast that
// races the snapshot is at worst a harmless duplicate/newer dump
// delivered right after — but subscribing after the snapshot send
// (the old order) let any update in that window vanish forever,
// since a tokio broadcast channel never delivers sends that
// predate subscribe(). That silently stuck clients (e.g. a fresh
// install's post-boot container scan) on a stale initial snapshot
// until a full page reload opened a new connection past the race.
let mut state_rx = state_manager.subscribe();
let initial_msg = state_manager.get_initial_message().await;
if let Ok(json_msg) = serde_json::to_string(&initial_msg) {
if let Err(e) = tx.send(Message::Text(json_msg)).await {
@ -47,8 +58,6 @@ impl ApiHandler {
}
debug!("Sent initial data dump at revision {}", initial_msg.rev);
}
let mut state_rx = state_manager.subscribe();
let ping_interval = tokio::time::interval(tokio::time::Duration::from_secs(30));
tokio::pin!(ping_interval);
let mut last_client_activity = Instant::now();

View File

@ -1,4 +1,6 @@
use super::{RpcHandler, DEV_DEFAULT_PASSWORD};
use super::RpcHandler;
#[cfg(debug_assertions)]
use super::DEV_DEFAULT_PASSWORD;
use anyhow::Result;
impl RpcHandler {
@ -14,7 +16,10 @@ impl RpcHandler {
let is_setup = self.auth_manager.is_setup().await?;
if !is_setup {
// Dev mode: allow default password so UI can log in without running setup
// Dev BUILDS only: allow the default password so the UI can log
// in without running setup. cfg-gated so no release binary can
// carry the bypass, whatever its runtime config says.
#[cfg(debug_assertions)]
if self.config.dev_mode && password == DEV_DEFAULT_PASSWORD {
tracing::info!("[onboarding] login via dev default password");
return Ok(serde_json::Value::Null);
@ -141,6 +146,19 @@ impl RpcHandler {
self.auth_manager.setup_user(password).await?;
tracing::info!("[onboarding] user setup complete");
// Persist the pending onboarding seed as the encrypted backup now that
// a passphrase (the login password) finally exists — otherwise "Reveal
// recovery phrase" has nothing to decrypt on this node, ever.
// Best-effort: a failure here must not break password setup.
match super::seed_rpc::save_pending_seed_encrypted(&self.config.data_dir, password).await {
Ok(true) => tracing::info!("[onboarding] encrypted seed backup saved"),
Ok(false) => tracing::info!(
"[onboarding] no pending mnemonic to back up (restored earlier or legacy node)"
),
Err(e) => tracing::warn!("[onboarding] encrypted seed backup failed: {e:#}"),
}
Ok(serde_json::json!(true))
}

View File

@ -862,7 +862,9 @@ async fn hydrate_tor_endpoint(data_dir: &Path, state: &mut BitcoinRelayState) {
let onion = onion.trim().trim_end_matches('/').to_string();
if !onion.is_empty() {
state.settings.tor_endpoint = Some(format!("http://{onion}/"));
let _ = save_relay_state(data_dir, state).await;
if let Err(e) = save_relay_state(data_dir, state).await {
tracing::warn!("Failed to persist relay tor endpoint: {e:#}");
}
}
}
}

View File

@ -267,13 +267,16 @@ impl RpcHandler {
.context("Failed to connect to peer")?;
// Record which transport actually reached the peer (B14) so the UI
// reflects FIPS vs Tor truthfully instead of always showing Tor/none.
let _ = crate::federation::record_peer_transport(
if let Err(e) = crate::federation::record_peer_transport(
&self.config.data_dir,
None,
Some(onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
if response.status() == reqwest::StatusCode::PAYMENT_REQUIRED {
let body: serde_json::Value = response.json().await.unwrap_or_default();
@ -348,13 +351,16 @@ impl RpcHandler {
.await
.context("Failed to connect to peer")?;
// Record which transport actually reached the peer (B14).
let _ = crate::federation::record_peer_transport(
if let Err(e) = crate::federation::record_peer_transport(
&self.config.data_dir,
None,
Some(onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
if !response.status().is_success() {
return Err(anyhow::anyhow!(
@ -497,13 +503,16 @@ impl RpcHandler {
}
};
// Record which transport actually reached the peer (B14).
let _ = crate::federation::record_peer_transport(
if let Err(e) = crate::federation::record_peer_transport(
&self.config.data_dir,
None,
Some(onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
if response.status() == reqwest::StatusCode::PAYMENT_REQUIRED {
// Payment was rejected by the seller. Surface the most likely cause
@ -763,13 +772,16 @@ impl RpcHandler {
}));
}
};
let _ = crate::federation::record_peer_transport(
if let Err(e) = crate::federation::record_peer_transport(
&self.config.data_dir,
None,
Some(onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
if response.status() == reqwest::StatusCode::PAYMENT_REQUIRED {
return Ok(serde_json::json!({
@ -951,13 +963,16 @@ impl RpcHandler {
}));
}
};
let _ = crate::federation::record_peer_transport(
if let Err(e) = crate::federation::record_peer_transport(
&self.config.data_dir,
None,
Some(onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
if response.status() == reqwest::StatusCode::PAYMENT_REQUIRED {
return Ok(serde_json::json!({
@ -1019,13 +1034,16 @@ impl RpcHandler {
.await
.context("Failed to connect to peer for preview")?;
// Record which transport actually reached the peer (B14).
let _ = crate::federation::record_peer_transport(
if let Err(e) = crate::federation::record_peer_transport(
&self.config.data_dir,
None,
Some(onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
if !response.status().is_success() {
return Err(anyhow::anyhow!(

View File

@ -223,6 +223,7 @@ impl RpcHandler {
"network.list-interfaces" => self.handle_network_list_interfaces().await,
"network.scan-wifi" => self.handle_network_scan_wifi().await,
"network.configure-wifi" => self.handle_network_configure_wifi(params).await,
"network.set-wifi-radio" => self.handle_network_set_wifi_radio(params).await,
"network.configure-ethernet" => self.handle_network_configure_ethernet(params).await,
"network.dns-status" => self.handle_network_dns_status().await,
"network.configure-dns" => self.handle_network_configure_dns(params).await,
@ -230,6 +231,13 @@ impl RpcHandler {
"router.info" => self.handle_router_info().await,
"router.configure" => self.handle_router_configure(params).await,
// OpenWrt / TollGate
"openwrt.scan" => self.handle_openwrt_scan(params).await,
"openwrt.get-status" => self.handle_openwrt_get_status(params).await,
"openwrt.provision-tollgate" => self.handle_openwrt_provision_tollgate(params).await,
"openwrt.scan-wifi" => self.handle_openwrt_scan_wifi(params).await,
"openwrt.configure-wan" => self.handle_openwrt_configure_wan(params).await,
// Ecash wallet
"wallet.ecash-balance" => self.handle_wallet_ecash_balance().await,
"wallet.ecash-mint" => self.handle_wallet_ecash_mint(params).await,

View File

@ -18,6 +18,24 @@ impl RpcHandler {
Ok(serde_json::json!({ "networks": networks }))
}
/// network.set-wifi-radio — turn the wifi adapter fully on or off (not just
/// disconnect from a network). Params: `{ "enabled": bool }`.
pub(super) async fn handle_network_set_wifi_radio(
&self,
params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
let params = params.ok_or_else(|| anyhow::anyhow!("Missing params"))?;
let enabled = params
.get("enabled")
.and_then(|v| v.as_bool())
.ok_or_else(|| anyhow::anyhow!("Missing required parameter: enabled"))?;
tracing::info!(enabled, "Setting wifi radio state");
set_wifi_radio(enabled).await?;
Ok(serde_json::json!({ "ok": true, "enabled": enabled }))
}
/// network.configure-wifi — connect to a WiFi network.
pub(super) async fn handle_network_configure_wifi(
&self,
@ -327,6 +345,27 @@ fn split_nmcli_escaped(line: &str, limit: usize) -> Vec<String> {
fields
}
/// Turn the wifi radio fully on or off using nmcli (a rfkill-level toggle, not
/// just disconnecting from the current network — the adapter stops scanning/
/// associating entirely until switched back on).
async fn set_wifi_radio(enabled: bool) -> Result<()> {
let state = if enabled { "on" } else { "off" };
let output = tokio::process::Command::new("nmcli")
.args(["radio", "wifi", state])
.output()
.await
.context("Failed to run nmcli radio wifi")?;
if !output.status.success() {
anyhow::bail!(
"nmcli radio wifi {} failed: {}",
state,
String::from_utf8_lossy(&output.stderr)
);
}
Ok(())
}
/// Connect to a WiFi network using nmcli.
async fn connect_wifi(ssid: &str, password: &str) -> Result<()> {
let conn_name = format!("archipelago-wifi-{ssid}");

View File

@ -1,6 +1,7 @@
use super::super::RpcHandler;
use crate::mesh;
use anyhow::Result;
use tracing::warn;
impl RpcHandler {
/// mesh.status — Get mesh radio status, device info, and peer count.
@ -283,7 +284,9 @@ impl RpcHandler {
let mut set = state.radio_contact_blocklist.write().await;
set.clear();
}
let _ = crate::mesh::save_ignored_radio_contacts(&data_dir, &[]).await;
if let Err(e) = crate::mesh::save_ignored_radio_contacts(&data_dir, &[]).await {
warn!("Failed to persist cleared radio-contact blocklist: {e:#}");
}
// Actually DELETE each radio contact from the firmware table (via
// CMD_REMOVE_CONTACT) so wiped peers don't just reappear on the next

View File

@ -67,6 +67,32 @@ pub(super) fn sanitize_error_message(msg: &str) -> String {
"Container",
"Image",
"Bitcoin address",
"No router",
"No OpenWrt",
"No space left",
"Not enough flash",
"Not enough space",
"TollGate installation failed",
"No pre-built TollGate",
"opkg not found",
"apk update failed",
"No wireless interface",
"No wireless radio",
"WiFi radio enabled but",
"Missing required field",
// seed.reveal / auth flows — user-actionable, no internals to leak.
// Without these the sanitizer collapsed every reveal failure into
// "Operation failed. Check server logs." (which isn't even a crash).
"Incorrect",
"This node has no encrypted seed",
"A 2FA code is required",
"2FA is enabled but",
"Could not decrypt the saved seed",
"Could not unlock 2FA",
"No mnemonic available",
"No pending seed generation",
"Submitted words",
"Already set up",
];
for prefix in &user_facing_prefixes {
if msg.starts_with(prefix) {
@ -86,6 +112,43 @@ pub(super) fn sanitize_error_message(msg: &str) -> String {
"Operation failed. Check server logs for details.".to_string()
}
#[cfg(test)]
mod sanitize_tests {
use super::sanitize_error_message;
#[test]
fn seed_reveal_errors_pass_through() {
// Every user-actionable seed.reveal failure must reach the user —
// masking them as "Check server logs" sent a real user hunting a
// crash that never happened.
for msg in [
"Incorrect password",
"This node has no encrypted seed backup, so the recovery phrase cannot be shown. It was only displayed once during setup.",
"A 2FA code is required to reveal the recovery phrase",
"2FA is enabled but no TOTP data found",
"Could not decrypt the saved seed. If you set a separate backup passphrase during setup, enter that passphrase.",
"Could not unlock 2FA with this password",
"No mnemonic available. Generate or restore a seed first.",
"Submitted words do not match generated seed",
"Already set up. Use auth.changePassword to change.",
] {
assert_ne!(
sanitize_error_message(msg),
"Operation failed. Check server logs for details.",
"masked: {msg}"
);
}
}
#[test]
fn internal_errors_stay_generic() {
assert_eq!(
sanitize_error_message("thread panicked at src/foo.rs:42"),
"Operation failed. Check server logs for details."
);
}
}
/// Derive a CSRF token from the session token via HMAC.
/// Deterministic: same session token always produces the same CSRF token.
/// Survives backend restarts because it depends only on the session token
@ -119,13 +182,91 @@ pub(super) fn extract_cookie(headers: &hyper::HeaderMap, name: &str) -> Option<S
None
}
/// Extract the client IP from request headers (X-Real-IP or X-Forwarded-For).
pub(super) fn extract_client_ip(headers: &hyper::HeaderMap) -> IpAddr {
/// The TCP peer address of the connection a request arrived on, injected
/// into request extensions by the server accept loop.
#[derive(Debug, Clone, Copy)]
pub struct PeerAddr(pub std::net::SocketAddr);
/// Extract the client IP for rate limiting.
///
/// `X-Real-IP`/`X-Forwarded-For` are only honored when the connection
/// itself comes from loopback — i.e. from our local nginx, which sets
/// `X-Real-IP $remote_addr`. On a direct connection (the FIPS peer
/// listener, or anything that isn't the local proxy) the headers are
/// client-supplied, so trusting them let an attacker rotate per-request
/// "IPs" and defeat the login rate limiter; there we use the socket
/// address instead.
pub(super) fn extract_client_ip(parts: &hyper::http::request::Parts) -> IpAddr {
let socket_ip = parts.extensions.get::<PeerAddr>().map(|p| p.0.ip());
match socket_ip {
Some(ip) if ip.is_loopback() => forwarded_client_ip(&parts.headers).unwrap_or(ip),
Some(ip) => ip,
// No socket info recorded (shouldn't happen in the server path);
// fall back to the pre-extension behavior.
None => forwarded_client_ip(&parts.headers)
.unwrap_or(IpAddr::V4(std::net::Ipv4Addr::LOCALHOST)),
}
}
/// The proxy-reported client IP, if a forwarded header carries one.
fn forwarded_client_ip(headers: &hyper::HeaderMap) -> Option<IpAddr> {
headers
.get("x-real-ip")
.or_else(|| headers.get("x-forwarded-for"))
.and_then(|v| v.to_str().ok())
.and_then(|s| s.split(',').next())
.and_then(|s| s.trim().parse::<IpAddr>().ok())
.unwrap_or(IpAddr::V4(std::net::Ipv4Addr::LOCALHOST))
}
#[cfg(test)]
mod client_ip_tests {
use super::*;
use std::net::SocketAddr;
fn parts_with(
peer: Option<&str>,
real_ip: Option<&str>,
) -> hyper::http::request::Parts {
let mut builder = hyper::Request::builder().uri("/rpc/v1");
if let Some(ip) = real_ip {
builder = builder.header("x-real-ip", ip);
}
let (mut parts, _) = builder.body(()).unwrap().into_parts();
if let Some(addr) = peer {
parts
.extensions
.insert(PeerAddr(addr.parse::<SocketAddr>().unwrap()));
}
parts
}
#[test]
fn loopback_connection_trusts_forwarded_header() {
// nginx on loopback forwards the real client IP — use it.
let parts = parts_with(Some("127.0.0.1:44412"), Some("192.168.1.50"));
assert_eq!(
extract_client_ip(&parts),
"192.168.1.50".parse::<IpAddr>().unwrap()
);
}
#[test]
fn direct_connection_ignores_spoofed_header() {
// A direct (non-proxy) client rotating X-Real-IP per request must
// still bucket under its socket address.
let parts = parts_with(Some("203.0.113.9:9999"), Some("10.0.0.1"));
assert_eq!(
extract_client_ip(&parts),
"203.0.113.9".parse::<IpAddr>().unwrap()
);
}
#[test]
fn loopback_connection_without_header_uses_socket_ip() {
let parts = parts_with(Some("127.0.0.1:5000"), None);
assert_eq!(
extract_client_ip(&parts),
"127.0.0.1".parse::<IpAddr>().unwrap()
);
}
}

View File

@ -23,6 +23,7 @@ mod names;
mod network;
mod node;
mod nostr;
mod openwrt;
mod package;
mod peers;
mod response;
@ -57,9 +58,13 @@ use middleware::{
derive_csrf_token, extract_client_ip, extract_cookie, sanitize_error_message,
CACHEABLE_METHODS, UNAUTHENTICATED_METHODS,
};
pub use middleware::PeerAddr;
use response::{cookie_header, json_response, ResponseCache, RpcError, RpcRequest, RpcResponse};
/// Default dev password when no user is set up (matches mock-backend).
/// Dev builds only — the pre-setup login bypass that reads this is
/// cfg-gated out of release binaries.
#[cfg(debug_assertions)]
pub(crate) const DEV_DEFAULT_PASSWORD: &str = "password123";
pub struct RpcHandler {
@ -368,7 +373,7 @@ impl RpcHandler {
// Rate limit login attempts
if rpc_req.method == "auth.login" {
let client_ip = extract_client_ip(&parts.headers);
let client_ip = extract_client_ip(&parts);
if !self.login_rate_limiter.check(client_ip).await {
return Ok(self.rate_limit_response());
}
@ -376,7 +381,7 @@ impl RpcHandler {
// Rate limit sensitive endpoints
{
let client_ip = extract_client_ip(&parts.headers);
let client_ip = extract_client_ip(&parts);
if !self
.endpoint_rate_limiter
.check(&rpc_req.method, client_ip)
@ -450,7 +455,7 @@ impl RpcHandler {
let mut response = json_response(StatusCode::OK, &resp_body);
// Post-dispatch: set cookies for auth-related methods
let client_ip = extract_client_ip(&parts.headers);
let client_ip = extract_client_ip(&parts);
self.apply_auth_cookies(
&rpc_req.method,
&mut rpc_resp,

View File

@ -0,0 +1,353 @@
use super::RpcHandler;
use anyhow::Result;
use archipelago_openwrt::{
detect,
router::Router,
tollgate::{self, TollGateConfig},
wan,
wifi_scan,
};
use crate::network::router as net_router;
/// Default port for the local Cashu mint (nutshell / cashu-mint app).
const LOCAL_MINT_PORT: u16 = 3338;
impl RpcHandler {
/// Scan the local subnet for OpenWrt routers.
///
/// Params: `{ "subnet": "192.168.1.0", "prefix": 24,
/// "ssh_user": "root", "ssh_password": "" }`
pub(super) async fn handle_openwrt_scan(
&self,
params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
let p = params.unwrap_or_default();
let subnet: [u8; 4] = parse_ipv4(
p.get("subnet").and_then(|v| v.as_str()).unwrap_or("192.168.1.0"),
)?;
let prefix = p.get("prefix").and_then(|v| v.as_u64()).unwrap_or(24) as u8;
let ssh_user = p
.get("ssh_user")
.and_then(|v| v.as_str())
.unwrap_or("root")
.to_string();
let ssh_password = p
.get("ssh_password")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string();
let routers = detect::scan_subnet(subnet, prefix, &ssh_user, &ssh_password).await;
let ips: Vec<String> = routers.iter().map(|ip| ip.to_string()).collect();
Ok(serde_json::json!({ "routers": ips }))
}
/// Read current settings from a saved or ad-hoc OpenWrt router via SSH/UCI.
///
/// Params (all optional): `{ "host": "...", "ssh_user": "root", "ssh_password": "" }`
/// If params are omitted the saved `router_config.json` credentials are used.
pub(super) async fn handle_openwrt_get_status(
&self,
params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
let saved = net_router::load_router_config(&self.config.data_dir).await?;
let p = params.unwrap_or_default();
let host_from_params = p.get("host").and_then(|v| v.as_str()).is_some();
let host = p
.get("host")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| if saved.configured { Some(saved.address.clone()) } else { None })
.ok_or_else(|| anyhow::anyhow!("No router configured — provide host or call router.configure first"))?;
let ssh_user = p
.get("ssh_user")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| saved.username.clone())
.unwrap_or_else(|| "root".to_string());
let ssh_password = p
.get("ssh_password")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| saved.password.clone())
.unwrap_or_default();
let router = Router::connect_password(&host, 22, &ssh_user, &ssh_password)?;
router.verify_openwrt()?;
// Persist the connection so other views (e.g. the Home dashboard's
// Network tile) can poll `openwrt.get-status` with no params instead
// of every caller needing to carry host/credentials around. Only do
// this when the host actually came from params — otherwise every
// no-args poll would re-save the same thing it just read.
if host_from_params {
let _ = net_router::configure_router(
&self.config.data_dir,
net_router::RouterType::OpenWrt,
&host,
None,
Some(&ssh_user),
Some(&ssh_password),
).await;
}
// System info
let release = router.run_ok("cat /etc/openwrt_release").unwrap_or_default();
let hostname = router
.uci_get("system.@system[0].hostname")
.unwrap_or_else(|_| "unknown".into());
let uptime_secs: u64 = router
.run_ok("cat /proc/uptime")
.unwrap_or_default()
.split_whitespace()
.next()
.and_then(|s| s.split('.').next())
.and_then(|s| s.parse().ok())
.unwrap_or(0);
// TollGate — check via opkg (≤24.x) or binary presence (25.x apk-native).
// The service binary is /usr/bin/tollgate-wrt (per its init.d script),
// not /usr/bin/tollgate-module-basic-go — that's only the opkg/apk
// *package* name, never an on-disk filename.
let tollgate_installed = router
.run("/usr/bin/opkg list-installed 2>/dev/null | grep -q '^tollgate-module-basic-go ' || \
test -f /usr/bin/tollgate-wrt 2>/dev/null")
.map(|(_, code)| code == 0)
.unwrap_or(false);
let tollgate = if tollgate_installed {
serde_json::json!({
"installed": true,
"enabled": router.uci_get("tollgate.main.enabled").map(|v| v == "1").unwrap_or(false),
"metric": router.uci_get("tollgate.main.metric").unwrap_or_default(),
"step_size_ms": router.uci_get("tollgate.main.step_size").ok().and_then(|v| v.parse::<u64>().ok()).unwrap_or(0),
"price_per_step":router.uci_get("tollgate.main.price_per_step").ok().and_then(|v| v.parse::<u64>().ok()).unwrap_or(0),
"min_steps": router.uci_get("tollgate.main.min_steps").ok().and_then(|v| v.parse::<u32>().ok()).unwrap_or(1),
"currency": router.uci_get("tollgate.main.currency").unwrap_or_default(),
"mint_url": router.uci_get("tollgate.main.mint_url").unwrap_or_default(),
})
} else {
serde_json::json!({ "installed": false })
};
// WiFi interfaces
let wifi_raw = router.run_ok("uci show wireless").unwrap_or_default();
let wifi_interfaces = parse_wifi_interfaces(&wifi_raw);
let wan_status = wan::get_wan_status(&router);
Ok(serde_json::json!({
"host": host,
"hostname": hostname,
"uptime_secs": uptime_secs,
"release": parse_release(&release),
"tollgate": tollgate,
"wifi_interfaces": wifi_interfaces,
"wan": wan_status,
}))
}
/// Provision TollGate on an OpenWrt router and create the "archipelago" SSID.
///
/// Params: `{ "host": "192.168.1.1", "ssh_user": "root", "ssh_password": "",
/// "price_sats": 10, "step_size_ms": 60000, "min_steps": 1,
/// "mint_url": "<optional override>" }`
///
/// `mint_url` defaults to `http://<this node's IP>:3338` — the local Cashu
/// mint that must be running as an Archy app before calling this endpoint.
pub(super) async fn handle_openwrt_provision_tollgate(
&self,
params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
let saved = net_router::load_router_config(&self.config.data_dir).await?;
let p = params.unwrap_or_default();
let host = p
.get("host")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| if saved.configured { Some(saved.address.clone()) } else { None })
.ok_or_else(|| anyhow::anyhow!("No router configured — provide host or call router.configure first"))?;
let ssh_user = p
.get("ssh_user")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| saved.username.clone())
.unwrap_or_else(|| "root".to_string());
let ssh_password = p
.get("ssh_password")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.or_else(|| saved.password.clone())
.unwrap_or_default();
let default_mint_url = format!("http://{}:{}", self.config.host_ip, LOCAL_MINT_PORT);
let mint_url = p
.get("mint_url")
.and_then(|v| v.as_str())
.unwrap_or(&default_mint_url)
.to_string();
let config = TollGateConfig {
ssid: "archipelago".to_string(),
mint_url,
price_sats: p.get("price_sats").and_then(|v| v.as_u64()).unwrap_or(10),
step_size_ms: p
.get("step_size_ms")
.and_then(|v| v.as_u64())
.unwrap_or(60_000),
min_steps: p
.get("min_steps")
.and_then(|v| v.as_u64())
.unwrap_or(1) as u32,
enabled: p.get("enabled").and_then(|v| v.as_bool()).unwrap_or(true),
};
let router = Router::connect_password(&host, 22, &ssh_user, &ssh_password)?;
router.verify_openwrt()?;
tollgate::provision(&router, &config).await?;
Ok(serde_json::json!({
"ok": true,
"host": host,
"ssid": config.ssid,
"mint_url": config.mint_url,
}))
}
/// Scan for visible WiFi networks from the router's radio.
///
/// Params: same host/credentials as other openwrt methods.
pub(super) async fn handle_openwrt_scan_wifi(
&self,
params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
let saved = net_router::load_router_config(&self.config.data_dir).await?;
let p = params.unwrap_or_default();
let host = p.get("host").and_then(|v| v.as_str()).map(|s| s.to_string())
.or_else(|| if saved.configured { Some(saved.address.clone()) } else { None })
.ok_or_else(|| anyhow::anyhow!("No router configured — provide host or call router.configure first"))?;
let ssh_user = p.get("ssh_user").and_then(|v| v.as_str()).map(|s| s.to_string())
.or_else(|| saved.username.clone()).unwrap_or_else(|| "root".to_string());
let ssh_password = p.get("ssh_password").and_then(|v| v.as_str()).map(|s| s.to_string())
.or_else(|| saved.password.clone()).unwrap_or_default();
let router = Router::connect_password(&host, 22, &ssh_user, &ssh_password)?;
router.verify_openwrt()?;
let networks = wifi_scan::scan_networks(&router)?;
let result: Vec<serde_json::Value> = networks
.iter()
.map(|n| serde_json::json!({
"ssid": n.ssid,
"bssid": n.bssid,
"signal": n.signal,
"channel": n.channel,
"encryption": n.encryption,
}))
.collect();
Ok(serde_json::json!({ "networks": result }))
}
/// Configure WAN/WISP — connect the router to an upstream WiFi network.
///
/// Params: host/credentials + `{ "ssid": "...", "password": "...", "encryption": "psk2" }`
pub(super) async fn handle_openwrt_configure_wan(
&self,
params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
let saved = net_router::load_router_config(&self.config.data_dir).await?;
let p = params.unwrap_or_default();
let host = p.get("host").and_then(|v| v.as_str()).map(|s| s.to_string())
.or_else(|| if saved.configured { Some(saved.address.clone()) } else { None })
.ok_or_else(|| anyhow::anyhow!("No router configured — provide host or call router.configure first"))?;
let ssh_user = p.get("ssh_user").and_then(|v| v.as_str()).map(|s| s.to_string())
.or_else(|| saved.username.clone()).unwrap_or_else(|| "root".to_string());
let ssh_password = p.get("ssh_password").and_then(|v| v.as_str()).map(|s| s.to_string())
.or_else(|| saved.password.clone()).unwrap_or_default();
let ssid = p.get("ssid").and_then(|v| v.as_str())
.ok_or_else(|| anyhow::anyhow!("Missing required field: ssid"))?.to_string();
let password = p.get("password").and_then(|v| v.as_str()).unwrap_or("").to_string();
let encryption = p.get("encryption").and_then(|v| v.as_str()).unwrap_or("psk2").to_string();
let dhcp_start = p.get("dhcp_start").and_then(|v| v.as_u64()).unwrap_or(100) as u32;
let dhcp_limit = p.get("dhcp_limit").and_then(|v| v.as_u64()).unwrap_or(150) as u32;
let masq = p.get("masq").and_then(|v| v.as_bool()).unwrap_or(true);
let router = Router::connect_password(&host, 22, &ssh_user, &ssh_password)?;
router.verify_openwrt()?;
let config = wan::WispConfig { ssid: ssid.clone(), password, encryption, dhcp_start, dhcp_limit, masq };
wan::configure_wisp(&router, &config)?;
Ok(serde_json::json!({ "ok": true, "host": host, "ssid": ssid }))
}
}
/// Parse /etc/openwrt_release key=value pairs into a JSON object.
fn parse_release(raw: &str) -> serde_json::Value {
let mut m = serde_json::Map::new();
for line in raw.lines() {
if let Some((k, v)) = line.split_once('=') {
m.insert(
k.to_lowercase(),
serde_json::Value::String(v.trim_matches('"').to_string()),
);
}
}
serde_json::Value::Object(m)
}
/// Extract AP wifi-iface sections from `uci show wireless` output.
fn parse_wifi_interfaces(raw: &str) -> Vec<serde_json::Value> {
use std::collections::HashMap;
let mut sections: HashMap<String, HashMap<String, String>> = HashMap::new();
for line in raw.lines() {
if let Some((lhs, rhs)) = line.trim().split_once('=') {
let parts: Vec<&str> = lhs.splitn(3, '.').collect();
if parts.len() == 3 && parts[0] == "wireless" {
sections
.entry(parts[1].to_string())
.or_default()
.insert(parts[2].to_string(), rhs.trim_matches('\'').to_string());
}
}
}
let mut ifaces: Vec<serde_json::Value> = sections
.into_iter()
.filter(|(_, f)| f.get("mode").map(|m| m == "ap").unwrap_or(false))
.map(|(name, f)| serde_json::json!({
"section": name,
"ssid": f.get("ssid").cloned().unwrap_or_default(),
"device": f.get("device").cloned().unwrap_or_default(),
"encryption": f.get("encryption").cloned().unwrap_or_else(|| "none".into()),
"network": f.get("network").cloned().unwrap_or_default(),
"disabled": f.get("disabled").map(|v| v == "1").unwrap_or(false),
}))
.collect();
ifaces.sort_by_key(|v| v["section"].as_str().unwrap_or("").to_string());
ifaces
}
fn parse_ipv4(s: &str) -> Result<[u8; 4]> {
let parts: Vec<&str> = s.split('.').collect();
if parts.len() != 4 {
anyhow::bail!("Invalid IPv4: {}", s);
}
Ok([
parts[0].parse()?,
parts[1].parse()?,
parts[2].parse()?,
parts[3].parse()?,
])
}

View File

@ -114,6 +114,31 @@ impl RpcHandler {
Err(e) => {
error!("package.install {} failed: {:#}", package_id_spawn, e);
install_log(&format!("INSTALL FAIL: {}{:#}", package_id_spawn, e)).await;
// Dependency-gate rejections happen BEFORE any resource
// (container/image/data dir) exists for this package, so
// keeping the optimistic entry would leave a phantom
// "Stopped" tile whose Start fails with `no such object`
// (the log-confirmed LND fresh-install failure). Remove
// the entry so the card reverts to installable, and
// surface the reason as a notification instead.
if let Some(gate) = e.downcast_ref::<super::dependencies::DependencyGateError>()
{
let (mut data, _) = handler.state_manager.get_snapshot().await;
data.package_data.remove(&package_id_spawn);
data.notifications.push(crate::data_model::Notification {
id: format!("install-deps-{package_id_spawn}"),
level: crate::data_model::NotificationLevel::Error,
title: format!("Could not install {package_id_spawn}"),
message: gate.to_string(),
timestamp: chrono::Utc::now().to_rfc3339(),
app_id: Some(package_id_spawn.clone()),
});
while data.notifications.len() > 20 {
data.notifications.remove(0);
}
handler.state_manager.update_data(data).await;
return;
}
// Don't remove the entry — that's what made the card
// vanish from My Apps mid-install / between retry-loop
// attempts (e.g. tailscale's entrypoint failure). Leave

View File

@ -94,35 +94,11 @@ async fn dynamic_app_config(
))
}
/// Trusted Docker registries. Only images from these sources are allowed.
#[allow(dead_code)]
pub(super) const TRUSTED_REGISTRIES: &[&str] = &[
"docker.io/",
"ghcr.io/",
"localhost/",
"git.tx1138.com/",
"146.59.87.168:3000/",
];
/// Validate Docker image against trusted registry allowlist.
/// Validate a Docker image reference. Delegates to the shared policy in
/// `container::image_policy` — the same rules the orchestrator enforces at
/// its pull sites, so the two layers can't drift apart.
pub(super) fn is_valid_docker_image(image: &str) -> bool {
if image.is_empty() || image.len() > 256 {
return false;
}
// Reject shell metacharacters
let dangerous_chars = ['&', '|', ';', '`', '$', '(', ')', '<', '>', '\n', '\r'];
if image.chars().any(|c| dangerous_chars.contains(&c)) {
return false;
}
// Must come from a trusted registry — match the exact domain, not just prefix
let registry = match image.split('/').next() {
Some(r) => r,
None => return false,
};
matches!(
registry,
"docker.io" | "ghcr.io" | "localhost" | "git.tx1138.com" | "146.59.87.168:3000"
)
crate::container::image_policy::is_valid_docker_image(image)
}
/// Per-app Linux capabilities needed beyond the default cap-drop=ALL.
@ -684,10 +660,15 @@ pub(super) async fn get_app_config(
),
"bitcoin-core" => (
vec![
"8332:8332".to_string(),
// RPC + ZMQ are auth-only/unauthenticated: host-local ONLY —
// never the LAN. In-node consumers dial the container's
// archy-net alias directly; never bind the archy-net gateway
// (10.89.0.1) — rootlessport can't hold it and podman run
// fails outright (2026-07-09, .228). P2P 8333 stays public.
"127.0.0.1:8332:8332".to_string(),
"8333:8333".to_string(),
"28332:28332".to_string(),
"28333:28333".to_string(),
"127.0.0.1:28332:28332".to_string(),
"127.0.0.1:28333:28333".to_string(),
],
vec!["/var/lib/archipelago/bitcoin:/home/bitcoin/.bitcoin".to_string()],
vec![],
@ -707,12 +688,17 @@ pub(super) async fn get_app_config(
// effectively pinned at 2 by --cpus=2 (now removed).
// -maxconnections=125 — default but explicit, so ops can
// tune downward on bandwidth-constrained nodes.
// Log volume: -printtoconsole=0 — bitcoind already writes
// debug.log in the datadir (self-shrunk on restart); echoing it
// to stdout too pushed every IBD "UpdateTip" line through
// conmon into journald (>1 GB/day on a fresh node). Deep
// debugging uses /var/lib/archipelago/bitcoin/debug.log.
Some(vec![
"-server=1".to_string(),
"-rpcbind=0.0.0.0".to_string(),
"-rpcallowip=0.0.0.0/0".to_string(),
"-rpcport=8332".to_string(),
"-printtoconsole=1".to_string(),
"-printtoconsole=0".to_string(),
"-datadir=/home/bitcoin/.bitcoin".to_string(),
format!("-dbcache={}", bitcoin_dbcache_mb()),
"-par=0".to_string(),
@ -721,10 +707,15 @@ pub(super) async fn get_app_config(
),
"bitcoin" | "bitcoin-knots" => (
vec![
"8332:8332".to_string(),
// RPC + ZMQ are auth-only/unauthenticated: host-local ONLY —
// never the LAN. In-node consumers dial the container's
// archy-net alias directly; never bind the archy-net gateway
// (10.89.0.1) — rootlessport can't hold it and podman run
// fails outright (2026-07-09, .228). P2P 8333 stays public.
"127.0.0.1:8332:8332".to_string(),
"8333:8333".to_string(),
"28332:28332".to_string(),
"28333:28333".to_string(),
"127.0.0.1:28332:28332".to_string(),
"127.0.0.1:28333:28333".to_string(),
],
vec!["/var/lib/archipelago/bitcoin:/home/bitcoin/.bitcoin".to_string()],
vec![],

View File

@ -58,6 +58,7 @@ fn archival_bitcoin_required_message(package_id: &str) -> String {
}
/// Snapshot of which dependency services are currently running.
#[derive(Debug)]
pub(super) struct RunningDeps {
pub has_bitcoin: bool,
pub has_electrumx: bool,
@ -227,6 +228,190 @@ pub(super) fn check_install_deps(package_id: &str, deps: &RunningDeps) -> Result
}
}
// ---------------------------------------------------------------------------
// Bounded dependency wait (install race fix)
// ---------------------------------------------------------------------------
//
// Confirmed race on fresh nodes: the user clicks "Install LND" while
// bitcoin-knots is itself still installing/starting. `check_install_deps`
// rejected instantly ("LND requires a running Bitcoin node…") even though
// Bitcoin came up 55s later. The fix: when the dependency is INSTALLED
// (container exists in `podman ps -a`, or the package state knows about it)
// but not Running yet, poll for up to DEP_WAIT_MAX_ATTEMPTS × DEP_WAIT_INTERVAL
// (~3 minutes) before failing, surfacing "Waiting for X to start…" via the
// install-progress message. If the dependency is not installed at all, fail
// fast with the canonical `check_install_deps` message — waiting can't help.
/// Poll interval while waiting for an installed dependency to start.
pub(super) const DEP_WAIT_INTERVAL: std::time::Duration = std::time::Duration::from_secs(5);
/// 36 × 5s = 3 minutes of bounded waiting.
pub(super) const DEP_WAIT_MAX_ATTEMPTS: u32 = 36;
/// Marker error: the install was rejected by the dependency gate BEFORE any
/// resource (container, image, data dir) was created for the package. The
/// async install wrapper (`async_lifecycle.rs`) downcasts to this to remove
/// the optimistic `Installing` state entry instead of leaving a phantom
/// "Stopped" tile whose Start fails with `no such object`.
#[derive(Debug)]
pub(in crate::api::rpc) struct DependencyGateError(pub String);
impl std::fmt::Display for DependencyGateError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.0)
}
}
impl std::error::Error for DependencyGateError {}
/// One unsatisfied install dependency: a user-facing label plus the container
/// name variants that would satisfy it.
struct MissingDep {
label: &'static str,
containers: &'static [&'static str],
}
/// Which dependencies `check_install_deps` would reject `package_id` over.
/// Must stay in lockstep with the match arms in `check_install_deps` (the
/// wait loop re-runs `check_install_deps` for the canonical error message).
fn missing_install_deps(package_id: &str, deps: &RunningDeps) -> Vec<MissingDep> {
const BITCOIN: MissingDep = MissingDep {
label: "Bitcoin",
containers: BITCOIN_NAMES,
};
const ELECTRUM: MissingDep = MissingDep {
label: "ElectrumX",
containers: ELECTRUM_NAMES,
};
let mut missing = Vec::new();
match package_id {
"electrumx" | "mempool-electrs" | "electrs" | "lnd" | "btcpay-server" | "btcpayserver" => {
if !deps.has_bitcoin {
missing.push(BITCOIN);
}
}
"mempool" | "mempool-web" => {
if !deps.has_bitcoin {
missing.push(BITCOIN);
}
if !deps.has_electrumx {
missing.push(ELECTRUM);
}
}
// fedimint deliberately absent: check_install_deps allows it without
// a local Bitcoin node (remote RPC configured in guardian setup).
_ => {}
}
missing
}
fn join_dep_labels(missing: &[MissingDep]) -> String {
missing
.iter()
.map(|d| d.label)
.collect::<Vec<_>>()
.join(" and ")
}
/// One snapshot of the dependency world, fed to [`wait_for_install_deps`].
pub(super) struct DepProbe {
/// Which dependency services are currently Running.
pub running: RunningDeps,
/// Container/package names that EXIST in any state — installed, but
/// possibly not running yet (`podman ps -a` package-state entries).
pub existing: Vec<String>,
}
/// All container names known to podman in any state (`podman ps -a`).
/// Conservative on probe failure: returns an empty list, which makes the
/// wait loop fall back to the pre-fix fail-fast behavior.
pub(super) async fn detect_existing_containers() -> Vec<String> {
let out = tokio::time::timeout(
std::time::Duration::from_secs(30),
tokio::process::Command::new("podman")
.args(["ps", "-a", "--format", "{{.Names}}"])
.output(),
)
.await;
match out {
Ok(Ok(o)) if o.status.success() => String::from_utf8_lossy(&o.stdout)
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect(),
_ => Vec::new(),
}
}
/// Bounded dependency gate. Returns the (satisfied) `RunningDeps` snapshot,
/// or a [`DependencyGateError`]:
/// - immediately, when a missing dependency is not installed at all
/// (canonical `check_install_deps` message), or
/// - after `max_attempts × interval`, when an installed dependency never
/// reached Running.
///
/// `probe` and `on_waiting` are injected so unit tests can drive the loop
/// without a podman runtime; production wires them to
/// `RpcHandler::dep_probe_for_install` / `set_install_message`.
pub(super) async fn wait_for_install_deps<P, PF, L, LF>(
package_id: &str,
mut probe: P,
mut on_waiting: L,
max_attempts: u32,
interval: std::time::Duration,
) -> Result<RunningDeps>
where
P: FnMut() -> PF,
PF: std::future::Future<Output = Result<DepProbe>>,
L: FnMut(String) -> LF,
LF: std::future::Future<Output = ()>,
{
let mut waited_attempts = 0u32;
loop {
let DepProbe { running, existing } = probe().await?;
let missing = missing_install_deps(package_id, &running);
if missing.is_empty() {
// Keep behavior in lockstep with the canonical gate (covers any
// future arm added there but not mirrored in missing_install_deps).
check_install_deps(package_id, &running)?;
return Ok(running);
}
// Fail fast if any missing dependency has no installed container
// under any name variant — waiting cannot satisfy it.
let some_dep_not_installed = missing
.iter()
.any(|dep| !dep.containers.iter().any(|c| existing.iter().any(|e| e == c)));
if some_dep_not_installed {
let msg = match check_install_deps(package_id, &running) {
Err(e) => e.to_string(),
Ok(()) => format!("{package_id} dependencies are not running"),
};
return Err(anyhow::Error::new(DependencyGateError(msg)));
}
if waited_attempts >= max_attempts {
let labels = join_dep_labels(&missing);
return Err(anyhow::Error::new(DependencyGateError(format!(
"{labels} is installed but did not reach the running state within \
{} seconds. Start {labels}, then install {package_id} again.",
u64::from(max_attempts) * interval.as_secs()
))));
}
waited_attempts += 1;
let labels = join_dep_labels(&missing);
if waited_attempts == 1 {
info!(
"Install {package_id}: dependency {labels} installed but not running yet — \
waiting up to {}s for it to start",
u64::from(max_attempts) * interval.as_secs()
);
}
on_waiting(format!("Waiting for {labels} to start…")).await;
tokio::time::sleep(interval).await;
}
}
/// ElectrumX and Mempool's Electrum backend need historical blocks from an
/// unpruned node while building their indexes. A pruned Bitcoin node can be
/// running and RPC-reachable but still leave them stuck with closed ports.
@ -287,7 +472,7 @@ pub(super) async fn check_bitcoin_pruning_compatibility(package_id: &str) -> Res
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
}
if detect_disk_gb() < ARCHIVAL_BITCOIN_DISK_GB {
if detect_disk_gb().await < ARCHIVAL_BITCOIN_DISK_GB {
anyhow::bail!(archival_bitcoin_required_message(package_id));
}
@ -312,10 +497,11 @@ fn check_blockchain_info_for_pruning(package_id: &str, json: &serde_json::Value)
Ok(())
}
fn detect_disk_gb() -> u64 {
let output = std::process::Command::new("df")
async fn detect_disk_gb() -> u64 {
let output = tokio::process::Command::new("df")
.args(["-BG", "/var/lib/archipelago"])
.output();
.output()
.await;
let Ok(output) = output else {
return u64::MAX;
};
@ -425,12 +611,33 @@ pub(super) async fn ordered_containers_for_start(package_id: &str) -> Result<Vec
/// error via `?`, every later member (the api + frontend) is then skipped,
/// leaving the stack down until the health monitor recovers it minutes later.
/// That was the source of mempool gate flakes #73 (frontend) / #74 (api).
/// Orchestrator APP ids for a multi-container stack, in dependency-start
/// order. Unlike `startup_order` (a union of CONTAINER-name variants across
/// install generations, sometimes including foreign dependencies), every
/// entry here is a real manifest app id the orchestrator can `start()` to
/// recreate the member from scratch.
///
/// Used when a stack has NO live containers: under quadlet, `package.stop`
/// removes every member container, so a subsequent `package.start` must
/// resurrect member-by-member. Falling back to just the package id left the
/// other members absent AND user-stopped (.228 indeedhub, 2026-07-09 — the
/// reconciler then correctly refused to revive them).
pub(super) fn stack_member_app_ids(package_id: &str) -> &'static [&'static str] {
// Canonical table moved to crate::app_ops (shared with the reconciler's
// in-flight-op guard).
crate::app_ops::stack_member_app_ids(package_id)
}
fn order_present_containers(package_id: &str, containers: Vec<String>) -> Vec<String> {
if containers.is_empty() {
// Nothing is live under any known name. Fall back to the package id so
// a single-container app whose container matches its id still gets one
// start attempt; multi-container stacks with no live members are
// surfaced as "no containers" by the caller's emptiness check.
// Nothing is live under any known name. For known stacks, resurrect
// every member via its app id (see stack_member_app_ids). Otherwise
// fall back to the package id so a single-container app whose
// container matches its id still gets one start attempt.
let members = stack_member_app_ids(package_id);
if !members.is_empty() {
return members.iter().map(|s| s.to_string()).collect();
}
return vec![package_id.to_string()];
}
let order = startup_order(package_id);
@ -546,10 +753,35 @@ mod tests {
}
#[test]
fn order_present_containers_empty_falls_back_to_package_id() {
fn order_present_containers_empty_resurrects_stack_members() {
// Under quadlet, package.stop removes every member container; the
// start fallback must name each member APP id so the orchestrator can
// recreate the whole stack (.228 indeedhub, 2026-07-09) — not just
// the umbrella/package id.
assert_eq!(
order_present_containers("mempool", vec![]),
vec!["mempool".to_string()]
vec!["archy-mempool-db", "mempool-api", "archy-mempool-web"]
);
assert_eq!(
order_present_containers("indeedhub", vec![]),
vec![
"indeedhub-postgres",
"indeedhub-redis",
"indeedhub-minio",
"indeedhub-relay",
"indeedhub-api",
"indeedhub-ffmpeg",
"indeedhub",
]
);
assert_eq!(
order_present_containers("immich", vec![]),
vec!["immich-postgres", "immich-redis", "immich"]
);
// Single-container apps keep the package-id fallback.
assert_eq!(
order_present_containers("vaultwarden", vec![]),
vec!["vaultwarden".to_string()]
);
}
@ -625,6 +857,218 @@ mod tests {
assert!(!manifest_declares_archival_bitcoin("does-not-exist"));
}
mod dep_wait {
use super::super::{wait_for_install_deps, DepProbe, DependencyGateError, RunningDeps};
use std::sync::atomic::{AtomicU32, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
fn deps(has_bitcoin: bool, has_electrumx: bool) -> RunningDeps {
RunningDeps {
has_bitcoin,
has_electrumx,
has_lnd: false,
}
}
fn probe(has_bitcoin: bool, has_electrumx: bool, existing: &[&str]) -> DepProbe {
DepProbe {
running: deps(has_bitcoin, has_electrumx),
existing: existing.iter().map(|s| s.to_string()).collect(),
}
}
/// Collects "Waiting for X to start…" labels emitted during the wait.
fn label_sink() -> (Arc<Mutex<Vec<String>>>, impl FnMut(String) -> std::future::Ready<()>)
{
let labels = Arc::new(Mutex::new(Vec::new()));
let sink = {
let labels = Arc::clone(&labels);
move |msg: String| {
labels.lock().unwrap().push(msg);
std::future::ready(())
}
};
(labels, sink)
}
#[tokio::test]
async fn passes_immediately_when_dependency_is_running() {
let (labels, sink) = label_sink();
let result = wait_for_install_deps(
"lnd",
|| async { Ok(probe(true, false, &["bitcoin-knots"])) },
sink,
3,
Duration::ZERO,
)
.await;
assert!(result.is_ok());
assert!(labels.lock().unwrap().is_empty(), "no waiting expected");
}
#[tokio::test]
async fn fails_fast_when_dependency_not_installed_at_all() {
let calls = AtomicU32::new(0);
let (labels, sink) = label_sink();
let err = wait_for_install_deps(
"lnd",
|| {
calls.fetch_add(1, Ordering::SeqCst);
async { Ok(probe(false, false, &["uptime-kuma"])) }
},
sink,
36,
Duration::ZERO,
)
.await
.unwrap_err();
// Single probe — no polling when waiting cannot help.
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert!(labels.lock().unwrap().is_empty());
// Canonical check_install_deps message, wrapped in the gate marker
// so async_lifecycle removes the optimistic Installing entry.
assert!(err.downcast_ref::<DependencyGateError>().is_some());
assert!(
err.to_string().contains("LND requires a running Bitcoin node"),
"unexpected message: {err}"
);
}
#[tokio::test]
async fn waits_while_installed_dependency_starts_then_passes() {
// Bitcoin container exists (installing/starting) but only reports
// Running from the 3rd probe onward — the log-confirmed LND race.
let calls = Arc::new(AtomicU32::new(0));
let (labels, sink) = label_sink();
let probe_calls = Arc::clone(&calls);
let result = wait_for_install_deps(
"lnd",
move || {
let n = probe_calls.fetch_add(1, Ordering::SeqCst);
async move { Ok(probe(n >= 2, false, &["bitcoin-knots"])) }
},
sink,
36,
Duration::ZERO,
)
.await;
assert!(result.is_ok(), "{result:?}");
assert_eq!(calls.load(Ordering::SeqCst), 3);
let labels = labels.lock().unwrap();
assert_eq!(labels.len(), 2, "one waiting label per polling attempt");
assert!(labels.iter().all(|l| l == "Waiting for Bitcoin to start…"));
}
#[tokio::test]
async fn times_out_when_installed_dependency_never_runs() {
let (labels, sink) = label_sink();
let err = wait_for_install_deps(
"lnd",
|| async { Ok(probe(false, false, &["bitcoin-knots"])) },
sink,
4,
Duration::ZERO,
)
.await
.unwrap_err();
assert!(err.downcast_ref::<DependencyGateError>().is_some());
assert!(
err.to_string()
.contains("did not reach the running state within 0 seconds"),
"unexpected message: {err}"
);
assert_eq!(labels.lock().unwrap().len(), 4);
}
#[tokio::test]
async fn mempool_waits_on_both_bitcoin_and_electrumx() {
let calls = Arc::new(AtomicU32::new(0));
let (labels, sink) = label_sink();
let probe_calls = Arc::clone(&calls);
let result = wait_for_install_deps(
"mempool",
move || {
let n = probe_calls.fetch_add(1, Ordering::SeqCst);
// Bitcoin comes up on probe 2, electrumx on probe 3.
async move { Ok(probe(n >= 1, n >= 2, &["bitcoin-knots", "electrumx"])) }
},
sink,
36,
Duration::ZERO,
)
.await;
assert!(result.is_ok(), "{result:?}");
let labels = labels.lock().unwrap();
assert_eq!(
labels.as_slice(),
&[
"Waiting for Bitcoin and ElectrumX to start…".to_string(),
"Waiting for ElectrumX to start…".to_string(),
]
);
}
#[tokio::test]
async fn mempool_fails_fast_when_one_dep_is_not_installed() {
// Bitcoin is installed (waiting could help) but ElectrumX is not
// installed at all — waiting can never satisfy the gate, so fail
// fast with the canonical message.
let (labels, sink) = label_sink();
let err = wait_for_install_deps(
"mempool",
|| async { Ok(probe(false, false, &["bitcoin-knots"])) },
sink,
36,
Duration::ZERO,
)
.await
.unwrap_err();
assert!(err.downcast_ref::<DependencyGateError>().is_some());
assert!(labels.lock().unwrap().is_empty());
assert!(
err.to_string().contains("Mempool requires"),
"unexpected message: {err}"
);
}
#[tokio::test]
async fn variant_container_names_count_as_installed() {
// bitcoin-core (not just bitcoin-knots) satisfies the "installed"
// check for the wait path.
let calls = Arc::new(AtomicU32::new(0));
let (_labels, sink) = label_sink();
let probe_calls = Arc::clone(&calls);
let result = wait_for_install_deps(
"electrumx",
move || {
let n = probe_calls.fetch_add(1, Ordering::SeqCst);
async move { Ok(probe(n >= 1, false, &["bitcoin-core"])) }
},
sink,
36,
Duration::ZERO,
)
.await;
assert!(result.is_ok(), "{result:?}");
}
#[tokio::test]
async fn apps_without_dependency_gate_pass_untouched() {
let (labels, sink) = label_sink();
let result = wait_for_install_deps(
"uptime-kuma",
|| async { Ok(probe(false, false, &[])) },
sink,
36,
Duration::ZERO,
)
.await;
assert!(result.is_ok());
assert!(labels.lock().unwrap().is_empty());
}
}
#[test]
fn mempool_api_is_directly_installable_and_covered_by_the_archival_gate() {
// `mempool-api` is a legitimate direct `package.install` target

View File

@ -3,9 +3,10 @@ use super::config::{
is_readonly_compatible, is_valid_docker_image,
};
use super::dependencies::{
check_bitcoin_pruning_compatibility, check_install_deps, configure_fedimint_lnd,
check_bitcoin_pruning_compatibility, configure_fedimint_lnd, detect_existing_containers,
detect_running_deps, detect_running_deps_from_package_data, log_optional_dep_info,
needs_archy_net, RunningDeps,
needs_archy_net, wait_for_install_deps, DepProbe, RunningDeps, DEP_WAIT_INTERVAL,
DEP_WAIT_MAX_ATTEMPTS,
};
use super::progress::parse_pull_progress;
use super::validation::validate_app_id;
@ -23,6 +24,12 @@ const IMAGE_INSPECT_TIMEOUT: Duration = Duration::from_secs(10);
/// Append a timestamped line to the persistent install log.
pub(in crate::api::rpc) async fn install_log(msg: &str) {
use tokio::io::AsyncWriteExt;
// Always mirror to tracing/journald: the file append below has been
// silently failing under the service sandbox (ProtectSystem=strict
// leaves /var/log/archipelago read-only — container-installs.log has
// been 0 bytes since April 2026), and these lifecycle breadcrumbs are
// the primary forensic trail for gate failures.
info!(target: "install_log", "{msg}");
let ts = chrono::Utc::now().format("%Y-%m-%d %H:%M:%S UTC");
let line = format!("[{}] {}\n", ts, msg);
if let Ok(mut f) = tokio::fs::OpenOptions::new()
@ -31,6 +38,8 @@ pub(in crate::api::rpc) async fn install_log(msg: &str) {
.open(INSTALL_LOG)
.await
{
// Fire-and-forget by design: install-log lines are advisory, and a
// per-line warn would spam the very log stream that's failing.
let _ = f.write_all(line.as_bytes()).await;
}
}
@ -265,8 +274,7 @@ impl RpcHandler {
.await;
if matches!(package_id, "mempool" | "mempool-web") {
let deps = self.running_deps_for_install(package_id).await?;
check_install_deps(package_id, &deps)?;
self.gate_install_deps(package_id).await?;
check_bitcoin_pruning_compatibility(package_id).await?;
}
@ -289,9 +297,11 @@ impl RpcHandler {
// Dependency checks. Prefer the scanner's cached package state so a
// congested Podman API does not turn an already-running dependency into
// a false install failure. Fall back to a bounded direct Podman probe
// only when the cache does not show the dependency.
let deps = self.running_deps_for_install(package_id).await?;
check_install_deps(package_id, &deps)?;
// only when the cache does not show the dependency. When the dependency
// is installed but not Running yet (the "clicked Install LND 55s before
// Bitcoin was up" race), wait up to ~3 minutes for it instead of
// failing instantly.
let deps = self.gate_install_deps(package_id).await?;
check_bitcoin_pruning_compatibility(package_id).await?;
log_optional_dep_info(package_id, &deps);
let repaired_bitcoin_conf =
@ -320,8 +330,20 @@ impl RpcHandler {
// mode).
// The adoption block is being phased out as apps move to the
// orchestrator path. Non-orchestrator apps still hit it.
let orchestrator_managed =
should_try_orchestrator_install(package_id, self.orchestrator.is_some());
let orchestrator_managed = match self.orchestrator.as_ref() {
// Migration allowlist OR any app whose manifest the orchestrator
// knows (disk or signed-catalog overlay). A manifest-driven app
// must never fall through to the legacy flow — it ignores the
// manifest entirely and creates a bare container with no ports or
// volumes (strfry, 2026-07-09).
Some(orch) => {
should_try_orchestrator_install(package_id, true)
|| orch
.knows_app(orchestrator_install_app_id(package_id))
.await
}
None => false,
};
// Check if container already exists (legacy adoption — non-orchestrator
// apps only).
@ -716,6 +738,24 @@ impl RpcHandler {
self.create_data_dirs(package_id, &volumes).await;
for port in &ports {
// Drop publishes whose bind address the host can't hold (e.g. the
// archy-net gateway 10.89.0.1 under rootless podman) — passing
// them through makes `podman run` itself fail and takes the app
// down. "ip:host:container" has 2 colons; plain "host:container"
// has 1 (no IPv6 binds in the legacy string table).
let bind = if port.matches(':').count() == 2 {
port.split(':').next().unwrap_or("")
} else {
""
};
if !archipelago_container::manifest::host_can_bind_publish_ip(bind) {
warn!(
app = %package_id,
publish = %port,
"dropping publish: bind address not assignable on this host"
);
continue;
}
run_args.push("-p");
run_args.push(port);
}
@ -945,6 +985,27 @@ impl RpcHandler {
}
}
/// Bounded dependency gate for installs: passes immediately when deps are
/// running, fails fast (with the phantom-tile marker) when a dependency
/// isn't installed at all, and otherwise waits up to
/// `DEP_WAIT_MAX_ATTEMPTS × DEP_WAIT_INTERVAL` for an installed-but-
/// starting dependency, surfacing "Waiting for X to start…" on the card.
pub(super) async fn gate_install_deps(&self, package_id: &str) -> Result<RunningDeps> {
wait_for_install_deps(
package_id,
|| async {
Ok(DepProbe {
running: self.running_deps_for_install(package_id).await?,
existing: detect_existing_containers().await,
})
},
|msg| async move { self.set_install_message(package_id, &msg).await },
DEP_WAIT_MAX_ATTEMPTS,
DEP_WAIT_INTERVAL,
)
.await
}
// -- Private helpers for install --
/// Pull the image from a registry or verify a local image exists.
@ -1295,6 +1356,11 @@ impl RpcHandler {
// Default to full archive — operators with 2TB+ drives shouldn't be
// silently pruned down to 550 MB. Users who want a pruned node can
// set `prune=N` in bitcoin.conf themselves after install.
//
// printtoconsole=0: bitcoind already writes debug.log in the datadir
// (self-shrunk on restart); duplicating it to stdout pushed every IBD
// "UpdateTip" line through conmon into journald (>1 GB/day). Deep
// debugging uses /var/lib/archipelago/bitcoin/debug.log.
let bitcoin_conf = format!(
"\
# rpcauth: salted hash only - no plaintext password in config or CLI\n\
@ -1304,7 +1370,7 @@ rpcallowip=0.0.0.0/0\n\
listen=1\n\
rpcthreads=16\n\
rpcworkqueue=256\n\
printtoconsole=1\n",
printtoconsole=0\n",
rpcauth_line
);
tokio::fs::create_dir_all(bitcoin_dir)
@ -2166,13 +2232,14 @@ async fn ensure_host_port_listener(
container_name: &str,
runtime_ports: &[String],
) -> Result<()> {
let Some(port) = runtime_ports
let mut port = runtime_ports
.first()
.and_then(|p| p.split(':').next())
.and_then(|p| p.parse::<u16>().ok())
.or_else(|| published_host_port(container_name))
.or_else(|| required_host_port(package_id))
else {
.and_then(|p| p.parse::<u16>().ok());
if port.is_none() {
port = published_host_port(container_name).await;
}
let Some(port) = port.or_else(|| required_host_port(package_id)) else {
return Ok(());
};
@ -2218,10 +2285,11 @@ async fn ensure_host_port_listener(
))
}
fn published_host_port(container_name: &str) -> Option<u16> {
let output = std::process::Command::new("podman")
async fn published_host_port(container_name: &str) -> Option<u16> {
let output = tokio::process::Command::new("podman")
.args(["port", container_name])
.output()
.await
.ok()?;
if !output.status.success() {
return None;

View File

@ -61,6 +61,31 @@ impl RpcHandler {
self.state_manager.update_data(data).await;
}
/// Set a user-facing install status message (e.g. "Waiting for Bitcoin
/// to start…") without disturbing the current phase/byte counters.
pub(super) async fn set_install_message(&self, package_id: &str, message: &str) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;
let entry = data
.package_data
.entry(package_id.to_string())
.or_insert_with(|| create_installing_entry(package_id));
if entry.state != PackageState::Updating {
entry.state = PackageState::Installing;
}
let (size, downloaded, phase) = entry
.install_progress
.as_ref()
.map(|p| (p.size, p.downloaded, p.phase))
.unwrap_or((0, 0, None));
entry.install_progress = Some(InstallProgress {
size,
downloaded,
phase,
message: Some(message.to_string()),
});
self.state_manager.update_data(data).await;
}
/// Clear install progress after pull completes or fails.
pub(super) async fn clear_install_progress(&self, package_id: &str) {
let (mut data, _rev) = self.state_manager.get_snapshot().await;

View File

@ -63,6 +63,8 @@ impl RpcHandler {
let to_start = if self.orchestrator.is_some() && uses_single_orchestrator_app(package_id) {
vec![orchestrator_app_id(package_id).to_string()]
} else if let Some(members) = orchestrator_stack_members(self.orchestrator.is_some(), package_id) {
members
} else {
ordered_containers_for_start(package_id).await?
};
@ -91,7 +93,10 @@ impl RpcHandler {
))
.await;
let op_lock = app_op_lock(package_id);
let data_dir = self.config.data_dir.clone();
tokio::spawn(async move {
let _op_guard = op_lock.lock().await;
let result = if let Some(orchestrator) = orchestrator.as_ref() {
do_orchestrator_package_start(orchestrator.as_ref(), &to_start).await
} else {
@ -102,6 +107,13 @@ impl RpcHandler {
reconcile_companions_for(&companion_app_id).await;
set_package_state(&state_manager, &package_id_owned, PackageState::Running)
.await;
cascade_restart_address_caching_dependents(
orchestrator.as_ref(),
&state_manager,
&data_dir,
&package_id_owned,
)
.await;
}
Err(e) => {
tracing::error!("package.start {} failed: {:#}", package_id_owned, e);
@ -151,6 +163,22 @@ impl RpcHandler {
return Err(anyhow::anyhow!("No containers found for {}", package_id));
}
// Orchestrator-managed stacks are stopped via member APP ids (reverse
// start order), never live container names: orchestrator.stop(app_id)
// stops the member's quadlet .service (systemd removes the --rm
// container), while a container NAME falls through the unknown-app-id
// fallback to a raw `podman stop` that races systemd over the unit
// (immich, gate 2026-07-09).
let to_stop_ids = if !single_orchestrator_app {
orchestrator_stack_members(self.orchestrator.is_some(), package_id)
.map(|mut members| {
members.reverse();
members
})
} else {
None
};
// Mark as user-stopped BEFORE the spawn so health monitor and
// crash recovery don't auto-restart mid-flight. Ordering is
// load-bearing — see runtime.rs:145-148 original note.
@ -158,9 +186,17 @@ impl RpcHandler {
for name in &containers {
crate::crash_recovery::mark_user_stopped(&self.config.data_dir, name).await;
}
// Stack members are marked under their app ids too, so the reconcile
// guard holds for members whose container is currently absent (a live
// container list can't name those).
if let Some(ids) = &to_stop_ids {
for id in ids {
crate::crash_recovery::mark_user_stopped(&self.config.data_dir, id).await;
}
}
let package_id_owned = package_id.to_string();
let to_stop = containers.clone();
let to_stop = to_stop_ids.unwrap_or_else(|| containers.clone());
let orchestrator = self.orchestrator.clone();
let state_manager = Arc::clone(&self.state_manager);
let pre_state =
@ -172,7 +208,9 @@ impl RpcHandler {
))
.await;
let op_lock = app_op_lock(package_id);
tokio::spawn(async move {
let _op_guard = op_lock.lock().await;
let result = if let Some(orchestrator) = orchestrator.as_ref() {
do_orchestrator_package_stop(orchestrator.as_ref(), &to_stop).await
} else {
@ -213,14 +251,21 @@ impl RpcHandler {
let single_orchestrator_app =
self.orchestrator.is_some() && uses_single_orchestrator_app(package_id);
let containers = if single_orchestrator_app {
let mut containers = if single_orchestrator_app {
vec![orchestrator_app_id(package_id).to_string()]
} else {
get_containers_for_app(package_id).await?
};
if containers.is_empty() {
tracing::warn!("package.restart {}: no containers found", package_id);
return Err(anyhow::anyhow!("No containers found for {}", package_id));
// A stack whose containers were all removed (quadlet stop) can
// still be restarted: resurrect it member-by-member, same as
// package.start's no-live-containers fallback.
let members = super::dependencies::stack_member_app_ids(package_id);
if members.is_empty() {
tracing::warn!("package.restart {}: no containers found", package_id);
return Err(anyhow::anyhow!("No containers found for {}", package_id));
}
containers = members.iter().map(|s| s.to_string()).collect();
}
// Restart does not mark user-stopped; user wants the app to keep
@ -235,6 +280,12 @@ impl RpcHandler {
let companion_app_id = package_id_owned.clone();
let to_restart = if single_orchestrator_app {
vec![orchestrator_app_id(package_id).to_string()]
} else if let Some(members) = orchestrator_stack_members(self.orchestrator.is_some(), package_id) {
// Restart stacks via member APP ids: restarting by live container
// name podman-stops the quadlet container (systemd --rm removes
// it) and the start half then finds no such container — a 5-min
// outage + RESTART FAIL on immich (gate 2026-07-09).
members
} else {
ordered_containers_for_start(package_id).await?
};
@ -249,7 +300,10 @@ impl RpcHandler {
))
.await;
let op_lock = app_op_lock(package_id);
let data_dir = self.config.data_dir.clone();
tokio::spawn(async move {
let _op_guard = op_lock.lock().await;
let result = if let Some(orchestrator) = orchestrator.as_ref() {
do_orchestrator_package_restart(orchestrator.as_ref(), &to_restart).await
} else {
@ -260,6 +314,13 @@ impl RpcHandler {
reconcile_companions_for(&companion_app_id).await;
set_package_state(&state_manager, &package_id_owned, PackageState::Running)
.await;
cascade_restart_address_caching_dependents(
orchestrator.as_ref(),
&state_manager,
&data_dir,
&package_id_owned,
)
.await;
}
Err(e) => {
tracing::error!("package.restart {} failed: {:#}", package_id_owned, e);
@ -765,7 +826,13 @@ async fn do_orchestrator_package_start(
match orchestrator.start(name).await {
Ok(()) => wait_after_orchestrator_start(name).await,
Err(e) if is_unknown_app_id_error(&e) => {
do_package_start(&[name.clone()]).await?;
// Collect instead of `?`: aborting here skipped every later
// stack member (mempool gate flakes #73/#74 — see
// order_present_containers).
if let Err(e) = do_package_start(&[name.clone()]).await {
tracing::error!(container = %name, error = %e, "fallback container start failed");
errors.push(format!("{}: {:#}", name, e));
}
}
Err(e) => {
tracing::error!(container = %name, error = %e, "orchestrator start failed");
@ -926,6 +993,8 @@ async fn inspect_runtime_container_state(container_name: &str) -> Result<Option<
fn is_missing_container_error(stderr: &str) -> bool {
stderr.contains("no such container")
|| stderr.contains("no container with name")
// podman 5.x `inspect` phrasing: `Error: no such object: "name"`
|| stderr.contains("no such object")
|| stderr.contains("does not exist")
|| stderr.contains("not found")
}
@ -1015,6 +1084,13 @@ async fn do_orchestrator_package_stop(
errors.push(format!("{}: {:#}", name, e));
}
}
// A member whose container is already gone IS stopped: quadlet
// stop removes the --rm container before the orchestrator's
// defensive `podman stop` fallback probes it, and a stack member
// can be absent outright (stopped earlier, or never revived).
Err(e) if is_missing_container_error(&format!("{:#}", e)) => {
tracing::debug!(container = %name, error = %e, "stop: container already absent — treating as stopped");
}
Err(e) => {
tracing::error!(container = %name, error = %e, "orchestrator stop failed");
errors.push(format!("{}: {:#}", name, e));
@ -1036,6 +1112,103 @@ fn orchestrator_app_id(package_id: &str) -> &str {
}
}
/// Member APP ids (start order) for an orchestrator-managed stack, or None
/// when there's no orchestrator / the id isn't a known stack. Lifecycle ops
/// on stacks must address members by app id so the orchestrator drives the
/// quadlet .service; addressing live container names falls through the
/// unknown-app-id fallback to raw podman stop/start, which races systemd's
/// --rm cleanup (the container is removed on stop, the start half then finds
/// nothing — immich restart, gate 2026-07-09).
fn orchestrator_stack_members(has_orchestrator: bool, package_id: &str) -> Option<Vec<String>> {
if !has_orchestrator {
return None;
}
let members = super::dependencies::stack_member_app_ids(package_id);
if members.is_empty() {
None
} else {
Some(members.iter().map(|s| s.to_string()).collect())
}
}
/// See crate::app_ops::address_caching_dependents — shared with the
/// reconciler, which cascades after its own recreates.
fn address_caching_dependents(package_id: &str) -> &'static [&'static str] {
crate::app_ops::address_caching_dependents(package_id)
}
/// After a backend start/restart lands, bounce its address-caching dependents
/// (see above). Runs inside the backend worker's op-lock scope and takes each
/// dependent's own op lock — lock order is always backend → dependent, and no
/// dependent op takes a backend lock, so the nesting cannot deadlock.
async fn cascade_restart_address_caching_dependents(
orchestrator: Option<&Arc<dyn crate::container::traits::ContainerOrchestrator>>,
state_manager: &crate::state::StateManager,
data_dir: &std::path::Path,
backend_id: &str,
) {
for dep in address_caching_dependents(backend_id) {
// A user-stopped or not-running dependent holds no live connection —
// starting it here would override an explicit operator decision.
if crate::crash_recovery::load_user_stopped(data_dir)
.await
.contains(*dep)
{
continue;
}
let running = match podman_control(&["ps", "--format", "{{.Names}}"]).await {
Ok(o) if o.status.success() => String::from_utf8_lossy(&o.stdout)
.lines()
.any(|l| l.trim() == *dep),
_ => false,
};
if !running {
continue;
}
install_log(&format!(
"CASCADE RESTART: {dep} (backend {backend_id} restarted; {dep} caches its resolved address)"
))
.await;
let op_lock = app_op_lock(dep);
let _op_guard = op_lock.lock().await;
let prev = flip_package_state(state_manager, dep, PackageState::Restarting).await;
let target = vec![dep.to_string()];
let result = if let Some(orch) = orchestrator {
do_orchestrator_package_restart(orch.as_ref(), &target).await
} else {
do_package_restart(&target).await
};
match result {
Ok(()) => {
reconcile_companions_for(dep).await;
set_package_state(state_manager, dep, PackageState::Running).await;
}
Err(e) => {
tracing::error!(dependent = %dep, backend = %backend_id, error = %e, "cascade restart failed");
install_log(&format!("CASCADE RESTART FAIL: {dep}{e:#}")).await;
if let Some(prev) = prev {
set_package_state(state_manager, dep, prev).await;
}
}
}
}
}
/// Per-app lifecycle-operation locks. package.start/stop/restart reply
/// immediately and run their multi-container sequences in spawned tasks;
/// unserialized, back-to-back RPCs interleave those sequences (gate run D,
/// 2026-07-09: package.start's member bring-up raced the still-running
/// package.stop's member shutdown — archy-mempool-db was stopped 2 seconds
/// after it started and the mempool stack finished with zero containers).
/// Workers take the app's lock as their first await; tokio's Mutex is fair
/// (FIFO), so queued operations run in RPC arrival order and the final
/// state matches the last request.
/// Registry lives in crate::app_ops so background actors (reconciler) can
/// probe in-flight ops without an api ↔ container dependency cycle.
fn app_op_lock(package_id: &str) -> Arc<tokio::sync::Mutex<()>> {
crate::app_ops::op_lock(orchestrator_app_id(package_id))
}
fn uses_single_orchestrator_app(package_id: &str) -> bool {
startup_order(package_id).is_empty()
&& matches!(
@ -1947,6 +2120,18 @@ pub(super) fn orchestrator_uninstall_app_ids(package_id: &str) -> Vec<String> {
"archy-btcpay-db".into(),
],
"fedimint" => vec!["fedimint".into(), "fedimint-gateway".into()],
// IndeedHub: 7-container stack; same rationale as immich below —
// without this, uninstalling "indeedhub" leaves the six companions
// enabled for the reconciler to resurrect forever.
"indeedhub" => vec![
"indeedhub-postgres".into(),
"indeedhub-redis".into(),
"indeedhub-minio".into(),
"indeedhub-relay".into(),
"indeedhub-api".into(),
"indeedhub-ffmpeg".into(),
"indeedhub".into(),
],
// Immich: multi-container stack, mirrors `immich_stack_app_ids` in
// stacks.rs. Without this, uninstalling "immich" only disabled the
// orchestrator-tracked "immich" app_id — "immich-postgres" and
@ -1966,6 +2151,22 @@ pub(super) fn orchestrator_uninstall_app_ids(package_id: &str) -> Vec<String> {
mod tests {
use super::*;
#[test]
fn missing_container_classifier_covers_podman5_phrasings() {
// Regression (.228 gate 2026-07-08): podman 5.x `inspect` on a missing
// container says `Error: no such object: "mempool"` — the classifier
// missed it, so a plain missing container surfaced as a hard inspect
// error and package.start aborted instead of proceeding to recreate.
assert!(is_missing_container_error(
"Error: no such object: \"mempool\""
));
assert!(is_missing_container_error("Error: no such container mempool"));
assert!(is_missing_container_error(
"Error: no container with name or id \"x\" found"
));
assert!(!is_missing_container_error("Error: OCI runtime error"));
}
#[test]
fn runtime_host_ports_are_manifest_derived_for_public_apps() {
assert_eq!(runtime_host_ports("photoprism"), vec![2342]);
@ -1978,6 +2179,25 @@ mod tests {
assert_eq!(runtime_host_ports("gitea"), vec![3001, 2222, 3000]);
}
#[test]
fn indeedhub_uninstall_covers_every_sibling_orchestrator_app_id() {
let ids = orchestrator_uninstall_app_ids("indeedhub");
for expected in [
"indeedhub-postgres",
"indeedhub-redis",
"indeedhub-minio",
"indeedhub-relay",
"indeedhub-api",
"indeedhub-ffmpeg",
"indeedhub",
] {
assert!(
ids.iter().any(|id| id == expected),
"missing {expected} in {ids:?}"
);
}
}
#[test]
fn immich_uninstall_covers_every_sibling_orchestrator_app_id() {
// Regression: uninstalling "immich" used to only disable the

View File

@ -130,6 +130,8 @@ async fn wait_for_stack_container_absent(container_name: &str, timeout: Duration
fn is_missing_container_error(stderr: &str) -> bool {
stderr.contains("no such container")
|| stderr.contains("no container with name")
// podman 5.x `inspect` phrasing: `Error: no such object: "name"`
|| stderr.contains("no such object")
|| stderr.contains("does not exist")
|| stderr.contains("not found")
}
@ -1009,9 +1011,9 @@ impl RpcHandler {
return Ok(adopted);
}
// Dependency check: Bitcoin must be running
let deps = super::dependencies::detect_running_deps().await?;
super::dependencies::check_install_deps("btcpay-server", &deps)?;
// Dependency check: Bitcoin must be running. Bounded wait covers the
// "installed but still starting" race instead of failing instantly.
self.gate_install_deps("btcpay-server").await?;
install_log("INSTALL START: btcpay-server (stack: postgres + nbxplorer + btcpay)").await;

View File

@ -172,18 +172,36 @@ impl RpcHandler {
/// Manual "check for updates": refresh the remote app catalog now. The
/// package scanner recomputes each app's `available-update` from the fresh
/// catalog on its next cycle and pushes it to the UI. Best-effort — a fetch
/// failure leaves the cached catalog in place and reports `refreshed: false`.
/// catalog on its next cycle and pushes it to the UI. When the catalog
/// bytes changed, the orchestrator's manifest overlay is reloaded in the
/// same call so catalog-shipped manifest fixes apply without a service
/// restart. Best-effort — a fetch failure leaves the cached catalog in
/// place and reports `refreshed: false`.
pub(in crate::api::rpc) async fn handle_package_check_updates(
&self,
_params: Option<serde_json::Value>,
) -> Result<serde_json::Value> {
match crate::container::app_catalog::refresh_catalog(&self.config.data_dir).await {
Ok(count) => Ok(serde_json::json!({
"status": "ok",
"refreshed": true,
"catalog_apps": count,
})),
Ok(refresh) => {
let mut manifests_reloaded = serde_json::Value::Null;
if refresh.changed {
if let Some(orch) = &self.orchestrator {
match orch.reload_manifests().await {
Ok(n) => manifests_reloaded = serde_json::json!(n),
Err(e) => tracing::warn!(
"check-updates: manifest reload after catalog change failed: {e}"
),
}
}
}
Ok(serde_json::json!({
"status": "ok",
"refreshed": true,
"catalog_apps": refresh.apps,
"catalog_changed": refresh.changed,
"manifests_reloaded": manifests_reloaded,
}))
}
Err(e) => Ok(serde_json::json!({
"status": "ok",
"refreshed": false,

View File

@ -26,6 +26,36 @@ impl Drop for OnboardingMnemonicState {
const MNEMONIC_TTL: std::time::Duration = std::time::Duration::from_secs(600); // 10 minutes
/// Persist the pending onboarding mnemonic as `identity/master_seed.enc`,
/// encrypted with `passphrase`. Called from `auth.setup` — the first moment a
/// user password exists — so "Reveal recovery phrase" works after onboarding
/// without the frontend having to remember a separate save step (it never
/// did, which left every onboarded node with no encrypted seed backup).
///
/// Deliberately ignores MNEMONIC_TTL: the mnemonic stays in memory until
/// overwritten regardless, so using it here widens nothing, and onboarding
/// legitimately takes longer than 10 minutes when the user carefully writes
/// down 24 words. Clears the in-memory copy on success — password setup is
/// the end of onboarding, so the plaintext no longer needs to linger.
///
/// Returns Ok(true) if a seed was saved, Ok(false) if none was pending.
pub(in crate::api::rpc) async fn save_pending_seed_encrypted(
data_dir: &std::path::Path,
passphrase: &str,
) -> Result<bool> {
let mut state = ONBOARDING_MNEMONIC.lock().await;
let Some(pending) = state.as_ref() else {
return Ok(false);
};
let mnemonic: bip39::Mnemonic = pending
.words
.parse()
.context("Invalid mnemonic in memory")?;
crate::seed::save_seed_encrypted(data_dir, &mnemonic, passphrase).await?;
*state = None;
Ok(true)
}
/// Best-effort: install fips.yaml + start archipelago-fips.service after the
/// seed onboarding has written the fips_key to disk. Runs in a detached task
/// so the user-facing RPC returns immediately — the systemctl calls can take
@ -208,6 +238,17 @@ impl RpcHandler {
let phrase = words.join(" ");
let (_mnemonic, seed) = crate::seed::MasterSeed::from_mnemonic_words(&phrase)?;
// Stash the restored words like seed.generate does, so auth.setup can
// persist the encrypted backup once the user's password exists and
// "Reveal recovery phrase" works on restored nodes too.
{
let mut state = ONBOARDING_MNEMONIC.lock().await;
*state = Some(OnboardingMnemonicState {
words: phrase.clone(),
created_at: std::time::Instant::now(),
});
}
// Derive and write node Ed25519 key.
let identity_dir = self.config.data_dir.join("identity");
crate::identity::NodeIdentity::from_seed(&identity_dir, &seed).await?;

View File

@ -575,7 +575,7 @@ impl RpcHandler {
// Restart the service via systemd
tokio::spawn(async {
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
let _ = std::process::Command::new("sudo")
let _ = tokio::process::Command::new("sudo")
.args(["systemctl", "restart", "archipelago"])
.spawn();
});

View File

@ -0,0 +1,128 @@
//! Cross-layer registry of per-app lifecycle-operation locks and stack
//! membership.
//!
//! The RPC layer's package.start/stop/restart workers serialize through
//! these locks (FIFO, see api::rpc::package::runtime). Background actors
//! (the reconciler; eventually the health monitor) must NOT act on an app
//! while a lifecycle op is mid-sequence: the reconciler once saw a stack
//! member "missing" between a restart worker's stop and start halves and
//! repair-recreated it behind systemd's back, killing the worker's fresh
//! container and leaving the unit down for minutes (.228 mempool frontend,
//! gate 2026-07-09). This module lives outside both layers so each can
//! consult the same state without an api ↔ container dependency cycle.
use std::sync::Arc;
static APP_OP_LOCKS: std::sync::LazyLock<
std::sync::Mutex<std::collections::HashMap<String, Arc<tokio::sync::Mutex<()>>>>,
> = std::sync::LazyLock::new(Default::default);
/// The per-app lifecycle-operation lock for a (normalized) app key. Workers
/// take this as their first await; tokio's Mutex is fair (FIFO), so queued
/// operations run in RPC arrival order and the final state matches the last
/// request.
pub fn op_lock(app_key: &str) -> Arc<tokio::sync::Mutex<()>> {
APP_OP_LOCKS
.lock()
.expect("APP_OP_LOCKS poisoned")
.entry(app_key.to_string())
.or_default()
.clone()
}
/// Member APP ids (start order) for orchestrator-managed stacks. Every entry
/// is a real manifest app id the orchestrator can `start()`/`stop()` so the
/// quadlet .service is driven instead of raw podman racing systemd's --rm
/// cleanup. Single source of truth — the RPC layer re-exports this.
pub fn stack_member_app_ids(package_id: &str) -> &'static [&'static str] {
match package_id {
"immich" => &["immich-postgres", "immich-redis", "immich"],
"indeedhub" => &[
"indeedhub-postgres",
"indeedhub-redis",
"indeedhub-minio",
"indeedhub-relay",
"indeedhub-api",
"indeedhub-ffmpeg",
"indeedhub",
],
"btcpay-server" | "btcpayserver" | "btcpay" => {
&["archy-btcpay-db", "archy-nbxplorer", "btcpay-server"]
}
"netbird" => &["netbird-server", "netbird-dashboard", "netbird"],
// The legacy umbrella id maps to the split stack (the orchestrator's
// umbrella alias handles this too; listing it here keeps the RPC
// layer's fan-out explicit).
"mempool" | "mempool-web" => {
&["archy-mempool-db", "mempool-api", "archy-mempool-web"]
}
_ => &[],
}
}
/// Dependents that resolve a backend's container address once at startup and
/// hold it: moving the backend's IP (restart OR recreate) strands them until
/// they restart too. lnd dials the bitcoin RPC address it resolved at boot
/// and never re-resolves (gate lnd getinfo test, .228 2026-07-09; hardening
/// plan §C). The RPC start/restart workers and the reconciler both consult
/// this — single source of truth, like the stack table above.
pub fn address_caching_dependents(package_id: &str) -> &'static [&'static str] {
match package_id {
"bitcoin" | "bitcoin-core" | "bitcoin-knots" => &["lnd"],
_ => &[],
}
}
/// The package whose lifecycle lock covers `app_id`: the stack package when
/// `app_id` is a member (RPC ops on "mempool" hold the "mempool" lock while
/// they drive archy-mempool-web), otherwise the app itself.
fn owning_package(app_id: &str) -> &str {
const STACKS: &[&str] = &["immich", "indeedhub", "btcpay-server", "netbird", "mempool"];
for stack in STACKS {
if stack_member_app_ids(stack).contains(&app_id) {
return stack;
}
}
app_id
}
/// True when a package.start/stop/restart worker currently holds the
/// lifecycle lock covering `app_id` (under its own key or its owning stack
/// package's key). Background actors use this to skip the app for a cycle
/// instead of interleaving with the worker's multi-step sequence. try_lock
/// on a fair tokio Mutex is non-blocking and does not queue.
pub fn lifecycle_op_in_flight(app_id: &str) -> bool {
let keys = [app_id, owning_package(app_id)];
for key in keys {
let lock = op_lock(key);
let held = lock.try_lock().is_err();
if held {
return true;
}
}
false
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn owning_package_maps_members_to_stack() {
assert_eq!(owning_package("archy-mempool-web"), "mempool");
assert_eq!(owning_package("immich-postgres"), "immich");
assert_eq!(owning_package("indeedhub-relay"), "indeedhub");
assert_eq!(owning_package("archy-nbxplorer"), "btcpay-server");
assert_eq!(owning_package("lnd"), "lnd");
}
#[tokio::test]
async fn in_flight_reflects_held_package_lock() {
assert!(!lifecycle_op_in_flight("archy-mempool-web"));
let lock = op_lock("mempool");
let _guard = lock.lock().await;
assert!(lifecycle_op_in_flight("archy-mempool-web"));
assert!(lifecycle_op_in_flight("mempool"));
assert!(!lifecycle_op_in_flight("jellyfin"));
}
}

View File

@ -101,19 +101,45 @@ fn friendly_transient_error(has_cached_state: bool, err_msg: &str) -> String {
.trim_end_matches('.');
let lower = detail.to_lowercase();
let state = if lower.contains("verifying blocks") {
"verifying blocks after restart"
Some("verifying blocks after restart")
} else if lower.contains("connection reset") {
Some("starting up and not yet accepting RPC connections")
} else if lower.contains("connection refused") || lower.contains("tcp connect error") {
"waiting for the Bitcoin RPC listener"
Some("waiting for the Bitcoin RPC listener")
} else if lower.contains("timed out") || lower.contains("timeout") {
"busy and not answering RPC before the timeout"
Some("busy and not answering RPC before the timeout")
} else {
"starting or busy syncing"
None
};
if has_cached_state {
format!("Bitcoin node is {state}; showing last known state and retrying. Detail: {detail}")
// Recognized transient causes get a clean human sentence only — the raw
// transport error (URLs, repeated "os error 104" chains) is operator
// noise that was ending up verbatim on the app card. Unrecognized errors
// keep a bounded detail so a genuinely new failure stays diagnosable.
let (state, detail) = match state {
Some(state) => (state, None),
None => (
"starting or busy syncing",
Some(if detail.len() > 120 {
let mut cut = 120;
while !detail.is_char_boundary(cut) {
cut -= 1;
}
format!("{}", &detail[..cut])
} else {
detail.to_string()
}),
),
};
let base = if has_cached_state {
format!("Bitcoin node is {state}; showing last known state and retrying.")
} else {
format!("Bitcoin node is {state}; retrying automatically. Detail: {detail}")
format!("Bitcoin node is {state}; retrying automatically.")
};
match detail {
Some(detail) => format!("{base} Detail: {detail}"),
None => base,
}
}
@ -278,4 +304,39 @@ mod tests {
assert!(msg.contains("busy and not answering RPC before the timeout"));
}
#[test]
fn connection_reset_gets_clean_message_without_raw_detail() {
// The exact string a fresh install showed on the app card: the raw
// reqwest chain (URL + repeated "os error 104") must not surface.
let msg = friendly_transient_error(
false,
"getblockchaininfo: Bitcoin RPC request failed: error sending request for url (http://127.0.0.1:8332/): connection error: Connection reset by peer (os error 104): connection error: Connection reset by peer (os error 104): Connection reset by peer (os error 104)",
);
assert!(msg.contains("starting up and not yet accepting RPC connections"));
assert!(!msg.contains("os error"));
assert!(!msg.contains("127.0.0.1"));
assert!(!msg.contains("Detail:"));
}
#[test]
fn recognized_causes_omit_detail_entirely() {
for raw in [
"x: Connection refused (os error 111)",
"x: operation timed out",
r#"x: {"error":{"code":-28,"message":"Verifying blocks..."}}"#,
] {
let msg = friendly_transient_error(false, raw);
assert!(!msg.contains("Detail:"), "leaked detail for: {raw}");
}
}
#[test]
fn unknown_errors_keep_bounded_detail() {
let long = format!("weird new failure {}", "x".repeat(300));
let msg = friendly_transient_error(false, &long);
assert!(msg.contains("Detail: weird new failure"));
assert!(msg.len() < 260);
}
}

View File

@ -39,6 +39,16 @@ const KIOSK_LAUNCHER: &str =
const KIOSK_SERVICE_PATH: &str = "/etc/systemd/system/archipelago-kiosk.service";
const KIOSK_LAUNCHER_PATH: &str = "/usr/local/bin/archipelago-kiosk-launcher";
// Journald log-volume policy (size cap + per-service rate limit). Fresh ISOs
// write the identical file at build time (image-recipe/_archived/
// build-auto-installer-iso.sh); this heals already-deployed nodes via OTA.
// A fresh node produced >1 GB/day of journal (bitcoind IBD console spam plus
// debug-level backend logging) — the cap bounds disk use and the rate limit
// keeps one chatty service from drowning everything else.
const JOURNALD_DROPIN: &str =
include_str!("../../../image-recipe/configs/journald-archipelago.conf");
const JOURNALD_DROPIN_PATH: &str = "/etc/systemd/journald.conf.d/10-archipelago-persistent.conf";
const NGINX_CONF_PATH: &str = "/etc/nginx/sites-available/archipelago";
const NGINX_ENABLED_CONF_PATH: &str = "/etc/nginx/sites-enabled/archipelago";
/// Per-app proxy snippet included by the HTTPS (:443) server block. Carries its
@ -120,6 +130,11 @@ pub async fn ensure_doctor_installed() {
Ok(false) => debug!("Bitcoin RPC bind settings already usable"),
Err(e) => warn!("Bitcoin RPC repair failed (non-fatal): {:#}", e),
}
match run_journald_dropin().await {
Ok(true) => info!("Installed journald log-volume policy drop-in"),
Ok(false) => debug!("journald log-volume policy already in place"),
Err(e) => warn!("journald drop-in bootstrap failed (non-fatal): {:#}", e),
}
match tighten_secrets_dir().await {
Ok(n) if n > 0 => info!(tightened = n, "Tightened mode on secret files"),
Ok(_) => debug!("Secrets directory already at expected mode"),
@ -408,6 +423,14 @@ ensure_line() {
ensure_line server=1
ensure_line rpcallowip=0.0.0.0/0
ensure_line listen=1
# Log-volume fix: printtoconsole=1 duplicated every log line (incl. per-block
# IBD "UpdateTip" spam) into journald via conmon on top of the datadir
# debug.log bitcoind already writes. Console off; debug.log stays (bitcoind
# self-shrinks it on restart).
if grep -q '^printtoconsole=1' "$conf"; then
sed -i 's/^printtoconsole=1$/printtoconsole=0/' "$conf"
changed=1
fi
[ "$changed" -eq 0 ] && exit 0
exit 2
"#;
@ -428,6 +451,44 @@ exit 2
}
}
/// Install the journald log-volume policy drop-in (JOURNALD_DROPIN) so nodes
/// deployed before the ISO shipped it get the size cap + rate limit via OTA.
/// Idempotent; restarts journald only when the file actually changed (safe:
/// the sockets are held by pid1, so at most a few messages queue briefly).
async fn run_journald_dropin() -> Result<bool> {
// Same dev-box guards as the doctor bootstrap: never touch /etc on
// contributors' laptops (symlinked or absent /home/archipelago/archy).
let home_archy = Path::new("/home/archipelago/archy");
if fs::symlink_metadata(home_archy)
.await
.map(|m| m.file_type().is_symlink())
.unwrap_or(false)
{
debug!("/home/archipelago/archy is a symlink — skipping journald bootstrap (dev box)");
return Ok(false);
}
if fs::metadata(home_archy).await.is_err() {
debug!("/home/archipelago/archy missing — skipping journald bootstrap");
return Ok(false);
}
let dropin_dir = "/etc/systemd/journald.conf.d";
let status = host_sudo(&["mkdir", "-p", dropin_dir])
.await
.with_context(|| format!("mkdir {}", dropin_dir))?;
if !status.success() {
anyhow::bail!("mkdir {} exited with {}", dropin_dir, status);
}
let changed = write_root_if_needed(JOURNALD_DROPIN_PATH, JOURNALD_DROPIN).await?;
if changed {
if let Err(e) = host_sudo(&["systemctl", "restart", "systemd-journald"]).await {
warn!("journald restart after drop-in update failed: {:#}", e);
}
}
Ok(changed)
}
async fn run() -> Result<bool> {
// Dev-box guard: on contributors' laptops `/home/archipelago/archy` is
// typically a symlink into the git checkout, and writing through it

View File

@ -19,6 +19,11 @@
//! Sign a JSON document (e.g. releases/app-catalog.json) in place: insert
//! `signature` + `signed_by` over the canonical form, matching exactly
//! what `trust::verify_detached` recomputes on every node.
//!
//! archipelago ceremony verify <file.json>
//! Verify a signed JSON document against the compiled-in release-root
//! anchor. Exits non-zero unless the signature verifies AND the signer
//! is the pinned anchor. Needs no mnemonic — used as the publish gate.
//! ```
use anyhow::{bail, Context, Result};
@ -47,9 +52,15 @@ pub fn run() -> Result<()> {
.context("usage: archipelago ceremony sign <file.json>")?;
cmd_sign(&file)
}
"verify" => {
let file = std::env::args()
.nth(3)
.context("usage: archipelago ceremony verify <file.json>")?;
cmd_verify(&file)
}
other => {
bail!(
"unknown ceremony subcommand {:?}; expected gen | pubkey | sign <file>",
"unknown ceremony subcommand {:?}; expected gen | pubkey | sign <file> | verify <file>",
other
)
}
@ -107,6 +118,33 @@ fn cmd_sign(path: &str) -> Result<()> {
Ok(())
}
fn cmd_verify(path: &str) -> Result<()> {
let body = std::fs::read_to_string(path).with_context(|| format!("read {path}"))?;
let value: serde_json::Value =
serde_json::from_str(&body).with_context(|| format!("parse {path} as JSON"))?;
match signed_doc::verify_detached(&value)? {
signed_doc::SignatureStatus::Verified {
signer_did,
anchored: true,
} => {
eprintln!("{path} verified — signed by the pinned release root");
eprintln!(" signed_by: {signer_did}");
Ok(())
}
signed_doc::SignatureStatus::Verified {
signer_did,
anchored: false,
} => {
// Only reachable if no anchor is compiled in/overridden — the
// signature is self-consistent but proves nothing about identity.
bail!("{path} signed by {signer_did}, but no release-root anchor is pinned to compare against")
}
signed_doc::SignatureStatus::Unsigned => {
bail!("{path} is NOT signed (no `signature` field)")
}
}
}
/// Derive the release-root signing key from the mnemonic in env/stdin.
fn load_release_root_key() -> Result<SigningKey> {
let phrase = read_mnemonic()?;

View File

@ -81,10 +81,11 @@ pub struct Config {
impl Config {
/// Detect primary host IP (first non-loopback IPv4)
fn detect_host_ip() -> Result<String> {
let output = std::process::Command::new("hostname")
async fn detect_host_ip() -> Result<String> {
let output = tokio::process::Command::new("hostname")
.args(["-I"])
.output()
.await
.context("Failed to run hostname -I")?;
let s = String::from_utf8_lossy(&output.stdout);
let ip = s
@ -210,7 +211,9 @@ impl Config {
if let Ok(ip) = std::env::var("ARCHIPELAGO_HOST_IP") {
config.host_ip = ip;
} else {
config.host_ip = Self::detect_host_ip().unwrap_or_else(|_| "127.0.0.1".to_string());
config.host_ip = Self::detect_host_ip()
.await
.unwrap_or_else(|_| "127.0.0.1".to_string());
}
// Ensure data directory exists

View File

@ -339,18 +339,31 @@ fn catalog_urls_from_mirrors(mirrors: &[crate::update::UpdateMirror]) -> Vec<Str
urls
}
/// Outcome of [`refresh_catalog`]: the app count of the fetched catalog and
/// whether the cached bytes actually changed. `changed` drives the manifest-
/// overlay reload — catalog manifests only take effect once `load_manifests`
/// re-runs, and reloading on every unchanged hourly poll would be pure churn.
#[derive(Debug, Clone, Copy)]
pub struct CatalogRefresh {
pub apps: usize,
pub changed: bool,
}
/// Fetch the catalog from the first reachable mirror and atomically write it to
/// `<data_dir>/app-catalog.json`. Returns the number of apps in the catalog on
/// success. Best-effort: a fetch failure leaves the existing cache untouched
/// `<data_dir>/app-catalog.json`. Returns the app count and whether the cache
/// changed. Best-effort: a fetch failure leaves the existing cache untouched
/// (origin-always-wins; updates simply aren't refreshed this cycle).
pub async fn refresh_catalog(data_dir: &Path) -> anyhow::Result<usize> {
pub async fn refresh_catalog(data_dir: &Path) -> anyhow::Result<CatalogRefresh> {
let mirrors = crate::update::load_mirrors(data_dir)
.await
.unwrap_or_default();
let urls = catalog_urls_from_mirrors(&mirrors);
if urls.is_empty() {
debug!("app-catalog: no mirror-derived URLs to fetch from");
return Ok(0);
return Ok(CatalogRefresh {
apps: 0,
changed: false,
});
}
let client = reqwest::Client::builder()
@ -360,13 +373,23 @@ pub async fn refresh_catalog(data_dir: &Path) -> anyhow::Result<usize> {
let mut last_err: Option<anyhow::Error> = None;
for url in &urls {
match fetch_one(&client, url).await {
Ok(catalog) => {
Ok((catalog, body)) => {
let count = catalog.apps.len();
write_cache(data_dir, &catalog)?;
// Invalidate the in-process cache so the next read re-parses.
*CACHE.lock().unwrap() = None;
info!("app-catalog: refreshed from {} ({} apps)", url, count);
return Ok(count);
let changed = write_cache(data_dir, &body)?;
if changed {
// Invalidate the in-process cache so the next read re-parses.
*CACHE.lock().unwrap() = None;
}
info!(
"app-catalog: refreshed from {} ({} apps{})",
url,
count,
if changed { ", changed" } else { ", unchanged" }
);
return Ok(CatalogRefresh {
apps: count,
changed,
});
}
Err(e) => {
debug!("app-catalog: fetch {} failed: {}", url, e);
@ -377,7 +400,10 @@ pub async fn refresh_catalog(data_dir: &Path) -> anyhow::Result<usize> {
Err(last_err.unwrap_or_else(|| anyhow::anyhow!("no catalog mirrors reachable")))
}
async fn fetch_one(client: &reqwest::Client, url: &str) -> anyhow::Result<AppCatalog> {
async fn fetch_one(
client: &reqwest::Client,
url: &str,
) -> anyhow::Result<(AppCatalog, String)> {
let resp = client.get(url).send().await?;
if !resp.status().is_success() {
anyhow::bail!("HTTP {}", resp.status());
@ -416,16 +442,29 @@ async fn fetch_one(client: &reqwest::Client, url: &str) -> anyhow::Result<AppCat
}
}
Ok(catalog)
Ok((catalog, body))
}
fn write_cache(data_dir: &Path, catalog: &AppCatalog) -> anyhow::Result<()> {
/// Atomically write the catalog cache. Caches the RAW fetched bytes — not a
/// re-serialization of the typed struct — for two reasons: the struct's
/// `apps` HashMap serializes in nondeterministic order (a re-serialized
/// comparison would report "changed" on every poll), and the raw bytes are
/// the signed preimage, so the cache stays signature-verifiable. Returns
/// `false` (skipping the write) when the bytes are identical to what's
/// already cached, so callers can tell a genuine catalog change from an
/// unchanged poll.
fn write_cache(data_dir: &Path, body: &str) -> anyhow::Result<bool> {
let dest = data_dir.join(APP_CATALOG_FILE);
if std::fs::read_to_string(&dest)
.map(|current| current == body)
.unwrap_or(false)
{
return Ok(false);
}
let tmp = data_dir.join(format!("{}.tmp", APP_CATALOG_FILE));
let json = serde_json::to_string_pretty(catalog)?;
std::fs::write(&tmp, json)?;
std::fs::write(&tmp, body)?;
std::fs::rename(&tmp, &dest)?;
Ok(())
Ok(true)
}
#[cfg(test)]
@ -462,6 +501,32 @@ mod tests {
assert_eq!(e.digest.as_deref(), Some("blake3:deadbeef"));
}
#[test]
fn write_cache_reports_changed_only_on_new_bytes() {
// The changed flag gates the runtime manifest-overlay reload: an
// unchanged hourly poll must NOT report changed (or every tick would
// rebuild the manifest map), while a genuinely new catalog must. Raw
// fetched bytes are compared — a re-serialized comparison would flap
// on the apps HashMap's nondeterministic key order (seen live on .228).
let dir = tempfile::tempdir().unwrap();
let body = r#"{"schema":1,"apps":{"demo":{"version":"1.0.0"}}}"#;
assert!(write_cache(dir.path(), body).unwrap(), "first write is a change");
assert!(
!write_cache(dir.path(), body).unwrap(),
"identical rewrite is not a change"
);
let body2 = r#"{"schema":1,"apps":{"demo":{"version":"1.0.1"}}}"#;
assert!(
write_cache(dir.path(), body2).unwrap(),
"new catalog bytes are a change"
);
assert_eq!(
std::fs::read_to_string(dir.path().join(APP_CATALOG_FILE)).unwrap(),
body2,
"cache holds the raw signed preimage bytes"
);
}
#[test]
fn entry_carries_embedded_manifest() {
let json = r#"{

View File

@ -424,7 +424,7 @@ fn build_unit(spec: &CompanionSpec, image: &str) -> QuadletUnit {
ports: spec
.ports
.iter()
.map(|(host, container)| (*host, *container, "tcp".into()))
.map(|(host, container)| (*host, *container, "tcp".into(), String::new()))
.collect(),
extra_podman_args: vec![],
depends_on: vec![],

View File

@ -0,0 +1,95 @@
//! Trusted-registry policy for container image references — the single
//! source of truth. The RPC boundary (`api::rpc::package::config`) and the
//! orchestrator's pull sites both validate against this, so a catalog- or
//! manifest-supplied ref can't reach `pull_image` unchecked (§A of the
//! 1.8.0 hardening plan).
/// Registries images may be pulled from with an explicit host part.
pub const TRUSTED_REGISTRIES: &[&str] = &[
"docker.io",
"ghcr.io",
"localhost",
"git.tx1138.com",
"146.59.87.168:3000",
];
/// Validate a container image reference.
///
/// Accepts:
/// * refs whose explicit registry host is on [`TRUSTED_REGISTRIES`]
/// (`docker.io/grafana/grafana`, `146.59.87.168:3000/archy/x:1`), and
/// * registry-less Docker Hub shorthand (`nginx`, `grafana/grafana`) —
/// the first segment has no `.`/`:` so it cannot name an attacker host;
/// resolution follows the host's registries.conf search order.
///
/// Rejects empty/oversized refs, shell metacharacters, and any ref whose
/// explicit registry host is not on the allowlist.
pub fn is_valid_docker_image(image: &str) -> bool {
if image.is_empty() || image.len() > 256 {
return false;
}
// Reject shell metacharacters
let dangerous_chars = [
'&', '|', ';', '`', '$', '(', ')', '<', '>', '\n', '\r', ' ', '\t',
];
if image.chars().any(|c| dangerous_chars.contains(&c)) {
return false;
}
let first_segment = match image.split('/').next() {
Some(r) if !r.is_empty() => r,
_ => return false,
};
if TRUSTED_REGISTRIES.contains(&first_segment) {
return true;
}
// No dot/colon in the first segment ⇒ it's a Docker Hub namespace or a
// bare repo name, not a registry host — allowed. Anything that *looks*
// like a host (has a dot or port) but isn't allowlisted is rejected.
!first_segment.contains('.') && !first_segment.contains(':')
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn accepts_trusted_registries() {
for img in [
"docker.io/library/nginx:1.25",
"ghcr.io/owner/app:latest",
"localhost/archy-dev:1",
"git.tx1138.com/lfg2025/x:2",
"146.59.87.168:3000/archy/bitcoin-knots:28.1",
] {
assert!(is_valid_docker_image(img), "{img} should be accepted");
}
}
#[test]
fn accepts_docker_hub_shorthand() {
for img in ["nginx", "grafana/grafana:11.2.0", "lightninglabs/lnd:v0.18"] {
assert!(is_valid_docker_image(img), "{img} should be accepted");
}
}
#[test]
fn rejects_untrusted_registry_hosts() {
for img in [
"evil.com/backdoor:latest",
"203.0.113.7:5000/x",
"registry.gitlab.com/x/y",
"quay.io/x/y",
] {
assert!(!is_valid_docker_image(img), "{img} should be rejected");
}
}
#[test]
fn rejects_malformed_refs() {
assert!(!is_valid_docker_image(""));
assert!(!is_valid_docker_image(&"a".repeat(257)));
assert!(!is_valid_docker_image("docker.io/x; rm -rf /"));
assert!(!is_valid_docker_image("docker.io/$(curl evil)"));
assert!(!is_valid_docker_image("/leading-slash"));
}
}

View File

@ -43,7 +43,11 @@ pub enum EnsureOutcome {
Unchanged,
}
pub async fn ensure_config(paths: &EnsurePaths, rpc_pass: &str) -> Result<EnsureOutcome> {
pub async fn ensure_config(
paths: &EnsurePaths,
rpc_pass: &str,
bitcoin_host: &str,
) -> Result<EnsureOutcome> {
fs::create_dir_all(&paths.data_dir)
.await
.with_context(|| format!("creating {}", paths.data_dir.display()))?;
@ -52,7 +56,7 @@ pub async fn ensure_config(paths: &EnsurePaths, rpc_pass: &str) -> Result<Ensure
let existing = fs::read_to_string(&paths.conf_path)
.await
.with_context(|| format!("reading {}", paths.conf_path.display()))?;
if has_required_lnd_flags(&existing, rpc_pass) {
if has_required_lnd_flags(&existing, rpc_pass, bitcoin_host) {
return Ok(EnsureOutcome::Unchanged);
}
}
@ -68,12 +72,11 @@ restlisten=0.0.0.0:8080\n\
bitcoin.active=true\n\
bitcoin.mainnet=true\n\
bitcoin.node=bitcoind\n\
bitcoind.rpchost=bitcoin-knots:8332\n\
bitcoind.rpchost={bitcoin_host}:8332\n\
bitcoind.rpcuser=archipelago\n\
bitcoind.rpcpass={}\n\
bitcoind.rpcpass={rpc_pass}\n\
bitcoind.rpcpolling=true\n\
bitcoind.estimatemode=ECONOMICAL\n",
rpc_pass
bitcoind.estimatemode=ECONOMICAL\n"
);
write_config_atomically(paths, &conf).await?;
@ -653,13 +656,14 @@ fn shell_quote(s: &str) -> String {
s.replace('\'', "'\\''")
}
fn has_required_lnd_flags(conf: &str, rpc_pass: &str) -> bool {
fn has_required_lnd_flags(conf: &str, rpc_pass: &str, bitcoin_host: &str) -> bool {
let rpc_pass_line = format!("bitcoind.rpcpass={rpc_pass}");
let rpc_host_line = format!("bitcoind.rpchost={bitcoin_host}:8332");
[
"bitcoin.active=true",
"bitcoin.mainnet=true",
"bitcoin.node=bitcoind",
"bitcoind.rpchost=bitcoin-knots:8332",
rpc_host_line.as_str(),
rpc_pass_line.as_str(),
]
.iter()
@ -678,7 +682,7 @@ mod tests {
conf_path: tmp.path().join("lnd/lnd.conf"),
};
let out = ensure_config(&paths, "secret").await.unwrap();
let out = ensure_config(&paths, "secret", "bitcoin-knots").await.unwrap();
assert_eq!(out, EnsureOutcome::Written);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoin.active=true"));
@ -697,17 +701,46 @@ mod tests {
};
assert_eq!(
ensure_config(&paths, "first").await.unwrap(),
ensure_config(&paths, "first", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
assert_eq!(
ensure_config(&paths, "second").await.unwrap(),
ensure_config(&paths, "second", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoind.rpcpass=second"));
}
#[tokio::test]
async fn ensure_config_repairs_bitcoin_host_drift() {
// A conf written against bitcoin-knots must be rewritten when the
// node's Bitcoin variant is bitcoin-core, or LND dials a hostname
// that doesn't exist on archy-net and dies on startup.
let tmp = tempfile::TempDir::new().unwrap();
let paths = EnsurePaths {
data_dir: tmp.path().join("lnd"),
conf_path: tmp.path().join("lnd/lnd.conf"),
};
assert_eq!(
ensure_config(&paths, "pw", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
assert_eq!(
ensure_config(&paths, "pw", "bitcoin-core").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();
assert!(conf.contains("bitcoind.rpchost=bitcoin-core:8332"));
assert!(!conf.contains("bitcoind.rpchost=bitcoin-knots:8332"));
assert_eq!(
ensure_config(&paths, "pw", "bitcoin-core").await.unwrap(),
EnsureOutcome::Unchanged
);
}
#[tokio::test]
async fn ensure_config_repairs_incomplete_existing_config() {
let tmp = tempfile::TempDir::new().unwrap();
@ -721,7 +754,7 @@ mod tests {
.unwrap();
assert_eq!(
ensure_config(&paths, "repaired").await.unwrap(),
ensure_config(&paths, "repaired", "bitcoin-knots").await.unwrap(),
EnsureOutcome::Written
);
let conf = fs::read_to_string(&paths.conf_path).await.unwrap();

View File

@ -7,6 +7,7 @@ pub mod dev_orchestrator;
pub mod docker_packages;
pub mod filebrowser;
pub mod hooks;
pub mod image_policy;
pub mod image_versions;
pub mod lnd;
pub mod prod_orchestrator;

File diff suppressed because it is too large Load Diff

View File

@ -140,8 +140,17 @@ pub struct QuadletUnit {
// Backend-manifest extensions (Phase 3.1). Companion units leave
// these defaulted; the renderer skips empty/false directives so a
// companion's rendered bytes are unchanged from before this PR.
pub ports: Vec<(u16, u16, String)>,
/// (host_port, container_port, protocol, bind). `bind` empty = 0.0.0.0.
pub ports: Vec<(u16, u16, String, String)>,
pub environment: Vec<String>,
/// Secret-backed env: (env_key, podman secret name). Rendered as
/// `Secret=<name>,type=env,target=<key>` so the VALUE never lands in
/// this unit file on disk — only a reference to the podman secret
/// store. The orchestrator registers the secrets before writing units.
pub secret_env: Vec<(String, String)>,
/// Container labels (`Label=k=v`). Carries the secret-env content hash
/// for rotation-drift detection.
pub labels: Vec<(String, String)>,
pub devices: Vec<String>,
pub add_hosts: Vec<(String, String)>,
pub network_aliases: Vec<String>,
@ -233,13 +242,19 @@ impl QuadletUnit {
// (exit 125). Skip publishing in host mode — matches the NetworkMode
// doc note that Podman discards port mappings under host networking.
if !matches!(self.network, NetworkMode::Host) {
for (host, container, proto) in &self.ports {
for (host, container, proto, bind) in &self.ports {
let p = if proto.is_empty() {
"tcp"
} else {
proto.as_str()
};
let _ = writeln!(s, "PublishPort={host}:{container}/{p}");
// Keep the rendered directive byte-identical for unbound
// ports so existing units don't read as drifted.
if bind.is_empty() {
let _ = writeln!(s, "PublishPort={host}:{container}/{p}");
} else {
let _ = writeln!(s, "PublishPort={bind}:{host}:{container}/{p}");
}
}
}
for env in &self.environment {
@ -247,6 +262,12 @@ impl QuadletUnit {
// accepts that form on a single Environment= line per pair.
let _ = writeln!(s, "Environment={}", quote_environment(env));
}
for (key, secret_name) in &self.secret_env {
let _ = writeln!(s, "Secret={secret_name},type=env,target={key}");
}
for (k, v) in &self.labels {
let _ = writeln!(s, "Label={k}={v}");
}
for dev in &self.devices {
let _ = writeln!(s, "AddDevice={dev}");
}
@ -412,9 +433,39 @@ impl QuadletUnit {
ports: app
.ports
.iter()
.map(|p| (p.host, p.container, p.protocol.clone()))
.filter(|p| {
let ok = archipelago_container::manifest::host_can_bind_publish_ip(&p.bind);
if !ok {
tracing::warn!(
app = %app.id,
bind = %p.bind,
host_port = p.host,
"dropping publish: bind address not assignable on this host \
(rootlessport would crash-loop the unit)"
);
}
ok
})
.map(|p| (p.host, p.container, p.protocol.clone(), p.bind.clone()))
.collect(),
environment: app.environment.clone(),
secret_env: app
.container
.secret_env_refs
.iter()
.map(|r| (r.env_key.clone(), r.secret_name.clone()))
.collect(),
labels: app
.container
.secret_env_hash
.iter()
.map(|h| {
(
archipelago_container::manifest::SECRET_ENV_HASH_LABEL.to_string(),
h.clone(),
)
})
.collect(),
devices: app.devices.clone(),
add_hosts: vec![("host.archipelago".into(), "10.89.0.1".into())],
// Container always answers to its own name; manifest extras add the
@ -487,13 +538,18 @@ fn translate_health_check(hc: &archipelago_container::HealthCheck) -> Option<Hea
format!("{url}{path}")
};
let helper_timeout = health_timeout_seconds(&hc.timeout);
let (tcp_host, tcp_port) = health_url_host_port(&final_url);
// Images vary wildly: SearXNG ships wget but no curl, while some
// Node images ship neither. Use whichever probe helper exists and
// skip Podman health if the image has none; host-side lifecycle
// probes still verify reachability.
// images (btcpay's dotnet base) ship neither but do have bash —
// fall through to a bash /dev/tcp connect probe there so the
// container can't report healthy while its port is dead
// (btcpay-server, 2026-07-10). Podman's own health timeout bounds
// the connect attempt. Only images with none of the three skip
// Podman health entirely; host-side lifecycle probes still verify
// reachability for those.
format!(
"if command -v wget >/dev/null 2>&1; then wget -q -T {1} -O /dev/null {0}; elif command -v curl >/dev/null 2>&1; then curl -fsS -m {1} {0}; else exit 0; fi",
final_url, helper_timeout
"if command -v wget >/dev/null 2>&1; then wget -q -T {1} -O /dev/null {0}; elif command -v curl >/dev/null 2>&1; then curl -fsS -m {1} {0}; elif command -v bash >/dev/null 2>&1; then bash -c 'exec 3<>/dev/tcp/{2}/{3}'; else exit 0; fi",
final_url, helper_timeout, tcp_host, tcp_port
)
}
"cmd" => hc.endpoint.as_deref()?.to_string(),
@ -507,6 +563,24 @@ fn translate_health_check(hc: &archipelago_container::HealthCheck) -> Option<Hea
})
}
/// Host and port of an http(s) health URL, for the bash /dev/tcp fallback
/// probe. Defaults to port 80/443 by scheme when the authority carries none.
fn health_url_host_port(url: &str) -> (String, u16) {
let (default_port, rest) = if let Some(rest) = url.strip_prefix("https://") {
(443, rest)
} else {
(80, url.strip_prefix("http://").unwrap_or(url))
};
let authority = rest.split('/').next().unwrap_or(rest);
match authority.rsplit_once(':') {
Some((host, port)) => match port.parse::<u16>() {
Ok(port) => (host.to_string(), port),
Err(_) => (authority.to_string(), default_port),
},
None => (authority.to_string(), default_port),
}
}
fn health_timeout_seconds(raw: &str) -> u64 {
let trimmed = raw.trim();
if trimmed.is_empty() {
@ -847,6 +921,24 @@ mod tests {
use super::*;
use tempfile::tempdir;
#[test]
fn render_emits_secret_env_by_reference_never_value() {
let u = QuadletUnit {
name: "t".into(),
description: "t".into(),
image: "img".into(),
secret_env: vec![("DB_PASS".into(), "archy-env-app-db_pass".into())],
labels: vec![("io.archipelago.secret-env-hash".into(), "abc123".into())],
..QuadletUnit::default()
};
let s = u.render();
assert!(s.contains("Secret=archy-env-app-db_pass,type=env,target=DB_PASS"));
assert!(s.contains("Label=io.archipelago.secret-env-hash=abc123"));
// the secret VALUE never had a path into this unit — but guard the
// env channel anyway: no Environment= line may mention the key
assert!(!s.contains("Environment=DB_PASS"));
}
fn sample_unit() -> QuadletUnit {
QuadletUnit {
name: "archy-bitcoin-ui".into(),
@ -908,7 +1000,7 @@ mod tests {
// Podman rejects PublishPort with Network=host (crash-loop exit 125).
let mut u = sample_unit();
u.network = NetworkMode::Host;
u.ports = vec![(3000, 3000, "tcp".into())];
u.ports = vec![(3000, 3000, "tcp".into(), String::new())];
let s = u.render();
assert!(s.contains("Network=host"));
assert!(!s.contains("PublishPort"));
@ -918,7 +1010,7 @@ mod tests {
fn render_non_host_network_emits_publish_ports() {
let mut u = sample_unit();
u.network = NetworkMode::Bridge("archy-net".into());
u.ports = vec![(3000, 3000, "tcp".into())];
u.ports = vec![(3000, 3000, "tcp".into(), String::new())];
let s = u.render();
assert!(s.contains("PublishPort=3000:3000/tcp"));
}
@ -1036,6 +1128,29 @@ mod tests {
assert_eq!(RestartPolicy::OnFailure.as_systemd(), "on-failure");
}
#[test]
fn render_bound_ports_prefix_the_bind_address() {
// Bitcoin RPC hardening: the same host port published on loopback +
// the archy-net gateway; an unbound port must render byte-identical
// to the legacy form so existing units don't read as drifted.
let u = QuadletUnit {
name: "bitcoin-knots".into(),
image: "registry/bitcoin-knots:latest".into(),
network: NetworkMode::Bridge("archy-net".into()),
ports: vec![
(8332, 8332, "tcp".into(), "127.0.0.1".into()),
(8332, 8332, "tcp".into(), "10.89.0.1".into()),
(8333, 8333, "tcp".into(), String::new()),
],
..QuadletUnit::default()
};
let s = u.render();
assert!(s.contains("PublishPort=127.0.0.1:8332:8332/tcp"));
assert!(s.contains("PublishPort=10.89.0.1:8332:8332/tcp"));
assert!(s.contains("PublishPort=8333:8333/tcp"));
assert!(!s.contains("PublishPort=8332:8332/tcp"));
}
#[test]
fn render_emits_backend_directives_when_set() {
let u = QuadletUnit {
@ -1045,7 +1160,10 @@ mod tests {
network: NetworkMode::Bridge("archy-net".into()),
cap_drop_all: true,
cap_add: vec!["NET_BIND_SERVICE".into()],
ports: vec![(8332, 8332, "tcp".into()), (8333, 8333, "tcp".into())],
ports: vec![
(8332, 8332, "tcp".into(), String::new()),
(8333, 8333, "tcp".into(), String::new()),
],
environment: vec![
"BITCOIN_RPC_USER=archipelago".into(),
"BITCOIN_RPC_PASS=secret".into(),
@ -1140,7 +1258,7 @@ app:
assert!(u.read_only_root);
assert!(u.no_new_privileges);
assert_eq!(u.cap_add, vec!["NET_BIND_SERVICE"]);
assert_eq!(u.ports, vec![(8332, 8332, "tcp".to_string())]);
assert_eq!(u.ports, vec![(8332, 8332, "tcp".to_string(), String::new())]);
assert_eq!(u.environment, vec!["BITCOIN_NETWORK=mainnet"]);
assert_eq!(u.bind_mounts.len(), 1);
assert_eq!(
@ -1364,7 +1482,22 @@ app:
let h = translate_health_check(&http).expect("http must translate");
assert_eq!(
h.cmd,
"if command -v wget >/dev/null 2>&1; then wget -q -T 3 -O /dev/null http://localhost:8080/health; elif command -v curl >/dev/null 2>&1; then curl -fsS -m 3 http://localhost:8080/health; else exit 0; fi"
"if command -v wget >/dev/null 2>&1; then wget -q -T 3 -O /dev/null http://localhost:8080/health; elif command -v curl >/dev/null 2>&1; then curl -fsS -m 3 http://localhost:8080/health; elif command -v bash >/dev/null 2>&1; then bash -c 'exec 3<>/dev/tcp/localhost/8080'; else exit 0; fi"
);
// The /dev/tcp fallback must target the URL's host:port, with
// scheme-default ports when the authority carries none.
assert_eq!(
health_url_host_port("http://localhost:8080/health"),
("localhost".to_string(), 8080)
);
assert_eq!(
health_url_host_port("http://127.0.0.1/"),
("127.0.0.1".to_string(), 80)
);
assert_eq!(
health_url_host_port("https://localhost/status"),
("localhost".to_string(), 443)
);
let cmdck = HealthCheck {

View File

@ -155,7 +155,9 @@ pub async fn load_registries(data_dir: &Path) -> Result<RegistryConfig> {
.collect::<Vec<_>>();
if changed {
// Persist so the next load doesn't have to re-merge.
let _ = save_registries(data_dir, &config).await;
if let Err(e) = save_registries(data_dir, &config).await {
tracing::warn!("Failed to persist migrated registry config: {e:#}");
}
}
Ok(config)
}

View File

@ -25,6 +25,26 @@ pub trait ContainerOrchestrator: Send + Sync {
/// to a manifest the orchestrator already knows about.
async fn install(&self, app_id: &str) -> Result<String>;
/// True when this orchestrator holds a manifest for `app_id` (disk or
/// signed-catalog overlay) — i.e. `install(app_id)` would not fail with
/// "unknown app_id". Lets the RPC layer route any manifest-driven app
/// through the orchestrator without a per-app allowlist. Defaults to
/// `false` so orchestrators without a manifest registry keep routing
/// through the legacy install flow.
async fn knows_app(&self, _app_id: &str) -> bool {
false
}
/// Rebuild the in-memory manifest map (disk + signed-catalog overlay).
/// Called after a runtime catalog refresh detects changed bytes so catalog
/// manifest changes take effect without a service restart — without this,
/// `load_manifests` only runs at startup and a freshly published manifest
/// sits dormant until the next restart. Returns the merged manifest count.
/// Defaults to a no-op for orchestrators without a manifest registry.
async fn reload_manifests(&self) -> Result<usize> {
Ok(0)
}
/// Start an already-created container.
async fn start(&self, app_id: &str) -> Result<()>;

View File

@ -49,6 +49,46 @@ pub fn is_recovery_complete() -> bool {
RECOVERY_COMPLETE.load(Ordering::SeqCst)
}
// ── Pending boot-start tracking ─────────────────────────────────────────
// Containers that boot recovery / the reconciler is about to start (or is
// starting right now). The package scanner overlays these as `Restarting`
// instead of the raw podman `Stopped`/`Exited`, so a freshly rebooted node
// doesn't tell the user their apps are "Stopped" while the sequential
// recovery pass (3s stagger + up to minutes for heavyweights like bitcoin)
// is still working through the queue. Writers register names when a pass
// begins and remove each name once its start attempt finishes, whatever
// the outcome — a container that truly failed goes back to showing its
// real state on the next scan.
static PENDING_BOOT_STARTS: std::sync::LazyLock<std::sync::RwLock<std::collections::HashSet<String>>> =
std::sync::LazyLock::new(|| std::sync::RwLock::new(std::collections::HashSet::new()));
/// Register container/app names an active recovery or reconcile pass
/// intends to start.
pub fn pending_boot_starts_add<I: IntoIterator<Item = String>>(names: I) {
if let Ok(mut set) = PENDING_BOOT_STARTS.write() {
set.extend(names);
}
}
/// A start attempt for `name` finished (success or failure) — stop
/// overlaying it.
pub fn pending_boot_start_done(name: &str) {
if let Ok(mut set) = PENDING_BOOT_STARTS.write() {
set.remove(name);
}
}
/// Whether `name` (a container name or scanner app id) is queued for a
/// boot/reconcile start. Container names may carry an `archy-` prefix the
/// scanner strips when deriving app ids, so check both forms.
pub fn is_pending_boot_start(name: &str) -> bool {
let Ok(set) = PENDING_BOOT_STARTS.read() else {
return false;
};
set.contains(name) || set.contains(&format!("archy-{name}"))
}
// ── User-stopped tracking ───────────────────────────────────────────────
// When a user explicitly stops a container via the UI, we record it here
// so crash recovery and health monitor don't auto-restart it.
@ -178,10 +218,17 @@ pub async fn check_for_crash(data_dir: &Path) -> Result<Option<Vec<RunningContai
old_pid
);
// Check if that PID is actually still running (zombie/stuck process)
// Check if that PID is actually still running (zombie/stuck process).
// Guard against PID reuse: after a reboot the old PID often belongs to an
// unrelated process (or, before the main.rs ordering fix, to OURSELVES) —
// only treat it as "previous instance still alive" if it's a live process
// that is not us and whose cmdline looks like the archipelago binary.
if !old_pid.is_empty() {
if let Ok(pid) = old_pid.parse::<u32>() {
if is_process_running(pid) {
if pid != std::process::id()
&& is_process_running(pid)
&& process_is_archipelago(pid)
{
warn!(
"Previous process (PID {}) is still running — not a crash, skipping recovery",
pid
@ -311,6 +358,8 @@ pub async fn recover_containers(containers: &[RunningContainerRecord]) -> Recove
failed: Vec::new(),
};
pending_boot_starts_add(containers.iter().map(|r| r.name.clone()));
for (i, record) in containers.iter().enumerate() {
info!(
"Recovering container: {} (image: {})",
@ -373,6 +422,7 @@ pub async fn recover_containers(containers: &[RunningContainerRecord]) -> Recove
if !started {
report.failed.push(record.name.clone());
}
pending_boot_start_done(&record.name);
}
report
@ -391,6 +441,16 @@ fn is_process_running(pid: u32) -> bool {
std::path::Path::new(&format!("/proc/{}", pid)).exists()
}
/// Whether the process at `pid` looks like an archipelago instance. Used to
/// tell "the previous instance is genuinely still alive" apart from PID
/// reuse by an unrelated process after a reboot.
fn process_is_archipelago(pid: u32) -> bool {
match std::fs::read(format!("/proc/{pid}/cmdline")) {
Ok(cmdline) => String::from_utf8_lossy(&cmdline).contains("archipelago"),
Err(_) => false,
}
}
/// Start all stopped containers that were previously installed.
/// Runs on every startup to ensure containers come back after clean reboots.
/// The crash recovery (PID-based) handles dirty shutdowns; this handles clean ones.
@ -425,16 +485,34 @@ async fn start_stopped_app_stacks(data_dir: &Path) -> RecoveryReport {
);
repair_stack_network_aliases(stack).await;
// Register the whole stack up front: the per-member dependency waits
// below can take minutes, and the UI should say "Restarting", not
// "Stopped", for members still queued behind them.
pending_boot_starts_add(
stack
.containers
.iter()
.filter(|c| !user_stopped.contains(**c))
.map(|c| (*c).to_string()),
);
for container in stack.containers {
if user_stopped.contains(*container) {
info!("Skipping user-stopped container: {}", container);
continue;
}
match container_state(container).await {
Some(state) if state == "running" => continue,
let state = container_state(container).await;
match state {
Some(state) if state == "running" => {
pending_boot_start_done(container);
continue;
}
Some(_) => {}
None => continue,
None => {
pending_boot_start_done(container);
continue;
}
}
repair_stack_network_aliases(stack).await;
@ -446,6 +524,7 @@ async fn start_stopped_app_stacks(data_dir: &Path) -> RecoveryReport {
} else {
report.failed.push((*container).to_string());
}
pending_boot_start_done(container);
}
}

View File

@ -166,7 +166,12 @@ pub async fn add_node(data_dir: &Path, node: FederatedNode) -> Result<Vec<Federa
}
// Explicitly (re-)adding a node clears any prior tombstone so the
// operator can intentionally bring back a previously removed peer.
let _ = untombstone_did(data_dir, &node.did).await;
// Propagate failure BEFORE mutating the node list: with the tombstone
// still in place, sync reconciliation would silently re-remove the
// node the operator just added.
untombstone_did(data_dir, &node.did)
.await
.context("clear removal tombstone")?;
nodes.push(node);
save_nodes(data_dir, &nodes).await?;
Ok(nodes)
@ -179,12 +184,16 @@ pub async fn remove_node(data_dir: &Path, did: &str) -> Result<Vec<FederatedNode
if nodes.len() == before {
anyhow::bail!("No federated node with DID {}", did);
}
save_nodes(data_dir, &nodes).await?;
// Tombstone the DID so transitive federation discovery (a still-federated
// peer advertising this DID as one of *its* trusted peers) can't silently
// re-add it. Best-effort: a failed tombstone write must not fail the
// remove the operator asked for.
let _ = tombstone_did(data_dir, did).await;
// re-add it. Tombstone FIRST and propagate failure: a remove whose
// tombstone never landed isn't a remove — the peer would quietly
// reappear after the next sync. Tombstoning is idempotent, so if the
// node-list save below fails the operator's retry works cleanly.
tombstone_did(data_dir, did)
.await
.context("persist removal tombstone")?;
save_nodes(data_dir, &nodes).await?;
Ok(nodes)
}

View File

@ -49,13 +49,16 @@ pub async fn sync_with_peer(
// Record transport used so the UI badge on this peer's card reflects
// the transport that actually carried the call, not a prediction.
let _ = super::storage::record_peer_transport(
if let Err(e) = super::storage::record_peer_transport(
data_dir,
Some(&peer.did),
Some(&peer.onion),
&transport.to_string(),
)
.await;
.await
{
tracing::warn!("Failed to persist peer transport badge: {e:#}");
}
let result: serde_json::Value = resp.json().await.context("Invalid response from peer")?;
let state_val = result

View File

@ -147,6 +147,21 @@ fn has_running_bitcoin_conflict(name: &str, containers: &[ContainerHealth]) -> b
})
}
/// True when a lifecycle op lock covering this container is held. Probes the
/// container name, the derived app id, and a '_'→'-' normalization of the
/// name — legacy stack containers are named with underscores
/// (immich_postgres) while op-lock keys and the stack-member table use the
/// hyphenated app ids.
fn lifecycle_op_covers_container(container_name: &str, app_id: &str) -> bool {
let hyphenated = container_name.replace('_', "-");
let mut keys = vec![container_name, app_id];
if hyphenated != container_name {
keys.push(hyphenated.as_str());
}
keys.iter()
.any(|key| crate::app_ops::lifecycle_op_in_flight(key))
}
/// Track restart attempts per container with exponential backoff and stability reset.
struct RestartTracker {
attempts: HashMap<String, u32>,
@ -1057,6 +1072,19 @@ pub fn spawn_health_monitor(state: Arc<StateManager>, data_dir: PathBuf) {
continue;
}
// A package.start/stop/restart worker is mid-sequence on this
// app (or its owning stack): the container is legitimately
// down between the worker's stop and start halves. Restarting
// it here races the worker the same way the reconciler's
// repair-recreate did (e275494a) — skip for this cycle.
if lifecycle_op_covers_container(&container.name, &container.app_id) {
debug!(
"Skipping auto-restart for {}: lifecycle operation in flight",
container.name
);
continue;
}
// When transitioning to a higher tier, wait briefly for previous tier to stabilize
if let Some(prev) = prev_tier {
if tier > prev {
@ -1456,6 +1484,30 @@ mod tests {
assert_eq!(container_tier("bitcoin-knots"), StartupTier::CoreInfra);
}
#[tokio::test]
async fn lifecycle_guard_covers_name_app_id_and_legacy_underscores() {
assert!(!lifecycle_op_covers_container("hm-guard-app", "hm-guard-app"));
// Held package lock covers the container directly and via stack
// membership; the '_'→'-' probe covers legacy underscore names.
let lock = crate::app_ops::op_lock("immich");
let _guard = lock.lock().await;
assert!(lifecycle_op_covers_container(
"immich-postgres",
"immich-postgres"
));
assert!(lifecycle_op_covers_container(
"immich_postgres",
"immich_postgres"
));
assert!(!lifecycle_op_covers_container("grafana", "grafana"));
// archy- prefixed containers probe via the stripped app id too.
let lnd = crate::app_ops::op_lock("lnd");
let _lnd_guard = lnd.lock().await;
assert!(lifecycle_op_covers_container("archy-lnd", "lnd"));
}
#[test]
fn test_container_tier_dependent() {
assert_eq!(container_tier("lnd"), StartupTier::DependentService);

View File

@ -26,6 +26,7 @@ use tokio::sync::Notify;
use tracing::info;
mod api;
mod app_ops;
mod auth;
mod avatar;
mod backup;
@ -98,11 +99,15 @@ async fn main() -> Result<()> {
let startup_start = std::time::Instant::now();
crash_recovery::init_start_time();
// Initialize tracing
// Initialize tracing. Default to `info`: production units don't set
// RUST_LOG, and the old `archipelago=debug` default flooded journald
// with per-request debug lines ("RPC method: …", cookie-flag notes) —
// part of a >1 GB/day journal on a fresh node. Set RUST_LOG (e.g.
// RUST_LOG=archipelago=debug) to get debug logs back when debugging.
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "archipelago=debug,info".into()),
.unwrap_or_else(|_| "info".into()),
)
.init();
@ -149,13 +154,18 @@ async fn main() -> Result<()> {
);
}
// Write PID marker early so we can detect crashes on next startup
// Check for a crash marker BEFORE writing our own. The old order wrote
// the marker first, so the check always read the CURRENT process's PID,
// found it alive, and skipped recovery — on every boot, forever.
let crash_containers = crash_recovery::check_for_crash(&config.data_dir).await;
// Now mark this instance as running so the next startup can detect a crash.
crash_recovery::write_pid_marker(&config.data_dir).await?;
// Run crash recovery before starting the manifest reconciler. Both paths
// mutate Podman; running them concurrently can corrupt transient runtime
// state and leave netavark/conmon unable to start containers.
match crash_recovery::check_for_crash(&config.data_dir).await {
match crash_containers {
Ok(Some(containers)) => {
info!(
"🔧 Recovering {} containers from previous crash...",
@ -212,7 +222,10 @@ async fn main() -> Result<()> {
)
.await
{
Ok(Ok(n)) => info!("🛰️ app-catalog refreshed before manifest load ({n} apps)"),
Ok(Ok(r)) => info!(
"🛰️ app-catalog refreshed before manifest load ({} apps)",
r.apps
),
Ok(Err(e)) => tracing::debug!("app-catalog pre-load refresh failed (using cache): {e}"),
Err(_) => tracing::debug!("app-catalog pre-load refresh timed out (using cache)"),
}

View File

@ -580,7 +580,9 @@ pub(super) async fn handle_identity_received(
// Update peer record
let peer = MeshPeer {
contact_id,
advert_name: format!("Archy-{}", &did[8..16.min(did.len())]),
// .get(): a malformed DID shorter than the "did:key:" prefix must
// not panic the listener on a radio-supplied string.
advert_name: format!("Archy-{}", did.get(8..16.min(did.len())).unwrap_or(did)),
did: Some(did.to_string()),
pubkey_hex: Some(ed_pubkey_hex.to_string()),
// The advert signature was verified above, so this is an authenticated

View File

@ -423,6 +423,7 @@ pub fn spawn_mesh_listener(
lora_region: Option<String>,
channel_name: Option<String>,
device_kind: Option<super::types::DeviceType>,
reticulum_tcp: Option<super::types::ReticulumTcpConfig>,
shutdown: tokio::sync::watch::Receiver<bool>,
cmd_rx: mpsc::Receiver<MeshCommand>,
) -> tokio::task::JoinHandle<()> {
@ -457,6 +458,7 @@ pub fn spawn_mesh_listener(
lora_region.as_deref(),
channel_name.as_deref(),
device_kind,
reticulum_tcp.clone(),
&mut shutdown,
&mut cmd_rx,
)

View File

@ -261,7 +261,7 @@ impl MeshRadioDevice {
/// `MeshConfig.device_kind` — see the plan's §2c reflashable-board note): only
/// that one device's probe runs, so a non-matching firmware's init bytes are
/// never injected into the port. `None` keeps the strict
/// Meshcore→Meshtastic→Reticulum probe order.
/// Reticulum→Meshcore→Meshtastic probe order.
async fn auto_detect_and_open(
data_dir: &Path,
our_ed_pubkey_hex: &str,
@ -274,6 +274,34 @@ async fn auto_detect_and_open(
}
for path in &paths {
debug!(path = %path, "Probing for mesh radio device");
// Tried FIRST: `ReticulumLink::open()` gates its expensive daemon
// spawn behind a cheap (~1s worst case) RNode KISS-detect probe, so a
// failed match here costs about as much as the Meshcore/Meshtastic
// probes below. Trying it first matters in practice: on a real RNode
// board, Meshcore's and Meshtastic's handshake bytes (~5s timeout
// each, ~10.6s combined) sitting on the wire before Reticulum ever
// gets a turn was observed to leave the RNode firmware unresponsive
// by the time its turn came — confirmed on hardware that responds
// correctly to the same KISS probe on a truly fresh port.
if device_kind.is_none_or(|k| k == DeviceType::Reticulum) {
match ReticulumLink::open(
path,
data_dir,
Some(our_ed_pubkey_hex),
Some(our_x25519_pubkey_hex),
)
.await
{
Ok(mut dev) => match dev.initialize().await {
Ok(info) => {
info!(path = %path, "Found Reticulum (RNode) device via auto-detect");
return Ok((path.clone(), MeshRadioDevice::Reticulum(dev), info));
}
Err(e) => debug!(path = %path, error = %e, "Reticulum daemon failed to initialize"),
},
Err(e) => debug!(path = %path, error = %e, "Not a Reticulum RNode"),
}
}
if device_kind.is_none_or(|k| k == DeviceType::Meshcore) {
match MeshcoreDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
@ -298,30 +326,6 @@ async fn auto_detect_and_open(
Err(e) => debug!(path = %path, error = %e, "Could not open serial port for Meshtastic"),
}
}
// Tried LAST: the same reflashable board (e.g. Heltec V3) can run
// Meshcore, Meshtastic, or RNode firmware, so each probe must fail
// strictly before the next is attempted. The RNode KISS-detect probe
// is the most expensive (spawns the supervised daemon on a match), so
// it goes after the two cheap firmware-specific handshakes above.
if device_kind.is_none_or(|k| k == DeviceType::Reticulum) {
match ReticulumLink::open(
path,
data_dir,
Some(our_ed_pubkey_hex),
Some(our_x25519_pubkey_hex),
)
.await
{
Ok(mut dev) => match dev.initialize().await {
Ok(info) => {
info!(path = %path, "Found Reticulum (RNode) device via auto-detect");
return Ok((path.clone(), MeshRadioDevice::Reticulum(dev), info));
}
Err(e) => debug!(path = %path, error = %e, "Reticulum daemon failed to initialize"),
},
Err(e) => debug!(path = %path, error = %e, "Not a Reticulum RNode"),
}
}
}
anyhow::bail!(
"No supported mesh radio found on {} candidate ports: {:?}",
@ -381,20 +385,10 @@ async fn open_preferred_path(
};
}
match MeshcoreDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => return Ok((MeshRadioDevice::Meshcore(dev), info)),
Err(e) => debug!(path = %path, error = %e, "Preferred path is not Meshcore"),
},
Err(e) => debug!(path = %path, error = %e, "Could not open preferred path as Meshcore"),
}
match MeshtasticDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => return Ok((MeshRadioDevice::Meshtastic(dev), info)),
Err(e) => debug!(path = %path, error = %e, "Preferred path is not Meshtastic"),
},
Err(e) => debug!(path = %path, error = %e, "Could not open preferred path as Meshtastic"),
}
// Reticulum first — see the matching comment on auto_detect_and_open:
// its cheap probe_rnode gate fails in ~1s for non-RNode firmware, while
// trying Meshcore/Meshtastic first was observed leaving a real RNode
// board unresponsive by the time Reticulum's turn came.
match ReticulumLink::open(
path,
data_dir,
@ -404,11 +398,69 @@ async fn open_preferred_path(
.await
{
Ok(mut dev) => match dev.initialize().await {
Ok(info) => Ok((MeshRadioDevice::Reticulum(dev), info)),
Err(e) => Err(e).context("Preferred path is not a working Reticulum RNode"),
Ok(info) => return Ok((MeshRadioDevice::Reticulum(dev), info)),
Err(e) => debug!(path = %path, error = %e, "Preferred path is not a working Reticulum RNode"),
},
Err(e) => Err(e).context("Could not open preferred path as Reticulum"),
Err(e) => debug!(path = %path, error = %e, "Could not open preferred path as Reticulum"),
}
match MeshcoreDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => return Ok((MeshRadioDevice::Meshcore(dev), info)),
Err(e) => debug!(path = %path, error = %e, "Preferred path is not Meshcore"),
},
Err(e) => debug!(path = %path, error = %e, "Could not open preferred path as Meshcore"),
}
match MeshtasticDevice::open(path).await {
Ok(mut dev) => match dev.initialize().await {
Ok(info) => Ok((MeshRadioDevice::Meshtastic(dev), info)),
Err(e) => Err(e).context("Preferred path is not a working Meshtastic device"),
},
Err(e) => Err(e).context("Could not open preferred path as Meshtastic"),
}
}
/// Bring up a Reticulum daemon over plain TCP — no physical RNode, no
/// `probe_rnode` KISS-detect gate. This is the radio-less counterpart to
/// `auto_detect_and_open`/`open_preferred_path`'s serial paths; see
/// `types::ReticulumTcpConfig`'s doc comment for scope (dev/verification,
/// loopback-only server bind).
async fn open_reticulum_tcp(
cfg: &ReticulumTcpConfig,
data_dir: &Path,
our_ed_pubkey_hex: &str,
our_x25519_pubkey_hex: &str,
) -> Result<(String, MeshRadioDevice, DeviceInfo)> {
let mut dev = match cfg {
ReticulumTcpConfig::Server { bind } => {
ReticulumLink::open_tcp_server(
bind,
data_dir,
Some(our_ed_pubkey_hex),
Some(our_x25519_pubkey_hex),
)
.await
.context("Could not open Reticulum TCP server interface")?
}
ReticulumTcpConfig::Client { connect } => {
ReticulumLink::open_tcp_client(
connect,
data_dir,
Some(our_ed_pubkey_hex),
Some(our_x25519_pubkey_hex),
)
.await
.context("Could not open Reticulum TCP client interface")?
}
};
let info = dev
.initialize()
.await
.context("Reticulum TCP interface failed to initialize")?;
let label = match cfg {
ReticulumTcpConfig::Server { bind } => format!("tcp-server:{bind}"),
ReticulumTcpConfig::Client { connect } => format!("tcp-client:{}", connect.join(",")),
};
Ok((label, MeshRadioDevice::Reticulum(dev), info))
}
/// ASCII marker for the original DM-via-channel format:
@ -627,11 +679,19 @@ async fn refresh_contacts(device: &mut MeshRadioDevice, state: &Arc<MeshState>)
advert_name: contact.advert_name.clone(),
did: existing.and_then(|p| p.did.clone()),
pubkey_hex: Some(contact.public_key_hex.clone()),
// Preserve any archipelago identity bound by an earlier
// identity advert — NEVER overwrite it with the firmware
// contact key, or a signed `!ai` query from this peer would
// fail authentication after the next contact refresh.
arch_pubkey_hex: existing.and_then(|p| p.arch_pubkey_hex.clone()),
// Reticulum carries the archipelago identity in-band with
// the contact snapshot itself (see `ParsedContact::
// arch_pubkey_hex`'s doc comment) — prefer it when this
// refresh's snapshot has one. Otherwise preserve whatever
// was bound by an earlier identity advert (Meshcore/
// Meshtastic's `bind_federation_twins` path): NEVER
// overwrite a known identity with the firmware contact
// key, or a signed `!ai` query from this peer would fail
// authentication after the next contact refresh.
arch_pubkey_hex: contact
.arch_pubkey_hex
.clone()
.or_else(|| existing.and_then(|p| p.arch_pubkey_hex.clone())),
x25519_pubkey: existing.and_then(|p| p.x25519_pubkey),
// Meshtastic-only today (see ParsedContact) — falls back to
// whatever was already known if this refresh's contact
@ -787,11 +847,17 @@ pub(super) async fn run_mesh_session(
lora_region: Option<&str>,
channel_name: Option<&str>,
device_kind: Option<DeviceType>,
reticulum_tcp: Option<ReticulumTcpConfig>,
shutdown: &mut tokio::sync::watch::Receiver<bool>,
cmd_rx: &mut mpsc::Receiver<MeshCommand>,
) -> Result<()> {
// Detect device — try preferred path first, fall back to auto-detect
let (device_path, mut device, device_info) = if let Some(path) = preferred_path {
// Detect device — TCP Reticulum config (radio-less) takes priority when
// set, otherwise try the preferred serial path, falling back to
// auto-detect. TCP mode is additive/dev-only; it never changes behavior
// for existing serial/RNode deployments where `reticulum_tcp` is None.
let (device_path, mut device, device_info) = if let Some(tcp_cfg) = &reticulum_tcp {
open_reticulum_tcp(tcp_cfg, data_dir, our_ed_pubkey_hex, our_x25519_pubkey_hex).await?
} else if let Some(path) = preferred_path {
match open_preferred_path(
path,
data_dir,

View File

@ -180,6 +180,12 @@ impl MeshtasticDevice {
"Failed to open serial port {} (permission denied? device busy?)",
path
))?;
// See probe_rnode() in reticulum.rs for why: ESP32-S3 native-USB
// boards reset on a DTR/RTS transition, so deassert both and settle
// before the handshake below.
let _ = port.set_dtr(false);
let _ = port.set_rts(false);
tokio::time::sleep(Duration::from_millis(300)).await;
info!(path = %path, baud = BAUD_RATE, "Opened Meshtastic serial port");
Ok(Self {
@ -976,6 +982,7 @@ impl MeshtasticDevice {
snr,
lat,
lon,
arch_pubkey_hex: None,
},
);
}
@ -1048,6 +1055,7 @@ fn packet_to_inbound_frame(
snr: None,
lat: None,
lon: None,
arch_pubkey_hex: None,
});
if packet.rx_rssi.is_some() {
contact.rssi = packet.rx_rssi.map(|v| v as i16);

View File

@ -258,7 +258,31 @@ impl TypedEnvelope {
}
}
/// Verify signature if present.
/// Signing preimage v2: binds the anti-replay `seq` so a radio MITM
/// can't reorder/replay a signed message under a different sequence
/// number. v1 (legacy) covers only (t, v, ts).
fn signing_preimage_v2(&self) -> Vec<u8> {
let mut sign_data = Vec::with_capacity(1 + self.v.len() + 4 + 8);
sign_data.push(self.t);
sign_data.extend_from_slice(&self.v);
sign_data.extend_from_slice(&self.ts.to_le_bytes());
sign_data.extend_from_slice(&self.seq.to_le_bytes());
sign_data
}
fn signing_preimage_v1(&self) -> Vec<u8> {
let mut sign_data = Vec::with_capacity(1 + self.v.len() + 4);
sign_data.push(self.t);
sign_data.extend_from_slice(&self.v);
sign_data.extend_from_slice(&self.ts.to_le_bytes());
sign_data
}
/// Verify signature if present. Accepts the seq-binding v2 preimage OR
/// the legacy (t, v, ts) preimage — senders still emit v1 until the
/// whole fleet verifies v2 (receivers hard-drop bad signatures, so
/// flipping the send side first would break mixed-fleet alerts). The
/// seq-tampering window closes only when the v1 arm is removed.
pub fn verify_signature(&self, verifying_key: &ed25519_dalek::VerifyingKey) -> Result<bool> {
let Some(sig_bytes) = &self.sig else {
return Ok(false);
@ -266,13 +290,14 @@ impl TypedEnvelope {
let signature =
ed25519_dalek::Signature::from_slice(sig_bytes).context("Invalid signature bytes")?;
let mut sign_data = Vec::with_capacity(1 + self.v.len() + 4);
sign_data.push(self.t);
sign_data.extend_from_slice(&self.v);
sign_data.extend_from_slice(&self.ts.to_le_bytes());
if verifying_key
.verify_strict(&self.signing_preimage_v2(), &signature)
.is_ok()
{
return Ok(true);
}
verifying_key
.verify_strict(&sign_data, &signature)
.verify_strict(&self.signing_preimage_v1(), &signature)
.context("Signature verification failed")?;
Ok(true)
}
@ -284,12 +309,25 @@ impl TypedEnvelope {
/// Set the outbound sequence number. Called by the send path after the
/// target's counter has been incremented. Safe to call AFTER `new_signed`
/// because the signature covers `(t, v, ts)` — not `seq`.
/// because the v1 signature covers `(t, v, ts)` — not `seq`. Once the
/// fleet is on a build whose `verify_signature` accepts the v2 preimage,
/// flip senders to sign AFTER seq allocation via `signed_with_seq`.
pub fn with_seq(mut self, seq: u64) -> Self {
self.seq = seq;
self
}
/// v2 sender path (NOT yet wired — see `verify_signature` for the fleet
/// migration order): set seq first, then sign binding it.
#[allow(dead_code)]
pub fn signed_with_seq(mut self, seq: u64, signing_key: &ed25519_dalek::SigningKey) -> Self {
use ed25519_dalek::Signer;
self.seq = seq;
let signature = signing_key.sign(&self.signing_preimage_v2());
self.sig = Some(signature.to_bytes().to_vec());
self
}
/// Encode to wire format: [0x02] [CBOR envelope].
pub fn to_wire(&self) -> Result<Vec<u8>> {
let mut buf = Vec::new();
@ -816,6 +854,37 @@ mod tests {
assert!(envelope.verify_signature(&key.verifying_key()).is_err());
}
#[test]
fn test_v2_seq_bound_signature() {
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
let key = SigningKey::generate(&mut OsRng);
let envelope = TypedEnvelope::new(MeshMessageType::Alert, b"test".to_vec())
.signed_with_seq(42, &key);
assert!(envelope.verify_signature(&key.verifying_key()).unwrap());
// v2 binds seq: replaying the signed envelope under a different
// sequence number must fail verification.
let mut replayed = envelope.clone();
replayed.seq = 43;
assert!(replayed.verify_signature(&key.verifying_key()).is_err());
}
#[test]
fn test_v1_signature_survives_seq_set_after_signing() {
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
// Mixed-fleet compatibility: current senders sign first (v1
// preimage, no seq) and allocate seq afterwards; verify must still
// accept that.
let key = SigningKey::generate(&mut OsRng);
let envelope = TypedEnvelope::new_signed(MeshMessageType::Alert, b"test".to_vec(), &key)
.with_seq(7);
assert!(envelope.verify_signature(&key.verifying_key()).unwrap());
}
#[test]
fn test_invoice_payload_roundtrip() {
let invoice = InvoicePayload {

View File

@ -392,6 +392,13 @@ pub struct MeshConfig {
/// strict-probe auto-detect.
#[serde(default)]
pub device_kind: Option<types::DeviceType>,
/// Optional plain-TCP Reticulum interface — radio-less dev/verification
/// alternative to the serial-RNode path (see `types::ReticulumTcpConfig`
/// doc comment). Not exposed via `mesh.configure`/frontend; hand-edit
/// this file to use it. `None` (default) preserves today's
/// serial-only/auto-detect behavior unchanged.
#[serde(default)]
pub reticulum_tcp: Option<types::ReticulumTcpConfig>,
}
fn default_assistant_backend() -> String {
@ -422,6 +429,7 @@ impl Default for MeshConfig {
assistant_backend: default_assistant_backend(),
assistant_allowed_contacts: Vec::new(),
device_kind: None,
reticulum_tcp: None,
}
}
}
@ -704,6 +712,7 @@ impl MeshService {
self.config.lora_region.clone(),
self.config.channel_name.clone(),
self.config.device_kind,
self.config.reticulum_tcp.clone(),
shutdown_rx,
cmd_rx,
);
@ -820,7 +829,9 @@ impl MeshService {
timestamp: header.timestamp,
announced_by: bha_did.clone(),
};
let _ = bha_cache.store_header(payload).await;
if let Err(e) = bha_cache.store_header(payload).await {
warn!("Failed to persist block-header cache: {e:#}");
}
// Build signed announcement and broadcast
match bitcoin_relay::build_block_header_announcement(
@ -2221,4 +2232,133 @@ mod tests {
assert!(loaded.enabled);
assert_eq!(loaded.device_path, Some("/dev/ttyUSB0".to_string()));
}
/// End-to-end: `MeshService::start()` spawns a real `reticulum-daemon` in
/// plain-TCP CLIENT mode (no serial RNode, no `probe_rnode`), dials a
/// second stand-alone daemon instance running in TCP SERVER mode (the
/// Aurora-side role — Aurora's `RnsTcpInterface` dials the same way), and
/// reaches `device_connected: true` via the exact `mesh.status`-backing
/// `MeshService::status()` call the RPC layer uses. This is the Rust-side
/// half of the archy<->Aurora TCP interop gate (see
/// docs/RETICULUM-TRANSPORT-PROGRESS.md); the LXMF wire-level proof
/// itself already passed via a scripted RNS/LXMF stand-in (Steps 1-2 of
/// that gate), so this test only needs to prove Rust's spawn/connect
/// path, not re-prove LXMF content delivery.
///
/// Requires `reticulum-daemon/.venv` (`python3 -m venv .venv &&
/// .venv/bin/pip install -r requirements.txt`) — skips (not fails) if
/// absent, since CI doesn't provision a Python/RNS/LXMF environment for
/// the rest of the mesh test suite either.
#[tokio::test]
#[ignore = "spawns real reticulum-daemon subprocesses over loopback TCP; run manually with `cargo test -p archipelago -- --ignored mesh_service_connects_over_reticulum_tcp`"]
async fn mesh_service_connects_over_reticulum_tcp_client() {
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
let repo_root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.and_then(|p| p.parent())
.expect("core/archipelago has two ancestors up to the repo root")
.to_path_buf();
let venv_py = repo_root.join("reticulum-daemon/.venv/bin/python");
let daemon_script = repo_root.join("reticulum-daemon/reticulum_daemon.py");
if !venv_py.exists() {
eprintln!(
"SKIP mesh_service_connects_over_reticulum_tcp_client: {} not found — \
run `python3 -m venv .venv && .venv/bin/pip install -r requirements.txt` \
in reticulum-daemon/",
venv_py.display()
);
return;
}
let root = tempfile::tempdir().unwrap();
// Stand-in "Aurora-side" server daemon: a second, independent
// reticulum-daemon instance in --tcp-listen mode. Any free loopback
// port works; this test doesn't need it fixed, so ask the OS for one.
let bind_addr = {
let sock = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let addr = sock.local_addr().unwrap();
drop(sock); // release it so the daemon can bind — small TOCTOU window, acceptable for a manual/ignored test
addr
};
let server_dir = root.path().join("standin-server");
std::fs::create_dir_all(&server_dir).unwrap();
let server_seed = SigningKey::generate(&mut OsRng);
std::fs::write(server_dir.join("seed.bin"), server_seed.to_bytes()).unwrap();
let mut server = tokio::process::Command::new(&venv_py)
.arg(&daemon_script)
.arg("--identity-key")
.arg(server_dir.join("seed.bin"))
.arg("--socket")
.arg(server_dir.join("server.sock"))
.arg("--rns-config")
.arg(server_dir.join("rns-config"))
.arg("--tcp-listen")
.arg(bind_addr.to_string())
.arg("--display-name")
.arg("standin-server")
.kill_on_drop(true)
.spawn()
.expect("failed to spawn stand-in reticulum-daemon (server)");
// Give the server daemon time to bind before the client dials it.
tokio::time::sleep(Duration::from_secs(2)).await;
// Point ReticulumLink::daemon_command's dev-fallback at our venv —
// these env vars are read fresh on every spawn (mesh/reticulum.rs),
// so setting them here (test-process-global, but this test owns the
// only mesh session in this process) is sufficient.
std::env::set_var("ARCHY_RETICULUM_DAEMON_PY", &venv_py);
std::env::set_var("ARCHY_RETICULUM_DAEMON_SCRIPT", &daemon_script);
// Force the dev fallback even if a packaged binary happens to exist
// on this machine's PATH convention — this test wants exactly the
// freshly-built venv daemon under test.
std::env::set_var("ARCHY_RETICULUM_DAEMON_BIN", "/nonexistent-force-dev-fallback");
let data_dir = root.path().join("node");
std::fs::create_dir_all(data_dir.join("identity")).unwrap();
let node_signing_key = SigningKey::generate(&mut OsRng);
std::fs::write(
data_dir.join("identity").join("node_key"),
node_signing_key.to_bytes(),
)
.unwrap();
let ed_pubkey_hex = hex::encode(node_signing_key.verifying_key().to_bytes());
let did = crate::identity::did_key_from_pubkey_hex(&ed_pubkey_hex).unwrap();
let config = MeshConfig {
enabled: true,
reticulum_tcp: Some(types::ReticulumTcpConfig::Client {
connect: vec![bind_addr.to_string()],
}),
..Default::default()
};
save_config(&data_dir, &config).await.unwrap();
let mut service = MeshService::new(&data_dir, &node_signing_key, &did, &ed_pubkey_hex)
.await
.expect("MeshService::new failed");
service.start().expect("MeshService::start failed");
let deadline = tokio::time::Instant::now() + Duration::from_secs(20);
let mut last_status = service.status().await;
while tokio::time::Instant::now() < deadline {
last_status = service.status().await;
if last_status.device_connected {
break;
}
tokio::time::sleep(Duration::from_millis(300)).await;
}
let _ = server.start_kill();
assert!(
last_status.device_connected,
"MeshService never reached device_connected: true over Reticulum TCP client \
mode within 20s (last status: {last_status:?})"
);
assert_eq!(last_status.device_type, DeviceType::Reticulum);
}
}

View File

@ -406,6 +406,15 @@ pub struct ParsedContact {
/// contact has shared one.
pub lat: Option<f64>,
pub lon: Option<f64>,
/// Archipelago ed25519 identity hex, when this transport carried it
/// in-band with the contact announce itself (Reticulum only today — the
/// RNS announce's app_data can embed an `ARCHY:n:` identity blob
/// alongside the destination hash in the same event, so there's no
/// ambiguity about which physical peer it belongs to). Meshcore/
/// Meshtastic identity adverts go out on a separate channel and are
/// correlated after the fact by `bind_federation_twins`'s advert_name
/// matching instead, so they always leave this `None`.
pub arch_pubkey_hex: Option<String>,
}
/// Parse RESP_CONTACT (0x03) response.
@ -457,6 +466,7 @@ pub fn parse_contact(data: &[u8]) -> Result<ParsedContact> {
snr: None,
lat: None,
lon: None,
arch_pubkey_hex: None,
})
}

View File

@ -47,7 +47,14 @@ const KISS_CMD_PLATFORM: u8 = 0x48;
const KISS_CMD_MCU: u8 = 0x49;
const PROBE_BAUD: u32 = 115200;
const PROBE_READ_TIMEOUT: Duration = Duration::from_millis(800);
// 800ms was too tight for real hardware: confirmed on a genuine Heltec V4
// RNode (firmware 1.86, verified via `rnodeconf --info`) that prints extra
// boot/status chatter on the same serial line before answering KISS
// commands — DETECT_RESP measured arriving ~1.05s after the probe write in
// that case. 2.5s leaves comfortable margin without meaningfully slowing
// down detection of non-RNode devices (Meshcore/Meshtastic already budget
// ~5s each).
const PROBE_READ_TIMEOUT: Duration = Duration::from_millis(2500);
/// Prefix marking an LXMF `content` string as base64 of a raw binary
/// typed-envelope payload rather than literal text. LXMF `content` travels
@ -64,6 +71,44 @@ const RETICULUM_BINARY_CONTENT_MARKER: &str = "\u{0}b64:";
/// packaging); during development it can point at the venv's interpreter
/// invoking `reticulum_daemon.py` directly. Overridable for testing/packaging.
///
/// Which Reticulum interface the daemon should bring up. `Serial` is the
/// original (and only, until now) path — a physical RNode over a serial
/// port, gated behind `probe_rnode`'s KISS-detect handshake in `open()`.
/// `TcpServer`/`TcpClient` are the radio-less, dev/verification-only
/// addition (see `types::ReticulumTcpConfig`'s doc comment): they let the
/// daemon speak plain Reticulum TCP — the same transport Aurora's
/// `RnsTcpInterface`/`RnsTcpServerInterface` use by default — without any
/// physical hardware. `TcpServer` is hard-gated to loopback by
/// `open_tcp_server`; `TcpClient` (outbound dial) is unrestricted, the same
/// risk class as Aurora's own hub uplinks.
pub enum ReticulumInterface<'a> {
Serial(&'a str),
TcpServer(&'a str),
TcpClient(&'a [String]),
}
impl ReticulumInterface<'_> {
/// Human-readable label used for both the `device_path` status field and
/// (sanitized) the per-instance RPC socket filename. For `Serial` this is
/// exactly the old bare path — zero behavior change for the existing
/// hardware flow.
fn label(&self) -> String {
match self {
Self::Serial(path) => path.to_string(),
Self::TcpServer(bind) => format!("tcp-server:{bind}"),
Self::TcpClient(targets) => format!("tcp-client:{}", targets.join(",")),
}
}
}
/// Bind-scope guard for TCP server mode — see `ReticulumInterface` doc
/// comment. Mirrors `reticulum_daemon.py`'s `_require_loopback`; enforced
/// here too (defense in depth) since this is the gate an operator/caller
/// actually goes through in Rust.
fn is_loopback_host(host: &str) -> bool {
matches!(host, "127.0.0.1" | "::1" | "localhost")
}
/// `archy_ed_pubkey_hex`/`archy_x25519_pubkey_hex` (when known) are embedded
/// by the daemon in its announce app_data as `ARCHY:2:{ed}:{x25519}` — the
/// SAME wire format meshcore/Meshtastic identity adverts use — so a
@ -72,7 +117,7 @@ const RETICULUM_BINARY_CONTENT_MARKER: &str = "\u{0}b64:";
/// satisfying cross-protocol DM convergence with zero new Rust dispatch code.
fn daemon_command(
socket_path: &Path,
serial_port: &str,
iface: &ReticulumInterface<'_>,
identity_key: &Path,
archy_ed_pubkey_hex: Option<&str>,
archy_x25519_pubkey_hex: Option<&str>,
@ -94,35 +139,72 @@ fn daemon_command(
cmd.arg("--identity-key")
.arg(identity_key)
.arg("--socket")
.arg(socket_path)
.arg("--serial-port")
.arg(serial_port);
.arg(socket_path);
match iface {
ReticulumInterface::Serial(path) => {
cmd.arg("--serial-port").arg(path);
}
ReticulumInterface::TcpServer(bind) => {
cmd.arg("--tcp-listen").arg(bind);
}
ReticulumInterface::TcpClient(targets) => {
for target in *targets {
cmd.arg("--tcp-connect").arg(target);
}
}
}
if let (Some(ed), Some(x)) = (archy_ed_pubkey_hex, archy_x25519_pubkey_hex) {
cmd.arg("--archy-ed-pubkey-hex")
.arg(ed)
.arg("--archy-x25519-pubkey-hex")
.arg(x);
}
cmd.kill_on_drop(true)
// Run the daemon as its own process-group leader. The packaged binary is
// a PyInstaller one-file bootloader that forks the real Python process;
// making it a group leader lets Drop signal the WHOLE group so the
// forked child can't be orphaned and keep holding the serial port.
.process_group(0)
// Run the daemon as its own process-group leader. The packaged binary is
// a PyInstaller one-file bootloader that forks the real Python process;
// making it a group leader lets shutdown signal the WHOLE group so the
// forked child can't be orphaned and keep holding the serial port.
// Deliberately NOT `kill_on_drop`: that SIGKILLs the bootloader the instant
// the `Child` drops, before it can delete its `_MEI*` extraction dir (48M
// leaked into TMPDIR per daemon restart — filled .116's 12G /tmp tmpfs).
// Shutdown goes through `terminate_group` instead, on every path.
cmd.process_group(0)
.stdin(std::process::Stdio::null())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped());
cmd
}
/// Gracefully stop a daemon process group: SIGTERM now (the daemon's handler
/// releases the RNode + socket, and the PyInstaller bootloader gets to delete
/// its `_MEI*` extraction dir once the Python child exits), SIGKILL from a
/// detached backstop thread a few seconds later so a wedged daemon still can't
/// survive or keep holding the serial port.
fn terminate_group(child: &Child) {
if let Some(pid) = child.id() {
let pgid = -(pid as i32);
unsafe {
libc::kill(pgid, libc::SIGTERM);
}
std::thread::spawn(move || {
std::thread::sleep(Duration::from_secs(5));
unsafe {
libc::kill(pgid, libc::SIGKILL);
}
});
}
}
/// One peer learned via an RNS announce (LXMF delivery destination).
#[derive(Clone)]
struct ReticulumPeer {
dest_hash: [u8; 16],
display_name: String,
/// Archy ed25519 identity hex, once carried in a verified announce
/// app-data blob. Not yet wired (TODO, lands with the signed-announce
/// work) — present so `get_contacts` has a stable shape to extend into.
/// Archy ed25519 identity hex, once carried in this peer's own announce
/// app-data blob (`ARCHY:n:...`) — see `handle_event`'s "announce" arm.
/// Unlike Meshcore/Meshtastic, this arrives in-band with the same event
/// as the destination hash, so it can be bound directly onto this
/// RNS-hash-keyed peer with no name-matching ambiguity (contrast
/// `bind_federation_twins`, which those two transports rely on instead).
arch_pubkey_hex: Option<String>,
reachable: bool,
}
@ -189,11 +271,62 @@ impl ReticulumLink {
our_x25519_pubkey_hex: Option<&str>,
) -> Result<Self> {
probe_rnode(path).await.context("RNode KISS detect failed")?;
Self::spawn(path, data_dir, our_ed_pubkey_hex, our_x25519_pubkey_hex).await
Self::spawn(
ReticulumInterface::Serial(path),
data_dir,
our_ed_pubkey_hex,
our_x25519_pubkey_hex,
)
.await
}
/// Bring up a daemon in plain-TCP server mode — no physical RNode, no
/// `probe_rnode` gate. `bind` (`host:port`) must be loopback; see
/// `ReticulumInterface`/`is_loopback_host`.
pub async fn open_tcp_server(
bind: &str,
data_dir: &Path,
our_ed_pubkey_hex: Option<&str>,
our_x25519_pubkey_hex: Option<&str>,
) -> Result<Self> {
let host = bind.rsplit_once(':').map(|(h, _)| h).unwrap_or(bind);
anyhow::ensure!(
is_loopback_host(host),
"reticulum TCP server bind must be loopback-only (127.0.0.1/::1/localhost) — \
got {bind}; WAN/LAN exposure needs its own security review"
);
Self::spawn(
ReticulumInterface::TcpServer(bind),
data_dir,
our_ed_pubkey_hex,
our_x25519_pubkey_hex,
)
.await
}
/// Bring up a daemon in plain-TCP client mode, dialing one or more
/// `host:port` targets — no physical RNode, no `probe_rnode` gate.
pub async fn open_tcp_client(
targets: &[String],
data_dir: &Path,
our_ed_pubkey_hex: Option<&str>,
our_x25519_pubkey_hex: Option<&str>,
) -> Result<Self> {
anyhow::ensure!(
!targets.is_empty(),
"reticulum TCP client mode needs at least one target"
);
Self::spawn(
ReticulumInterface::TcpClient(targets),
data_dir,
our_ed_pubkey_hex,
our_x25519_pubkey_hex,
)
.await
}
async fn spawn(
path: &str,
iface: ReticulumInterface<'_>,
data_dir: &Path,
our_ed_pubkey_hex: Option<&str>,
our_x25519_pubkey_hex: Option<&str>,
@ -212,13 +345,23 @@ impl ReticulumLink {
let _ = tokio::fs::set_permissions(&runtime_dir, std::fs::Permissions::from_mode(0o700))
.await;
}
let socket_path = runtime_dir.join(format!(
"{}.sock",
path.replace(['/', ' '], "_")
));
let label = iface.label();
let iface_key = label.replace(['/', ' ', ':', ','], "_");
let socket_path = runtime_dir.join(format!("{iface_key}.sock"));
if socket_path.exists() {
let _ = std::fs::remove_file(&socket_path);
}
// Private TMPDIR for the PyInstaller one-file bootloader, wiped on every
// (re)spawn: even a hard-killed or power-lost daemon can never accumulate
// stale 48M `_MEI*` extraction dirs, and they land under the data dir
// instead of the small tmpfs /tmp. Per-interface so the live daemons of
// two radios never share a dir (the previous daemon for the SAME
// interface is guaranteed dead before we respawn it).
let tmp_dir = runtime_dir.join("tmp").join(&iface_key);
let _ = tokio::fs::remove_dir_all(&tmp_dir).await;
tokio::fs::create_dir_all(&tmp_dir)
.await
.context("Failed to create reticulum daemon tmp dir")?;
let identity_key = data_dir.join("identity").join("node_key");
if !identity_key.exists() {
anyhow::bail!(
@ -229,58 +372,70 @@ impl ReticulumLink {
let mut cmd = daemon_command(
&socket_path,
path,
&iface,
&identity_key,
our_ed_pubkey_hex,
our_x25519_pubkey_hex,
);
let mut child = cmd
cmd.env("TMPDIR", &tmp_dir);
let child = cmd
.spawn()
.context("Failed to spawn reticulum-daemon — is it installed/packaged?")?;
// Wait for the socket to appear, then for the daemon's "ready" event.
let deadline = tokio::time::Instant::now() + Duration::from_secs(15);
let stream = loop {
if tokio::time::Instant::now() > deadline {
let _ = child.start_kill();
anyhow::bail!("reticulum-daemon did not create its RPC socket in time");
// Runs as a block so every failure path tears the just-spawned daemon
// group down via `terminate_group` (the child has no `kill_on_drop`).
let init = async {
let deadline = tokio::time::Instant::now() + Duration::from_secs(15);
let stream = loop {
if tokio::time::Instant::now() > deadline {
anyhow::bail!("reticulum-daemon did not create its RPC socket in time");
}
match UnixStream::connect(&socket_path).await {
Ok(s) => break s,
Err(_) => tokio::time::sleep(Duration::from_millis(150)).await,
}
};
let (read_half, write_half) = stream.into_split();
let mut reader = BufReader::new(read_half);
let mut line = String::new();
tokio::time::timeout(Duration::from_secs(10), reader.read_line(&mut line))
.await
.context("Timed out waiting for reticulum-daemon ready event")?
.context("reticulum-daemon RPC connection closed before ready")?;
let ready: Value = serde_json::from_str(line.trim())
.context("reticulum-daemon sent a non-JSON ready line")?;
if ready.get("event").and_then(Value::as_str) != Some("ready") {
anyhow::bail!("reticulum-daemon's first message was not 'ready': {ready}");
}
match UnixStream::connect(&socket_path).await {
Ok(s) => break s,
Err(_) => tokio::time::sleep(Duration::from_millis(150)).await,
let dest_hash_hex = ready
.get("dest_hash")
.and_then(Value::as_str)
.context("ready event missing dest_hash")?;
let dest_hash = parse_hash16(dest_hash_hex)?;
let display_name = ready
.get("display_name")
.and_then(Value::as_str)
.map(str::to_string);
info!(
iface = %label,
dest_hash = %dest_hash_hex,
"Reticulum daemon ready"
);
Ok((write_half, reader, dest_hash, display_name))
};
let (write_half, reader, dest_hash, display_name) = match init.await {
Ok(parts) => parts,
Err(e) => {
terminate_group(&child);
return Err(e);
}
};
let (read_half, write_half) = stream.into_split();
let mut reader = BufReader::new(read_half);
let mut line = String::new();
tokio::time::timeout(Duration::from_secs(10), reader.read_line(&mut line))
.await
.context("Timed out waiting for reticulum-daemon ready event")?
.context("reticulum-daemon RPC connection closed before ready")?;
let ready: Value = serde_json::from_str(line.trim())
.context("reticulum-daemon sent a non-JSON ready line")?;
if ready.get("event").and_then(Value::as_str) != Some("ready") {
anyhow::bail!("reticulum-daemon's first message was not 'ready': {ready}");
}
let dest_hash_hex = ready
.get("dest_hash")
.and_then(Value::as_str)
.context("ready event missing dest_hash")?;
let dest_hash = parse_hash16(dest_hash_hex)?;
let display_name = ready
.get("display_name")
.and_then(Value::as_str)
.map(str::to_string);
info!(
path = %path,
dest_hash = %dest_hash_hex,
"Reticulum daemon ready"
);
let mut link = Self {
device_path: path.to_string(),
device_path: label,
socket_path,
child,
writer: write_half,
@ -544,6 +699,7 @@ impl ReticulumLink {
snr: None,
lat: None,
lon: None,
arch_pubkey_hex: p.arch_pubkey_hex.clone(),
})
.collect())
}
@ -622,17 +778,21 @@ impl ReticulumLink {
.filter(|s| !s.is_empty());
// If the announce app_data is an ARCHY:n: identity blob (see
// daemon_command's doc comment), bind it onto this peer AND
// surface it through the SAME channel-text path
// meshcore/Meshtastic identity adverts use
// daemon_command's doc comment), bind the ed25519 hex directly
// onto this RNS-hash-keyed peer below (unambiguous — it came
// in the same event as `hash`) AND surface it through the same
// channel-text path meshcore/Meshtastic identity adverts use
// (frames::handle_channel_payload -> parse_identity_broadcast
// -> handle_identity_received -> bind_federation_twins), so a
// Reticulum-carried identity merges into the same conversation
// as that node's other-transport twins — zero new bind logic.
let is_identity_blob = app_data_text
// -> handle_identity_received), so it also lands on that
// transport's federation-twin peer for UI/contact purposes.
// `group_peer_twins` can then collapse the two rows since both
// now carry the same `arch_pubkey_hex`, instead of relying on
// `bind_federation_twins`'s advert_name matching, which never
// matches here — see `display_name` below.
let parsed_identity = app_data_text
.as_deref()
.map(|t| protocol::parse_identity_broadcast(t).is_some())
.unwrap_or(false);
.and_then(protocol::parse_identity_broadcast);
let is_identity_blob = parsed_identity.is_some();
if is_identity_blob {
let text = app_data_text.clone().unwrap();
let mut data = Vec::with_capacity(7 + text.len());
@ -645,6 +805,7 @@ impl ReticulumLink {
bytes_consumed: 0,
});
}
let arch_pubkey_hex = parsed_identity.map(|(_did, ed_pubkey, _x25519)| ed_pubkey);
let display_name = app_data_text
.filter(|_| !is_identity_blob)
@ -654,11 +815,14 @@ impl ReticulumLink {
.and_modify(|p| {
p.display_name = display_name.clone();
p.reachable = true;
if arch_pubkey_hex.is_some() {
p.arch_pubkey_hex = arch_pubkey_hex.clone();
}
})
.or_insert(ReticulumPeer {
dest_hash: hash,
display_name,
arch_pubkey_hex: None,
arch_pubkey_hex,
reachable: true,
});
self.persist_peers();
@ -904,6 +1068,14 @@ fn parse_hash16(hex_str: &str) -> Result<[u8; 16]> {
async fn probe_rnode(path: &str) -> Result<()> {
let port = serial2_tokio::SerialPort::open(path, PROBE_BAUD)
.with_context(|| format!("Failed to open {} for Reticulum probe", path))?;
// ESP32-S3 native-USB boards (Heltec V3/V4 etc. — no separate USB-UART
// bridge chip) treat a DTR/RTS transition on open as a reset signal, the
// same mechanism esptool uses to force bootloader entry. Deassert both
// and let the board settle before writing the probe, or the reboot eats
// the DETECT_RESP window below.
let _ = port.set_dtr(false);
let _ = port.set_rts(false);
tokio::time::sleep(Duration::from_millis(300)).await;
let probe: [u8; 13] = [
KISS_FEND,
KISS_CMD_DETECT,
@ -950,22 +1122,12 @@ fn contains_detect_resp(buf: &[u8]) -> bool {
impl Drop for ReticulumLink {
fn drop(&mut self) {
// The packaged daemon is a PyInstaller one-file bootloader that forks the
// real Python process into the same (leader) group we spawned it in.
// SIGKILLing only the bootloader (what `start_kill`/`kill_on_drop` do)
// orphans that child, which keeps holding the serial port — the root
// cause of daemons piling up across reconnects and jamming the RNode.
// Signal the whole process group instead: SIGTERM is caught by the
// daemon's handler (clean RNode + socket release), and SIGKILL is the
// hard backstop so a wedged daemon can never survive.
if let Some(pid) = self.child.id() {
let pgid = -(pid as i32);
unsafe {
libc::kill(pgid, libc::SIGTERM);
libc::kill(pgid, libc::SIGKILL);
}
}
let _ = self.child.start_kill();
// Group-wide SIGTERM with a delayed SIGKILL backstop (`terminate_group`).
// The old immediate SIGTERM+SIGKILL never let the PyInstaller bootloader
// run its exit cleanup, stranding a 48M `_MEI*` extraction dir on every
// reconnect/restart until /tmp filled; the grace period fixes that, and
// the per-spawn TMPDIR wipe in `spawn` covers any dir that still leaks.
terminate_group(&self.child);
let _ = std::fs::remove_file(&self.socket_path);
}
}

View File

@ -176,7 +176,9 @@ async fn fire_due(scheduler: &Arc<MeshScheduler>, state: &Arc<MeshState>) {
}
q.retain(|m| !to_remove.contains(&m.id));
}
let _ = scheduler.save().await;
if let Err(e) = scheduler.save().await {
warn!("Failed to persist mesh outbox after sweep: {e:#}");
}
}
/// Hand a due message to the radio. Returns true if it was sent (or should be

View File

@ -57,6 +57,12 @@ impl MeshcoreDevice {
"Failed to open serial port {} (permission denied? device busy?)",
path
))?;
// See probe_rnode() in reticulum.rs for why: ESP32-S3 native-USB
// boards reset on a DTR/RTS transition, so deassert both and settle
// before the handshake below.
let _ = port.set_dtr(false);
let _ = port.set_rts(false);
tokio::time::sleep(Duration::from_millis(300)).await;
info!(path = %path, baud = BAUD_RATE, "Opened serial port");

View File

@ -28,6 +28,23 @@ impl std::fmt::Display for DeviceType {
}
}
/// Optional plain-TCP Reticulum interface, radio-less alternative to the
/// serial-RNode path. Dev/verification surface for now (e.g. proving
/// interop with Aurora's `RnsTcpInterface`/`RnsTcpServerInterface`, which is
/// its default connectivity mode) — not exposed through `mesh.configure`/the
/// frontend. `Server` is hard-gated to loopback in both the daemon and Rust
/// (`reticulum::is_loopback_host`); WAN/LAN exposure is a separate, deliberate
/// future decision (archy is otherwise Tor-first for inter-node traffic).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "mode", rename_all = "snake_case")]
pub enum ReticulumTcpConfig {
/// Bind a `TCPServerInterface`. `bind` is `host:port`, host must be
/// loopback (127.0.0.1/::1/localhost).
Server { bind: String },
/// Dial one or more `TCPClientInterface` targets (`host:port`).
Client { connect: Vec<String> },
}
/// The per-message transport pill label for a radio-delivered message: the
/// active device's own name, since one session owns exactly one device.
/// Federation sends/receives are labelled "fips"/"tor" elsewhere — this only

View File

@ -267,7 +267,10 @@ impl Server {
info!("📡 Auto-detected mesh radio: {:?} — enabling mesh", devices);
mesh_config.enabled = true;
mesh_config.device_path = Some(devices[0].clone());
let _ = crate::mesh::save_config(&data_dir, &mesh_config).await;
if let Err(e) = crate::mesh::save_config(&data_dir, &mesh_config).await
{
warn!("Failed to persist auto-detected mesh config: {e:#}");
}
}
}
@ -1031,9 +1034,13 @@ async fn accept_loop(
let permit = active_connections.clone().acquire_owned().await;
tokio::spawn(async move {
let _permit = permit;
let service = service_fn(move |req: hyper::Request<hyper::Body>| {
let service = service_fn(move |mut req: hyper::Request<hyper::Body>| {
let handler = handler.clone();
async move {
// Record the TCP peer so rate limiting only trusts
// forwarded headers on loopback (nginx) connections.
req.extensions_mut()
.insert(crate::api::rpc::PeerAddr(peer_addr));
if peer_only && !is_peer_allowed_path(req.uri().path()) {
let resp = hyper::Response::builder()
.status(hyper::StatusCode::NOT_FOUND)
@ -1177,6 +1184,30 @@ fn merge_preserving_transitional(
{
fresh.state.clone()
}
// A user-initiated stop whose container podman now reports settled
// (the scanner maps exited+user-stopped → Stopped) has visibly
// completed — report it. The stop worker still owns cleanup, but it
// can trail the actual container exit by minutes when it is queued
// behind the orchestrator's per-app lock (reconcile host-port repair
// holds it through multi-minute stability waits). Holding the card
// (and the lifecycle gate) in "Stopping" that whole time reports a
// completed stop as stuck (vaultwarden/jellyfin, gate runs C/D,
// .228 2026-07-09).
(crate::data_model::PackageState::Stopping, crate::data_model::PackageState::Stopped)
if user_stop_requested =>
{
fresh.state.clone()
}
// Same reasoning for start: once podman reports Running, the start
// has visibly succeeded. The start worker keeps Starting through its
// full readiness wait (host-port probe budgets reach 420s for
// uptime-kuma — longer than any UI/test patience) even though the
// container is up and its live health is already shown separately
// (uptime-kuma, gate run E). Restarting is deliberately NOT mapped:
// mid-restart Running readings are the pre-stop container.
(crate::data_model::PackageState::Starting, crate::data_model::PackageState::Running) => {
fresh.state.clone()
}
// Removing with a live running container is stale: uninstall either
// failed or Archipelago restarted before the spawned task could revert
// state. Let the scanner recover the UI immediately instead of
@ -1203,6 +1234,21 @@ fn merge_preserving_transitional(
}
}
/// Package ids whose `Restarting` state was written by the scanner's
/// pending-boot-start overlay (not by an RPC restart task). For these, the
/// scan is the owner: once podman reports a settled state and the id is no
/// longer queued for a boot start, the fresh state wins immediately instead
/// of being preserved for the transitional-stuck timeout.
static SCANNER_RESTARTING: std::sync::LazyLock<std::sync::Mutex<std::collections::HashSet<String>>> =
std::sync::LazyLock::new(|| std::sync::Mutex::new(std::collections::HashSet::new()));
fn take_scanner_restarting(id: &str) -> bool {
SCANNER_RESTARTING
.lock()
.map(|mut set| set.remove(id))
.unwrap_or(false)
}
fn is_podman_scan_timeout(error: &anyhow::Error) -> bool {
let msg = format!("{:#}", error);
msg.contains("podman ps") && msg.contains("timed out")
@ -1223,6 +1269,31 @@ async fn scan_and_update_packages(
pkg.state = crate::data_model::PackageState::Stopped;
pkg.exit_code = None;
}
// A down container that boot recovery / the reconciler is queued to
// start is "Restarting", not "Stopped" — after a reboot the sequential
// recovery pass can take minutes to reach heavyweights, and telling
// the user their app stopped when it's about to come back is wrong.
// Ids overlaid here are recorded in SCANNER_RESTARTING so the merge
// below knows this Restarting is scanner-authored (resolve it as soon
// as podman reports a settled state) and not owned by an RPC restart
// task (whose transitional state must be preserved).
// …but never for a user-stopped app: the stop marker is written
// before the stop runs, and a reconcile pass that queued the app
// moments earlier must not repaint the user's deliberate Stopped as
// Restarting (gate iteration-5 vaultwarden stop-wait, 2026-07-09 —
// the overlay held 'restarting' past the 120s window).
if matches!(
pkg.state,
crate::data_model::PackageState::Stopped | crate::data_model::PackageState::Exited
) && crate::crash_recovery::is_pending_boot_start(id)
&& !user_stopped.contains(id)
{
pkg.state = crate::data_model::PackageState::Restarting;
pkg.exit_code = None;
if let Ok(mut set) = SCANNER_RESTARTING.lock() {
set.insert(id.clone());
}
}
}
normalize_reachable_package_health(&mut packages).await;
@ -1273,6 +1344,19 @@ async fn scan_and_update_packages(
absence_tracker.remove(id);
let existing = merged.get(id);
let overwrite = match existing {
// Scanner-authored Restarting (the pending-boot-start overlay)
// resolves as soon as the fresh scan reports anything else: the
// scan is its owner — no RPC task will ever write a final state
// back. Without this, a successfully recovered container would
// sit wedged in "Restarting" until the 20-minute stuck timeout.
Some(existing_entry)
if existing_entry.state == crate::data_model::PackageState::Restarting
&& pkg.state != crate::data_model::PackageState::Restarting
&& take_scanner_restarting(id) =>
{
transitional_since.remove(id);
true
}
Some(existing_entry) if is_transitional(&existing_entry.state) => {
let entered = *transitional_since.entry(id.clone()).or_insert(now);
let timeout = transitional_stuck_timeout(&existing_entry.state);
@ -1654,6 +1738,49 @@ mod merge_tests {
assert_eq!(merged.health.as_deref(), Some("healthy"));
}
#[test]
fn user_stop_resolves_when_container_has_exited() {
// The container exited and the scanner already normalized
// exited+user-stopped to Stopped — the stop visibly completed, even
// if the stop worker is still queued behind the per-app lock.
let existing = make_entry(PackageState::Stopping, Some("unknown"));
let fresh = make_entry(PackageState::Stopped, None);
let merged = merge_preserving_transitional(&existing, &fresh, true);
assert_eq!(merged.state, PackageState::Stopped);
}
#[test]
fn non_user_stopping_with_exited_container_stays_owned() {
// No user-stop marker → this Stopping belongs to some other flow;
// don't resolve it from a scan.
let existing = make_entry(PackageState::Stopping, Some("unknown"));
let fresh = make_entry(PackageState::Stopped, None);
let merged = merge_preserving_transitional(&existing, &fresh, false);
assert_eq!(merged.state, PackageState::Stopping);
}
#[test]
fn starting_resolves_when_container_is_running() {
// Start worker may still be inside its readiness wait (up to 420s for
// uptime-kuma) — but podman reporting Running means the start visibly
// succeeded; live health is merged separately.
let existing = make_entry(PackageState::Starting, Some("starting"));
let fresh = make_entry(PackageState::Running, Some("healthy"));
let merged = merge_preserving_transitional(&existing, &fresh, false);
assert_eq!(merged.state, PackageState::Running);
assert_eq!(merged.health.as_deref(), Some("healthy"));
}
#[test]
fn restarting_is_not_resolved_by_running_scan() {
// Mid-restart the pre-stop container still reads Running — the
// restart worker owns this state until it finishes.
let existing = make_entry(PackageState::Restarting, Some("healthy"));
let fresh = make_entry(PackageState::Running, Some("healthy"));
let merged = merge_preserving_transitional(&existing, &fresh, false);
assert_eq!(merged.state, PackageState::Restarting);
}
#[test]
fn merges_fresh_observability_fields() {
// Non-state observability fields (health, exit_code, installed)

View File

@ -23,26 +23,36 @@ use std::time::{Duration, SystemTime, UNIX_EPOCH};
/// TTL keeps the result responsive to daemon flaps without pounding DBus.
const AVAILABILITY_CACHE_TTL: Duration = Duration::from_secs(10);
/// Availability cache shared with the background probe thread, so the
/// sync `is_available()` hot path never blocks on `systemctl`.
struct AvailabilityCache {
available: AtomicBool,
probed_at_ms: AtomicU64,
probe_in_flight: AtomicBool,
}
pub struct FipsTransport {
identity_dir: PathBuf,
available_cached: AtomicBool,
available_cached_at_ms: AtomicU64,
availability: std::sync::Arc<AvailabilityCache>,
}
impl FipsTransport {
pub fn new(identity_dir: &Path) -> Self {
Self {
identity_dir: identity_dir.to_path_buf(),
available_cached: AtomicBool::new(false),
available_cached_at_ms: AtomicU64::new(0),
availability: std::sync::Arc::new(AvailabilityCache {
available: AtomicBool::new(false),
probed_at_ms: AtomicU64::new(0),
probe_in_flight: AtomicBool::new(false),
}),
}
}
fn probe_daemon_active() -> bool {
// Cheap blocking probe: spawn `systemctl is-active` synchronously.
// Short-circuit if either the archipelago-managed unit or the
// upstream fips.service is active — legacy/dev nodes run only the
// upstream unit.
// Blocking probe — only ever run on a dedicated background thread
// (see is_available), never on a tokio worker. Short-circuit if
// either the archipelago-managed unit or the upstream fips.service
// is active — legacy/dev nodes run only the upstream unit.
for unit in [
crate::fips::SERVICE_UNIT,
crate::fips::UPSTREAM_SERVICE_UNIT,
@ -70,14 +80,30 @@ impl NodeTransport for FipsTransport {
.duration_since(UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
let cached_at = self.available_cached_at_ms.load(Ordering::Relaxed);
let cached_at = self.availability.probed_at_ms.load(Ordering::Relaxed);
let cached = self.availability.available.load(Ordering::Relaxed);
if now_ms.saturating_sub(cached_at) < AVAILABILITY_CACHE_TTL.as_millis() as u64 {
return self.available_cached.load(Ordering::Relaxed);
return cached;
}
let val = Self::probe_daemon_active();
self.available_cached.store(val, Ordering::Relaxed);
self.available_cached_at_ms.store(now_ms, Ordering::Relaxed);
val
// Cache is stale. This sync trait method is called from async
// route(), so running the ~50ms systemctl probe inline stalls a
// tokio worker. Serve the stale value and refresh on a background
// thread instead — the transport supervisor's warm loop keeps this
// fresh in steady state, so staleness is bounded to one probe round.
let cache = std::sync::Arc::clone(&self.availability);
if !cache.probe_in_flight.swap(true, Ordering::Relaxed) {
std::thread::spawn(move || {
let val = Self::probe_daemon_active();
let probed_ms = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_millis() as u64)
.unwrap_or(0);
cache.available.store(val, Ordering::Relaxed);
cache.probed_at_ms.store(probed_ms, Ordering::Relaxed);
cache.probe_in_flight.store(false, Ordering::Relaxed);
});
}
cached
}
fn send<'a>(

View File

@ -16,9 +16,13 @@ use ed25519_dalek::VerifyingKey;
/// Hex of the pinned Ed25519 release-root public key (32 bytes / 64 hex chars).
///
/// TODO(dht Phase 0): bake the real value here after the signing ceremony.
/// Generate it with: `scripts/release-root-ceremony.sh pubkey`.
pub const RELEASE_ROOT_PUBKEY_HEX: Option<&str> = None;
/// Pinned 2026-07-02 from the release-root signing ceremony
/// (signer did:key:z6MkkidEnEpo6qHMCNSZoNKWtvQvxq3whnaME9wGgEFhq7ur). The
/// corresponding mnemonic is held offline by the publisher — see
/// `docs/workstream-b-signing-runbook.md`. Regenerate/verify with:
/// `RELEASE_MASTER_MNEMONIC=… archipelago ceremony pubkey`.
pub const RELEASE_ROOT_PUBKEY_HEX: Option<&str> =
Some("5d15cbee8a108f7dd288c02d29a1d9d71f198acc99186aad8008b4f28d469951");
const ENV_OVERRIDE: &str = "ARCHY_RELEASE_ROOT_PUBKEY";
@ -51,9 +55,14 @@ mod tests {
use super::*;
#[test]
fn unset_constant_is_none() {
// Default build ships no pinned anchor yet.
assert!(RELEASE_ROOT_PUBKEY_HEX.is_none());
fn pinned_constant_parses_to_a_valid_key() {
// The release-root anchor is pinned (ceremony 2026-07-02); it must be
// present and a well-formed 32-byte Ed25519 key.
let hex = RELEASE_ROOT_PUBKEY_HEX.expect("release-root anchor must be pinned");
assert!(
parse_pubkey_hex(hex).is_some(),
"pinned RELEASE_ROOT_PUBKEY_HEX is not a valid Ed25519 key"
);
}
#[test]

View File

@ -149,6 +149,18 @@ mod tests {
doc
}
/// Pin `test_key` as the release-root anchor for this process via the env
/// override. Needed because the baked-in `RELEASE_ROOT_PUBKEY_HEX` is the
/// real ceremony key, which no unit test can produce signatures for — so to
/// exercise the anchored-verification path we pin a key we can sign with.
/// Every call sets the same value, so parallel tests stay consistent.
fn pin_test_key_as_anchor() {
std::env::set_var(
"ARCHY_RELEASE_ROOT_PUBKEY",
hex::encode(test_key().verifying_key().to_bytes()),
);
}
#[test]
fn unsigned_document_reports_unsigned() {
let doc = json!({"schema": 1, "apps": {}});
@ -156,18 +168,31 @@ mod tests {
}
#[test]
fn roundtrip_verifies() {
fn roundtrip_verifies_and_anchors_to_pinned_key() {
// With the anchor pinned to the signer, verification succeeds AND
// reports anchored == true (signer identity confirmed).
pin_test_key_as_anchor();
let signed = sign_into(&test_key(), json!({"schema": 1, "n": 42}));
match verify_detached(&signed).unwrap() {
// No anchor pinned in the default test build → anchored == false.
SignatureStatus::Verified { anchored, .. } => assert!(!anchored),
SignatureStatus::Verified { anchored, .. } => assert!(anchored),
other => panic!("expected Verified, got {:?}", other),
}
}
#[test]
fn signature_from_non_anchor_key_is_rejected() {
// A self-consistent signature from a key that is NOT the pinned anchor
// must hard-reject — this is what stops a mirror swapping in its own key.
pin_test_key_as_anchor();
let other_key = SigningKey::from_bytes(&[11u8; 32]);
let signed = sign_into(&other_key, json!({"schema": 1, "n": 42}));
assert!(verify_detached(&signed).is_err());
}
#[test]
fn signature_survives_key_reordering() {
// Re-emitting the document with shuffled keys must not break the sig.
pin_test_key_as_anchor();
let signed = sign_into(&test_key(), json!({"b": 2, "a": 1}));
let reparsed: Value =
serde_json::from_str(&serde_json::to_string(&signed).unwrap()).unwrap();
@ -179,6 +204,9 @@ mod tests {
#[test]
fn tampered_payload_is_rejected() {
// Pin the signer so verification reaches the signature check (not an
// anchor-identity short-circuit), proving tamper detection itself.
pin_test_key_as_anchor();
let mut signed = sign_into(&test_key(), json!({"schema": 1, "n": 42}));
signed
.as_object_mut()

View File

@ -160,7 +160,9 @@ pub async fn load_mirrors(data_dir: &Path) -> Result<Vec<UpdateMirror>> {
force_ovh_update_primary(&mut list);
changed = changed || before_order != list.iter().map(|m| m.url.clone()).collect::<Vec<_>>();
if changed {
let _ = save_mirrors(data_dir, &list).await;
if let Err(e) = save_mirrors(data_dir, &list).await {
warn!("Failed to persist migrated update-mirrors list: {e:#}");
}
}
Ok(list)
}
@ -338,6 +340,26 @@ fn rewrite_manifest_origins(manifest: &mut UpdateManifest, manifest_url: &str) {
}
}
/// Parse a fetched manifest body, verifying its detached release-root
/// signature if one is present. Returns the manifest plus whether it was
/// signed by the pinned release-root anchor.
///
/// * Present-but-invalid signature (tampered payload, wrong signer, bad
/// hex…) → `Err`; the caller treats the mirror as failed. This is the
/// teeth: a MITM can strip the signature but can never forge one.
/// * Unsigned → accepted with `signed == false` during the migration
/// window; the scheduler refuses to auto-apply such manifests and the
/// state surfaces `manifest_signed` to the UI.
fn parse_and_verify_manifest(raw: serde_json::Value) -> Result<(UpdateManifest, bool)> {
let signed = match crate::trust::verify_detached(&raw).context("manifest signature")? {
crate::trust::SignatureStatus::Verified { anchored, .. } => anchored,
crate::trust::SignatureStatus::Unsigned => false,
};
let manifest: UpdateManifest =
serde_json::from_value(raw).context("parse update manifest")?;
Ok((manifest, signed))
}
/// Which manifest URL to try FIRST — operator override via env wins,
/// otherwise the first entry in the mirrors list, otherwise the hard
/// default. Callers that need the full mirror walk should use
@ -394,6 +416,14 @@ pub struct UpdateState {
/// their node actually hit (vs. just which is configured primary).
#[serde(default)]
pub manifest_mirror: Option<String>,
/// True when `available_update` came from a manifest whose detached
/// Ed25519 signature verified against the pinned release-root anchor
/// (`trust::verify_detached`). Unsigned manifests are still offered for
/// MANUAL apply during the signing-migration window, but the scheduler
/// refuses to auto-apply them — otherwise a mirror/MITM could push an
/// arbitrary root binary to the fleet unattended.
#[serde(default)]
pub manifest_signed: bool,
}
impl Default for UpdateState {
@ -406,6 +436,7 @@ impl Default for UpdateState {
rollback_available: false,
schedule: UpdateSchedule::DailyCheck,
manifest_mirror: None,
manifest_signed: false,
}
}
}
@ -482,8 +513,64 @@ async fn probe_frontend_once() -> Result<()> {
anyhow::bail!("frontend probe returned HTTP {}", status);
}
/// Probe the backend RPC API through nginx. An unauthenticated call is
/// EXPECTED to get 401/403 — any such response proves the Rust HTTP stack
/// is alive behind nginx. 5xx (backend dead → nginx 502), 404 (proxy
/// misroute), or connection errors mean the API is down even though the
/// static frontend may still serve — exactly the failure mode the plain
/// `GET /` probe waved through.
async fn probe_backend_once() -> Result<()> {
let client = reqwest::Client::builder()
.danger_accept_invalid_certs(true)
.timeout(std::time::Duration::from_secs(5))
.build()
.context("build probe client")?;
let body = serde_json::json!({ "method": "update.status" });
let resp = match client
.post("https://127.0.0.1/rpc/v1")
.json(&body)
.send()
.await
{
Ok(resp) => resp,
Err(e) if e.is_connect() => client
.post("http://127.0.0.1/rpc/v1")
.json(&body)
.send()
.await
.context("probe POST http://127.0.0.1/rpc/v1 (https not bound on loopback)")?,
Err(e) => return Err(e).context("probe POST https://127.0.0.1/rpc/v1"),
};
let status = resp.status();
if status.is_server_error() || status == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!("backend RPC probe returned HTTP {}", status);
}
Ok(())
}
/// Probe that the rootless container runtime is reachable from the new
/// binary (uid mapping / podman socket intact after the swap). A healthy
/// node answers `podman ps` in well under a second.
async fn probe_container_runtime_once() -> Result<()> {
let out = tokio::process::Command::new("podman")
.args(["ps", "--format", "{{.Names}}"])
.output()
.await
.context("spawn podman ps")?;
if !out.status.success() {
anyhow::bail!(
"podman ps exited {}: {}",
out.status,
String::from_utf8_lossy(&out.stderr).trim()
);
}
Ok(())
}
/// Called from main.rs startup. If a pending-verification marker is
/// present, probe the frontend; on failure, trigger rollback and
/// present, verify the node actually works on the new version — binary
/// version matches the marker, frontend serves, backend RPC answers,
/// rootless podman is reachable. On failure, trigger rollback and
/// restart the service so the OLD binary boots.
///
/// This is the "post-OTA auto-rollback" guardrail. If ANY problem in
@ -516,34 +603,59 @@ pub async fn verify_pending_update(data_dir: &Path) {
info!(
new_version = %marker.new_version,
previous_version = %marker.previous_version,
"Post-OTA verification: probing frontend at https://127.0.0.1/"
"Post-OTA verification: probing frontend, backend RPC, and container runtime"
);
// Give the new service time to bind its listeners + nginx to
// pick up any config changes. 15s matches what we observed on
// .116 during the v1.7.40 rollout recovery.
tokio::time::sleep(std::time::Duration::from_secs(15)).await;
let deadline =
std::time::Instant::now() + std::time::Duration::from_secs(PENDING_VERIFY_WINDOW_SECS);
// Binary identity check: if the running binary's version isn't the one
// the marker says we applied, the swap silently failed (or half-applied
// — new frontend with old binary). Deterministic, so no retry loop:
// fall through straight to rollback to restore a matched pair.
let running = env!("CARGO_PKG_VERSION");
let mut attempt = 0u32;
let mut last_err: Option<String> = None;
let version_ok = running == marker.new_version;
if !version_ok {
last_err = Some(format!(
"running binary is {} but marker says {} was applied — binary swap failed",
running, marker.new_version
));
} else {
// Give the new service time to bind its listeners + nginx to
// pick up any config changes. 15s matches what we observed on
// .116 during the v1.7.40 rollout recovery.
tokio::time::sleep(std::time::Duration::from_secs(15)).await;
while std::time::Instant::now() < deadline {
attempt += 1;
match probe_frontend_once().await {
Ok(()) => {
info!(attempt, "Post-OTA verification succeeded — clearing marker");
clear_pending_verification(data_dir).await;
return;
}
Err(e) => {
let msg = e.to_string();
tracing::warn!(attempt, error = %msg, "Post-OTA probe failed, retrying");
last_err = Some(msg);
let deadline = std::time::Instant::now()
+ std::time::Duration::from_secs(PENDING_VERIFY_WINDOW_SECS);
while std::time::Instant::now() < deadline {
attempt += 1;
// All three must pass in the same attempt: static frontend via
// nginx, backend RPC liveness, and rootless-podman reachability.
// (Individual app containers are NOT asserted — the pre-Quadlet
// service restart legitimately takes them down and the boot
// reconciler can need minutes to bring heavy apps back.)
let result = match probe_frontend_once().await {
Ok(()) => match probe_backend_once().await {
Ok(()) => probe_container_runtime_once().await,
Err(e) => Err(e),
},
Err(e) => Err(e),
};
match result {
Ok(()) => {
info!(attempt, "Post-OTA verification succeeded — clearing marker");
clear_pending_verification(data_dir).await;
return;
}
Err(e) => {
let msg = format!("{e:#}");
tracing::warn!(attempt, error = %msg, "Post-OTA probe failed, retrying");
last_err = Some(msg);
}
}
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
}
tokio::time::sleep(std::time::Duration::from_secs(5)).await;
}
tracing::error!(
@ -621,6 +733,7 @@ pub async fn load_state(data_dir: &Path) -> Result<UpdateState> {
// if there's genuinely something newer.
state.available_update = None;
state.manifest_mirror = None;
state.manifest_signed = false;
changed = true;
}
@ -710,19 +823,33 @@ pub async fn check_for_updates(data_dir: &Path) -> Result<UpdateState> {
tokio::time::sleep(std::time::Duration::from_secs(2)).await;
}
match client.get(manifest_url).send().await {
Ok(resp) if resp.status().is_success() => match resp.json::<UpdateManifest>().await
Ok(resp) if resp.status().is_success() => match resp
.json::<serde_json::Value>()
.await
.map_err(anyhow::Error::from)
.and_then(parse_and_verify_manifest)
{
Ok(mut manifest) => {
Ok((mut manifest, signed)) => {
rewrite_manifest_origins(&mut manifest, manifest_url);
if is_newer(&manifest.version, &state.current_version) {
if !signed {
warn!(
available = %manifest.version,
mirror = %manifest_url,
"Update manifest is NOT signed by the release root — \
offering for manual apply only; auto-apply is refused"
);
}
info!(
current = %state.current_version,
available = %manifest.version,
mirror = %manifest_url,
signed,
"Update available"
);
state.available_update = Some(manifest);
state.manifest_mirror = Some(manifest_url.clone());
state.manifest_signed = signed;
} else {
// Manifest version matches us or is behind
// us — either we're current, or this mirror
@ -737,6 +864,7 @@ pub async fn check_for_updates(data_dir: &Path) -> Result<UpdateState> {
);
state.manifest_mirror = None;
state.available_update = None;
state.manifest_signed = false;
handled = true;
continue 'mirrors;
}
@ -1130,14 +1258,22 @@ async fn download_component_resumable(
}
}
if canceled {
// Best-effort: the partial file is re-validated (size + sha)
// when the download resumes, so a lost tail here is harmless.
let _ = file.flush().await;
drop(file);
DOWNLOAD_TOTAL.store(0, Ordering::Relaxed);
DOWNLOAD_BYTES.store(0, Ordering::Relaxed);
anyhow::bail!("Download canceled");
}
let _ = file.flush().await;
let _ = file.sync_all().await;
// A failed flush/sync surfaces later as a size/sha mismatch and a
// resume, but log it — this is where ENOSPC first shows itself.
if let Err(e) = file.flush().await {
warn!("Staging file flush failed: {e:#}");
}
if let Err(e) = file.sync_all().await {
warn!("Staging file sync failed: {e:#}");
}
drop(file);
if stream_err {
continue;
@ -1220,7 +1356,9 @@ pub async fn cancel_download(data_dir: &Path) -> Result<()> {
if let Ok(mut state) = load_state(data_dir).await {
if state.update_in_progress {
state.update_in_progress = false;
let _ = save_state(data_dir, &state).await;
if let Err(e) = save_state(data_dir, &state).await {
warn!("Failed to persist cleared update_in_progress: {e:#}");
}
cleared_marker = true;
}
}
@ -1759,6 +1897,23 @@ async fn apply_per_app_auto_updates(
}
}
/// After a catalog refresh that changed the cached bytes, rebuild the
/// orchestrator's manifest map so registry-shipped manifest changes take
/// effect now instead of at the next service restart.
async fn reload_manifests_if_changed(
refresh: crate::container::app_catalog::CatalogRefresh,
orchestrator: &Option<std::sync::Arc<dyn crate::container::traits::ContainerOrchestrator>>,
) {
if !refresh.changed {
return;
}
let Some(orch) = orchestrator else { return };
match orch.reload_manifests().await {
Ok(n) => info!("Update scheduler: catalog changed, reloaded {n} manifest(s)"),
Err(e) => warn!("Update scheduler: manifest reload after catalog change failed: {e}"),
}
}
pub async fn run_update_scheduler(
data_dir: std::path::PathBuf,
orchestrator: Option<std::sync::Arc<dyn crate::container::traits::ContainerOrchestrator>>,
@ -1770,11 +1925,12 @@ pub async fn run_update_scheduler(
// Refresh the app catalog once at startup so per-app "update available"
// badges appear without waiting for the first hourly tick.
if let Err(e) = crate::container::app_catalog::refresh_catalog(&data_dir).await {
debug!(
match crate::container::app_catalog::refresh_catalog(&data_dir).await {
Ok(refresh) => reload_manifests_if_changed(refresh, &orchestrator).await,
Err(e) => debug!(
"Update scheduler: initial app-catalog refresh failed: {}",
e
);
),
}
loop {
@ -1784,8 +1940,11 @@ pub async fn run_update_scheduler(
// populates per-app update availability (the "Update" button still has
// to be clicked — nothing auto-applies). Best-effort; on failure the
// previously cached catalog stays in place (origin-always-wins).
if let Err(e) = crate::container::app_catalog::refresh_catalog(&data_dir).await {
debug!("Update scheduler: app-catalog refresh failed: {}", e);
// A changed catalog also reloads the orchestrator's manifest overlay so
// catalog-shipped manifest fixes apply without a service restart.
match crate::container::app_catalog::refresh_catalog(&data_dir).await {
Ok(refresh) => reload_manifests_if_changed(refresh, &orchestrator).await,
Err(e) => debug!("Update scheduler: app-catalog refresh failed: {}", e),
}
// Per-app auto-update-to-latest (multi-version support). Runs every tick
@ -1846,6 +2005,23 @@ pub async fn run_update_scheduler(
info!("Update scheduler: 3 AM auto-apply window");
match check_for_updates(&data_dir).await {
Ok(s) if s.available_update.is_some() => {
if !s.manifest_signed {
// Unattended apply of an unauthenticated manifest is
// the §A supply-chain hole: a mirror/MITM could ship
// an arbitrary root binary fleet-wide at 3 AM. Manual
// apply from the UI remains possible during the
// signing-migration window; auto-apply does not.
warn!(
available = %s
.available_update
.as_ref()
.map(|m| m.version.as_str())
.unwrap_or("?"),
"Update scheduler: manifest is not signed by the \
release root refusing unattended auto-apply"
);
continue;
}
info!("Update scheduler: downloading update");
if let Err(e) = download_update(&data_dir).await {
debug!("Update scheduler: download failed: {}", e);
@ -1884,6 +2060,68 @@ pub async fn run_update_scheduler(
mod tests {
use super::*;
fn sample_manifest_value() -> serde_json::Value {
serde_json::json!({
"version": "9.9.9",
"release_date": "2026-07-02",
"changelog": ["test release"],
"components": [{
"name": "archipelago",
"current_version": "0.0.0",
"new_version": "9.9.9",
"download_url": "http://example.invalid/archipelago",
"sha256": "00",
"size_bytes": 1
}]
})
}
/// Same key + env-pin convention as `trust::signed_doc::tests` — every
/// caller pins the identical value, so parallel tests stay consistent.
fn test_release_key() -> ed25519_dalek::SigningKey {
let key = ed25519_dalek::SigningKey::from_bytes(&[7u8; 32]);
std::env::set_var(
"ARCHY_RELEASE_ROOT_PUBKEY",
hex::encode(key.verifying_key().to_bytes()),
);
key
}
fn sign_value(key: &ed25519_dalek::SigningKey, mut doc: serde_json::Value) -> serde_json::Value {
let (sig, did) = crate::trust::signed_doc::sign_detached(key, &doc).unwrap();
let obj = doc.as_object_mut().unwrap();
obj.insert("signed_by".into(), serde_json::json!(did));
obj.insert("signature".into(), serde_json::json!(sig));
doc
}
#[test]
fn unsigned_manifest_parses_but_reports_unsigned() {
let (manifest, signed) = parse_and_verify_manifest(sample_manifest_value()).unwrap();
assert_eq!(manifest.version, "9.9.9");
assert!(!signed, "unsigned manifest must not report as signed");
}
#[test]
fn anchor_signed_manifest_reports_signed() {
let key = test_release_key();
let doc = sign_value(&key, sample_manifest_value());
let (manifest, signed) = parse_and_verify_manifest(doc).unwrap();
assert_eq!(manifest.version, "9.9.9");
assert!(signed);
}
#[test]
fn tampered_signed_manifest_is_rejected() {
let key = test_release_key();
let mut doc = sign_value(&key, sample_manifest_value());
// Version swap after signing — exactly what a malicious mirror would do.
doc.as_object_mut()
.unwrap()
.insert("version".into(), serde_json::json!("99.0.0"));
assert!(parse_and_verify_manifest(doc).is_err());
}
#[test]
fn test_update_schedule_default_is_daily_check() {
let schedule = UpdateSchedule::default();
@ -2043,6 +2281,7 @@ mod tests {
rollback_available: true,
schedule: UpdateSchedule::AutoApply,
manifest_mirror: None,
manifest_signed: false,
};
let json = serde_json::to_string(&state).unwrap();
let deserialized: UpdateState = serde_json::from_str(&json).unwrap();
@ -2170,6 +2409,7 @@ mod tests {
"https://git.tx1138.com/lfg2025/archy/raw/branch/main/releases/manifest.json"
.to_string(),
),
manifest_signed: false,
};
save_state(dir.path(), &state).await.unwrap();
let loaded = load_state(dir.path()).await.unwrap();

View File

@ -19,6 +19,8 @@ chrono = { version = "0.4", features = ["serde"] }
uuid = { version = "1.0", features = ["v4"] }
log = "0.4"
tracing = "0.1"
sha2 = "0.10"
hex = "0.4"
[lib]
name = "archipelago_container"

View File

@ -0,0 +1,207 @@
//! Container image signature verification (cosign).
//!
//! The manifest/catalog `image_signature` field is a *claim* that the image
//! is signed with the fleet's cosign key. Verification runs at the pull
//! choke points (`PodmanClient::pull_image`, `DockerRuntime::pull_image`);
//! a declared signature that cannot be verified hard-fails the pull.
//!
//! Every manifest has carried the literal placeholder `cosign://...` since
//! the field was introduced — that means "not signed yet" and is treated as
//! no claim, so enforcement stays dormant until the signing ceremony
//! publishes real signatures AND nodes carry the pinned cosign public key.
//! Ship order matters: key + cosign binary reach the fleet first, real
//! signature values in the catalog come after.
use anyhow::{bail, Context, Result};
use std::path::PathBuf;
/// The literal placeholder every pre-ceremony manifest carries.
pub const SIGNATURE_PLACEHOLDER: &str = "cosign://...";
/// Env override for the pinned cosign public key path (tests, staging).
pub const COSIGN_PUBKEY_ENV: &str = "ARCHIPELAGO_COSIGN_PUBKEY";
const DEFAULT_PUBKEY_PATH: &str = "/etc/archipelago/cosign.pub";
const COSIGN_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum SignatureClaim {
/// No signature declared (field absent or empty).
None,
/// The literal `cosign://...` placeholder — manifest predates real signing.
Placeholder,
/// A real declared signature reference; MUST verify or the pull fails.
Declared(String),
}
pub fn classify_signature(signature: Option<&str>) -> SignatureClaim {
match signature.map(str::trim) {
None | Some("") => SignatureClaim::None,
Some(SIGNATURE_PLACEHOLDER) => SignatureClaim::Placeholder,
Some(s) => SignatureClaim::Declared(s.to_string()),
}
}
fn pinned_pubkey_path() -> PathBuf {
std::env::var(COSIGN_PUBKEY_ENV)
.map(PathBuf::from)
.unwrap_or_else(|_| PathBuf::from(DEFAULT_PUBKEY_PATH))
}
/// Verify a declared image signature with the fleet's pinned cosign key.
/// Any failure — missing key, missing cosign binary, verification error —
/// is a hard error: an image that CLAIMS to be signed must never be pulled
/// on a node that can't prove the claim.
pub async fn verify_declared_signature(
image: &str,
sig_ref: &str,
allow_insecure_registry: bool,
) -> Result<()> {
verify_with_key_path(image, sig_ref, &pinned_pubkey_path(), allow_insecure_registry).await
}
async fn verify_with_key_path(
image: &str,
sig_ref: &str,
key_path: &std::path::Path,
allow_insecure_registry: bool,
) -> Result<()> {
if !key_path.exists() {
bail!(
"Image '{image}' declares signature '{sig_ref}' but the pinned cosign \
public key is missing at {} (override with {COSIGN_PUBKEY_ENV}). \
Refusing to pull an image whose signature claim cannot be verified.",
key_path.display()
);
}
// Self-managed key => signatures aren't in the public Rekor transparency
// log, so tlog verification must be disabled explicitly (cosign v2
// defaults it on and would fail every private-key signature otherwise).
let mut cmd = tokio::process::Command::new("cosign");
cmd.arg("verify")
.arg("--key")
.arg(key_path)
.arg("--insecure-ignore-tlog=true");
if allow_insecure_registry {
// podman's --tls-verify=false covers both plain HTTP and bad TLS;
// cosign splits those into two flags — pass both to match.
cmd.arg("--allow-insecure-registry");
cmd.arg("--allow-http-registry");
}
cmd.arg(image);
let output = tokio::time::timeout(COSIGN_TIMEOUT, cmd.output())
.await
.map_err(|_| {
anyhow::anyhow!(
"cosign verify timed out after {}s for image '{image}'",
COSIGN_TIMEOUT.as_secs()
)
})?
.with_context(|| {
format!(
"Failed to run cosign for image '{image}' which declares signature \
'{sig_ref}' is cosign installed? A declared signature cannot be \
skipped."
)
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!(
"Signature verification FAILED for image '{image}' (declared: '{sig_ref}'): \
{stderr}"
);
}
tracing::info!("cosign signature verified for image {image}");
Ok(())
}
/// Shared pull-site gate: decide whether the pull may proceed.
/// Returns Ok(()) for unsigned/placeholder claims (with a log line) and only
/// after successful cosign verification for declared ones.
pub async fn enforce_signature_claim(
image: &str,
signature: Option<&str>,
allow_insecure_registry: bool,
) -> Result<()> {
match classify_signature(signature) {
SignatureClaim::None => {
tracing::debug!("image {image}: no signature declared, pulling unverified");
Ok(())
}
SignatureClaim::Placeholder => {
tracing::debug!(
"image {image}: signature is the pre-ceremony placeholder, pulling unverified"
);
Ok(())
}
SignatureClaim::Declared(sig_ref) => {
verify_declared_signature(image, &sig_ref, allow_insecure_registry).await
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn absent_and_empty_signatures_are_no_claim() {
assert_eq!(classify_signature(None), SignatureClaim::None);
assert_eq!(classify_signature(Some("")), SignatureClaim::None);
assert_eq!(classify_signature(Some(" ")), SignatureClaim::None);
}
#[test]
fn literal_placeholder_is_not_a_claim() {
assert_eq!(
classify_signature(Some("cosign://...")),
SignatureClaim::Placeholder
);
assert_eq!(
classify_signature(Some(" cosign://... ")),
SignatureClaim::Placeholder
);
}
#[test]
fn real_values_are_declared_claims() {
assert_eq!(
classify_signature(Some("cosign://sha256-abc.sig")),
SignatureClaim::Declared("cosign://sha256-abc.sig".to_string())
);
// Unknown schemes still count as a claim — better to fail closed on
// a value we don't understand than to pull unverified.
assert_eq!(
classify_signature(Some("sigstore://whatever")),
SignatureClaim::Declared("sigstore://whatever".to_string())
);
}
#[tokio::test]
async fn declared_signature_without_pinned_key_hard_fails() {
let err = verify_with_key_path(
"registry.example/app:1.0",
"cosign://sha256-abc.sig",
std::path::Path::new("/nonexistent/cosign.pub"),
false,
)
.await
.unwrap_err();
assert!(err.to_string().contains("pinned cosign public key is missing"));
}
#[tokio::test]
async fn unsigned_and_placeholder_claims_pass_the_gate() {
enforce_signature_claim("registry.example/app:1.0", None, false)
.await
.unwrap();
enforce_signature_claim("registry.example/app:1.0", Some("cosign://..."), false)
.await
.unwrap();
}
}

View File

@ -1,5 +1,6 @@
pub mod bitcoin_simulator;
pub mod health_monitor;
pub mod image_verify;
pub mod manifest;
pub mod podman_client;
pub mod port_manager;

View File

@ -243,6 +243,22 @@ pub struct ContainerConfig {
/// Example: `"100070:100070"` for Postgres' mapped subuid.
#[serde(default)]
pub data_uid: Option<String>,
/// Runtime-resolved secret env entries (never serialized, never in a
/// manifest file). Populated by the orchestrator's env-resolution
/// chokepoint; the backends turn each ref into a podman secret
/// reference (`secret_env` in the API spec / `Secret=…,type=env` in
/// Quadlet) so the VALUE never lands in `podman inspect` output or a
/// unit file on disk. The dev-only docker fallback injects `value` as
/// plain env — docker has no rootless secret store.
#[serde(skip)]
pub secret_env_refs: Vec<SecretEnvRef>,
/// sha256 over all resolved secret env pairs, stamped onto the
/// container as a label so rotation is detectable as drift without
/// exposing values. None when the app has no secret env.
#[serde(skip)]
pub secret_env_hash: Option<String>,
}
/// Derived-env entry. The template is rendered against `HostFacts` at
@ -265,6 +281,75 @@ pub struct SecretEnv {
pub secret_file: String,
}
/// A fully resolved secret env entry, produced at apply time. `value` lives
/// only in memory on its way to the podman secret store (or, on the dev
/// docker fallback, straight into the container env).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SecretEnvRef {
pub env_key: String,
/// Podman secret object name: `archy-env-<app>-<KEY>`.
pub secret_name: String,
pub value: String,
}
/// Container label carrying the combined secret-env content hash, used by
/// the reconciler to detect secret rotation as env drift.
pub const SECRET_ENV_HASH_LABEL: &str = "io.archipelago.secret-env-hash";
/// Podman secret label carrying the individual secret's content hash, used
/// to skip re-registration when nothing changed.
pub const SECRET_HASH_LABEL: &str = "io.archipelago.hash";
/// Expand `${KEY}` placeholders across plain + secret values, then split
/// the result into (plain env, secret-bearing pairs). Any plain entry whose
/// value interpolates a secret key is *tainted* — it embeds the secret and
/// must travel as a secret itself (btcpay's
/// `BTCPAY_POSTGRES=…Password=${BTCPAY_DB_PASS};…` pattern). Secret values
/// themselves are taken verbatim: a generated secret that happens to
/// contain `${` must not be mangled by expansion.
pub fn expand_and_partition_env(
plain: Vec<String>,
secrets: Vec<(String, String)>,
) -> (Vec<String>, Vec<(String, String)>) {
let plain_values: std::collections::HashMap<String, String> = plain
.iter()
.filter_map(|entry| {
let (key, value) = entry.split_once('=')?;
Some((key.to_string(), value.to_string()))
})
.collect();
let mut out_plain = Vec::with_capacity(plain.len());
let mut out_secret: Vec<(String, String)> = Vec::with_capacity(secrets.len());
for entry in plain {
let Some((key, value)) = entry.split_once('=') else {
out_plain.push(entry);
continue;
};
let mut expanded = value.to_string();
let mut tainted = false;
for (k, v) in &plain_values {
expanded = expanded.replace(&format!("${{{k}}}"), v);
}
for (k, v) in &secrets {
let placeholder = format!("${{{k}}}");
if expanded.contains(&placeholder) {
expanded = expanded.replace(&placeholder, v);
tainted = true;
}
}
if tainted {
out_secret.push((key.to_string(), expanded));
} else {
out_plain.push(format!("{key}={expanded}"));
}
}
out_secret.extend(secrets);
(out_plain, out_secret)
}
/// How a [`GeneratedSecret`] is produced. Each kind is deterministic in shape
/// (so the orchestrator knows which files to expect) but random in value.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
@ -418,6 +503,13 @@ pub struct PortMapping {
pub container: u16,
#[serde(default)]
pub protocol: String,
/// Host address to bind the publish to. Empty = all interfaces
/// (0.0.0.0). Set `127.0.0.1` to keep a port host-local; list the same
/// host/container pair twice with different binds to serve several
/// addresses (e.g. loopback + the archy-net gateway `10.89.0.1` so
/// containers keep reaching it via `host.archipelago`).
#[serde(default)]
pub bind: String,
}
impl From<(u16, u16)> for PortMapping {
@ -426,10 +518,38 @@ impl From<(u16, u16)> for PortMapping {
host,
container,
protocol: "tcp".to_string(),
bind: String::new(),
}
}
}
/// True when the host can actually bind a publish to `ip` right now.
///
/// Rootless podman forwards published ports via rootlessport, which listens
/// in the HOST network namespace — an address that only exists inside the
/// rootless netns (e.g. the archy-net bridge gateway `10.89.0.1`) fails with
/// "cannot assign requested address" and systemd crash-loops the whole unit
/// (bitcoin went down fleet-wide-capable this way, 2026-07-09 on .228).
/// Callers drop such publishes instead of passing them through: a publish
/// that cannot bind provides nothing, while the failed bind takes the
/// container down with it.
///
/// Empty (= 0.0.0.0), wildcard, and loopback binds are always accepted
/// without probing. Anything else is probed with an ephemeral-port bind.
/// Unparseable addresses return false — podman would reject them anyway.
pub fn host_can_bind_publish_ip(ip: &str) -> bool {
if ip.is_empty() {
return true;
}
let Ok(addr) = ip.parse::<std::net::IpAddr>() else {
return false;
};
if addr.is_unspecified() || addr.is_loopback() {
return true;
}
std::net::TcpListener::bind((addr, 0)).is_ok()
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Volume {
#[serde(rename = "type")]
@ -890,7 +1010,16 @@ fn validate_ports(ports: &[PortMapping]) -> Result<(), ManifestError> {
"ports[{i}].protocol must be tcp or udp"
)));
}
if !seen_host.insert((port.host, protocol.to_string())) {
if !port.bind.is_empty() && port.bind.parse::<std::net::IpAddr>().is_err() {
return Err(ManifestError::Invalid(format!(
"ports[{i}].bind must be an IP address, got '{}'",
port.bind
)));
}
// The same host port may be listed more than once with different bind
// addresses (e.g. loopback + the archy-net gateway); identical
// (host, protocol, bind) triples are still rejected.
if !seen_host.insert((port.host, protocol.to_string(), port.bind.clone())) {
return Err(ManifestError::Invalid(format!(
"ports contains duplicate host binding {}/{}",
port.host, protocol
@ -1209,6 +1338,19 @@ impl ContainerConfig {
&self,
provider: &dyn SecretsProvider,
) -> Result<Vec<String>, ManifestError> {
Ok(self
.resolve_secret_env_pairs(provider)?
.into_iter()
.map(|(k, v)| format!("{k}={v}"))
.collect())
}
/// Like `resolve_secret_env` but returns (key, value) pairs — the shape
/// the podman-secret pipeline needs.
pub fn resolve_secret_env_pairs(
&self,
provider: &dyn SecretsProvider,
) -> Result<Vec<(String, String)>, ManifestError> {
let mut out = Vec::with_capacity(self.secret_env.len());
for e in &self.secret_env {
let v = provider.read(&e.secret_file)?;
@ -1220,12 +1362,28 @@ impl ContainerConfig {
e.key, e.secret_file
)));
}
out.push(format!("{}={}", e.key, v));
out.push((e.key.clone(), v));
}
Ok(out)
}
}
/// Deterministic content hash over resolved secret env pairs (sorted by
/// key), used for the container drift label and per-secret labels.
pub fn secret_env_content_hash(pairs: &[(String, String)]) -> String {
use sha2::{Digest, Sha256};
let mut sorted: Vec<&(String, String)> = pairs.iter().collect();
sorted.sort_by(|a, b| a.0.cmp(&b.0));
let mut hasher = Sha256::new();
for (k, v) in sorted {
hasher.update(k.as_bytes());
hasher.update([0u8]);
hasher.update(v.as_bytes());
hasher.update([0u8]);
}
hex::encode(hasher.finalize())
}
#[cfg(test)]
mod tests {
use super::*;
@ -1233,6 +1391,72 @@ mod tests {
use std::fs;
use std::path::{Path, PathBuf};
#[test]
fn host_can_bind_accepts_empty_wildcard_and_loopback_without_probing() {
assert!(host_can_bind_publish_ip(""));
assert!(host_can_bind_publish_ip("0.0.0.0"));
assert!(host_can_bind_publish_ip("127.0.0.1"));
assert!(host_can_bind_publish_ip("::"));
assert!(host_can_bind_publish_ip("::1"));
}
#[test]
fn host_can_bind_rejects_addresses_absent_from_the_host() {
// TEST-NET-1 (RFC 5737) is never assigned to a real interface. The
// production case is 10.89.0.1 (podman bridge gateway, exists only
// inside the rootless netns), but that could legitimately bind on a
// rootful host, so the test probes an address that can't.
assert!(!host_can_bind_publish_ip("192.0.2.1"));
assert!(!host_can_bind_publish_ip("not-an-ip"));
}
#[test]
fn partition_taints_plain_entries_that_interpolate_secrets() {
let plain = vec![
"PLAIN=1".to_string(),
"COMPOSED=${BASE}/x".to_string(),
"DB_URL=postgres://u:${DB_PASS}@db/x".to_string(),
"BASE=/srv".to_string(),
"UNKNOWN=${NOT_DEFINED}".to_string(),
];
let secrets = vec![("DB_PASS".to_string(), "s3cret".to_string())];
let (p, s) = expand_and_partition_env(plain, secrets);
// plain-from-plain expansion stays plain; unknown placeholders stay
// literal (legacy expander parity)
assert!(p.contains(&"PLAIN=1".to_string()));
assert!(p.contains(&"COMPOSED=/srv/x".to_string()));
assert!(p.contains(&"UNKNOWN=${NOT_DEFINED}".to_string()));
// the tainted entry moved out of plain, fully expanded
assert!(!p.iter().any(|e| e.starts_with("DB_URL=")));
assert!(s.contains(&("DB_URL".to_string(), "postgres://u:s3cret@db/x".to_string())));
// the original secret rides along verbatim
assert!(s.contains(&("DB_PASS".to_string(), "s3cret".to_string())));
}
#[test]
fn secret_values_are_never_expanded() {
// A generated secret containing `${` must pass through untouched.
let secrets = vec![("WEIRD".to_string(), "pa${PLAIN}ss".to_string())];
let (_, s) = expand_and_partition_env(vec!["PLAIN=1".to_string()], secrets);
assert!(s.contains(&("WEIRD".to_string(), "pa${PLAIN}ss".to_string())));
}
#[test]
fn secret_env_hash_is_order_independent() {
let a = vec![
("K1".to_string(), "v1".to_string()),
("K2".to_string(), "v2".to_string()),
];
let b = vec![
("K2".to_string(), "v2".to_string()),
("K1".to_string(), "v1".to_string()),
];
assert_eq!(secret_env_content_hash(&a), secret_env_content_hash(&b));
let c = vec![("K1".to_string(), "CHANGED".to_string())];
assert_ne!(secret_env_content_hash(&a), secret_env_content_hash(&c));
}
#[test]
fn hooks_parse_and_validate() {
let yaml = r#"
@ -1765,6 +1989,8 @@ app:
generated_secrets: vec![],
generated_certs: vec![],
data_uid: None,
secret_env_refs: vec![],
secret_env_hash: None,
};
let facts = HostFacts {
host_ip: "192.168.1.116".to_string(),
@ -1817,6 +2043,8 @@ app:
generated_secrets: vec![],
generated_certs: vec![],
data_uid: None,
secret_env_refs: vec![],
secret_env_hash: None,
};
let p = MapSecretsProvider {
data: HashMap::from([
@ -1855,6 +2083,8 @@ app:
generated_secrets: vec![],
generated_certs: vec![],
data_uid: None,
secret_env_refs: vec![],
secret_env_hash: None,
};
let p = MapSecretsProvider {
data: HashMap::from([("bitcoin-rpc-password".to_string(), " \n".to_string())]),
@ -1959,6 +2189,35 @@ app:
"#,
"duplicate host binding",
),
(
"duplicate host port with same bind",
r#"
app:
id: bad-port-bind
name: Bad
version: 1.0.0
container:
image: test/image:latest
ports:
- { host: 8332, container: 8332, protocol: tcp, bind: 127.0.0.1 }
- { host: 8332, container: 8332, protocol: tcp, bind: 127.0.0.1 }
"#,
"duplicate host binding",
),
(
"non-IP bind address",
r#"
app:
id: bad-bind
name: Bad
version: 1.0.0
container:
image: test/image:latest
ports:
- { host: 8332, container: 8332, protocol: tcp, bind: localhost }
"#,
"bind must be an IP address",
),
(
"bad device",
r#"
@ -1998,6 +2257,31 @@ app:
}
}
#[test]
fn same_host_port_with_distinct_binds_is_valid() {
// Bitcoin RPC hardening: 8332 published on loopback + the archy-net
// gateway as two mappings of the same host port.
let m = AppManifest::parse(
r#"
app:
id: bitcoin-knots
name: Bitcoin Knots
version: 1.0.0
container:
image: test/bitcoin:latest
ports:
- { host: 8332, container: 8332, protocol: tcp, bind: 127.0.0.1 }
- { host: 8332, container: 8332, protocol: tcp, bind: 10.89.0.1 }
- { host: 8333, container: 8333, protocol: tcp }
"#,
)
.expect("distinct binds on one host port must validate");
assert_eq!(m.app.ports.len(), 3);
assert_eq!(m.app.ports[0].bind, "127.0.0.1");
assert_eq!(m.app.ports[1].bind, "10.89.0.1");
assert_eq!(m.app.ports[2].bind, "");
}
#[test]
fn reviewed_host_bind_exceptions_parse() {
let yaml = r#"

View File

@ -252,7 +252,17 @@ impl PodmanClient {
// ─── Container Operations ────────────────────────────────────
pub async fn pull_image(&self, image: &str, _signature: Option<&str>) -> Result<()> {
pub async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()> {
// A declared (non-placeholder) signature must verify before we fetch
// anything; placeholder/absent claims pull unverified until the
// signing ceremony ships real signatures (see image_verify).
crate::image_verify::enforce_signature_claim(
image,
signature,
image_uses_insecure_registry(image),
)
.await?;
// Image pull uses CLI — it's a streaming operation that the API handles differently
let mut cmd = tokio::process::Command::new("podman");
cmd.arg("pull");
@ -281,6 +291,15 @@ impl PodmanClient {
// Build the container spec for the API
let mut port_mappings = Vec::new();
for port in &manifest.app.ports {
if !crate::manifest::host_can_bind_publish_ip(&port.bind) {
tracing::warn!(
app = %manifest.app.id,
bind = %port.bind,
host_port = port.host,
"dropping publish: bind address not assignable on this host"
);
continue;
}
// Honour the manifest's protocol (default tcp). netbird's STUN port
// is 3478/udp; forcing tcp here would publish the wrong protocol and
// silently break relay discovery.
@ -289,11 +308,15 @@ impl PodmanClient {
"sctp" => "sctp",
_ => "tcp",
};
port_mappings.push(serde_json::json!({
let mut mapping = serde_json::json!({
"container_port": port.container,
"host_port": port.host,
"protocol": protocol,
}));
});
if !port.bind.is_empty() {
mapping["host_ip"] = serde_json::json!(port.bind);
}
port_mappings.push(mapping);
}
let mut mounts = Vec::new();
@ -372,12 +395,32 @@ impl PodmanClient {
manifest.app.security.network_policy.as_str(),
);
// Secret env travels by reference (podman injects the value at
// start), so it never shows up in `podman inspect` output. The
// combined content hash rides as a label for rotation-drift checks.
let mut secret_env_map = serde_json::Map::new();
for r in &manifest.app.container.secret_env_refs {
secret_env_map.insert(
r.env_key.clone(),
serde_json::Value::String(r.secret_name.clone()),
);
}
let mut labels_map = serde_json::Map::new();
if let Some(hash) = &manifest.app.container.secret_env_hash {
labels_map.insert(
crate::manifest::SECRET_ENV_HASH_LABEL.to_string(),
serde_json::Value::String(hash.clone()),
);
}
let mut body = serde_json::json!({
"name": name,
"image": image_ref,
"portmappings": port_mappings,
"mounts": mounts,
"env": env_map,
"secret_env": secret_env_map,
"labels": labels_map,
"entrypoint": manifest.app.container.entrypoint.clone(),
"command": manifest.app.container.custom_args.clone(),
"hostadd": [

View File

@ -2,7 +2,6 @@ use crate::manifest::{AppManifest, BuildConfig};
use crate::podman_client::{ContainerState, ContainerStatus, PodmanClient};
use anyhow::{Context, Result};
use async_trait::async_trait;
use std::process::Command;
use std::time::Duration;
use tokio::process::Command as TokioCommand;
@ -83,6 +82,18 @@ pub trait ContainerRuntime: Send + Sync {
/// `create_container` / `image_exists` calls. Stdout/stderr are collected
/// and included in the error on failure; on success they are discarded.
async fn build_image(&self, config: &BuildConfig) -> Result<()>;
/// Register the app's resolved secret-env entries in the runtime's
/// secret store (idempotent — skips entries whose content hash already
/// matches). Called before `create_container` for manifests with
/// `secret_env_refs`, so the create can reference secrets by name and
/// the values never appear in inspect output or unit files. Default is
/// a no-op for runtimes without a secret store (mocks, docker — the
/// docker fallback injects plain env in `create_container` instead).
async fn ensure_env_secrets(&self, refs: &[crate::manifest::SecretEnvRef]) -> Result<()> {
let _ = refs;
Ok(())
}
}
pub struct PodmanRuntime {
@ -132,6 +143,68 @@ impl ContainerRuntime for PodmanRuntime {
self.client.pull_image(image, signature).await
}
async fn ensure_env_secrets(&self, refs: &[crate::manifest::SecretEnvRef]) -> Result<()> {
use crate::manifest::{secret_env_content_hash, SECRET_HASH_LABEL};
use tokio::io::AsyncWriteExt;
for r in refs {
let hash = secret_env_content_hash(&[(r.env_key.clone(), r.value.clone())]);
// Skip the write when the stored secret already has this content
// (label round-trip beats rewriting the secret store every
// reconcile). Any inspect failure just falls through to create.
let fmt = format!("{{{{ index .Spec.Labels \"{SECRET_HASH_LABEL}\" }}}}");
if let Ok(out) = self
.podman_cli(&["secret", "inspect", &r.secret_name, "--format", &fmt])
.await
{
if out.status.success()
&& String::from_utf8_lossy(&out.stdout).trim() == hash
{
continue;
}
}
// Value goes via stdin — never argv, never a temp file.
let mut cmd = TokioCommand::new("podman");
cmd.args([
"secret",
"create",
"--replace",
"--label",
&format!("{SECRET_HASH_LABEL}={hash}"),
&r.secret_name,
"-",
]);
cmd.stdin(std::process::Stdio::piped());
cmd.stdout(std::process::Stdio::piped());
cmd.stderr(std::process::Stdio::piped());
cmd.kill_on_drop(true);
let mut child = cmd
.spawn()
.with_context(|| format!("spawning podman secret create {}", r.secret_name))?;
{
let mut stdin = child
.stdin
.take()
.context("podman secret create stdin unavailable")?;
stdin.write_all(r.value.as_bytes()).await?;
// drop closes the pipe so podman sees EOF
}
let out = tokio::time::timeout(PODMAN_CLI_DEFAULT_TIMEOUT, child.wait_with_output())
.await
.with_context(|| format!("podman secret create {} timed out", r.secret_name))??;
if !out.status.success() {
anyhow::bail!(
"podman secret create {} failed: {}",
r.secret_name,
String::from_utf8_lossy(&out.stderr)
);
}
}
Ok(())
}
async fn create_container(
&self,
manifest: &AppManifest,
@ -499,6 +572,8 @@ fn is_missing_container_error(stderr: &str) -> bool {
let stderr = stderr.to_ascii_lowercase();
stderr.contains("no container with name or id")
|| stderr.contains("no such container")
// podman 5.x `inspect` phrasing: `Error: no such object: "name"`
|| stderr.contains("no such object")
|| stderr.contains("does not exist")
|| stderr.contains("not found")
}
@ -547,7 +622,12 @@ impl DockerRuntime {
#[async_trait]
impl ContainerRuntime for DockerRuntime {
async fn pull_image(&self, image: &str, _signature: Option<&str>) -> Result<()> {
async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()> {
// Same signature gate as the podman path — the docker fallback is
// dev-only, but a declared signature must never be skippable by
// switching runtimes.
crate::image_verify::enforce_signature_claim(image, signature, false).await?;
let mut cmd = self.docker_async();
cmd.arg("pull").arg(image);
@ -617,6 +697,12 @@ impl ContainerRuntime for DockerRuntime {
for env in &manifest.app.environment {
cmd.arg("-e").arg(env);
}
// Dev-only fallback: docker has no rootless secret store, so secret
// env rides as plain env here. The podman path (production) passes
// these by secret reference instead — see ensure_env_secrets.
for r in &manifest.app.container.secret_env_refs {
cmd.arg("-e").arg(format!("{}={}", r.env_key, r.value));
}
// Resource limits
if let Some(cpu) = manifest.app.resources.cpu_limit {
@ -853,11 +939,11 @@ pub struct AutoRuntime {
impl AutoRuntime {
pub async fn new(user: String) -> Result<Self> {
// Try Podman first
if Self::check_podman_available() {
if Self::check_podman_available().await {
Ok(Self {
runtime: Box::new(PodmanRuntime::new(user)),
})
} else if Self::check_docker_available() {
} else if Self::check_docker_available().await {
Ok(Self {
runtime: Box::new(DockerRuntime::new(user)),
})
@ -866,12 +952,20 @@ impl AutoRuntime {
}
}
fn check_podman_available() -> bool {
Command::new("podman").arg("--version").output().is_ok()
async fn check_podman_available() -> bool {
TokioCommand::new("podman")
.arg("--version")
.output()
.await
.is_ok()
}
fn check_docker_available() -> bool {
Command::new("docker").arg("--version").output().is_ok()
async fn check_docker_available() -> bool {
TokioCommand::new("docker")
.arg("--version")
.output()
.await
.is_ok()
}
}
@ -881,6 +975,10 @@ impl ContainerRuntime for AutoRuntime {
self.runtime.pull_image(image, signature).await
}
async fn ensure_env_secrets(&self, refs: &[crate::manifest::SecretEnvRef]) -> Result<()> {
self.runtime.ensure_env_secrets(refs).await
}
async fn create_container(
&self,
manifest: &AppManifest,
@ -939,6 +1037,16 @@ mod tests {
use super::*;
use std::collections::HashMap;
#[test]
fn missing_container_classifier_covers_podman5_phrasings() {
// podman 5.x `inspect` phrasing for a missing container.
assert!(is_missing_container_error(
"Error: no such object: \"mempool\""
));
assert!(is_missing_container_error("Error: no such container x"));
assert!(!is_missing_container_error("Error: OCI runtime error"));
}
fn cfg(context: &str, tag: &str, dockerfile: &str, args: &[(&str, &str)]) -> BuildConfig {
BuildConfig {
context: context.to_string(),

23
core/openwrt/Cargo.toml Normal file
View File

@ -0,0 +1,23 @@
[package]
name = "archipelago-openwrt"
version = "0.1.0"
edition = "2021"
description = "OpenWrt gateway integration for Archipelago — TollGate provisioning over SSH/UCI"
[lib]
name = "archipelago_openwrt"
path = "src/lib.rs"
[dependencies]
tokio = { version = "1", features = ["full"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
anyhow = "1.0"
thiserror = "1.0"
tracing = "0.1"
ssh2 = "0.9"
async-trait = "0.1"
reqwest = { version = "0.11", default-features = false, features = ["json", "rustls-tls"] }
[dev-dependencies]
tokio-test = "0.4"

View File

@ -0,0 +1,72 @@
use anyhow::Result;
use std::net::{IpAddr, SocketAddr, TcpStream};
use std::time::Duration;
use tracing::{debug, info};
use crate::Router;
const SSH_PORT: u16 = 22;
const PROBE_TIMEOUT: Duration = Duration::from_millis(500);
/// Scan a CIDR subnet and return IP addresses of OpenWrt routers.
///
/// Probes TCP/22, then verifies /etc/openwrt_release over SSH.
/// `ssh_user` and `ssh_password` are used for the verification probe only.
pub async fn scan_subnet(
subnet_base: [u8; 4],
prefix_len: u8,
ssh_user: &str,
ssh_password: &str,
) -> Vec<IpAddr> {
let host_count = host_count_for_prefix(prefix_len);
let base_u32 = u32::from_be_bytes(subnet_base);
let mask = !((1u32 << (32 - prefix_len)) - 1);
let network = base_u32 & mask;
let mut candidates = Vec::new();
for i in 1..host_count {
let ip_u32 = network + i;
let ip = IpAddr::V4(std::net::Ipv4Addr::from(ip_u32));
if tcp_reachable(ip, SSH_PORT) {
candidates.push(ip);
}
}
info!("{} hosts with TCP/22 open in /{}", candidates.len(), prefix_len);
let mut routers = Vec::new();
for ip in candidates {
match verify_openwrt(ip, ssh_user, ssh_password) {
Ok(true) => {
info!("OpenWrt detected at {}", ip);
routers.push(ip);
}
Ok(false) => debug!("{} is not OpenWrt", ip),
Err(e) => debug!("{} probe failed: {}", ip, e),
}
}
routers
}
/// Check whether a known IP is an OpenWrt router.
pub fn probe(ip: IpAddr, ssh_user: &str, ssh_password: &str) -> Result<bool> {
verify_openwrt(ip, ssh_user, ssh_password)
}
fn tcp_reachable(ip: IpAddr, port: u16) -> bool {
TcpStream::connect_timeout(&SocketAddr::new(ip, port), PROBE_TIMEOUT).is_ok()
}
fn verify_openwrt(ip: IpAddr, user: &str, password: &str) -> Result<bool> {
let router = Router::connect_password(&ip.to_string(), SSH_PORT, user, password)?;
let (out, code) = router.run("cat /etc/openwrt_release")?;
Ok(code == 0 && out.contains("OpenWrt"))
}
fn host_count_for_prefix(prefix_len: u8) -> u32 {
if prefix_len >= 32 {
return 1;
}
1u32 << (32 - prefix_len)
}

9
core/openwrt/src/lib.rs Normal file
View File

@ -0,0 +1,9 @@
pub mod detect;
pub mod opkg;
pub mod router;
pub mod tollgate;
pub mod uci;
pub mod wan;
pub mod wifi_scan;
pub use router::Router;

90
core/openwrt/src/opkg.rs Normal file
View File

@ -0,0 +1,90 @@
use anyhow::Result;
use tracing::info;
use crate::Router;
/// Which package manager is available on this router.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PkgManager {
/// Traditional opkg (OpenWrt ≤24.x).
Opkg,
/// OpenWrt 25.x+ — apk is the native manager, opkg is not in repos.
ApkNative,
}
impl Router {
/// Detect which package manager is available.
///
/// - If `/usr/bin/opkg` exists → `PkgManager::Opkg` (nothing to do).
/// - If `/usr/bin/apk` exists → run `apk update` (switching repos to HTTP
/// first to work around missing CA bundle on fresh images), then try
/// `apk add opkg`. If opkg is in the repos → `Opkg`. If not (OpenWrt
/// 25.x) → `ApkNative`.
/// - Neither found → error.
pub fn opkg_check(&self) -> Result<PkgManager> {
let (_, code) = self.run("test -x /usr/bin/opkg")?;
if code == 0 {
return Ok(PkgManager::Opkg);
}
let (_, apk_code) = self.run("test -x /usr/bin/apk")?;
if apk_code == 0 {
info!("[{}] opkg not found — using apk (OpenWrt 25.x+)", self.host);
// Fresh images ship without a CA bundle; switch repos to HTTP so
// apk's wget can reach the package index without TLS verification.
self.run_ok("sed -i 's|https://|http://|g' /etc/apk/repositories 2>/dev/null || true")?;
let (update_out, update_code) = self.run("/usr/bin/apk update 2>&1")?;
if update_code != 0 {
anyhow::bail!(
"apk update failed (exit {}) — router may have no internet access. \
Ensure WAN/internet is working on the router before provisioning.\n{}",
update_code,
update_out.trim()
);
}
// Try to install opkg (only available on some 25.x builds).
let (add_out, add_code) = self.run("/usr/bin/apk add opkg 2>&1")?;
if add_code == 0 {
return Ok(PkgManager::Opkg);
}
if add_out.contains("no such package") || add_out.contains("unable to select") {
info!("[{}] opkg not in apk repos — staying in apk-native mode", self.host);
return Ok(PkgManager::ApkNative);
}
anyhow::bail!("apk add opkg failed (exit {}): {}", add_code, add_out.trim());
}
anyhow::bail!(
"opkg not found at /usr/bin/opkg — this router's firmware may not \
support package management (TollGate requires a standard OpenWrt build)"
);
}
/// `opkg update` — refresh package lists.
pub fn opkg_update(&self) -> Result<()> {
info!("[{}] opkg update", self.host);
self.run_ok("/usr/bin/opkg update")?;
Ok(())
}
/// Install a package, skipping if already installed.
pub fn opkg_install(&self, package: &str) -> Result<()> {
// Check if already installed to avoid unnecessary network traffic.
let (_, code) = self.run(&format!("/usr/bin/opkg list-installed | grep -q '^{} '", package))?;
if code == 0 {
info!("[{}] {} already installed", self.host, package);
return Ok(());
}
info!("[{}] opkg install {}", self.host, package);
self.run_ok(&format!("/usr/bin/opkg install {}", package))?;
Ok(())
}
/// Remove a package.
pub fn opkg_remove(&self, package: &str) -> Result<()> {
info!("[{}] opkg remove {}", self.host, package);
self.run_ok(&format!("/usr/bin/opkg remove {}", package))?;
Ok(())
}
}

View File

@ -0,0 +1,87 @@
use anyhow::{Context, Result};
use ssh2::Session;
use std::io::Read;
use std::net::TcpStream;
use std::path::Path;
use tracing::debug;
/// An active SSH connection to an OpenWrt router.
pub struct Router {
pub host: String,
pub port: u16,
session: Session,
}
impl Router {
/// Connect to an OpenWrt router via SSH using a private key.
pub fn connect(host: &str, port: u16, user: &str, key_path: &Path) -> Result<Self> {
let addr = format!("{}:{}", host, port);
let tcp = TcpStream::connect(&addr)
.with_context(|| format!("TCP connect to {}", addr))?;
let mut session = Session::new().context("create SSH session")?;
session.set_tcp_stream(tcp);
session.handshake().context("SSH handshake")?;
session
.userauth_pubkey_file(user, None, key_path, None)
.with_context(|| format!("SSH auth as {} with key {:?}", user, key_path))?;
Ok(Self {
host: host.to_string(),
port,
session,
})
}
/// Connect using a password (fallback for routers not yet provisioned with a key).
pub fn connect_password(host: &str, port: u16, user: &str, password: &str) -> Result<Self> {
let addr = format!("{}:{}", host, port);
let tcp = TcpStream::connect(&addr)
.with_context(|| format!("TCP connect to {}", addr))?;
let mut session = Session::new().context("create SSH session")?;
session.set_tcp_stream(tcp);
session.handshake().context("SSH handshake")?;
session
.userauth_password(user, password)
.with_context(|| format!("SSH password auth as {}", user))?;
Ok(Self {
host: host.to_string(),
port,
session,
})
}
/// Run a command and return (stdout, exit_code).
pub fn run(&self, cmd: &str) -> Result<(String, i32)> {
debug!("ssh [{}] $ {}", self.host, cmd);
let mut channel = self.session.channel_session().context("open channel")?;
channel.exec(cmd).with_context(|| format!("exec: {}", cmd))?;
let mut stdout = String::new();
channel.read_to_string(&mut stdout).context("read stdout")?;
channel.wait_close().context("wait close")?;
let exit = channel.exit_status().context("exit status")?;
Ok((stdout, exit))
}
/// Run a command, fail if exit code is non-zero.
pub fn run_ok(&self, cmd: &str) -> Result<String> {
let (out, code) = self.run(cmd)?;
if code != 0 {
anyhow::bail!("command `{}` exited with code {}: {}", cmd, code, out.trim());
}
Ok(out)
}
/// Verify the remote device is actually running OpenWrt.
pub fn verify_openwrt(&self) -> Result<String> {
let release = self
.run_ok("cat /etc/openwrt_release")
.context("read /etc/openwrt_release — is this an OpenWrt device?")?;
Ok(release)
}
}

View File

@ -0,0 +1,59 @@
use anyhow::Result;
use serde::{Deserialize, Serialize};
use crate::Router;
/// TollGate provisioning parameters.
///
/// `mint_url` must be the externally-reachable URL of the Archy Cashu mint —
/// TollGate customers connect from outside the Archy node's loopback, so
/// localhost URLs will not work. Resolve this from the running mint app before
/// calling `provision`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TollGateConfig {
/// SSID name for the pay-as-you-go network.
pub ssid: String,
/// Externally-reachable URL of the Archy Cashu mint.
pub mint_url: String,
/// Price in satoshis per `step_size` interval.
pub price_sats: u64,
/// Step size in milliseconds (default: 60000 = 1 minute).
pub step_size_ms: u64,
/// Minimum steps a customer must purchase at once.
pub min_steps: u32,
/// Whether the TollGate service should be running and enabled at boot.
pub enabled: bool,
}
impl Default for TollGateConfig {
fn default() -> Self {
Self {
ssid: "archipelago".to_string(),
mint_url: String::new(), // must be set by caller from the running mint app
price_sats: 10,
step_size_ms: 60_000,
min_steps: 1,
enabled: true,
}
}
}
/// Write TollGate UCI configuration and commit.
///
/// Maps TIP-01 / TIP-02 fields onto UCI keys used by tollgate-module-basic-go.
pub fn apply(router: &Router, cfg: &TollGateConfig) -> Result<()> {
router.uci_apply(
"tollgate",
&[
("tollgate.main", "tollgate"),
("tollgate.main.enabled", if cfg.enabled { "1" } else { "0" }),
("tollgate.main.metric", "milliseconds"),
("tollgate.main.step_size", &cfg.step_size_ms.to_string()),
("tollgate.main.min_steps", &cfg.min_steps.to_string()),
("tollgate.main.price_per_step", &cfg.price_sats.to_string()),
("tollgate.main.currency", "sat"),
("tollgate.main.mint_url", &cfg.mint_url),
],
)?;
Ok(())
}

View File

@ -0,0 +1,254 @@
use anyhow::Result;
use tracing::info;
use crate::Router;
/// The OpenWrt package name for the TollGate reference implementation.
const TOLLGATE_PACKAGE: &str = "tollgate-module-basic-go";
/// Direct-download fallback URLs by opkg architecture string.
/// Used when the package is not in any configured feed.
/// Source: https://github.com/OpenTollGate/tollgate-module-basic-go/releases/tag/v0.2.0
fn ipk_url(arch: &str) -> Option<&'static str> {
match arch {
"mips_24kc" => Some("https://github.com/OpenTollGate/tollgate-module-basic-go/releases/download/v0.2.0/mips_24kc.ipk"),
"mipsel_24kc" => Some("https://github.com/OpenTollGate/tollgate-module-basic-go/releases/download/v0.2.0/mipsel_24kc.ipk"),
"aarch64_cortex-a53" => Some("https://github.com/OpenTollGate/tollgate-module-basic-go/releases/download/v0.2.0/aarch64_cortex-a53.ipk"),
"aarch64_cortex-a72" => Some("https://github.com/OpenTollGate/tollgate-module-basic-go/releases/download/v0.2.0/aarch64_cortex-a72.ipk"),
"arm_cortex-a7" => Some("https://github.com/OpenTollGate/tollgate-module-basic-go/releases/download/v0.2.0/arm_cortex-a7.ipk"),
_ => None,
}
}
/// Install tollgate-module-basic-go via opkg (OpenWrt ≤24.x).
///
/// Tries opkg first (works if a custom feed is configured). Falls back to
/// downloading the .ipk directly from GitHub releases if opkg can't find it.
/// Caller is responsible for running `opkg_update` first.
pub fn install_tollgate(router: &Router) -> Result<()> {
info!("[{}] Installing {}", router.host, TOLLGATE_PACKAGE);
// Fast path: standard opkg install (or already installed).
if router.opkg_install(TOLLGATE_PACKAGE).is_ok() {
return Ok(());
}
// Package not in any feed — download the .ipk directly.
let arch = router
.run_ok("/usr/bin/opkg print-architecture | grep -v all | grep -v noarch | tail -1 | awk '{print $2}'")?;
let arch = arch.trim();
let url = ipk_url(arch).ok_or_else(|| {
anyhow::anyhow!(
"No pre-built TollGate package for architecture '{}'. \
Add a custom opkg feed or build from source.",
arch
)
})?;
info!("[{}] Downloading TollGate for {} from GitHub releases", router.host, arch);
router.run_ok(&format!("wget --no-check-certificate -O /tmp/tollgate.ipk '{}' 2>&1", url))?;
install_ipk(router, "/tmp/tollgate.ipk")
}
/// Install tollgate-module-basic-go on OpenWrt 25.x where opkg is not available.
///
/// Downloads the .ipk from GitHub releases and extracts it manually using
/// BusyBox `ar` and `tar` (both present on all OpenWrt images).
pub fn install_tollgate_apk_native(router: &Router) -> Result<()> {
info!("[{}] Installing {} (apk-native mode)", router.host, TOLLGATE_PACKAGE);
// Already installed? The service binary is /usr/bin/tollgate-wrt (per its
// init.d script) — TOLLGATE_PACKAGE is only the opkg/apk package name,
// never an on-disk filename, so it can't be used for the file-existence
// fallback below.
let (_, code) = router.run(&format!(
"apk list --installed 2>/dev/null | grep -q '^{}' || \
test -f /usr/bin/tollgate-wrt 2>/dev/null",
TOLLGATE_PACKAGE
))?;
if code == 0 {
info!("[{}] {} already installed", router.host, TOLLGATE_PACKAGE);
return Ok(());
}
// Get architecture from /etc/openwrt_release.
// The variable is DISTRIB_ARCH on most builds; OPENWRT_ARCH on some.
// Fall back to apk --print-arch, then uname -m.
let arch_raw = router.run_ok(
". /etc/openwrt_release 2>/dev/null \
&& a=\"${DISTRIB_ARCH:-${OPENWRT_ARCH:-}}\" \
&& [ -n \"$a\" ] && echo \"$a\" \
|| /usr/bin/apk --print-arch 2>/dev/null \
|| uname -m"
)?;
// Normalise: uname -m returns bare "mipsel"/"mips"; map to 24kc variant
// which is the standard for home-router MIPS builds.
let arch = match arch_raw.trim() {
"mipsel" => "mipsel_24kc",
"mips" => "mips_24kc",
other => other,
};
info!("[{}] detected arch: {:?}", router.host, arch);
if arch.is_empty() {
anyhow::bail!("Could not determine router architecture");
}
let url = ipk_url(arch).ok_or_else(|| {
anyhow::anyhow!(
"No pre-built TollGate package for architecture '{}'. \
Add a custom feed or build from source.",
arch
)
})?;
info!("[{}] Downloading TollGate for {} from GitHub releases", router.host, arch);
// --no-check-certificate: fresh OpenWrt 25.x images ship without a CA bundle;
// GitHub serves releases over HTTPS so wget would otherwise reject the cert.
let (dl_out, dl_code) = router.run(&format!(
"wget --no-check-certificate -O /tmp/tollgate.ipk '{}' 2>&1", url
))?;
if dl_code != 0 {
anyhow::bail!("TollGate download failed: {}", dl_out.trim());
}
// Sanity-check: a real .ipk is at least 50 KB.
// If wget captured an HTML error page it will be tiny.
let (size_out, _) = router.run("wc -c < /tmp/tollgate.ipk 2>/dev/null")?;
let size: u64 = size_out.trim().parse().unwrap_or(0);
if size < 50_000 {
anyhow::bail!(
"Downloaded TollGate package is only {}B — wget likely captured an error page. \
Check router internet access and that the release URL is reachable.",
size
);
}
install_ipk(router, "/tmp/tollgate.ipk")
}
/// Extract and install an .ipk file without opkg.
///
/// An .ipk is an `ar` archive containing `data.tar.gz` (package files) and
/// `control.tar.gz` (metadata + postinst script).
fn install_ipk(router: &Router, ipk_path: &str) -> Result<()> {
// Check for disk space first (rough: need at least ~1 MB free on /overlay).
// TollGate is a Go binary — typically 58 MB on flash.
let (df_out, _) = router.run("df /overlay 2>/dev/null | awk 'NR==2{print $4}'")?;
let free_kb: u64 = df_out.trim().parse().unwrap_or(u64::MAX);
if free_kb < 5120 {
anyhow::bail!(
"Not enough flash space for TollGate: only {}kB free on /overlay \
(need 5MB). Free up space first or use a router with more storage.",
free_kb
);
}
router.run_ok("rm -rf /tmp/_tg_install && mkdir -p /tmp/_tg_install")?;
// OpenWrt 25.x BusyBox does not include `ar` — install binutils via
// whichever package manager is available before trying to unpack the ipk.
let (_, ar_found) = router.run("command -v ar >/dev/null 2>&1")?;
if ar_found != 0 {
info!("[{}] ar not found, installing binutils", router.host);
let (pkg_out, pkg_code) = router.run(
"apk add binutils 2>&1 || opkg install binutils 2>&1"
)?;
if pkg_code != 0 {
anyhow::bail!(
"TollGate installation failed: ar not available and binutils install failed: {}",
pkg_out.trim()
);
}
}
// Try standard opkg ar format first (ar archive → data.tar.gz inside).
let (ar_out, ar_code) = router.run(&format!(
"cd /tmp/_tg_install && ar x {} 2>&1", ipk_path
))?;
if ar_code != 0 {
// Fallback: some builds produce the .ipk as a gzip tarball rather than
// a classic `ar` archive. This can still contain the same three ipk
// members (debian-binary/data.tar.gz/control.tar.gz) one level deep —
// just gzip-tarred together instead of ar'd — or, less commonly, a
// flat tarball of the real package files with no ipk structure at
// all. Extract to the scratch dir and check which shape it is before
// deciding how to install it.
info!("[{}] ar failed ({}), trying tar -xzf", router.host, ar_out.trim());
// List contents first — validates format without writing anything.
let (list_out, list_code) = router.run(&format!(
"tar -tzf {} 2>&1 | head -30", ipk_path
))?;
if list_code != 0 {
anyhow::bail!(
"TollGate installation failed: file is not an ar archive or gzip tar.\n\
ar: {}\ntar -t: {}",
ar_out.trim(), list_out.trim()
);
}
info!("[{}] ipk contents:\n{}", router.host, list_out.trim());
router.run_ok(&format!("tar -xzf {} -C /tmp/_tg_install 2>&1", ipk_path))?;
let (_, nested) = router.run("test -f /tmp/_tg_install/data.tar.gz")?;
if nested != 0 {
// Genuinely flat tarball, no ipk structure — its contents are the
// real package files, already unpacked into the scratch dir.
let (ov_df, _) = router.run("df / 2>/dev/null | awk 'NR==2{print $4}'")?;
let overlay_free_kb: u64 = ov_df.trim().parse().unwrap_or(0);
if overlay_free_kb < 5120 {
anyhow::bail!(
"Not enough space to install TollGate: only {}kB free on /. \
Need at least 5MB. Free up flash space on the router first \
(e.g. remove unused packages with `apk del `).",
overlay_free_kb
);
}
let (cp_out, cp_code) = router.run("cp -a /tmp/_tg_install/. / 2>&1")?;
if cp_code != 0 {
anyhow::bail!("TollGate installation failed: file copy failed: {}", cp_out.trim());
}
// No package-manager postinst ran for these files either — see
// the uci-defaults note below.
router.run_ok(
"for f in /etc/uci-defaults/*; do \
[ -f \"$f\" ] && ( cd \"$(dirname \"$f\")\" && . \"$f\" ) && rm -f \"$f\"; \
done; uci commit 2>/dev/null; true"
)?;
router.run_ok(&format!("rm -rf /tmp/_tg_install {}", ipk_path))?;
return Ok(());
}
// Nested ipk-member layout — fall through to the shared unpack below.
}
// Unpack data.tar.gz (the real payload) from either the `ar`-extracted or
// gzip-tar-extracted scratch dir, then run control.tar.gz's postinst.
let (tar_out, tar_code) = router.run(
"tar -xzf /tmp/_tg_install/data.tar.gz -C / 2>&1"
)?;
if tar_code != 0 {
anyhow::bail!("TollGate installation failed: data extract failed: {}", tar_out.trim());
}
// Run postinst if present (optional — failures are non-fatal).
router.run_ok(
"if tar -xzf /tmp/_tg_install/control.tar.gz -C /tmp/_tg_install 2>/dev/null; then \
chmod +x /tmp/_tg_install/postinst 2>/dev/null; \
/tmp/_tg_install/postinst configure 2>/dev/null || true; \
fi"
)?;
// `default_postinst` (what most packages' postinst calls, including
// this one) only runs pending /etc/uci-defaults/* scripts for packages
// it finds in opkg/apk's own file-list records. Since these files were
// extracted manually rather than through a real package-manager install,
// no such record exists, so run any pending scripts directly — this is
// exactly what opkg's install path (or the next reboot) would otherwise
// do for them, just without waiting for either.
router.run_ok(
"for f in /etc/uci-defaults/*; do \
[ -f \"$f\" ] && ( cd \"$(dirname \"$f\")\" && . \"$f\" ) && rm -f \"$f\"; \
done; uci commit 2>/dev/null; true"
)?;
router.run_ok(&format!("rm -rf /tmp/_tg_install {}", ipk_path))?;
Ok(())
}

View File

@ -0,0 +1,61 @@
pub mod config;
pub mod install;
pub mod wifi;
pub use config::TollGateConfig;
pub use install::install_tollgate;
pub use wifi::provision_ssid;
use anyhow::Result;
use tracing::info;
use crate::{opkg::PkgManager, Router};
/// Full TollGate provisioning sequence:
/// 1. Install tollgate-module-basic-go
/// 2. Write TollGate UCI config (pricing, mint URL)
/// 3. Create the pay-as-you-go WiFi SSID
/// 4. Restart affected services
pub async fn provision(router: &Router, config: &TollGateConfig) -> Result<()> {
info!("[{}] Starting TollGate provisioning", router.host);
let pkg_mgr = router.opkg_check()?;
match pkg_mgr {
PkgManager::Opkg => {
router.opkg_update()?;
install_tollgate(router)?;
}
PkgManager::ApkNative => {
install::install_tollgate_apk_native(router)?;
}
}
config::apply(router, config)?;
wifi::provision_ssid(router, config)?;
restart_services(router, config.enabled)?;
info!("[{}] TollGate provisioning complete", router.host);
Ok(())
}
/// Applies `enabled` to the actual running service, not just the UCI value —
/// the tollgate-wrt init script doesn't consult `tollgate.main.enabled`
/// itself, so toggling it requires an explicit enable/start or disable/stop.
///
/// The service's init script is `/etc/init.d/tollgate-wrt` (its actual
/// on-disk name — "tollgate" alone does not exist).
fn restart_services(router: &Router, enabled: bool) -> Result<()> {
if enabled {
router.run_ok("/etc/init.d/tollgate-wrt enable")?;
router.run_ok(
"/etc/init.d/tollgate-wrt restart || /etc/init.d/tollgate-wrt start"
)?;
} else {
router.run_ok("/etc/init.d/tollgate-wrt stop || true")?;
router.run_ok("/etc/init.d/tollgate-wrt disable || true")?;
}
router.run_ok("/etc/init.d/network restart")?;
// Reload wireless so wireless.tollgate.disabled takes effect on the radio —
// `network restart` alone doesn't reliably reconfigure wifi interfaces.
router.run_ok("wifi down 2>&1; wifi up 2>&1")?;
Ok(())
}

View File

@ -0,0 +1,109 @@
use anyhow::{Context, Result};
use tracing::info;
use crate::tollgate::TollGateConfig;
use crate::Router;
/// Create (or update) the dedicated pay-as-you-go WiFi interface for TollGate.
///
/// Uses a fixed named section (`wireless.tollgate`) rather than `uci add`, so
/// re-provisioning (e.g. editing price/mint URL after install) updates the
/// same interface in place instead of piling up a new `wifi-iface` section —
/// and therefore a new duplicate broadcast SSID — on every call.
pub fn provision_ssid(router: &Router, cfg: &TollGateConfig) -> Result<()> {
let radio = detect_radio(router).context("detect WiFi radio")?;
info!("[{}] Using radio {} for TollGate SSID", router.host, radio);
router.uci_apply(
"wireless",
&[
("wireless.tollgate", "wifi-iface"),
("wireless.tollgate.device", &radio),
("wireless.tollgate.mode", "ap"),
("wireless.tollgate.ssid", &cfg.ssid),
("wireless.tollgate.encryption", "none"),
("wireless.tollgate.network", "tollgate"),
// Disable 802.11r/k/v — unnecessary for transient pay-as-you-go clients.
("wireless.tollgate.ieee80211r", "0"),
// Stop broadcasting entirely when disabled, rather than leaving an
// open SSID up that leads nowhere once the backend is stopped.
("wireless.tollgate.disabled", if cfg.enabled { "0" } else { "1" }),
],
)?;
provision_network(router)?;
provision_firewall(router)?;
Ok(())
}
/// Add a `tollgate` network interface (isolated LAN for TollGate clients).
fn provision_network(router: &Router) -> Result<()> {
router.uci_apply(
"network",
&[
("network.tollgate", "interface"),
("network.tollgate.proto", "static"),
("network.tollgate.ipaddr", "192.168.99.1"),
("network.tollgate.netmask", "255.255.255.0"),
],
)?;
// Enable DHCP for the tollgate interface.
router.uci_apply(
"dhcp",
&[
("dhcp.tollgate", "dhcp"),
("dhcp.tollgate.interface", "tollgate"),
("dhcp.tollgate.start", "100"),
("dhcp.tollgate.limit", "150"),
("dhcp.tollgate.leasetime", "5m"),
],
)?;
Ok(())
}
/// Add firewall zone for the tollgate interface.
///
/// TollGate itself gates forwarding via iptables; the firewall zone isolates
/// tollgate clients from other LAN segments.
fn provision_firewall(router: &Router) -> Result<()> {
// Zone
router.uci_apply(
"firewall",
&[
("firewall.tollgate_zone", "zone"),
("firewall.tollgate_zone.name", "tollgate"),
("firewall.tollgate_zone.network", "tollgate"),
("firewall.tollgate_zone.input", "ACCEPT"),
("firewall.tollgate_zone.output", "ACCEPT"),
("firewall.tollgate_zone.forward", "REJECT"),
],
)?;
// Forwarding rule: tollgate → wan (TollGate manages which clients can forward)
router.uci_apply(
"firewall",
&[
("firewall.tollgate_fwd", "forwarding"),
("firewall.tollgate_fwd.src", "tollgate"),
("firewall.tollgate_fwd.dest", "wan"),
],
)?;
Ok(())
}
/// Return the first available wireless radio device name (e.g. "radio0").
fn detect_radio(router: &Router) -> Result<String> {
let out = router.run_ok("uci show wireless | grep -o 'wireless\\.radio[0-9]*\\.type' | head -1")?;
// Extract "radioN" from "wireless.radioN.type"
let radio = out
.trim()
.split('.')
.nth(1)
.unwrap_or("radio0")
.to_string();
Ok(radio)
}

65
core/openwrt/src/uci.rs Normal file
View File

@ -0,0 +1,65 @@
use anyhow::Result;
use crate::Router;
/// Thin wrappers around `uci` CLI commands over SSH.
impl Router {
/// `uci get <key>` — returns trimmed value.
pub fn uci_get(&self, key: &str) -> Result<String> {
let out = self.run_ok(&format!("uci get {}", key))?;
Ok(out.trim().to_string())
}
/// `uci set <key>=<value>`
pub fn uci_set(&self, key: &str, value: &str) -> Result<()> {
self.run_ok(&format!("uci set {}={}", key, shell_quote(value)))?;
Ok(())
}
/// `uci add <config> <type>` — returns the new section name.
pub fn uci_add(&self, config: &str, section_type: &str) -> Result<String> {
let out = self.run_ok(&format!("uci add {} {}", config, section_type))?;
Ok(out.trim().to_string())
}
/// `uci add_list <key>=<value>`
pub fn uci_add_list(&self, key: &str, value: &str) -> Result<()> {
self.run_ok(&format!("uci add_list {}={}", key, shell_quote(value)))?;
Ok(())
}
/// `uci delete <key>`
pub fn uci_delete(&self, key: &str) -> Result<()> {
self.run_ok(&format!("uci delete {}", key))?;
Ok(())
}
/// `uci commit [<config>]`
pub fn uci_commit(&self, config: Option<&str>) -> Result<()> {
match config {
Some(c) => self.run_ok(&format!("uci commit {}", c))?,
None => self.run_ok("uci commit")?,
};
Ok(())
}
/// Batch: apply a list of `(key, value)` pairs then commit the config.
pub fn uci_apply(&self, config: &str, pairs: &[(&str, &str)]) -> Result<()> {
// `uci set config.section=type` fails with "Entry not found" if
// /etc/config/<config> doesn't exist yet — true for any config file
// shipped by the base system (wireless, network, dhcp, ...) but not
// for a package-defined namespace like "tollgate" that nothing has
// created a default for. `touch` is a no-op if it already exists.
self.run_ok(&format!("touch /etc/config/{}", config))?;
for (key, value) in pairs {
self.uci_set(key, value)?;
}
self.uci_commit(Some(config))?;
Ok(())
}
}
/// Wrap a value in single quotes, escaping any embedded single quotes.
fn shell_quote(s: &str) -> String {
format!("'{}'", s.replace('\'', r"'\''"))
}

218
core/openwrt/src/wan.rs Normal file
View File

@ -0,0 +1,218 @@
use anyhow::Result;
use tracing::info;
use crate::Router;
pub struct WispConfig {
pub ssid: String,
pub password: String,
pub encryption: String, // psk2 | psk | sae | none
pub dhcp_start: u32, // first address in DHCP pool (default 100 → .100)
pub dhcp_limit: u32, // pool size (default 150 → .100.249)
pub masq: bool, // enable NAT on WAN zone (almost always true)
}
pub fn configure_wisp(router: &Router, config: &WispConfig) -> Result<()> {
info!("[{}] Configuring WISP → ssid={}", router.host, config.ssid);
let radio = detect_radio(router)?;
// Ensure the radio is enabled (disabled=1 by default on fresh flash)
router.uci_set("wireless.radio0.disabled", "0")?;
// Create/update named sta wifi-iface "wwan" (idempotent: uci set creates if absent)
router.uci_set("wireless.wwan", "wifi-iface")?;
router.uci_set("wireless.wwan.device", &radio)?;
router.uci_set("wireless.wwan.mode", "sta")?;
router.uci_set("wireless.wwan.ssid", &config.ssid)?;
router.uci_set("wireless.wwan.network", "wwan")?;
router.uci_set("wireless.wwan.disabled", "0")?;
router.uci_set("wireless.wwan.encryption", &config.encryption)?;
if config.encryption != "none" && !config.password.is_empty() {
router.uci_set("wireless.wwan.key", &config.password)?;
}
router.uci_commit(Some("wireless"))?;
// Create/update wwan network interface (DHCP)
router.uci_set("network.wwan", "interface")?;
router.uci_set("network.wwan.proto", "dhcp")?;
router.uci_commit(Some("network"))?;
// Add wwan to the WAN firewall zone (walk zones by name)
ensure_wwan_in_wan_zone(router)?;
// Configure LAN DHCP pool
router.uci_set("dhcp.lan.start", &config.dhcp_start.to_string())?;
router.uci_set("dhcp.lan.limit", &config.dhcp_limit.to_string())?;
router.uci_commit(Some("dhcp"))?;
// Ensure masquerade on WAN zone so LAN clients reach the internet
if config.masq {
ensure_masq_on_wan_zone(router)?;
}
// Full wifi cycle so wpa_supplicant restarts cleanly with the new config.
// "wifi reload" is not enough on some drivers — it keeps stale state.
let (down_out, down_code) = router.run("wifi down 2>&1")?;
if down_code != 0 {
info!("[{}] wifi down failed ({}): {}", router.host, down_code, down_out.trim());
}
let (up_out, up_code) = router.run("wifi up 2>&1")?;
if up_code != 0 {
info!("[{}] wifi up failed ({}): {} — falling back to network restart", router.host, up_code, up_out.trim());
router.run_ok("/etc/init.d/network restart 2>&1")?;
}
Ok(())
}
pub fn get_wan_status(router: &Router) -> serde_json::Value {
let configured = router
.uci_get("network.wwan.proto")
.map(|v| v == "dhcp")
.unwrap_or(false);
let ssid = router.uci_get("wireless.wwan.ssid").unwrap_or_default();
let encryption = router.uci_get("wireless.wwan.encryption").unwrap_or_default();
let radio0_disabled = router
.uci_get("wireless.radio0.disabled")
.map(|v| v == "1")
.unwrap_or(false);
// Find the active sta-mode interface and its association state
let iw_out = router.run_ok("iw dev 2>/dev/null").unwrap_or_default();
let (sta_iface, assoc_ssid) = parse_sta_iface(&iw_out);
// Interface operstate (up / down / absent)
let sta_state = if !sta_iface.is_empty() {
router
.run_ok(&format!("cat /sys/class/net/{}/operstate 2>/dev/null", sta_iface))
.unwrap_or_else(|_| "unknown".into())
.trim()
.to_string()
} else {
"absent".to_string()
};
// Source IP for reaching 8.8.8.8 — empty if no default route yet
let ip = router
.run_ok("ip -4 route get 8.8.8.8 2>/dev/null | awk '{for(i=1;i<=NF;i++) if($i==\"src\"){print $(i+1); exit}}'")
.unwrap_or_default()
.trim()
.to_string();
// Recent wifi-related kernel/syslog lines for quick diagnosis
let wifi_log = router
.run_ok("logread 2>/dev/null | grep -iE 'wlan|wwan|wifi|assoc|deauth|auth fail|CTRL-EVENT|wpa_supplicant' | tail -8 2>/dev/null")
.unwrap_or_default()
.trim()
.to_string();
// LAN info for the DHCP setup display
let lan_ip = router.uci_get("network.lan.ipaddr").unwrap_or_else(|_| "192.168.1.1".into());
let lan_netmask = router.uci_get("network.lan.netmask").unwrap_or_else(|_| "255.255.255.0".into());
let dhcp_start = router.uci_get("dhcp.lan.start").unwrap_or_else(|_| "100".into());
let dhcp_limit = router.uci_get("dhcp.lan.limit").unwrap_or_else(|_| "150".into());
// Masquerade: check WAN zone
let masq = {
let script = "for i in $(seq 0 9); do \
n=$(uci get firewall.@zone[$i].name 2>/dev/null) || break; \
if [ \"$n\" = \"wan\" ]; then \
uci get firewall.@zone[$i].masq 2>/dev/null; break; \
fi; done";
router.run_ok(script).unwrap_or_default().trim().to_string() == "1"
};
info!("[{}] WAN status: configured={} ssid={:?} assoc={:?} sta_iface={:?} sta_state={:?} ip={:?} lan={} masq={}",
router.host, configured, ssid, assoc_ssid, sta_iface, sta_state, ip, lan_ip, masq);
if !wifi_log.is_empty() {
info!("[{}] wifi_log: {}", router.host, wifi_log.replace('\n', " | "));
}
serde_json::json!({
"configured": configured,
"ssid": ssid,
"assoc_ssid": assoc_ssid,
"encryption": encryption,
"ip": ip,
"internet": !ip.is_empty(),
"radio0_disabled": radio0_disabled,
"sta_iface": sta_iface,
"sta_state": sta_state,
"wifi_log": wifi_log,
"lan_ip": lan_ip,
"lan_netmask": lan_netmask,
"dhcp_start": dhcp_start,
"dhcp_limit": dhcp_limit,
"masq": masq,
})
}
fn parse_sta_iface(iw_out: &str) -> (String, String) {
let mut result_iface = String::new();
let mut result_ssid = String::new();
let mut current_iface = String::new();
let mut current_type = String::new();
let mut current_ssid = String::new();
for line in iw_out.lines() {
let line = line.trim();
if let Some(name) = line.strip_prefix("Interface ") {
// Save previous interface if it was a sta
if current_type == "managed" && result_iface.is_empty() {
result_iface = current_iface.clone();
result_ssid = current_ssid.clone();
}
current_iface = name.trim().to_string();
current_type.clear();
current_ssid.clear();
} else if let Some(t) = line.strip_prefix("type ") {
current_type = t.trim().to_string();
} else if let Some(s) = line.strip_prefix("ssid ") {
current_ssid = s.trim().to_string();
}
}
// Handle last block
if current_type == "managed" && result_iface.is_empty() {
result_iface = current_iface;
result_ssid = current_ssid;
}
(result_iface, result_ssid)
}
fn detect_radio(router: &Router) -> Result<String> {
// radio0 is universal; verify it exists
let out = router.uci_get("wireless.radio0").unwrap_or_default();
if !out.is_empty() {
return Ok("radio0".to_string());
}
anyhow::bail!("No wireless radio (radio0) found in UCI config")
}
fn ensure_masq_on_wan_zone(router: &Router) -> Result<()> {
let script = "for i in $(seq 0 9); do \
name=$(uci get firewall.@zone[$i].name 2>/dev/null) || break; \
if [ \"$name\" = \"wan\" ]; then \
uci set firewall.@zone[$i].masq=1 2>/dev/null; \
uci commit firewall; \
break; \
fi; \
done; echo ok";
router.run_ok(script)?;
Ok(())
}
fn ensure_wwan_in_wan_zone(router: &Router) -> Result<()> {
// Walk zones 0-9, find the one named "wan", add wwan to its network list
let script = "for i in $(seq 0 9); do \
name=$(uci get firewall.@zone[$i].name 2>/dev/null) || break; \
if [ \"$name\" = \"wan\" ]; then \
uci add_list firewall.@zone[$i].network=wwan 2>/dev/null; \
uci commit firewall; \
break; \
fi; \
done; echo ok";
router.run_ok(script)?;
Ok(())
}

View File

@ -0,0 +1,177 @@
use anyhow::Result;
use crate::Router;
pub struct ScannedNetwork {
pub ssid: String,
pub bssid: String,
pub signal: i32,
pub channel: u8,
pub encryption: String,
}
pub fn scan_networks(router: &Router) -> Result<Vec<ScannedNetwork>> {
let (iface, temp) = find_wireless_iface(router)?;
let output = router.run_ok(&format!("iwinfo {} scan 2>&1", iface))?;
let result = if output.contains("Scanning not possible") {
// Vendor MediaTek `mt_wifi` driver (see find_wireless_iface) doesn't
// support scanning through iwinfo/nl80211 at all. Fall back to its own
// private ioctl site-survey, which works on the same interface.
scan_via_mtk_site_survey(router, &iface)
} else if output.contains("No scan results") || output.trim().is_empty() {
Ok(vec![])
} else {
parse_iwinfo_scan(&output)
};
if temp {
let _ = router.run(&format!("iw dev {} del 2>/dev/null", iface));
}
result
}
fn scan_via_mtk_site_survey(router: &Router, iface: &str) -> Result<Vec<ScannedNetwork>> {
let _ = router.run(&format!("iwpriv {} set SiteSurvey=1 2>/dev/null", iface));
std::thread::sleep(std::time::Duration::from_secs(4));
let output = router.run_ok(&format!("iwpriv {} get_site_survey 2>&1", iface))?;
parse_mtk_site_survey(&output)
}
/// Parses MediaTek's `iwpriv <iface> get_site_survey` fixed-width table.
/// Column offsets come from the header row layout, which is part of the
/// vendor SDK's ioctl response format shared across OEMs (GL.iNet, etc.),
/// not something set per-device.
fn parse_mtk_site_survey(output: &str) -> Result<Vec<ScannedNetwork>> {
let mut networks = Vec::new();
for line in output.lines() {
if !line.trim_start().as_bytes().first().is_some_and(u8::is_ascii_digit) {
continue; // skip header/summary lines; data rows start with an index
}
let ssid = line.get(8..41).unwrap_or("").trim().to_string();
if ssid.is_empty() {
continue;
}
let bssid = line.get(41..61).unwrap_or("").trim().to_string();
let security = line.get(61..84).unwrap_or("");
let channel: u8 = line.get(4..8).and_then(|s| s.trim().parse().ok()).unwrap_or(0);
let signal: i32 = line.get(84..92).and_then(|s| s.trim().parse().ok()).unwrap_or(-100);
networks.push(ScannedNetwork {
ssid,
bssid,
signal,
channel,
encryption: normalize_encryption(security),
});
}
networks.sort_by(|a, b| b.signal.cmp(&a.signal));
Ok(networks)
}
/// Returns `(interface_name, is_temporary)`.
/// If no interface exists, creates a temporary managed one directly on the PHY
/// so we can scan without needing any UCI wifi-iface sections.
fn find_wireless_iface(router: &Router) -> Result<(String, bool)> {
// Fast path: an interface already exists (radio was enabled previously)
let (out, _) = router.run("iw dev 2>/dev/null | awk '/Interface/{print $2}' | head -1")?;
if !out.trim().is_empty() {
return Ok((out.trim().to_string(), false));
}
// Some vendor wifi drivers (e.g. MediaTek's out-of-tree `mt_wifi`/`mtk` SDK
// driver used by GL.iNet and others) never register with cfg80211/mac80211,
// so they have no `iw dev` entry and no /sys/class/ieee80211 phy even though
// the radio is real and already up. `iwinfo` abstracts over those vendor
// backends too, so fall back to its device listing before concluding there's
// no radio at all.
let (iwinfo_out, _) = router.run("iwinfo 2>/dev/null | awk '/^[A-Za-z]/{print $1; exit}'")?;
if !iwinfo_out.trim().is_empty() {
return Ok((iwinfo_out.trim().to_string(), false));
}
// Find the phy — if this is empty the device has no WiFi hardware at all
let (phy_out, _) = router.run("ls /sys/class/ieee80211/ 2>/dev/null | head -1")?;
let phy = phy_out.trim().to_string();
if phy.is_empty() {
anyhow::bail!("No wireless radio found on this router");
}
// Create a temporary managed interface directly on the PHY. This bypasses
// netifd entirely so it works even when there are no wifi-iface sections in
// UCI (common on a freshly-flashed device).
tracing::info!("[{}] Creating temporary scan interface on {}", router.host, phy);
// Remove any stale scan0 from a previous attempt, then add fresh
let _ = router.run("iw dev scan0 del 2>/dev/null");
router.run_ok(&format!(
"iw phy {} interface add scan0 type managed 2>&1 && ip link set scan0 up 2>&1",
phy
))?;
Ok(("scan0".to_string(), true))
}
fn parse_iwinfo_scan(output: &str) -> Result<Vec<ScannedNetwork>> {
let mut networks: Vec<ScannedNetwork> = Vec::new();
let mut current: Option<ScannedNetwork> = None;
for line in output.lines() {
let line = line.trim();
if line.starts_with("Cell ") {
if let Some(n) = current.take() {
if !n.ssid.is_empty() {
networks.push(n);
}
}
let bssid = line.split("Address:").nth(1).unwrap_or("").trim().to_string();
current = Some(ScannedNetwork {
ssid: String::new(),
bssid,
signal: -100,
channel: 0,
encryption: "none".to_string(),
});
} else if let Some(ref mut n) = current {
if let Some(rest) = line.strip_prefix("ESSID:") {
n.ssid = rest.trim().trim_matches('"').to_string();
} else if line.contains("Channel:") && !line.starts_with("Encryption") {
if let Some(ch_part) = line.split("Channel:").nth(1) {
n.channel = ch_part.trim().split_whitespace().next()
.and_then(|s| s.parse().ok())
.unwrap_or(0);
}
} else if line.starts_with("Signal:") {
if let Some(dbm_str) = line.split_whitespace().nth(1) {
n.signal = dbm_str.parse().unwrap_or(-100);
}
} else if let Some(rest) = line.strip_prefix("Encryption:") {
n.encryption = normalize_encryption(rest.trim());
}
}
}
if let Some(n) = current {
if !n.ssid.is_empty() {
networks.push(n);
}
}
networks.sort_by(|a, b| b.signal.cmp(&a.signal));
Ok(networks)
}
fn normalize_encryption(raw: &str) -> String {
let lower = raw.to_lowercase();
if lower.contains("wpa3") || lower.contains("sae") {
"sae".to_string()
} else if lower.contains("wpa2") || lower.contains("psk2") {
"psk2".to_string()
} else if lower.contains("wpa") {
// CCMP/AES is WPA2's cipher suite — even if iwinfo labels it "WPA PSK (CCMP)"
// it's a WPA2 network and we must use psk2 to associate correctly.
if lower.contains("ccmp") || lower.contains("aes") {
"psk2".to_string()
} else {
"psk".to_string()
}
} else if lower.contains("none") || lower.contains("open") || lower.is_empty() {
"none".to_string()
} else {
lower
}
}

View File

@ -1,111 +0,0 @@
// Container image signature verification using Cosign
// Verifies that container images are signed and trusted
use anyhow::{Context, Result};
use std::process::Command;
use tracing::{info, warn};
pub struct ImageVerifier {
cosign_public_key: Option<String>,
require_signatures: bool,
}
impl ImageVerifier {
pub fn new(cosign_public_key: Option<String>) -> Self {
Self {
cosign_public_key,
require_signatures: false,
}
}
/// Create a verifier that requires all images to be signed.
pub fn new_strict(cosign_public_key: Option<String>) -> Self {
Self {
cosign_public_key,
require_signatures: true,
}
}
/// Verify a container image signature
pub async fn verify_image(&self, image: &str, signature: Option<&str>) -> Result<bool> {
if signature.is_none() && self.cosign_public_key.is_none() {
if self.require_signatures {
return Err(anyhow::anyhow!(
"Image '{}' has no signature and no cosign key is configured. \
All container images must be signed for production use.",
image
));
}
warn!("No signature provided for image: {}", image);
return Ok(false);
}
// Check if cosign is available
let cosign_available = Command::new("cosign").arg("version").output().is_ok();
if !cosign_available {
if self.require_signatures {
return Err(anyhow::anyhow!(
"Cosign binary not found. Install cosign to verify container image signatures."
));
}
warn!("Cosign not available, skipping signature verification");
return Ok(false);
}
// If public key is provided, use it for verification
if let Some(ref public_key) = self.cosign_public_key {
let output = Command::new("cosign")
.arg("verify")
.arg("--key")
.arg(public_key)
.arg(image)
.output()
.context("Failed to run cosign verify")?;
if output.status.success() {
info!("Image signature verified: {}", image);
return Ok(true);
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(anyhow::anyhow!("Signature verification failed: {}", stderr));
}
}
// If signature URL is provided, verify using that
if let Some(sig_url) = signature {
if sig_url.starts_with("cosign://") {
// Extract signature reference
let sig_ref = sig_url.strip_prefix("cosign://").unwrap();
let output = Command::new("cosign")
.arg("verify")
.arg("--signature")
.arg(sig_ref)
.arg(image)
.output()
.context("Failed to run cosign verify")?;
if output.status.success() {
info!("Image signature verified: {}", image);
return Ok(true);
} else {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(anyhow::anyhow!("Signature verification failed: {}", stderr));
}
}
}
Ok(false)
}
/// Check if an image has a signature
pub async fn has_signature(&self, image: &str) -> bool {
// Try to find signature in registry
let output = Command::new("cosign")
.arg("triangulate")
.arg(image)
.output();
output.is_ok() && output.unwrap().status.success()
}
}

View File

@ -1,7 +1,5 @@
pub mod container_policies;
pub mod image_verifier;
pub mod secrets_manager;
pub use container_policies::ContainerPolicyGenerator;
pub use image_verifier::ImageVerifier;
pub use secrets_manager::SecretsManager;

View File

@ -0,0 +1,411 @@
# Archipelago 1.8.0 — Release Hardening Plan & Tracker
> **The one living checklist for shipping 1.8.0.** Derived from a full-system deep
> audit (2026-07-02): backend security, backend code-quality, frontend, mesh,
> tests/release pipeline, and the ISO build. Supersedes nothing — it *sits above*
> `docs/UNIFIED-TASK-TRACKER.md` (day-to-day) as the release exit-criteria list.
> **Keep it updated: tick a box the moment an item lands, with the commit sha.**
**Definition of done for 1.8.0:** the supply chain is authenticated end-to-end
(§A), OTA self-update is safe and rollback-proven on real hardware (§B), no
secrets ship in the image (§F), and the single-node gate stays 5/5 green through
all of it. Everything else is polish that should not block the tag.
**Legend:** `[ ]` open · `[~]` in progress · `[x]` done · 🔴 critical · 🟠 high ·
🟡 medium · 🟢 low/polish · ⛔ blocked on you.
---
## 🎯 The single most important insight
The **release signing ceremony (Workstream B) is the linchpin.** ✅ The ceremony
KEY was generated (user confirmed 2026-07-02) — the hard offline part is done. But
the outputs are **not yet wired into the repo**: `anchor.rs:21` is still `None` and
`releases/app-catalog.json` carries no `signature`/`signed_by` (its `image_signature`
fields are literal `"cosign://..."` placeholders). Three mechanical steps remain,
split by who can run them: **(1)** pin the pubkey — needs only the *public* hex, can
be done in-repo now; **(2)** sign the catalog with the `RELEASE_MASTER_MNEMONIC`
only the publisher, secret never touches a host; **(3)** implement + flip cosign
enforcement on the pull path. Until (1)+(2) land, every "verify the signature" task
below is written but not enforced. **This is still the critical path; §A converges on it.**
---
## §A — Supply-chain authentication (🔴 THE release blocker)
Today an attacker who controls the mirror IP (or any MITM on the plaintext HTTP
path) can ship an arbitrary root binary, arbitrary container images, and an
arbitrary app catalog to the entire fleet — fully unattended under
`auto_apply`. These four items are one story and must land together.
- [x] 🔴 **Pin `RELEASE_ROOT_PUBKEY_HEX` + sign the catalog** — DONE 2026-07-02.
`anchor.rs` pinned to `5d15cbee…d469951` (signer
`did:key:z6MkkidEnEpo6qHMCNSZoNKWtvQvxq3whnaME9wGgEFhq7ur`); trust tests updated (16/16
green). `releases/app-catalog.json` signed in place (`signed_by` matches, 64-byte sig);
two blocking floats fixed en route (`archy-btcpay-db` version→string, `cpu_limit` 0.25→1).
Ship order (backward-compatible): signed catalog goes out first (old binaries still accept
it), pinned-anchor binary follows in the next build/OTA. **Still ahead:** (a) the
pinned-anchor binary must actually be built + shipped for enforcement to be live on nodes;
(b) flip "accept unsigned" → "reject unsigned" only after the whole fleet is on the pinned
binary (`container/app_catalog.rs:397`, the `Unsigned` arm) — see the next item.
- [~] 🔴 **Enforce a signature on the OTA manifest before trusting it.** Signature
verification LANDED 2026-07-02: `check_for_updates` now fetches raw JSON and runs
`trust::verify_detached` — a present-but-invalid/wrong-signer signature hard-rejects
the mirror; unsigned manifests are offered for MANUAL apply only (`manifest_signed`
surfaced in `UpdateState`) and **auto-apply refuses them**. Publisher side:
`create-release.sh` signs the manifest inline (ceremony), `publish-release-assets.sh`
hard-refuses to ship unsigned (grep + `ceremony verify` crypto gate), and
`scripts/sign-manifest.sh` exists for re-signs. **Still open:** move the mirror
to HTTPS + pinned cert (tracked with the next item); flip unsigned-manual-apply →
hard-reject once the fleet is on a pinned-anchor binary.
- [x] 🔴 **Implement container image signature verification (cosign).** DONE 2026-07-04
(code path; enforcement dormant until the ceremony): new `container::image_verify`
gates BOTH pull sites (`PodmanClient::pull_image` + the dev-only `DockerRuntime`).
Claims classify as None / the literal `cosign://...` placeholder (every fleet
manifest today → pull proceeds, logged) / Declared → `cosign verify --key
/etc/archipelago/cosign.pub --insecure-ignore-tlog=true` (+ both insecure-registry
flags for the HTTP mirror; flags verified against cosign docs), hard-fail on missing
key, missing cosign binary, timeout, or bad signature — a declared signature can
never be skipped, on either runtime. Key path overridable via
`ARCHIPELAGO_COSIGN_PUBKEY`. Deleted the caller-less, blocking, wrong-CLI
`security::ImageVerifier`. **Activation = ceremony work**: pin cosign.pub on nodes +
install cosign + publish real `image_signature` values (in that order); tracked with
the Workstream B signing ceremony item.
- [ ] 🟠 **Move the image mirror to HTTPS; drop `--tls-verify=false`.**
`podman_client.rs:641` `INSECURE_REGISTRY_HOSTS = ["146.59.87.168:3000"]` +
`config.rs:104,124` allowlist pull images over unauthenticated HTTP. Remove the raw-IP
entries; give the mirror a valid/pinned cert. (Same host also baked insecurely into
the ISO — see §F.)
- [x] 🟠 **Validate every image string at the pull site, not just the RPC boundary.**
DONE 2026-07-03: policy extracted to `container::image_policy` (single source of truth;
RPC-boundary check delegates to it) and BOTH orchestrator pull sites (`install_fresh` +
`ensure_resolved_source_available`) hard-bail on refs that fail it. Policy accepts
trusted-registry refs + registry-less Docker Hub shorthand (`grafana/grafana` — used by
8 manifests, can't name an attacker host); rejects any explicit non-allowlisted
registry host, shell metachars, malformed refs. 4 new unit tests; container 159 /
package 46 green.
---
## §B — OTA self-update safety (🔴 1.8.0's headline feature is untested live)
The apply path itself is well-built (resumable download, staged-complete marker,
atomic swap, single-depth backup). The gaps are **authenticity** (§A) and
**verification depth** — plus the fact that the upgrade path has never run
end-to-end on real hardware.
- [x] 🔴 **Deepen the post-OTA health check.** DONE 2026-07-03: `verify_pending_update`
now requires, in the same attempt, (1) frontend 2xx/3xx via nginx, (2) backend RPC
liveness — unauthenticated POST `/rpc/v1`; 401/403 = alive, 5xx/404/refused = dead,
so a 502-behind-static-files release now rolls back, (3) rootless `podman ps`
reachability; plus a pre-loop binary-version==marker assertion that catches a silent
or half swap (new frontend + old binary) deterministically. Per-app container
assertions deliberately EXCLUDED — the pre-Quadlet service restart legitimately kills
containers and the reconciler can need minutes (false-rollback risk); revisit after
the Phase-3 flip. LND-unlock-level checks remain out of scope for the 90s window.
- [ ] 🟠 **Run one real upgrade-from-vN-1 soak on hardware before tagging.**
No test installs the previous version, points it at a staged 1.8.0 manifest, applies,
and asserts health + rollback. This is the top release risk for an OTA release. A
two-VM (or two-node) harness is enough.
- [x] 🟡 **Guard the frontend-build-no-op in the *actual* release path.** DONE 2026-07-08
(`e77ccff0`): the grep guard is folded directly into `create-release.sh` right after its
own `npm run build` (post-version-bump, pre-packaging — calling `run.sh --with-build` at
stage 0 would have checked the pre-bump version). Verified both ways against the real
dist: passes on the current version, trips on a missing one.
- [x] 🟢 **publish-release-assets verifies size, not sha256.** DONE 2026-07-08 (`e77ccff0`):
the verify loop now downloads each published asset and compares sha256 (and size)
against the manifest. Verified live against the v1.7.99-alpha assets on the vps2 mirror.
---
## §C — Backend robustness (🟠 stability, mostly low-effort/high-ROI)
Note: the `.unwrap()`/`panic!` worry is a **non-issue** — nearly all are in test
modules; production request/boot paths are essentially panic-free. The real risks:
- [x] 🟠 **Log swallowed persistence writes.** DONE 2026-07-02 (full-workspace re-inventory
found 19 production sites): 16 converted to `if let Err(e) = … { warn!(…) }` — mesh
config (`server.rs`), relay tor endpoint (`bitcoin_relay.rs`), update mirrors/state +
staging flush/sync (`update.rs`), registry config, radio-contact blocklist, mesh outbox
sweep (`scheduler.rs`), block-header cache (`mesh/mod.rs`), 7× peer-transport badge
(`sync.rs` + `content.rs`). Federation tombstone/untombstone upgraded to hard errors
(see §I). Install-log line write left fire-and-forget with an explanatory comment.
- [x] 🟠 **Remove blocking `std::process::Command` from async handlers.** DONE 2026-07-03:
converted to `tokio::process``published_host_port` (install), `detect_disk_gb`
(dependencies), factory-reset restart (system/handlers), `config.rs detect_host_ip`,
the orchestrator host-facts helpers (`detect_host_ip/mdns/disk_gb`, `bitcoin_host`,
`resolve_dynamic_env` now async through all 6 call sites), and `AutoRuntime::new`
probes. `transport/fips.rs is_available()` (sync trait method on the async route path)
now serves the cached value and refreshes via a background thread (stale-while-
revalidate) instead of blocking on systemctl. `image_verifier.rs` cosign sites have no
callers yet — handled with the §A cosign item. Tests: container 155 / transport 29 /
config 29 / package 46 all green.
- [x] 🟡 **Restrict Bitcoin RPC exposure.** DECIDED (user, 2026-07-08: break external
wallets) + DONE `dd61a204`: manifest port mappings grew a validated `bind` field;
bitcoin-knots/-core publish 8332 on `127.0.0.1` + the archy-net gateway `10.89.0.1`
only (in-node consumers dial host.archipelago/host.containers.internal → 10.89.0.1,
unaffected; P2P 8333 stays public); legacy config.rs strings get the same incl.
unauthenticated ZMQ 28332/28333. Unbound ports render byte-identical (no false-drift
wave). **Still to roll out:** catalog regen + re-sign for catalog-covered nodes;
each node's bitcoin container recreates once on deploy → restart lnd after (IP cache).
- [x] 🟡 **Move secret env out of plaintext channels → podman secrets.** DONE 2026-07-05,
**VERIFIED ON .228 2026-07-08**: recreate wave settled (only fedimint-gateway lagged —
restart-sensitive, converged via a controlled quadlet restart, healthy, 0 plaintext
password vars in inspect); btcpay inspect carries no secret values; 17 quadlet unit
files use `Secret=` lines. Original notes:
secret env no longer merges into `environment` — it would land in `podman inspect`
AND as plaintext `Environment=` lines in Quadlet unit files on disk (the worse leak).
New pipeline: `expand_and_partition_env` taints plain entries that interpolate
secrets (btcpay's `Password=${BTCPAY_DB_PASS}` connection strings travel as secrets
too), values register as podman secrets (stdin, `--replace`, content-hash label,
per-app cache so steady-state reconciles are podman-free), containers reference
them via `secret_env` (API) / `Secret=…,type=env` (Quadlet). Verified empirically
on fleet podman 5.4.2: value absent from inspect, runtime injection works. Rotation
drift via `io.archipelago.secret-env-hash` container label; pre-upgrade containers
lack the label → ONE-TIME recreate wave on first reconcile after deploy (by design —
scrubs plaintext secrets from existing container configs). Docker dev fallback keeps
plain env (no secret store). `/proc/<pid>/environ` inside the container is unchanged
(env is the app-compat contract); the closed leaks are inspect output + unit files.
- [x] 🟡 **Harden rate-limit IP extraction.** DONE 2026-07-03: the accept loop injects the
TCP `PeerAddr` into request extensions; `extract_client_ip` honors
`X-Real-IP`/`X-Forwarded-For` ONLY when the connection is from loopback (our nginx,
which sets `X-Real-IP $remote_addr`) — direct connections (e.g. the FIPS peer
listener) bucket under their socket IP, so per-request header rotation no longer
defeats the login limiter. 3 unit tests.
- [x] 🟢 **Include `seq` in the mesh signed preimage.** DONE 2026-07-04 (receiver half):
`verify_signature` accepts a v2 preimage `(t,v,ts,seq)` alongside legacy v1 `(t,v,ts)`;
`signed_with_seq()` is the v2 sender path, deliberately NOT yet wired — receivers
hard-drop bad signatures, so senders stay on v1 until the whole fleet verifies v2.
The seq-tampering window closes only when the v1 arm is removed (track as a
post-fleet-rollout follow-up). Unit tests cover v2 verify, v2 seq-tamper rejection,
and v1 sign-then-set-seq compatibility.
- [x] 🟢 **Guard the short-DID slice panic** (`mesh/listener/decode.rs:566`) and gate the
dev-mode `password123` bypass (`auth.rs:18`) behind `#[cfg]`. DONE 2026-07-04:
advert_name uses `.get()` fallback (malformed radio-supplied DID can't panic the
listener); the pre-setup dev-password login + the constant itself are
`#[cfg(debug_assertions)]` — no release binary carries the bypass regardless of
runtime config.
- [ ] 🟢 **Apply the seccomp/apparmor profile**`security/src/container_policies.rs:71` is a
TODO; the profile is defined but never applied to podman.
- [ ] 🟡 **Manifests that hardcode secrets in plain `environment:` bypass the whole secret
pipeline** (found during the 2026-07-08 .228 leak check): indeedhub-api/-ffmpeg ship
`AES_MASTER_SECRET=0123456789abcdef…` and photoprism `PHOTOPRISM_ADMIN_PASSWORD=archipelago`
as literal plaintext in quadlet unit files; grafana's `GF_SECURITY_ADMIN_PASSWORD=$${…}`
reference never expands. Fix = declare them as `generated_secrets`/`secret_env` in the
manifests — which means catalog regen + re-sign + republish (catalog overlay supremacy).
- [x] 🟠 **Legacy `mempool` umbrella id destroys the split stack on stop→start (quadlet).**
FOUND by the first quadlet-mode gate run on .228 (2026-07-08), FIXED `161a6e4d`:
orchestrator start/stop/restart now alias `mempool` → the split members whenever the
umbrella manifest was dropped (same alias install already used); podman-5
`no such object` phrasing added to all 3 `is_missing_container_error` classifiers.
Follow-up (open): reconciler-side cleanup of an orphan umbrella `mempool` container
when the split stack owns the frontend, so the stale-tile state can't arise at all.
- [x] 🟠 **Transitional package state sticks past the gate window on legacy apps**
(vaultwarden:stop run C, jellyfin:stop run D, uptime-kuma:start run E — .228
2026-07-09), FIXED `dd3afbba`: the scanner already saw the settled container every
60s but `merge_preserving_transitional` refused to report it until the RPC worker
wrote back — and workers legitimately trail the container by minutes (stop workers
queue behind the orchestrator app_lock that reconcile's host-port repair holds
through multi-minute stability waits; start workers hold `Starting` through
readiness budgets up to 420s for uptime-kuma against a 240s gate window, with the
20-minute Installing stuck-timeout as the only escape). New merge rules:
(Stopping, Stopped)+user-stop-marker → Stopped; (Starting, Running) → Running;
Restarting deliberately unresolved. Follow-up (open, same family as the op-lock
known-limit): repair/readiness waits should abort early when a user-stop marker
appears mid-wait, so an explicit stop is never queued behind a multi-minute repair.
- [x] 🟢 **`install_log()` has been a no-op since April** — `/var/log/archipelago/
container-installs.log` is 0 bytes: the service sandbox leaves /var/log read-only,
the open() fails, fire-and-forget drops every line. FIXED `c3f0a306`: every line
now mirrors to tracing/journald; file append stays best-effort.
- [x] 🟢 **Gate tests 123/124 false-fail on user-stopped apps with lingering quadlet
units** (run E: the inactive bitcoin-core of the multi-version pair), FIXED
`2683ad4f0`: `use-quadlet-backends-install.bats` active-state asserts now honour
`user-stopped.json`.
- [ ] 🟠 **Backend recreate must cascade to dependent apps** (found on .228 2026-07-08):
when bitcoin-knots was recreated mid-gate (one-time secret-env recreate) it got a new
archy-net IP; **lnd caches the resolved backend IP** and kept dialing the dead one
("no route to host") for 30+ min — chain-blind with open channels, silently (container
"running", health green). Repair was a manual lnd restart. Fix = reconciler/health
monitor restarts (or at least alerts on) apps whose declared backend container was
recreated; same class applies to electrumx/btcpay/nbxplorer → bitcoin links.
---
## §D — Frontend security & performance (🟠)
The untrusted mesh/LoRa chat path is **safe** (interpolation, no `v-html` — good).
The real issues are the app-bridge origin model and a bloated bundle.
- [x] 🟠 **Validate `event.origin` + add consent gates in the NIP-07 nostr bridge.**
DONE 2026-07-02: `handleNostrRequest` rejects senders whose `event.origin` doesn't match
the open app's URL origin, and ALL identity-sensitive methods (`getPublicKey`, `signEvent`,
`nip04`/`nip44` encrypt+decrypt) now go through the consent/approved-origins gate, not just
`signEvent`. Verified present in the built bundle.
- [x] 🟠 **Origin-check the `share-to-mesh` handler.** DONE 2026-07-02: `App.vue`
`onShareToMeshMessage` now requires `ev.origin === window.location.origin` (matching
`Chat.vue`).
- [ ] 🟡 **Decide the app-iframe isolation model.** `AppSessionFrame.vue:54` /
`AppLauncherOverlay.vue:79` embed apps same-origin with no meaningful `sandbox`; a
same-origin app can read the CSRF cookie + `localStorage`. Ideal fix (serve apps from a
per-app subdomain origin) is architectural — at minimum decide + document for 1.8.0.
- [ ] 🟡 **Shrink the 93 MB dist.** `assets/video/video-intro.mp4` is **14.7 MB**
(precached by the service worker → blocks PWA install), plus ~18 MB of ~1 MB full-screen
JPEGs. Convert backgrounds to WebP/AVIF at responsive sizes, lazy/stream the intro video,
and exclude video/audio from the Workbox precache. Biggest, easiest perf win.
- [x] 🟢 **DOMPurify the `Server.vue` QR SVG / guard `Mesh.vue` pollInterval / surface
`curatedApps.ts` fetch failures.** DONE 2026-07-03: WireGuard peer QR now sanitized with
the same `USE_PROFILES: {svg}` call as TwoFactorSection; Mesh poll interval guarded +
nulled on unmount; catalog fetch failures log per-URL console.warn incl. the
all-sources-failed fallback. Bundle-verified.
---
## §E — Mesh transports (🟢 mostly done — verify & polish)
Confirmed **fixed in HEAD:** B8 (1970 timestamps), B6 (inbound RX surfacing), the
per-message transport pill, and the archy↔archy plain-TEXT-DM E2E fix. Remaining:
- [ ] 🟠 **Active Reticulum daemon-death detection.** `reticulum.rs:589` only `warn!`s on
socket EOF and `try_recv_frame` then returns `Ok(None)` forever; nothing calls
`child.try_wait()`. On an idle link a crashed daemon is invisible for up to 30 min (the
RX-stall timeout). Treat socket EOF as `Err` → immediate respawn. (Pairs with the current
`fix/reticulum-daemon-pdeathsig` branch work.)
- [ ] 🟡 **Persist chat history across restarts.** `state.messages` boots empty
(`listener/mod.rs:283`) while outbox/scheduler/peers survive — inconsistent; bubbles
vanish on restart. Add `mesh-messages.json` mirroring the `scheduler.rs`/`outbox.rs`
pattern (or explicitly accept the loss).
- [ ] 🟡 **Tighten the 30 s legacy dedup** (`listener/mod.rs:383-389`) — it silently drops a
peer legitimately sending identical text twice within 30 s.
- [ ] 🟢 **Wire the PyInstaller daemon binary into the release tarball / deploy script**
(Rust expects `/usr/local/bin/archy-reticulum-daemon`, `reticulum.rs:80`); add the RNode
udev rule; finish `ARCHY:2:` announce→`arch_pubkey_hex` binding (`reticulum.rs:119`).
- [ ] 🟢 **Duty-cycle guard for LoRa TX** — none exists; EU 868 is legally 1%. At minimum an
airtime budget/warning.
---
## §F — ISO / image build (🔴 one secret leak; otherwise 🟠 hardening)
`image-recipe/_archived/build-auto-installer-iso.sh` (3604 lines) is the real
builder; OTA is the normal update path but the ISO is what produces installable
media (latest artifact only one minor behind).
- [ ] ⛔🔴 **Anthropic API key — INTENTIONAL for alpha/beta, hard GO-LIVE gate.**
`build-auto-installer-iso.sh:2645` bakes a live `sk-ant-…` key into `claude-api-proxy.service`
so alpha/beta testers get frictionless AI (deliberate — per user 2026-07-02). **Do NOT
remove for alpha/beta.** Before public GA it MUST be removed + rotated + injected at runtime
(a second copy also exists in a worktree). Track it here so it can't be forgotten at launch.
- [ ] 🔴 **Per-device secrets on first boot.** The self-signed TLS **private key is generated
at build time** (`:426`) → every device ships the same key; SSH host keys likewise not
regenerated. Generate TLS + SSH host keys on first boot.
- [ ] 🟠 **Kill default credentials.** `archipelago`/`archipelago` (SSH+root), web `password123`,
and SSH `PasswordAuthentication yes` (`:411`) all ship. Lock root, force credential
creation in onboarding, disable SSH password auth (or force-change on first login).
- [ ] 🟠 **Sign + checksum the ISO.** Pipeline ends at `xorriso` with no `SHA256SUMS`, no
GPG/minisign, no Secure Boot (`BOOTX64.EFI` is unsigned though `grub-efi-amd64-signed` is
installed). Emit + sign checksums; wire signed Secure Boot.
- [ ] 🟠 **Registries over HTTPS in the image too**`146.59.87.168:3000` / `git.tx1138.com`
are baked `insecure=true`/`tls_verify:false` (`:216`, `:2308`). (Ties to §A.)
- [ ] 🟡 **Add `unattended-upgrades` + a default-deny nftables firewall** (allow 22/80/443 +
mesh/WG). Neither exists today; OS packages drift until reflash and there is no host
firewall.
- [ ] 🟡 **Pin the build for reproducibility.** FIPS daemon is built from unpinned upstream
`main`, Tailscale from its live apt repo, and `scripts/image-versions.sh` uses many
`:latest`/`stable` tags (+ `bitcoin-ui:1.7.84-alpha`, 15 behind). Pin to commits/versions;
snapshot apt. Wire ISO version to `Cargo.toml` so it can't drift.
- [ ] 🟢 **Harden LUKS + roadmap A/B partitioning.** The LUKS data key sits in plaintext on the
unencrypted root (`:2137`); add TPM2/passphrase binding. Longer-term: A/B (or
factory-reset) partitions for safe OTA rollback, and a real install-time TUI
(`docs/archive/INSTALL-SCREENS-DESIGN.md` exists but the installer is headless "press Enter").
---
## §G — Refactor & code health (🟢 not release-blocking; do after the tag or opportunistically)
- [ ] 🟢 **Manifest-drive per-app special-casing.** App names are branched on across 5-7 Rust
files (`config.rs` 36 match arms, `runtime.rs` 17, `install.rs:275-287` dispatch,
`prod_orchestrator.rs:54-83` baseline/restart-sensitive lists). Move `baseline`,
`restart_sensitive`, `stack_members`, `multi_container` into the manifest schema; collapse
the five near-identical `install_*_stack()` wrappers into one generic call. **Biggest
maintainability win.** (Grew again 2026-07-09: `stack_member_app_ids` in
`package/dependencies.rs` — the quadlet stack-resurrection fallback — is a fifth
per-app map that must fold into the same manifest field.)
- [ ] 🟢 **Route all podman/systemctl through `podman_client`.** 113 raw `Command::new("podman")`
+ 32 `systemctl` calls bypass the existing 952-LOC wrapper → untestable + the blocking-call
risk (§C). Consolidating also unlocks unit tests for the thinly-tested `package/` handlers
(`stacks.rs` 1 test, `config.rs` 2, `runtime.rs` 3, `install.rs` 7).
- [ ] 🟢 **Split the god-modules.** `prod_orchestrator.rs` (5,263 LOC) → `orchestrator/{reconcile,
host_ports,ownership,hooks}.rs`; `Mesh.vue` (2,485 LOC / 241 KB chunk) → sub-components.
Both are well-tested, so safe.
- [ ] 🟢 **Delete dead code.** ~4,100 LOC of orphan StartOS crates (`js-engine`, `models`,
`helpers`, `container-init`) not in the workspace or linked; the committed AppleDouble
`._*.rs` files; the committed `.venv/`/`build/`/`__pycache__` under the duplicate
`reticulum-daemon/` tree; promote `MeshRadioDevice` enum → trait.
- [ ] 🟢 **Resolve the Quadlet flag & dep hygiene.** Decide `use_quadlet_backends`' fate
(flip default + delete the legacy `create_container` branch, or freeze as experimental —
don't ship both half-maintained). Consolidate the mixed hyper 0.14/1.x ecosystem; bump
stale majors (reqwest, base64, thiserror, tokio-tungstenite).
---
## §H — Testing gaps that gate confidence (🟠)
- [ ] 🟠 **Add the OTA upgrade soak** (same as §B item 2) — the highest-value missing test.
- [ ] 🟡 **Add a host-reboot survival tier** — every app is `○` (untested) for reboot in
`TESTING.md:138`; the gate can't reboot the node it runs on. Run SSH-`reboot`-then-reprobe
out-of-band per node.
- [ ] 🟡 **Make the release gate run the full Rust suite** (or hard-require a green CI sha).
`tests/release/run.sh:101` runs only a 6-module slice because the full 1000-test suite
hangs PTYs on the dev box → 994 tests unverified at release time if CI is stale.
- [x] 🟡 **Add `--max-time` to `node_rpc()`.** DONE 2026-07-08 (`380f4f19`): login + rpc get
`--connect-timeout 10 --max-time 120` (override `MULTINODE_RPC_TIMEOUT`). Verified live:
.116 login/rpc OK; an unroutable node fails in 10s instead of hanging.
- [x] 🟢 **De-hardcode creds in tests.** DONE 2026-07-08 (`380f4f19`): multinode suites no
longer commit node passwords — `*_PW` env required, auto-loaded from git-ignored
`tests/multinode/.env` (`.env.example` documents the shape). Still open from this
bullet: snapshot/restore node baseline between destructive iterations
(teardown currently only clears `/tmp` session files).
---
## §I — Carried-over open items (from `UNIFIED-TASK-TRACKER.md`, still valid)
- [~] 🟠 **Multinode gate pass** — 5× destructive gate was launched on node `.5`; bring the
rest of the fleet to precondition, then run the existing (undocumented-but-present)
`tests/multinode/{smoke,meshtastic}.sh` cross-node suites.
- [~] 🟠 **Federation `remove-node` tombstone regression.** Code fix DONE 2026-07-02:
`remove_node` now tombstones BEFORE trimming the node list and propagates the write
error (idempotent, so retries are clean); `add_node`'s untombstone likewise propagates
before mutating. **Still open: `tests/multinode/smoke.sh` re-verify on real nodes.**
- [ ] 🟠 **Phase-3 Quadlet default-flip** — validated + opt-in on .228/.198; flip
`config.rs:256` once the .5 gate reports clean.
- [ ] 🟠 **Developer CLI suite** (`archy app validate/render/install/test`) — gates external
app publishing (`APP-PACKAGING-MIGRATION-PLAN.md` step 5).
- [ ] 🟡 **Version bump + tag** — DECIDED (user, 2026-07-08): the release ships as
**`1.8.0-alpha`**. Remaining work is the mechanical bump + `create-release.sh` run
when the gate criteria are met.
- [ ] 🟢 **Bitcoin multi-version fleet OTA** — DECIDED (user, 2026-07-08): timing doesn't
matter; fold the branch into the next fleet OTA (`docs/bitcoin-version-bulletproof-rollout.md`).
- [x] ~~⛔🟢 **3ccc stock-Meshtastic RF validation**~~ — DROPPED per user 2026-07-08; the
code fix stays in, no live-radio validation will be scheduled.
---
## Suggested order of attack
1. **The critical path:** §A signing ceremony → then turn on manifest/catalog/image
signature enforcement (§A) + OTA HTTPS/signature + deeper health check (§B).
2. **Cheap high-ROI stability:** §C swallowed-writes + blocking-calls; §D nostr-bridge
+ share-to-mesh origin checks; §H OTA soak + reboot tier.
3. **Image hardening:** rest of §F (per-device secrets, default creds, ISO signing,
firewall/unattended-upgrades, pinning).
4. **Polish, post-tag:** §G refactors, §E mesh persistence/dedup, §D bundle shrink.
5. **Decisions you own (⛔):** version name, signing mnemonic, bitcoin OTA timing, 3ccc test.
6. **Before public GA only (NOT alpha/beta):** remove + rotate the Anthropic key (§F) —
intentionally left in for frictionless AI during alpha/beta.
*Last updated: 2026-07-08 PM (hardening session 3: §C secrets verified on .228,
§B release-path guards `e77ccff0`, §H test creds/timeouts `380f4f19`, mempool
umbrella lifecycle fix `161a6e4d` from the first quadlet-mode gate run; new
finding: manifests hardcoding secrets in plain `environment:`). Update this
line + tick boxes with commit shas as items land.*

View File

@ -32,7 +32,7 @@ As of the current `1.8-alpha` workstream:
- `apps/*/manifest.yml` is the source of truth for runtime app definitions.
- The Rust manifest parser validates app identity, image-vs-build source selection, safe environment/secrets, safe ports, safe bind/named/tmpfs volumes, generated files under declared bind mounts, devices, and security/network policy values.
- Manifest-owned generated files exist through `app.files` and have been used for app config material such as Meshtastic config regeneration.
- Manifest-owned generated files exist through `app.files` and have been used for app config material (e.g. strfry, netbird config regeneration).
- Local image builds are represented with `container.build`; pulled images are represented with `container.image`.
- Data ownership repair is represented with `container.data_uid`.
- Derived host facts and secret-file-backed environment variables are represented with `container.derived_env` and `container.secret_env`.
@ -41,7 +41,7 @@ As of the current `1.8-alpha` workstream:
- Runtime package listings now derive LAN launch URLs from manifest-owned `interfaces.main` declarations or HTTP app ports before falling back to legacy compatibility aliases.
- Release drift checking is implemented by `scripts/check-app-catalog-drift.py --release --strict`.
- The canonical catalog and the UI public catalog are expected to remain byte-for-byte synced after generation.
- Runtime validation has already moved many simple and moderate apps into the manifest/orchestrator path, including Filebrowser, Vaultwarden, Portainer, Uptime Kuma, Grafana, Gitea, Nextcloud, SearXNG, Nostr Relay, PhotoPrism, Jellyfin, Meshtastic, and several Bitcoin-adjacent apps.
- Runtime validation has already moved many simple and moderate apps into the manifest/orchestrator path, including Filebrowser, Vaultwarden, Portainer, Uptime Kuma, Grafana, Gitea, Nextcloud, SearXNG, Nostr Relay, PhotoPrism, Jellyfin, and several Bitcoin-adjacent apps.
The remaining migration work is mostly orchestration quality: post-reboot adoption, progress reporting, stale scanner-state handling, update policy, multi-container stack ownership, proxy route generation, and cleanup of obsolete legacy installers/fallbacks.
@ -327,6 +327,14 @@ Once a complex stack is clean, the app system is credible.
## Implementation Phases
**Status (2026-07-08):** Phases 13 ✅ DONE (per-member manifests won over a
compose subset; all five real multi-container stacks — btcpay, mempool,
immich, netbird, indeedhub — install via `install_stack_via_orchestrator`).
Phase 5 mostly done (orchestrator-first with legacy fallback + per-app
adoption/repair). Phase 4 (routing via `proxy/routes.yml`) NOT started —
routing is still host-nginx driven. Phase 6 (cleanup + developer CLI) NOT
started; the CLI gates external app publishing.
### Phase 1: Package Contract
- Use `apps/<app-id>/manifest.yml` as the package contract.
@ -340,7 +348,7 @@ Once a complex stack is clean, the app system is credible.
- Continue hardening package install for one-container apps.
- Compile manifests to rootless Podman/Quadlet runtime behavior.
- Support ports, env, generated files, devices, volumes, resources, health checks, data UID repair, image pull/build availability checks, and launch metadata.
- Keep Filebrowser, Vaultwarden, Portainer, Uptime Kuma, Grafana, SearXNG, Jellyfin, PhotoPrism, Meshtastic, and similar apps as regression proofs.
- Keep Filebrowser, Vaultwarden, Portainer, Uptime Kuma, Grafana, SearXNG, Jellyfin, PhotoPrism, and similar apps as regression proofs.
### Phase 3: Multi-Container Runtime

View File

@ -54,7 +54,7 @@ real nodes. Until then, this plan is the priority.
- **~40 apps are manifest-based and Quadlet-migrated** (survive
`archipelago.service` restart + reboot). Exhaustive per-app table:
`docs/app-registry-status-2026-06-21.md`.
`docs/archive/app-registry-status-2026-06-21.md`.
- **Legacy holdout: immich** — the one app with **no manifest** and a hardcoded
Rust stack installer (in-cgroup, not Quadlet). 3 containers, healthy, live data.
The migration proof case.
@ -77,7 +77,7 @@ real nodes. Until then, this plan is the priority.
| E | **Production test gate** — 5× lifecycle on **.228**, per-app L1/L2 matrix; multinode is split out → `multinode-testing-plan.md` | `tests/lifecycle/TESTING.md`, `bulletproof-containers.md` | **✅ .228 5×-GREEN (110/110 ×5, 0 not-ok, 2026-06-23)** — but this is DESTRUCTIVE-tier / ~8 core apps only; see §6c for the coverage gaps |
| F | **Lifecycle perfection — cascade + progress + ALL apps** — extend the gate to uninstall/reinstall (cascade), real install/uninstall progress UI, and EVERY installed app (not just the 8 core). The "insanely-perfect OS/container environment" bar. | §6c (below), `tests/lifecycle/TESTING.md` | **IN PROGRESS (2026-06-26)** — root bug FIXED: uninstall could hang → ghost/stuck-bar/reinstall-block (`71cc9ac4`, unbounded systemctl/podman in `quadlet::disable_remove`); `cascade-uninstall.bats` **7/7 green on .228** w/ binary `ae349a75`. Remaining: wire CASCADE into the canonical gate run, progress-UI truthfulness, all-apps matrix, guardian/IBD state. |
**Orchestrator architecture** (foundation for A/B): `rust-orchestrator-migration.md`
**Orchestrator architecture** (foundation for A/B): `archive/rust-orchestrator-migration.md`
(ProdContainerOrchestrator, BootReconciler 30s level-triggered reconcile, adoption
scan, Quadlet rendering) and `bulletproof-containers.md` (the six container failure
modes FM1FM6 + the desired-state-first reconciler that fixes them).
@ -159,7 +159,7 @@ the next release. **Authoritative detail + exact remaining steps + test matrix
filebrowser). It explicitly **SKIPS uninstall/reinstall** (the CASCADE tier is gated behind
`ARCHY_ALLOW_CASCADE_DESTRUCTIVE`, which `run-gate.sh` never sets) and has **zero coverage**
for the other ~30 apps (grafana, jellyfin, vaultwarden, penpot, nextcloud, photoprism,
uptime-kuma, homeassistant, … — see `app-registry-status-2026-06-21.md`). So uninstall,
uptime-kuma, homeassistant, … — see `archive/app-registry-status-2026-06-21.md`). So uninstall,
reinstall, install-progress UI, and most apps were never under test.
**Real bugs found in manual multinode testing on .198 (2026-06-23) — the motivating evidence:**
@ -313,7 +313,7 @@ phases 26 (`dual-ecash-design.md`).
**Branch/worktree:** currently on `bitcoin-version-bulletproof`, not `main`. Worktree is dirty.
Do **not** discard mesh changes: they include E2E/transport indicator plumbing and the Meshtastic
receive-path fixes below. Separate recovery note: `docs/SESSION-1.8.0-OTA-PROGRESS.md`.
receive-path fixes below. Separate recovery note: `docs/archive/SESSION-1.8.0-OTA-PROGRESS.md`.
**What was done this session:**
1. ✅ **Local Rust release gate fixed and green.** `cargo test -p archipelago --bin archipelago` is
@ -902,13 +902,13 @@ This master plan is the hub. Authoritative standalone docs (linked above), kept:
- **Design:** `architecture.md`, `app-developer-guide.md`,
`APP-PACKAGING-MIGRATION-PLAN.md`, `registry-manifest-design.md`,
`marketplace-protocol.md`, `dht-distribution-design.md`,
`multi-node-architecture.md`, `rust-orchestrator-migration.md`,
`bulletproof-containers.md`, `three-mode-ui-design.md`, `dual-ecash-design.md`,
`multi-node-architecture.md`, `archive/rust-orchestrator-migration.md`,
`bulletproof-containers.md`, `archive/three-mode-ui-design.md`, `dual-ecash-design.md`,
`meshroller-integration-design.md`, `phase4-streaming-ecash-plan.md`, `adr/*`.
- **Reference:** `app-manifest-spec.md`, `api-reference.md`, `developer-guide.md`,
`operations-runbook.md`, `troubleshooting.md`, `user-walkthrough.md`,
`bitcoin-rpc-relay.md`, `security-code-audit-2026-03.md`, `GAMEPAD-NAV.md`,
`SEED-VERIFICATION.md`, `hotfix-process.md`, `app-registry-status-2026-06-21.md`.
`bitcoin-rpc-relay.md`, `archive/security-code-audit-2026-03.md`, `GAMEPAD-NAV.md`,
`SEED-VERIFICATION.md`, `hotfix-process.md`, `archive/app-registry-status-2026-06-21.md`.
All dated handoffs/resumes/transcripts/superseded trackers were consolidated here
and removed (recoverable via git) on 2026-06-21.

View File

@ -8,7 +8,7 @@ Full plan: `.claude/plans/enchanted-strolling-rocket.md`. Memory pointer:
`project_reticulum_transport_plan.md` (auto-memory index).
**Coordination note (2026-06-30):** a separate agent owns concurrent Meshtastic work, scoped to
`mesh/meshtastic.rs` + `mesh/protocol.rs` (see `docs/SESSION-1.8.0-OTA-PROGRESS.md`) and explicitly
`mesh/meshtastic.rs` + `mesh/protocol.rs` (see `docs/archive/SESSION-1.8.0-OTA-PROGRESS.md`) and explicitly
avoiding `mesh/listener/session.rs` transport plumbing + `mesh/mod.rs` routing, which this work
owns. Stay out of `meshtastic.rs`/`protocol.rs` to avoid collisions.
@ -25,6 +25,7 @@ owns. Stay out of `meshtastic.rs`/`protocol.rs` to avoid collisions.
| 2c | `MeshConfig.device_kind` reflashable-board pin | ✅ **DONE** this session (was the one open Phase-2 item) |
| 3 | Frontend (~8 label/CSS spots) | ✅ DONE (scoped down — see note below) |
| 4 | Multi-device (run all 3 radios at once) + per-network channels | ⏳ not started (follow-on, after 03) |
| 5 | Aurora interop — optional plain-TCP Reticulum interface (radio-less) | ✅ **DONE + verified 2026-07-03** — see checkpoint below. Real Aurora GUI test still open (manual follow-up). |
## Checkpoint 2026-06-30 (late session — read this first if cut off)
@ -290,3 +291,82 @@ Phase 0 gates #1#3 are now **all passed**. What's left:
second board will likely report the same `0001` stock serial since CP2102 modules commonly
ship with an unprogrammed default, so this may still need a different disambiguator.
6. Phase 4 (run all 3 radios at once) — still not started, follow-on after the above.
## Checkpoint 2026-07-03 — Phase 5: Aurora interop via plain-TCP Reticulum (radio-less)
**Why:** `~/aurora` (a separate Flutter off-grid messenger) already runs real RNS + LXMF
(`LxmfRouter`, comment "interop with Sideband/NomadNet/MeshChat" in `rns_service.dart`), and its
**default** connectivity mode is plain TCP (`RnsTcpInterface`/`RnsTcpServerInterface`), not radio —
it ships a static bootstrap list of public RNS hubs on port 4242. Archy's daemon could previously
only bring up a serial-RNode interface, so it was unreachable by Aurora (or any TCP-based RNS/LXMF
client) at all, and every interop proof was bottlenecked on scarce LoRa hardware. This phase adds
an **optional, additive, loopback-only plain-TCP interface**, proves interop with a scripted
RNS/LXMF stand-in (the same class of proof the Sideband gate already established), and leaves the
serial/RNode path completely unchanged.
**Done, all verified:**
1. `reticulum-daemon/reticulum_daemon.py``_write_rns_config()` gained a third branch
(`--tcp-listen HOST:PORT``TCPServerInterface`, `--tcp-connect HOST:PORT` repeatable →
`TCPClientInterface`), mutually exclusive with `--serial-port`. `--tcp-listen` is hard-gated to
loopback (`_require_loopback`) — archy is otherwise Tor-first for inter-node traffic, so a
WAN/LAN-exposed Reticulum port is a deliberate future decision, not something this phase does
silently. Verified: `--selftest` regression still passes; two daemon processes (server +
client, throwaway identities) reached `connected: true` on both sides via `mesh.status`-daemon
RPC, live `TCPServerInterface`/`TCPClientInterface` visible in `get_interface_stats()`.
2. **Bidirectional LXMF DM gate against a scripted Aurora stand-in** (Python RNS+LXMF client
dialing as a `TCPClientInterface` + running its own `LXMRouter` — a legitimate protocol-level
proxy for Aurora's Dart stack, same wire format): forward (stand-in → archy daemon) and reverse
(archy daemon → stand-in) both delivered with matching content and correct source/dest hashes,
confirmed via the daemon's own `recv`/`delivered` RPC events. Direct TCP analogue of the
already-passed Sideband gate (RF → TCP, Sideband → scripted stand-in).
3. **Rust wiring**, fully additive — the serial/RNode path is byte-for-byte unchanged:
- `mesh/reticulum.rs`: new `ReticulumInterface` enum (`Serial`/`TcpServer`/`TcpClient`) threads
through `daemon_command()`/`spawn()`; `open()` (serial) now just wraps
`ReticulumInterface::Serial` — same `probe_rnode` gate as before. New
`open_tcp_server()`/`open_tcp_client()` associated fns skip `probe_rnode` entirely (the
"spawn without a physical RNode" path); `open_tcp_server` hard-enforces
`is_loopback_host()` (mirrors the Python-side guard).
- `mesh/types.rs`: new `ReticulumTcpConfig` enum (`Server { bind }` / `Client { connect }`).
- `mesh/mod.rs`: `MeshConfig.reticulum_tcp: Option<ReticulumTcpConfig>` (`#[serde(default)]`,
`None` by default — no migration, zero behavior change when unset); threaded into
`start()``spawn_mesh_listener`.
- `listener/mod.rs` / `listener/session.rs`: `reticulum_tcp` param threaded through
`spawn_mesh_listener`/`run_mesh_session`; new leading branch — if set, a new
`open_reticulum_tcp()` helper dispatches to `open_tcp_server`/`open_tcp_client`; otherwise
falls through to the **untouched** existing `preferred_path`/`auto_detect_and_open` logic.
- Deliberately **not** wired into `mesh.configure`/the frontend — dev/verification-only surface
for now (hand-edit `mesh-config.json`), consistent with how narrowly scoped this phase is.
- `cargo check -p archipelago` + `cargo test -p archipelago` (mesh module): **108 passed, 0
failed, 1 ignored** (the pre-existing hardware-gated `probe_rnode_detects_real_hardware`) —
zero regression to the serial/RNode path, provable without any hardware.
4. **End-to-end Rust integration test** (`mesh::tests::mesh_service_connects_over_reticulum_tcp_client`,
`#[ignore]`d — spawns real subprocesses, skipped in the default `cargo test` run the same way
the rest of the mesh suite skips hardware-gated tests): a real `MeshService::start()` spawns the
daemon in TCP **client** mode (no serial probe at all), dials a second stand-alone daemon
instance in TCP **server** mode (the Aurora-side role), and reaches `device_connected: true` /
`device_type: Reticulum` via the exact `MeshService::status()` call the `mesh.status` RPC uses.
Passed in ~2.6s. Run manually: `cargo test -p archipelago -- --ignored
mesh_service_connects_over_reticulum_tcp` (needs `reticulum-daemon/.venv`, see below).
**Environment note:** this session's Rust toolchain drift — system `rustc` (apt, 1.85.0) is too
old for code already on `main` (`u32::is_multiple_of` in `health_monitor.rs`, stabilized upstream
after 1.85); a pre-installed rustup toolchain at
`~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu` (1.96.0) builds clean. Not something this
phase's changes caused — pre-existing, just newly hit. Put that toolchain's `bin/` first on `PATH`
if `cargo check`/`test` reports `E0658 unsigned_is_multiple_of`.
**Explicitly NOT done (out of scope for this phase, see plan non-goals):**
- Real Aurora Flutter GUI verification — this dev sandbox has no `flutter`, no `$DISPLAY`, and no
`reticulum-dart` sibling checked out (Aurora's actual RNS implementation lives in that separate
repo; Aurora's CI clones it fresh at build time). The scripted-stand-in gate above is the
protocol-level substitute. **Manual follow-up**: point a real Aurora build's TCP hub list (or an
ad hoc connect) at an archy node's `--tcp-listen` address and confirm an LXMF DM in the actual
app UI.
- Any non-loopback (LAN/WAN) TCP bind — hard-gated off on purpose; a real "Aurora hub" deployment
needs its own security review given archy's Tor-first posture for inter-node traffic.
- LXMF propagation-node / always-on-hub role for archy (bridging Aurora's offline BLE peers) —
bigger architectural + storage commitment.
- Identity unification between archy's and Aurora's independent Nostr/secp256k1 keys — both
already have separate Nostr identities with no derivation link; out of scope here.
- `mesh.configure` RPC / frontend exposure of `reticulum_tcp` — stays hand-edit-only until/unless
it becomes user-facing.

97
docs/ROADMAP.md Normal file
View File

@ -0,0 +1,97 @@
# Archipelago Roadmap
_Last updated: 2026-07-08. This is the public-facing summary. The live,
priority-ordered engineering list is [`UNIFIED-TASK-TRACKER.md`](UNIFIED-TASK-TRACKER.md);
the narrative plan behind it is [`PRODUCTION-MASTER-PLAN.md`](PRODUCTION-MASTER-PLAN.md)._
## North star
A world-class, **developer-ready app platform**: every app manifest-driven,
manifests distributed via a **signed registry**, and third-party developers
publishing through an **external/decentralized marketplace** — all rootless,
secure, robust, and 100%-uptime-capable.
Five pillars every app must satisfy: Quadlet-everywhere · level-triggered
reconciler · lifecycle-bulletproof (full test matrix, repeatedly green) ·
data-driven (no host changes, no per-app binary code) · rootless +
security-first.
## ✅ Shipped
- **Single-node production gate GREEN** (2026-06-23) — the full destructive
lifecycle matrix (install / UI / stop / start / restart / reinstall /
reboot-survive / backend-restart-survive / uninstall) passed 5 consecutive
times with zero failures on real hardware. This was the first exit criterion.
- **Manifest-driven app platform** — 50+ apps as declarative manifests; all
multi-container stacks (BTCPay, Mempool, Immich, NetBird, IndeeHub) install
through the orchestrator; generated-secrets system replaces per-app secret code.
- **Rust orchestrator + level-triggered reconciler** — the bash-script era is
retired; drift self-heals on a 30-second loop.
- **Registry-distributed manifests** — the signed catalog embeds full manifests
per app; nodes verify against the pinned release-root key and overlay
catalog manifests over disk files (catalog wins).
- **Release signing ceremony** (2026-07-02) — release-root Ed25519 key pinned
in the binary; OTA release manifests and the app catalog are signed;
auto-apply refuses unsigned manifests.
- **1.8.0 hardening batches** — supply-chain signature verification, deepened
post-OTA health checks, async-executor and secret-handling fixes, browser
origin checks, dist shrink.
- **Reticulum third mesh transport** — RNS/LXMF over real LoRa hardware
(RNode), interop verified against Sideband; joins Meshtastic and MeshCore
behind one chat UI with X3DH + double-ratchet E2E, attachments, and mesh AI.
- **Bitcoin multi-version** — Core and Knots with per-app version pinning and
safe switching (fleet rollout pending below).
- **Decentralized marketplace backend** — Nostr NIP-78 discovery, DID-signed
manifests, federation-weighted trust scoring, Lightning purchase invoices.
- **Quadlet migration validated** — all backends run as `user.slice` Quadlet
services on the canary node (default flip pending below).
- **Public demo** — multi-visitor sandbox deployed.
## 🔄 In progress
- **Multinode pass** — run the same production gate across the whole test
fleet, plus cross-node federation/mesh suites
([`multinode-testing-plan.md`](multinode-testing-plan.md)). This is the
current exit criterion.
- **Quadlet default flip** — flip the validated Quadlet path from opt-in to
default fleet-wide; eliminates the last container-flapping root cause.
- **Container-flapping elimination** — reconciler churn and failed-unit
self-healing gaps observed on live nodes.
- **Reticulum tail** — ship the daemon binary inside the release tarball;
final fleet redeploys.
## ⏳ Release-blocking for 1.8.0
Tracked in detail in [`1.8.0-RELEASE-HARDENING-PLAN.md`](1.8.0-RELEASE-HARDENING-PLAN.md):
- **OTA upgrade-from-previous-version soak** on real hardware — the top
untested release risk.
- **ISO/image hardening** — per-device TLS/SSH keys on first boot, remove
default credentials and SSH password auth, signed + checksummed ISO,
registries over HTTPS, unattended-upgrades and firewall defaults.
- **Bitcoin multi-version fleet OTA** — code done; rollout timing is a
deliberate hold.
- Version bump to `1.8.0-alpha` + tag once the pre-tag items close.
## 🔭 Planned (post-1.8.0)
- **Developer CLI**`archy app validate / render / install / test`; the gate
for opening third-party app publishing.
- **External marketplace maturation** — publishing tooling, trust UX, and
reputation surfaces on top of the shipped backend
([`marketplace-protocol.md`](marketplace-protocol.md)).
- **DHT/P2P distribution** — releases and app images over iroh-based swarm
([`dht-distribution-design.md`](dht-distribution-design.md); feature-gated
skeleton exists).
- **P2P encrypted voice/video** over Tor between federated nodes.
- **Dual ecash** — Fedimint + Cashu phases 26, networking-sats
([`dual-ecash-design.md`](dual-ecash-design.md)).
- **Paid streaming** — streaming ecash for content
([`phase4-streaming-ecash-plan.md`](phase4-streaming-ecash-plan.md)).
- **Hardware signer** support ([`hardware-signer-design.md`](hardware-signer-design.md)).
- **Code health** — split god-modules, remove dead crates, route all
podman/systemctl calls through the wrapper.
## Release pipeline
Feature Testing (internal) → User Testing (controlled hardware) → Beta Live (public).

View File

@ -2,7 +2,7 @@
Single working list for everything left before 1.8.0 ships and the next master-plan
exit criteria (multinode + workstreams B/C/D) are met. Supersedes the open-task
sections of `docs/SESSION-1.8.0-OTA-PROGRESS.md` and `docs/PRODUCTION-MASTER-PLAN.md`
sections of `docs/archive/SESSION-1.8.0-OTA-PROGRESS.md` and `docs/PRODUCTION-MASTER-PLAN.md`
as the day-to-day tracker — those docs remain the historical record / detailed
narrative and are still linked from here where useful. **Ordered fastest/simplest
first** so we work top-down instead of hunting across docs.
@ -113,8 +113,80 @@ those are marked ✅ below with the commit that did it, so we stop re-litigating
manifest-driven apps, never stacks; fedimint/fedimint-gateway/fedimint-clientd
are 3 separate single-container apps with manifest dependency edges, not a
coordinated stack. Workstream A's stack-migration tail is fully closed.
- [ ] **Container thrashing/flapping + reconciler churn** (added 2026-07-04 — was
implicit across other tracks, now an explicit pre-tag concern). The root cause
of restart-storm flapping is pre-Quadlet architecture: restarting
`archipelago.service` SIGKILLs every container in its cgroup, then the
reconciler rebuilds the world over several minutes (the post-OTA health check
deliberately skips per-app container assertions because of exactly this).
Consolidated lever list, in order of impact:
- **Phase-3 Quadlet default-flip** (tracked above) — removes the SIGKILL-the-world
behavior entirely; the single biggest fix.
- **Workstream F lifecycle items** — immich/grafana uninstall hangs + ghost
containers, grafana reinstall stops, fedimint guardian sync
(`docs/PRODUCTION-MASTER-PLAN.md` workstream F).
- **Reconciler churn observability** — no metric/log today distinguishes "settling
after restart" from "flapping"; add a per-app restart counter + log line when an
app restarts >N times in M minutes so thrash is visible instead of anecdotal.
- **Failed-unit self-healing gap (observed live 2026-07-06 on .228)**: fedimint's
quadlet unit exited 255 at 21:21 and sat `failed` for 7+ hours — the reconciler
never revived it (it repairs missing/drifted containers but doesn't
`reset-failed`+start failed .services). Same for the indeedhub trio after the
gate run. The health monitor also can't help (container is gone when the unit
fails). Add a reconcile step: quadlet-backed app whose .service is `failed` and
not user-stopped → reset-failed + start, with backoff.
- Already landed, don't re-do: boot-reconciler circuit breaker (2026-07-01),
indeedhub crashloop fix (2026-07-01), async blocking-Command pass (`4c75bb3d`,
removes executor stalls that made the API janky under reconcile load),
quadlet entrypoint-split false-drift fix (2026-07-08 — `container_command_drifted`
compared entrypoint/cmd halves separately, but quadlet folds `sh -lc` into
`Entrypoint=sh` + `Exec=-lc …`, so every quadlet-created app with a
multi-element entrypoint read as permanently drifted; electrumx on .228
recreated 114×/6h until the comparator was switched to concatenated argv).
- Perf polish riding along: 93 MB frontend dist shrink (hardening plan §D 🟡).
- [ ] **Developer tooling CLI suite** (validate/render/local-install/lifecycle-test) —
APP-PACKAGING-MIGRATION-PLAN.md step 5, needed before external devs can publish.
- [x] ~~**Consolidated deploy 2026-07-01**: merged PR #67 (reticulum daemon
process-group fix, `469b0203`), the UI/UX work (`8256fde1` — mesh/web5/apps
layout, modal, search UX), and `archy-openwrt` (TollGate/OpenWrt gateway
integration — new `core/openwrt` crate, RPC surface, `OpenWrtGateway.vue`)
into `main`, alongside the indeedhub self-heal fix~~ — all merged clean, no
conflicts. **Found + fixed 2 real build-breaking issues during
verification, not caught by whoever authored them**: a vestigial unused
`ref` in `Web5ConnectedNodes.vue` that broke `vue-tsc`, and a stale
`MeshMap.test.ts` mock missing `federatedPositions` (predated this
session's Mesh Map feature) that crashed on mount. Full test suite green
(667 passed) after fixes. **Deployed fleet-wide 2026-07-01, all 5 nodes
sha256-verified**: .116, .198, .228, .5 (recovered cleanly from one
truncated-transfer hiccup, caught via checksum before it hit the live
service), 100.82.34.38 (non-Quadlet node — all containers survived the
restart intact, unlike the worst-case risk flagged beforehand). Also
built an unbundled installer ISO from this same merged source
(`archipelago-installer-1.7.99-alpha-unbundled-x86_64.iso`, 2.4GB) —
the ISO pipeline was archived from the release process at v1.7.43-alpha
(OTA tarballs are now primary) but the wrapper script still works.
- [ ] **⚠️ NOT YET DEPLOYED — start here next session.** After the fleet deploy
above, found that PR #67 ("kill whole daemon process group on drop",
branch `fix/reticulum-daemon-process-group`, head `be50c886`) is a
**different, separate** reticulum-daemon fix from the one already
deployed (`469b0203` on `fix/reticulum-daemon-pdeathsig`) — I'd
conflated the two by topic similarity and only merged/deployed the
Python-level `pdeathsig` fix, missing PR #67's Rust-level
kill-whole-process-group-on-`Drop` fix entirely. Merged PR #67 into
`main` (`7a7fec21`, clean, `cargo check` green, complementary not
conflicting with the already-deployed fix) and separately fixed a real
bug found live: `OpenWrtGateway.vue`'s back button had no `@click`
handler at all (`7d7ba573`, `vue-tsc` clean). **Both committed + pushed
to `main` but genuinely NOT deployed to any node** — user asked to hold
off deploying to restart their computer. Also spot-checked
`openwrt.scan` live on .116: RPC plumbing works, but no physical
OpenWrt router was available to confirm true-positive detection, and
`detect::scan_subnet` does blocking TCP/SSH calls inside an `async fn`
with no `.await` — untested at scale, worth hardening. **Next steps**:
build release binary + frontend from current `main`, deploy to all 5
fleet nodes (.116/.198/.228/.5/100.82.34.38) the same way as the
earlier consolidated deploy, then verify the back button + (if a real
OpenWrt router is available) router detection live.
- [~] **Cross-node federation/mesh/transport suites** — **big find 2026-07-01: these
already exist**, just aren't wired into the gate or documented as existing:
`tests/multinode/smoke.sh` (federation pairing/sync, FIPS anchor, peer content
@ -145,13 +217,12 @@ those are marked ✅ below with the commit that did it, so we stop re-litigating
## Tier 3 — Blocked on a decision or resource only you can supply
- [ ] **Version naming decision (1.7.99-alpha → 1.8.0 vs 1.8.00-alpha)** — code is
otherwise ready to tag; this is a one-line decision, then a mechanical bump +
tag + push. **Needs your call**, not more engineering.
- [ ] **Workstream B signing ceremony**`core/archipelago/src/trust/anchor.rs:21`
still has `RELEASE_ROOT_PUBKEY_HEX = None`. Needs the offline
`RELEASE_MASTER_MNEMONIC` to run `docs/workstream-b-signing-runbook.md`'s
4-step ceremony — can't be automated by me.
- [x] ~~Version naming decision~~**decided 2026-07-08: `1.8.0-alpha`.** Remaining
work is the mechanical bump + tag + push once the pre-tag items above close.
- [x] ~~Workstream B signing ceremony~~**done 2026-07-02.** `anchor.rs` pins
`RELEASE_ROOT_PUBKEY_HEX = 5d15cbee…9951` (signer
`did:key:z6MkkidEnEpo6qHMCNSZoNKWtvQvxq3whnaME9wGgEFhq7ur`); mnemonic held
offline per `docs/workstream-b-signing-runbook.md`.
- [ ] **Bitcoin multi-version fleet-wide OTA**`.228` fully working on branch,
per your prior gating this rollout is explicitly held for your decision on
timing (`docs/bitcoin-version-bulletproof-rollout.md`).
@ -172,6 +243,6 @@ those are marked ✅ below with the commit that did it, so we stop re-litigating
---
*Historical narrative and detailed per-session logs remain in
`docs/SESSION-1.8.0-OTA-PROGRESS.md` and `docs/PRODUCTION-MASTER-PLAN.md` §6/§8b —
`docs/archive/SESSION-1.8.0-OTA-PROGRESS.md` and `docs/PRODUCTION-MASTER-PLAN.md` §6/§8b —
this doc is the live "what's left, in priority order" list. Update it (don't just
append to the old docs) as items close or new ones surface.*

View File

@ -110,8 +110,12 @@ app:
| `app.container.network` | Podman network setting such as `archy-net` or `pasta`; dangerous namespace-sharing modes are rejected |
| `app.container.entrypoint` / `custom_args` | Entrypoint and command override |
| `app.container.derived_env` | Environment values rendered from allowed host facts such as `HOST_IP`, `HOST_MDNS`, and `DISK_GB` |
| `app.container.secret_env` | Environment values read from `/var/lib/archipelago/secrets/<secret_file>` |
| `app.container.secret_env` | Environment values read from `/var/lib/archipelago/secrets/<secret_file>`, injected as podman secrets (never visible in `podman inspect` or unit files) |
| `app.container.generated_secrets` | Secrets the orchestrator creates on first use (`hex16`/`hex32`/`base64`/`bcrypt`) — self-healing, 0600, no host provisioning |
| `app.container.generated_certs` | Self-signed TLS certs materialised before create; CN/SANs rendered from host facts |
| `app.container.network_aliases` | Extra DNS names on the app network so stack members answer to short baked-in hostnames (`api`, `minio`, `relay`) |
| `app.container.data_uid` | UID:GID ownership repair for app data directories |
| `app.hooks` | Allow-listed lifecycle hooks (`post_install`: `exec` inside the app's own container, `copy_from_host` from allow-listed roots) — see `manifest-hooks-design.md` |
| `app.dependencies` | Storage requirements and app dependencies |
| `app.resources` | CPU, memory, and disk limits |
| `app.security` | Capabilities, read-only root, no-new-privileges, network policy, optional AppArmor profile |

Some files were not shown because too many files have changed in this diff Show More