Record candidate deployment and corrected payment evidence

Every paid download logged "filing into filebrowser/Music/... failed
(non-fatal): Permission denied". The purchase played in-app but never
appeared in Files. FileBrowser's folders belong to its rootless container
range (host uid 100000, mode 755). This service is host uid 1000, outside
that range, so it can read them but not create files in them.

New container::filebrowser::save_new_file:
- Writes directly when the folder allows it.
- Otherwise writes through `podman unshare`, where that uid range is
  ours: to a temp file, then chowned to the folder's owner, set to 0644,
  and hard-linked into place. FileBrowser never sees a partial file and an
  existing file is never replaced. A missing folder is created and given
  its parent's owner. No sudo.
- Keeps the "name (2).ext" de-duplication the RPC did inline.

Checked the unshare script on amishparadise in a scratch folder owned
like FileBrowser's: new folder + file OK, owner/mode right, no clobber,
no temp file left, and the service can read the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

AGENTS.md

@@ -29,3 +29,14 @@ unrestricted `cargo test` on a node with installed apps: older mocked-runtime
 tests still reached real service commands. The runner isolates wallet data,
 service buses, container storage, networking, and process IDs. Compilation with
 `cargo test --no-run` is safe. Keep separately authorized live checks explicit.
+
+## Active release regression checklist
+
+Before resuming release work, read
+`docs/post-1.8.22-regressions-20261001.md` and retain its unfinished tasks.
+The operator requested that every reported issue be tracked, fixed and tested
+before another OTA/ISO. Keep source/unit-test results separate from actual-node
+acceptance. In particular, paid-file recovery must not send another payment,
+and app cleanup must preserve wallets, persistent data and uninstall decisions.
+Do not mark the new paid-file incident resolved merely because the earlier
+Framework LND startup incident was closed.

CHANGELOG.md

@@ -1,6 +1,34 @@
 # Changelog
 
-## Unreleased
+## v1.8.22-alpha (2026-09-30)
+
+- Fixed Nginx Proxy Manager launch readiness choosing a proxy listener instead of its admin port after container recreation.
+
+- Network diagnostic failures no longer stop all apps or rebuild shared container networking.
+- Prevented orphaned companion dashboards from repeatedly reinstalling themselves after their backend app was removed.
+- Fixed companion dashboard builds still referencing a retired image registry.
+
+- Fixed Angor Indexer health checks choosing IPv6 localhost for an IPv4 listener and unnecessarily restarting the working service.
+
+- Prevented false app restarts by probing each published port at its actual bind address; Nginx Proxy Manager now checks its internal admin API.
+- Added a backed-up migration for the recognized legacy Nginx Proxy Manager tunnel/LND port conflict in both OTA and ISO startup paths.
+
+- Checked Bitcoin and Electrum companion dashboards instead of backend protocol ports, preserving dashboard access during initial sync.
+- Removed web-interface waiting messages from headless services such as Phoenixd and clarified which interface is unavailable for launchable apps.
+
+- Finished runtime app-file promotion before manifest loading, preventing startup catalog refresh from forgetting disk-only apps.
+
+- Named the app in compact readiness messages and kept app-card actions aligned at the bottom.
+- Removed duplicate Mempool cards caused by frontend container aliases in restored inventory.
+
+- Kept installed apps visible through restarts and hard refreshes, and delayed app launches until their web interface is ready.
+- Made Bitcoin version selection readable and usable in the ThinkPad kiosk, above the pruning settings.
+- Restored GitWorkshop build files in installation/update payloads and made slow image-pull progress clearer.
+- Fixed same-node Gitea access from Portainer, with persistent runtime migration, state backups and recovery after failed restarts.
+- Preserved Gitea configuration and SSH operation during fresh setup and upgrades.
+- Improved paid-file delivery, saved-file permissions and repeat-download compatibility; verified Tor-only payment with change, rejection refunds and free repeat downloads.
+- Added a headless Angor Indexer service using the existing Mempool/ElectrumX stack, and an optional separate Angor relay.
+- Prevented manifest command arguments containing apostrophes from being corrupted in generated services.
 
 ## v1.8.21-alpha (2026-09-30)
 

app-catalog/catalog.json

@@ -644,6 +644,35 @@
           "/var/lib/archipelago/vaultwarden:/data"
         ]
       }
+    },
+    {
+      "id": "angor-indexer",
+      "title": "Angor Indexer",
+      "version": "1.0.1",
+      "description": "Headless Bitcoin indexer endpoint for Angor. Reuses this node’s Mempool and Electrum index; requires a synced, unpruned Bitcoin node. Add this service’s address as the custom indexer in Angor settings. A relay is optional and installed separately.",
+      "dockerImage": "source.archipelago-foundation.org/chaum/angor-indexer:1.0.1",
+      "author": "Angor / Archipelago",
+      "requires": [
+        "Mempool API",
+        "Unpruned Bitcoin"
+      ],
+      "category": "money",
+      "tier": "optional",
+      "icon": "/assets/img/app-icons/angor-green.png",
+      "repoUrl": "https://github.com/block-core/angor"
+    },
+    {
+      "id": "angor-relay",
+      "title": "Angor Relay",
+      "version": "1.1.2",
+      "description": "Optional dedicated Nostr relay for Angor project metadata. Separate storage and access settings keep the node’s internal relay private. Add this service’s address to Angor’s relay settings; use WSS for browser clients.",
+      "dockerImage": "source.archipelago-foundation.org/chaum/angor-relay:1.1.2",
+      "author": "Angor / Archipelago",
+      "requires": [],
+      "category": "nostr",
+      "tier": "optional",
+      "icon": "/assets/img/app-icons/angor-green.png",
+      "repoUrl": "https://github.com/hoytech/strfry"
     }
   ]
 }

apps/angor-indexer/README.md

@@ -0,0 +1,57 @@
+# Angor Indexer
+
+Headless mainnet API endpoint for Angor. The service reuses this node's Mempool
+backend and Electrum index instead of creating a second blockchain database.
+An unpruned, fully synced Bitcoin node is required. Installing against a pruned
+node must show the existing archival-node requirement; it must never silently
+unprune or replace its Bitcoin data.
+
+## Connect Angor
+
+Install **Angor Indexer** in the store. Its API appears under **Services**.
+In Angor settings, use `http://<node-address>:8998/` as the custom indexer origin.
+The `/health` endpoint reports readiness against Mempool's indexed block height;
+it returns 503 while that backend is unavailable. Index building may take time.
+
+Browser clients require a reachable HTTPS origin with a trusted certificate.
+Configure your HTTPS reverse proxy to forward to port 8998, then use that HTTPS
+origin in Angor. Do not disable browser TLS checks. The API supports both
+`/api/v1/` and `/api/` paths, transaction broadcast, and CORS without cookies.
+
+This endpoint intentionally exposes public blockchain queries and transaction
+broadcast through the app gate without dashboard-cookie login. It has no Bitcoin
+RPC password, wallet keys, or persistent wallet data. The backend stays on the
+managed container network; its private port does not become publicly exposed.
+You can change network access using the node's normal access controls.
+
+## Relay
+
+A relay is optional. Angor can continue using its configured external relays.
+Install **Angor Relay** separately to host project metadata locally, then add
+`ws://<node-address>:8091/` in Angor, or a trusted `wss://` proxy origin for browser
+clients. Its storage and configuration are separate from the node's internal
+relay; installing or uninstalling it does not change the internal relay.
+
+## Packaging
+
+Build the pinned image with:
+
+```
+podman build -t source.archipelago-foundation.org/chaum/angor-indexer:1.0.1 apps/angor-indexer/container
+```
+
+The image runs as UID 101 with a read-only root filesystem and no capabilities.
+Only temporary nginx state is writable. Runtime DNS is read from resolv.conf so
+Mempool recreation does not require editing IP addresses or restarting this app.
+No app-specific Rust installer is required.
+
+Source documentation: [Angor's official deployment guide](https://github.com/block-core/angor/blob/869dd43cf38332dd7128a284a6bf4c1cac44c1a7/docker/DEPLOY-INDEXER-AND-RELAY.md).
+The app icon is based on [Angor’s dark-mode app icon](https://angor.io/images/app-icon-dark-mode.png), retrieved 2026-09-30. At the operator’s request, the outer corners use the same green as the background. The built-in imagegen edit preserved the black mark and filled the square green; the project asset is `neode-ui/public/assets/img/app-icons/angor-green.png`.
+
+Tests and release acceptance are recorded in the next-release checklist. The
+health probe establishes backend availability, not a guarantee that every
+address query is indexed at the latest Bitcoin tip.
+
+Install Mempool Explorer first. The declarative `install_prerequisites` check
+refuses a new adapter installation if its Mempool API component is absent, before
+creating an installed-app record. It does not install or resync Bitcoin for you.

apps/angor-indexer/container/Dockerfile

@@ -0,0 +1,6 @@
+FROM docker.io/library/nginx:1.31.3-alpine@sha256:1d40e3eb3bf4f138de1d67193f2aa5309fcaf343eb5ffadbf5e9439de1eb1ebb
+COPY nginx.conf /etc/angor-nginx.conf.template
+COPY entrypoint.sh /usr/local/bin/angor-indexer
+USER 101:101
+EXPOSE 8080
+ENTRYPOINT ["/usr/local/bin/angor-indexer"]

apps/angor-indexer/container/entrypoint.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+set -eu
+# Resolve through the container runtime's DNS, including after dependency
+# recreation. Never bake a container IP into the indexer endpoint.
+DNS_RESOLVER=$(awk '/^nameserver[[:space:]]/ {print $2; exit}' /etc/resolv.conf)
+case "$DNS_RESOLVER" in
+  ''|*[!0-9a-fA-F.:]*) echo 'Container DNS resolver is unavailable' >&2; exit 1 ;;
+esac
+case "$DNS_RESOLVER" in *:*) DNS_RESOLVER="[$DNS_RESOLVER]" ;; esac
+export DNS_RESOLVER
+envsubst '${DNS_RESOLVER}' < /etc/angor-nginx.conf.template > /tmp/nginx.conf
+exec nginx -c /tmp/nginx.conf -g 'daemon off;'

apps/angor-indexer/container/nginx.conf

@@ -0,0 +1,63 @@
+worker_processes 1;
+pid /tmp/nginx.pid;
+error_log /dev/stderr warn;
+events { worker_connections 512; }
+http {
+    access_log off;
+    server_tokens off;
+    client_body_temp_path /tmp/client_temp;
+    proxy_temp_path /tmp/proxy_temp;
+    fastcgi_temp_path /tmp/fastcgi_temp;
+    uwsgi_temp_path /tmp/uwsgi_temp;
+    scgi_temp_path /tmp/scgi_temp;
+    resolver ${DNS_RESOLVER} valid=10s ipv6=off;
+    upstream mempool_backend {
+        zone mempool_backend 64k;
+        server mempool-api:8999 resolve;
+    }
+    server {
+        listen 8080;
+        client_max_body_size 4m;
+        proxy_connect_timeout 5s;
+        proxy_read_timeout 60s;
+        proxy_send_timeout 30s;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header Connection "";
+        proxy_set_header Authorization "";
+        proxy_set_header Cookie "";
+        proxy_hide_header Access-Control-Allow-Origin;
+        add_header Access-Control-Allow-Origin '*' always;
+        add_header Access-Control-Allow-Methods 'GET, HEAD, POST, OPTIONS' always;
+        add_header Access-Control-Allow-Headers 'Content-Type' always;
+        add_header Cache-Control 'no-store' always;
+        if ($request_method = OPTIONS) { return 204; }
+        # Mempool's backend uses /api/v1. Match its frontend's shorter /api
+        # surface too, without doubling already-versioned Angor URLs.
+        rewrite ^/api/(?!v1/)(.*)$ /api/v1/$1 last;
+        location = / {
+            default_type application/json;
+            return 200 '{"service":"Angor Indexer","network":"mainnet","api":"/api/v1","health":"/health"}\n';
+        }
+        # Readiness checks the indexing backend, not this gateway's process.
+        location = /health {
+            limit_except GET { deny all; }
+            proxy_pass http://mempool_backend/api/v1/blocks/tip/height;
+            proxy_intercept_errors on;
+            error_page 500 502 503 504 =503 @waiting;
+        }
+        location @waiting {
+            default_type application/json;
+            return 503 '{"status":"waiting","message":"Waiting for Bitcoin and Mempool indexing"}\n';
+        }
+        location ~ ^/api/(v1/)?tx$ {
+            limit_except GET POST { deny all; }
+            proxy_pass http://mempool_backend;
+        }
+        location /api/ {
+            limit_except GET { deny all; }
+            proxy_pass http://mempool_backend;
+        }
+        location / { return 404; }
+    }
+}

apps/angor-indexer/manifest.yml

@@ -0,0 +1,68 @@
+app:
+  id: angor-indexer
+  name: Angor Indexer
+  version: 1.0.1
+  description: Headless Bitcoin indexer endpoint for Angor. Reuses this node’s Mempool
+    and Electrum index; requires a synced, unpruned Bitcoin node. Add this service’s
+    address as the custom indexer in Angor settings. A relay is optional and installed
+    separately.
+  category: money
+  install_prerequisites:
+  - mempool-api
+  upstream:
+    kind: github
+    repo: block-core/angor
+  container:
+    image: source.archipelago-foundation.org/chaum/angor-indexer:1.0.1
+    pull_policy: if-not-present
+    network: archy-net
+  dependencies:
+  - app_id: mempool-api
+    version: '>=3.0.0'
+  - bitcoin:archival
+  resources:
+    cpu_limit: 1
+    memory_limit: 128Mi
+    disk_limit: 128Mi
+  security:
+    capabilities: []
+    readonly_root: true
+    no_new_privileges: true
+    user: 101
+    network_policy: isolated
+  ports:
+  - host: 8998
+    container: 8080
+    protocol: tcp
+    bind: 127.0.0.1
+    auth: open
+    auth_rationale: Public Bitcoin chain-data API and validated transaction broadcast for Angor clients; no wallet keys or node RPC credentials are exposed. Browser cookie login would break machine clients.
+  interfaces:
+    main:
+      name: Angor Indexer API
+      description: Use this origin as Angor’s custom mainnet indexer URL. HTTPS is
+        required for browser clients.
+      type: api
+      port: 8998
+      protocol: http
+      path: /
+  health_check:
+    type: http
+    endpoint: http://127.0.0.1:8080
+    path: /health
+    interval: 30s
+    timeout: 8s
+    retries: 3
+  bitcoin_integration:
+    rpc_access: none
+    sync_required: true
+    pruning_support: false
+  metadata:
+    icon: /assets/img/app-icons/angor-green.png
+    tier: optional
+    repo: https://github.com/block-core/angor
+    features:
+    - Angor mainnet API
+    - Reuses existing Mempool indexing
+    - No separate blockchain database
+    - Optional independent relay

apps/angor-relay/README.md

@@ -0,0 +1,21 @@
+# Angor Relay
+
+Optional standalone strfry relay for Angor's public project metadata. See
+[Angor Indexer setup](../angor-indexer/README.md) for client URLs and HTTPS/WSS.
+
+The gate exposes port 8091 for Nostr clients. strfry validates event signatures;
+this is a public relay, not a private messaging archive. It mounts only
+`/var/lib/archipelago/angor-relay` and its separate configuration directory.
+It never opens, reconfigures or shares the node's internal strfry database.
+
+The configuration is seeded only when absent, preserving operator changes.
+Stop the service before making a consistent backup of its event database.
+Ordinary start/restart/recreation preserves both mounts. Use the standard app
+lifecycle; do not manually recreate a systemd-managed container.
+
+## Image provenance
+
+Mirrored from `docker.io/dockurr/strfry:1.1.2`, upstream manifest digest
+`sha256:e81d238db13507f6ef24c49d47cd0b0ea58ff207961f10581fa2a7c901054df4`.
+The public Angor policy is supplied by this app's own configuration; it does not
+reuse the internal relay's event whitelist.

apps/angor-relay/manifest.yml

@@ -0,0 +1,223 @@
+app:
+  id: angor-relay
+  name: Angor Relay
+  version: 1.1.2
+  upstream:
+    kind: github
+    repo: hoytech/strfry
+  description: Optional dedicated Nostr relay for Angor project metadata. Separate
+    storage and access settings keep the node’s internal relay private. Add this service’s
+    address to Angor’s relay settings; use WSS for browser clients.
+  container:
+    image: source.archipelago-foundation.org/chaum/angor-relay:1.1.2
+    pull_policy: if-not-present
+  dependencies:
+  - storage: 5Gi
+  resources:
+    cpu_limit: 1
+    memory_limit: 512Mi
+    disk_limit: 5Gi
+  security:
+    capabilities: []
+    readonly_root: true
+    no_new_privileges: true
+    seccomp_profile: default
+    network_policy: isolated
+    apparmor_profile: nostr-relay
+  ports:
+  - host: 8091
+    container: 7777
+    protocol: tcp
+    bind: 127.0.0.1
+    auth: open
+    auth_rationale: Dedicated public Nostr relay for Angor project metadata; strfry verifies event signatures. It has separate storage from the private node relay and no wallet or node credentials.
+  volumes:
+  - type: bind
+    source: /var/lib/archipelago/angor-relay
+    target: /app/strfry-db
+    options:
+    - rw
+  - type: bind
+    source: /var/lib/archipelago/angor-relay-config/angor-relay.conf
+    target: /etc/strfry.conf
+    options:
+    - ro
+  files:
+  - path: /var/lib/archipelago/angor-relay-config/angor-relay.conf
+    overwrite: false
+    content: |
+      ##
+      ## Default strfry config
+      ##
+
+      # Directory that contains the strfry LMDB database (restart required)
+      db = "./strfry-db/"
+
+      dbParams {
+          # Maximum number of threads/processes that can simultaneously have LMDB transactions open (restart required)
+          maxreaders = 256
+
+          # Size of mmap() to use when loading LMDB (default is 10TB, does *not* correspond to disk-space used) (restart required)
+          mapsize = 10995116277760
+
+          # Disables read-ahead when accessing the LMDB mapping. Reduces IO activity when DB size is larger than RAM. (restart required)
+          noReadAhead = false
+      }
+
+      events {
+          # Maximum size of normalised JSON, in bytes
+          maxEventSize = 65536
+
+          # Events newer than this will be rejected
+          rejectEventsNewerThanSeconds = 900
+
+          # Events older than this will be rejected
+          rejectEventsOlderThanSeconds = 94608000
+
+          # Ephemeral events older than this will be rejected
+          rejectEphemeralEventsOlderThanSeconds = 60
+
+          # Ephemeral events will be deleted from the DB when older than this
+          ephemeralEventsLifetimeSeconds = 300
+
+          # Maximum number of tags allowed
+          maxNumTags = 2000
+
+          # Maximum size for tag values, in bytes
+          maxTagValSize = 1024
+      }
+
+      relay {
+          # Interface to listen on. Use 0.0.0.0 to listen on all interfaces (restart required)
+          bind = "0.0.0.0"
+
+          # Port to open for the nostr websocket protocol (restart required)
+          port = 7777
+
+          # Set OS-limit on maximum number of open files/sockets (if 0, don't attempt to set) (restart required)
+          nofiles = 0
+
+          # HTTP header that contains the client's real IP, before reverse proxying (ie x-real-ip) (MUST be all lower-case)
+          realIpHeader = ""
+
+          info {
+              # NIP-11: Name of this server. Short/descriptive (< 30 characters)
+              name = "Angor Relay"
+
+              # NIP-11: Detailed information about relay, free-form
+              description = "Dedicated public relay for Angor project metadata."
+
+              # NIP-11: Administrative nostr pubkey, for contact purposes
+              pubkey = ""
+
+              # NIP-11: Alternative administrative contact (email, website, etc)
+              contact = ""
+
+              # NIP-11: URL pointing to an image to be used as an icon for the relay
+              icon = ""
+
+              # List of supported lists as JSON array, or empty string to use default. Example: "[1,2]"
+              nips = ""
+          }
+
+          # Maximum accepted incoming websocket frame size (should be larger than max event) (restart required)
+          maxWebsocketPayloadSize = 131072
+
+          # Maximum number of filters allowed in a REQ
+          maxReqFilterSize = 200
+
+          # Websocket-level PING message frequency (should be less than any reverse proxy idle timeouts) (restart required)
+          autoPingSeconds = 55
+
+          # If TCP keep-alive should be enabled (detect dropped connections to upstream reverse proxy)
+          enableTcpKeepalive = false
+
+          # How much uninterrupted CPU time a REQ query should get during its DB scan
+          queryTimesliceBudgetMicroseconds = 10000
+
+          # Maximum records that can be returned per filter
+          maxFilterLimit = 500
+
+          # Maximum number of subscriptions (concurrent REQs) a connection can have open at any time
+          maxSubsPerConnection = 20
+
+          writePolicy {
+              # If non-empty, path to an executable script that implements the writePolicy plugin logic
+              plugin = ""
+          }
+
+          compression {
+              # Use permessage-deflate compression if supported by client. Reduces bandwidth, but slight increase in CPU (restart required)
+              enabled = true
+
+              # Maintain a sliding window buffer for each connection. Improves compression, but uses more memory (restart required)
+              slidingWindow = true
+          }
+
+          logging {
+              # Dump all incoming messages
+              dumpInAll = false
+
+              # Dump all incoming EVENT messages
+              dumpInEvents = false
+
+              # Dump all incoming REQ/CLOSE messages
+              dumpInReqs = false
+
+              # Log performance metrics for initial REQ database scans
+              dbScanPerf = false
+
+              # Log reason for invalid event rejection? Can be disabled to silence excessive logging
+              invalidEvents = true
+          }
+
+          numThreads {
+              # Ingester threads: route incoming requests, validate events/sigs (restart required)
+              ingester = 3
+
+              # reqWorker threads: Handle initial DB scan for events (restart required)
+              reqWorker = 3
+
+              # reqMonitor threads: Handle filtering of new events (restart required)
+              reqMonitor = 3
+
+              # negentropy threads: Handle negentropy protocol messages (restart required)
+              negentropy = 2
+          }
+
+          negentropy {
+              # Support negentropy protocol messages
+              enabled = true
+
+              # Maximum records that sync will process before returning an error
+              maxSyncEvents = 1000000
+          }
+      }
+  health_check:
+    type: http
+    endpoint: http://127.0.0.1:7777
+    path: /health
+    interval: 30s
+    timeout: 5s
+    retries: 3
+  nostr_integration:
+    relay_type: public
+    monetization_enabled: false
+  category: nostr
+  interfaces:
+    main:
+      name: Angor Relay
+      description: Nostr WebSocket endpoint; use ws:// for LAN or wss:// through your
+        HTTPS domain.
+      type: api
+      port: 8091
+      protocol: http
+      path: /
+  metadata:
+    icon: /assets/img/app-icons/angor-green.png
+    tier: optional
+    repo: https://github.com/hoytech/strfry
+    features:
+    - Angor project metadata
+    - Separate from the node relay
+    - Persistent Nostr event storage

apps/gitea/manifest.yml

@@ -15,6 +15,9 @@ app:
     image: source.archipelago-foundation.org/lfg2025/gitea:1.27.3
     pull_policy: if-not-present
 
+  # Preserve repositories, database, keys and configuration during runtime repairs.
+  backup_before_runtime_change: true
+
   dependencies:
     # Source history, LFS objects, release artifacts and OCI layers all share
     # this persistent store. 500Mi was only suitable for an empty demo node.
@@ -25,7 +28,7 @@ app:
     disk_limit: 50Gi
 
   security:
-    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE, NET_BIND_SERVICE]
+    capabilities: [CHOWN, FOWNER, SETUID, SETGID, DAC_OVERRIDE, NET_BIND_SERVICE, SYS_CHROOT]
     readonly_root: false
     no_new_privileges: false
     network_policy: bridge
@@ -62,6 +65,17 @@ app:
       target: /etc/gitea
       options: [rw]
 
+  # Seed a fresh installation with the same origin advertised by the app gate.
+  # Existing app.ini (including custom HTTPS/domain settings) is never replaced.
+  files:
+    - path: /var/lib/archipelago/gitea/data/gitea/conf/app.ini
+      overwrite: false
+      content: |
+        [server]
+        DOMAIN = {{HOST_IP}}
+        SSH_DOMAIN = {{HOST_IP}}
+        ROOT_URL = http://{{HOST_IP}}:3001/
+
   environment:
     - GITEA__database__DB_TYPE=sqlite3
     - GITEA__server__SSH_PORT=2222
@@ -106,10 +120,3 @@ app:
       - Issue tracking and pull requests
       - CI/CD via Gitea Actions
       - Lightweight SQLite deployment
-
-  nginx_proxy:
-    listen: 3000
-    proxy_pass: http://127.0.0.1:3001
-    extra_headers:
-      - proxy_hide_header X-Frame-Options
-      - proxy_hide_header Content-Security-Policy

apps/nginx-proxy-manager/manifest.yml

@@ -64,9 +64,11 @@ app:
 
   environment: []
 
+  # Probe the admin API inside the container, independent of optional
+  # tunnel listeners. This also verifies the Node backend is ready.
   health_check:
-    type: tcp
-    endpoint: localhost:81
+    type: http
+    endpoint: http://127.0.0.1:81/api/
     interval: 30s
     timeout: 5s
     retries: 3

apps/portainer/manifest.yml

@@ -14,8 +14,16 @@ app:
   container:
     image: source.archipelago-foundation.org/lfg2025/portainer:2.45.0
     pull_policy: if-not-present
+    # Portainer fetches Git sources and images from services on this same node.
+    # Rootless pasta copies the host LAN address into its namespace, so a LAN
+    # URL points back at Portainer itself. Give it a private address with the
+    # supported rootless slirp backend; public app URLs still traverse the gate.
+    network: slirp4netns
     data_uid: "1000:1000"
 
+  # Snapshot state before an upgrade recreates this app with new networking.
+  backup_before_runtime_change: true
+
   dependencies:
     - storage: 1Gi
 

core/Cargo.lock

@@ -104,7 +104,7 @@ dependencies = [
 
 [[package]]
 name = "archipelago"
-version = "1.8.21-alpha"
+version = "1.8.22-alpha"
 dependencies = [
  "anyhow",
  "archipelago-container",

core/archipelago/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "archipelago"
-version = "1.8.21-alpha"
+version = "1.8.22-alpha"
 edition = "2021"
 license.workspace = true
 description = "Archipelago Bitcoin Node OS - Native backend"

core/archipelago/src/api/handler/content.rs

@@ -74,7 +74,7 @@ impl ApiHandler {
         let invoice_hash = headers
             .get("x-invoice-hash")
             .and_then(|v| v.to_str().ok())
-            .map(|s| s.to_string())
+            .map(|s| s.to_ascii_lowercase())
             .or_else(|| {
                 headers
                     .get("x-onchain-address")
@@ -98,6 +98,46 @@ impl ApiHandler {
             None => false,
         };
 
+        // Payment settlement is verified on the seller even when no status
+        // poll preceded this download (e.g. direct payment from another node).
+        let requires_payment = if !owner_session && headers.contains_key("x-invoice-hash") {
+            content_server::load_catalog(&config.data_dir)
+                .await?
+                .items
+                .iter()
+                .any(|item| {
+                    item.id == content_id
+                        && matches!(item.access, content_server::AccessControl::Paid { .. })
+                })
+        } else {
+            false
+        };
+        if requires_payment {
+            if let Some(hash) = headers.get("x-invoice-hash").and_then(|v| v.to_str().ok()) {
+                if hash.len() != 64 || !hash.bytes().all(|c| c.is_ascii_hexdigit()) {
+                    return Ok(build_response(
+                        StatusCode::BAD_REQUEST,
+                        "text/plain",
+                        hyper::Body::from("Invalid payment hash"),
+                    ));
+                }
+                if let Err(error) = self
+                    .rpc_handler
+                    .settle_content_invoice(hash, content_id)
+                    .await
+                {
+                    tracing::warn!("Cannot verify peer-file invoice settlement: {error:#}");
+                    return Ok(build_response(
+                        StatusCode::SERVICE_UNAVAILABLE,
+                        "application/json",
+                        hyper::Body::from(
+                            r#"{"error":"Payment verification is temporarily unavailable. Retry the download without paying again."}"#,
+                        ),
+                    ));
+                }
+            }
+        }
+
         // Parse Range header for streaming support
         let range = headers
             .get("range")
@@ -249,7 +289,13 @@ impl ApiHandler {
             .await
         {
             Ok((bolt11, payment_hash)) if !payment_hash.is_empty() => {
-                crate::content_invoice::record_pending(&payment_hash, content_id, price_sats).await;
+                crate::content_invoice::record_pending(
+                    &self.config.data_dir,
+                    &payment_hash,
+                    content_id,
+                    price_sats,
+                )
+                .await?;
                 let body = serde_json::json!({
                     "bolt11": bolt11,
                     "payment_hash": payment_hash,
@@ -309,26 +355,10 @@ impl ApiHandler {
             ));
         }
 
-        // The hash must be one we issued for exactly this content item.
-        match crate::content_invoice::lookup(payment_hash).await {
-            Some((cid, _)) if cid == content_id => {}
-            _ => {
-                return Ok(build_response(
-                    StatusCode::NOT_FOUND,
-                    "application/json",
-                    hyper::Body::from(r#"{"error":"Unknown invoice"}"#),
-                ))
-            }
-        }
-
-        // Already paid? Otherwise ask our LND and persist the result.
-        let mut paid = crate::content_invoice::is_paid_for(payment_hash, content_id).await;
-        if !paid {
-            if let Ok(true) = self.rpc_handler.invoice_is_settled(payment_hash).await {
-                crate::content_invoice::mark_paid(payment_hash).await;
-                paid = true;
-            }
-        }
+        let paid = self
+            .rpc_handler
+            .settle_content_invoice(payment_hash, content_id)
+            .await?;
 
         let body = serde_json::json!({ "paid": paid });
         Ok(build_response(
@@ -389,7 +419,13 @@ impl ApiHandler {
 
         match self.rpc_handler.new_onchain_address().await {
             Ok(address) if !address.is_empty() => {
-                crate::content_invoice::record_pending(&address, content_id, price_sats).await;
+                crate::content_invoice::record_pending(
+                    &self.config.data_dir,
+                    &address,
+                    content_id,
+                    price_sats,
+                )
+                .await?;
                 let body = serde_json::json!({
                     "address": address,
                     "amount_sats": price_sats,
@@ -439,7 +475,7 @@ impl ApiHandler {
             ));
         }
         // The address must be one we issued for exactly this content item.
-        let price = match crate::content_invoice::lookup(address).await {
+        let price = match crate::content_invoice::lookup(&self.config.data_dir, address).await? {
             Some((cid, price)) if cid == content_id => price,
             _ => {
                 return Ok(build_response(
@@ -450,10 +486,11 @@ impl ApiHandler {
             }
         };
 
-        let mut paid = crate::content_invoice::is_paid_for(address, content_id).await;
+        let mut paid =
+            crate::content_invoice::is_paid_for(&self.config.data_dir, address, content_id).await;
         if !paid {
             if let Ok(true) = self.rpc_handler.onchain_received(address, price).await {
-                crate::content_invoice::mark_paid(address).await;
+                crate::content_invoice::mark_paid(&self.config.data_dir, address).await?;
                 paid = true;
             }
         }

core/archipelago/src/api/rpc/content.rs

@@ -73,13 +73,20 @@ fn paid_content_response(bytes: &[u8], mime: &str, paid_sats: u64) -> serde_json
     })
 }
 
-/// FileBrowser owns its files through a rootless UID mapping. Use its authenticated
-/// API rather than writing host paths with the backend's unrelated UID. Its
-/// override=false upload atomically refuses existing names, including races.
+// Updated clients open the persisted file through the Range-capable HTTP
+// endpoint. Avoid putting two base64 copies of a large video in a JSON reply.
+// Keep older clients compatible until both sides have upgraded.
+fn invoice_download_response(bytes: &[u8], mime: &str, cache_only: bool) -> serde_json::Value {
+    if cache_only {
+        serde_json::json!({ "owned": true, "mime_type": mime, "size_bytes": bytes.len() })
+    } else {
+        paid_content_response(bytes, mime, 0)
+    }
+}
+
+/// File purchases through an atomic no-clobber write in Files' own namespace.
 async fn file_purchase_in_files(
-    client: &reqwest::Client,
-    base_url: &str,
-    token: &str,
+    data_dir: &std::path::Path,
     filename: &str,
     mime: &str,
     bytes: &[u8],
@@ -91,59 +98,24 @@ async fn file_purchase_in_files(
     } else {
         "Documents"
     };
-    let mut folder_url = reqwest::Url::parse(base_url)?;
-    folder_url
-        .path_segments_mut()
-        .map_err(|_| anyhow::anyhow!("Invalid Files URL"))?
-        .extend(["api", "resources", folder, ""]);
-    let response = client
-        .get(folder_url.clone())
-        .header("X-Auth", token)
-        .send()
-        .await?;
-    if response.status() == reqwest::StatusCode::NOT_FOUND {
-        let response = client
-            .post(folder_url.clone())
-            .header("X-Auth", token)
-            .send()
-            .await?;
-        if response.status() != reqwest::StatusCode::CONFLICT {
-            response.error_for_status()?;
-        }
-    } else {
-        response.error_for_status()?;
-    }
-    let base = std::path::Path::new(filename)
+    let root = data_dir.join("filebrowser");
+    anyhow::ensure!(
+        tokio::fs::metadata(&root).await?.is_dir(),
+        "Files storage is unavailable"
+    );
+    let name = std::path::Path::new(filename)
         .file_name()
         .and_then(|n| n.to_str())
         .filter(|n| !n.is_empty())
         .unwrap_or("download");
-    let (stem, extension) = match base.rsplit_once('.') {
-        Some((stem, ext)) if !stem.is_empty() => (stem, format!(".{ext}")),
-        _ => (base, String::new()),
-    };
-    for attempt in 1..=100 {
-        let name = if attempt == 1 {
-            base.to_string()
-        } else {
-            format!("{stem} ({attempt}){extension}")
-        };
-        let mut url = folder_url.clone();
-        url.path_segments_mut().unwrap().pop_if_empty().push(&name);
-        url.query_pairs_mut().append_pair("override", "false");
-        let response = client
-            .post(url)
-            .header("X-Auth", token)
-            .body(bytes.to_vec())
-            .send()
-            .await?;
-        if response.status() == reqwest::StatusCode::CONFLICT {
-            continue;
-        }
-        response.error_for_status()?;
-        return Ok(format!("{folder}/{name}"));
-    }
-    anyhow::bail!("Too many existing copies; purchased file remains in the purchase cache")
+    let path =
+        crate::container::filebrowser::save_new_file(&root.join(folder), name, bytes).await?;
+    Ok(format!(
+        "{folder}/{}",
+        path.file_name()
+            .and_then(|n| n.to_str())
+            .context("Invalid Files name")?
+    ))
 }
 
 impl RpcHandler {
@@ -755,28 +727,8 @@ impl RpcHandler {
 
         // The durable purchased-content cache above is primary. A Files copy
         // remains optional: a stopped FileBrowser must not undo a paid download.
-        let filed = async {
-            let auth = self.handle_filebrowser_token().await?;
-            let token = auth
-                .get("token")
-                .and_then(|v| v.as_str())
-                .context("FileBrowser omitted its authentication token")?;
-            let client = reqwest::Client::builder()
-                .no_proxy()
-                .redirect(reqwest::redirect::Policy::none())
-                .timeout(std::time::Duration::from_secs(30))
-                .build()?;
-            file_purchase_in_files(
-                &client,
-                "http://127.0.0.1:8083",
-                token,
-                &filename,
-                &mime_type,
-                &bytes,
-            )
-            .await
-        }
-        .await;
+        let filed =
+            file_purchase_in_files(&self.config.data_dir, &filename, &mime_type, &bytes).await;
         match filed {
             Ok(path) => tracing::info!("paid download: filed into Files/{path}"),
             Err(error) => tracing::warn!(
@@ -929,10 +881,29 @@ impl RpcHandler {
         if !is_valid_v3_onion(onion) {
             return Err(anyhow::anyhow!("Invalid v3 onion address"));
         }
-        if payment_hash.is_empty() || !payment_hash.chars().all(|c| c.is_ascii_hexdigit()) {
+        if payment_hash.len() != 64 || !payment_hash.chars().all(|c| c.is_ascii_hexdigit()) {
             return Err(anyhow::anyhow!("Invalid payment_hash"));
         }
 
+        let cache_only = params
+            .get("cache_only")
+            .and_then(|v| v.as_bool())
+            .unwrap_or(false);
+        if let Some((mime, bytes)) =
+            crate::content_owned::read_owned(&self.config.data_dir, onion, content_id).await
+        {
+            return Ok(invoice_download_response(&bytes, &mime, cache_only));
+        }
+        // Older sellers only mark settlement during status polling. Always
+        // perform that handshake before requesting bytes; retries never pay.
+        // The download gate remains authoritative: a file may have become
+        // free, and newer sellers verify directly if status polling fails.
+        let _ = self
+            .handle_content_invoice_status(Some(serde_json::json!({
+                "onion": onion, "content_id": content_id, "payment_hash": payment_hash,
+            })))
+            .await;
+
         let (data, _) = self.state_manager.get_snapshot().await;
         let local_did = crate::identity::did_key_from_pubkey_hex(&data.server_info.pubkey)?;
         let fips_npub = crate::federation::fips_npub_for_onion(&self.config.data_dir, onion).await;
@@ -971,7 +942,7 @@ impl RpcHandler {
 
         if response.status() == reqwest::StatusCode::PAYMENT_REQUIRED {
             return Ok(serde_json::json!({
-                "error": "Seller has not registered this payment yet — wait for settlement and retry."
+                "error": "The seller has not confirmed access yet. Retry the download without paying again."
             }));
         }
         if !response.status().is_success() {
@@ -980,16 +951,45 @@ impl RpcHandler {
             }));
         }
 
+        let mime = response
+            .headers()
+            .get(reqwest::header::CONTENT_TYPE)
+            .and_then(|v| v.to_str().ok())
+            .unwrap_or("application/octet-stream")
+            .split(';')
+            .next()
+            .unwrap_or("application/octet-stream")
+            .to_string();
         let bytes = response
             .bytes()
             .await
-            .context("Failed to read response body")?;
-        use base64::Engine;
-        let encoded = base64::engine::general_purpose::STANDARD.encode(&bytes);
-        Ok(serde_json::json!({
-            "data": encoded,
-            "size": bytes.len(),
-        }))
+            .context("Paid file transfer interrupted; retry the download without paying again")?;
+        let filename = params
+            .get("filename")
+            .and_then(|v| v.as_str())
+            .unwrap_or(content_id);
+        crate::content_owned::record_purchase(
+            &self.config.data_dir,
+            onion,
+            content_id,
+            filename,
+            &mime,
+            &bytes,
+            params
+                .get("price_sats")
+                .and_then(|v| v.as_u64())
+                .unwrap_or(0),
+            "lightning",
+            &chrono::Utc::now().to_rfc3339(),
+        )
+        .await
+        .context("Paid file could not be saved; retry the download without paying again")?;
+        if let Err(error) =
+            file_purchase_in_files(&self.config.data_dir, filename, &mime, &bytes).await
+        {
+            tracing::warn!("Lightning purchase cached; optional Files copy failed: {error:#}");
+        }
+        Ok(invoice_download_response(&bytes, &mime, cache_only))
     }
 
     /// Buyer side (#46): ask the seller for a fresh on-chain address to pay.
@@ -1464,3 +1464,19 @@ impl RpcHandler {
 #[cfg(test)]
 #[path = "content_tests.rs"]
 mod tests;
+
+#[cfg(test)]
+mod invoice_delivery_response_tests {
+    use super::*;
+    #[test]
+    fn cached_delivery_avoids_base64_but_keeps_old_clients_compatible() {
+        let cached = invoice_download_response(b"paid bytes", "video/mp4", true);
+        assert_eq!(cached["owned"], true);
+        assert_eq!(cached["size_bytes"], 10);
+        assert!(cached.get("data").is_none());
+        assert!(cached.get("data_base64").is_none());
+        let legacy = invoice_download_response(b"paid bytes", "video/mp4", false);
+        assert_eq!(legacy["data"], "cGFpZCBieXRlcw==");
+        assert_eq!(legacy["data"], legacy["data_base64"]);
+    }
+}

core/archipelago/src/api/rpc/content_tests.rs

@@ -1,69 +1,4 @@
 use super::*;
-use hyper::{
-    service::{make_service_fn, service_fn},
-    Body, Response, Server,
-};
-use std::{
-    collections::VecDeque,
-    convert::Infallible,
-    sync::{Arc, Mutex},
-};
-
-struct FilesApi {
-    url: String,
-    seen: Arc<Mutex<Vec<(String, String, Vec<u8>)>>>,
-    task: tokio::task::JoinHandle<()>,
-}
-impl Drop for FilesApi {
-    fn drop(&mut self) {
-        self.task.abort();
-    }
-}
-fn files_api(statuses: Vec<u16>) -> FilesApi {
-    let statuses = Arc::new(Mutex::new(VecDeque::from(statuses)));
-    let seen = Arc::new(Mutex::new(Vec::new()));
-    let history = seen.clone();
-    let server = Server::bind(&([127, 0, 0, 1], 0).into());
-    let address = server.local_addr();
-    let service = make_service_fn(move |_| {
-        let statuses = statuses.clone();
-        let seen = history.clone();
-        async move {
-            Ok::<_, Infallible>(service_fn(move |request: hyper::Request<Body>| {
-                let statuses = statuses.clone();
-                let seen = seen.clone();
-                async move {
-                    assert_eq!(request.headers().get("X-Auth").unwrap(), "test-session");
-                    let method = request.method().to_string();
-                    let uri = request.uri().to_string();
-                    let body = hyper::body::to_bytes(request.into_body())
-                        .await
-                        .unwrap()
-                        .to_vec();
-                    seen.lock().unwrap().push((method, uri, body));
-                    let status = statuses
-                        .lock()
-                        .unwrap()
-                        .pop_front()
-                        .expect("unexpected extra Files request");
-                    Ok::<_, Infallible>(
-                        Response::builder()
-                            .status(status)
-                            .body(Body::empty())
-                            .unwrap(),
-                    )
-                }
-            }))
-        }
-    });
-    FilesApi {
-        url: format!("http://{address}"),
-        seen,
-        task: tokio::spawn(async move {
-            server.serve(service).await.unwrap();
-        }),
-    }
-}
 
 #[test]
 fn first_and_cached_paid_downloads_have_the_same_client_payload_contract() {
@@ -85,82 +20,39 @@ fn first_and_cached_paid_downloads_have_the_same_client_payload_contract() {
 }
 
 #[tokio::test]
-async fn files_copy_uses_authenticated_api_and_preserves_existing_names() {
-    let api = files_api(vec![200, 409, 200]);
-    let client = reqwest::Client::new();
-    let path = file_purchase_in_files(
-        &client,
-        &api.url,
-        "test-session",
-        "../my #file?.txt",
-        "text/plain",
-        b"paid bytes",
-    )
-    .await
-    .unwrap();
-    assert_eq!(path, "Documents/my #file? (2).txt");
-    let seen = api.seen.lock().unwrap();
-    assert_eq!(seen[0].0, "GET");
-    assert_eq!(seen[0].1, "/api/resources/Documents/");
-    assert_eq!(seen.len(), 3);
-    for (_, uri, body) in &seen[1..] {
-        assert!(uri.contains("override=false"));
-        assert!(uri.contains("%23file%3F"));
-        assert!(!uri.contains("../"));
-        assert_eq!(body, b"paid bytes");
-    }
-}
-
-#[tokio::test]
-async fn files_copy_creates_missing_media_folder() {
+async fn files_copy_routes_media_and_sanitizes_the_filename() {
+    let dir = tempfile::tempdir().unwrap();
+    tokio::fs::create_dir(dir.path().join("filebrowser"))
+        .await
+        .unwrap();
     for (mime, folder) in [
         ("image/png", "Photos"),
         ("video/mp4", "Photos"),
-        ("audio/ogg", "Music"),
+        ("audio/mpeg", "Music"),
+        ("text/plain", "Documents"),
     ] {
-        let api = files_api(vec![404, 200, 200]);
-        let path = file_purchase_in_files(
-            &reqwest::Client::new(),
-            &api.url,
-            "test-session",
-            "file",
-            mime,
-            b"bytes",
-        )
-        .await
-        .unwrap();
-        assert_eq!(path, format!("{folder}/file"));
-        let seen = api.seen.lock().unwrap();
-        assert_eq!(seen[1].0, "POST");
-        assert!(seen[1].1.ends_with('/'));
-        assert!(seen[1].2.is_empty());
-        assert_eq!(seen[2].2, b"bytes");
+        let relative = file_purchase_in_files(dir.path(), "../name #?.bin", mime, b"paid")
+            .await
+            .unwrap();
+        assert!(relative.starts_with(&format!("{folder}/name #?")));
+        assert_eq!(
+            tokio::fs::read(dir.path().join("filebrowser").join(relative))
+                .await
+                .unwrap(),
+            b"paid"
+        );
     }
 }
 
 #[tokio::test]
-async fn files_copy_fails_without_overwriting_or_claiming_success_on_errors() {
-    for statuses in [
-        vec![401],
-        vec![503],
-        vec![404, 500],
-        vec![200, 507],
-        vec![200, 403],
-    ] {
-        let expected = statuses.len();
-        let api = files_api(statuses);
-        assert!(file_purchase_in_files(
-            &reqwest::Client::new(),
-            &api.url,
-            "test-session",
-            "file.txt",
-            "text/plain",
-            b"bytes"
-        )
-        .await
-        .is_err());
-        assert_eq!(api.seen.lock().unwrap().len(), expected);
-    }
+async fn unavailable_files_storage_is_reported_without_creating_a_fake_installation() {
+    let dir = tempfile::tempdir().unwrap();
+    assert!(
+        file_purchase_in_files(dir.path(), "name", "text/plain", b"bytes")
+            .await
+            .is_err()
+    );
+    assert!(!dir.path().join("filebrowser").exists());
 }
 
 #[test]

core/archipelago/src/api/rpc/lnd/channels.rs

@@ -473,6 +473,7 @@ impl RpcHandler {
             ));
         }
 
+        let fee_query = close_channel_fee_query(&params)?;
         let force = params
             .get("force")
             .and_then(|v| v.as_bool())
@@ -498,13 +499,11 @@ impl RpcHandler {
             .build()
             .context("Failed to create streaming HTTP client")?;
 
-        let url = format!(
-            "{LND_REST_BASE_URL}/v1/channels/{}/{}?force={}",
-            parts[0], parts[1], force
-        );
+        let url = format!("{LND_REST_BASE_URL}/v1/channels/{}/{}", parts[0], parts[1]);
 
         let mut resp = client
             .delete(&url)
+            .query(&fee_query)
             .header("Grpc-Metadata-macaroon", &macaroon_hex)
             .send()
             .await
@@ -572,3 +571,101 @@ impl RpcHandler {
         }
     }
 }
+
+/// LND's CloseChannel REST endpoint takes fee selection as query parameters.
+/// With neither parameter LND uses a lax target; keep legacy clients on our
+/// explicit Standard target rather than silently accepting that default.
+fn close_channel_fee_query(params: &serde_json::Value) -> Result<Vec<(&'static str, String)>> {
+    let force = match params.get("force") {
+        None | Some(serde_json::Value::Null) => false,
+        Some(value) => value
+            .as_bool()
+            .ok_or_else(|| anyhow::anyhow!("force must be a boolean"))?,
+    };
+    let integer = |key: &str, max: u64| -> Result<Option<u64>> {
+        match params.get(key) {
+            None | Some(serde_json::Value::Null) => Ok(None),
+            Some(value) => {
+                let n = value
+                    .as_u64()
+                    .ok_or_else(|| anyhow::anyhow!("{key} must be a positive whole number"))?;
+                anyhow::ensure!((1..=max).contains(&n), "{key} must be between 1 and {max}");
+                Ok(Some(n))
+            }
+        }
+    };
+    let target = integer("target_conf", 1008)?;
+    let rate = integer("sat_per_vbyte", 5000)?;
+    anyhow::ensure!(
+        target.is_none() || rate.is_none(),
+        "Specify either target_conf or sat_per_vbyte, not both"
+    );
+    anyhow::ensure!(
+        !force || (target.is_none() && rate.is_none()),
+        "Closing fee selection requires a cooperative close"
+    );
+    let mut query = vec![("force", force.to_string())];
+    if !force {
+        if let Some(rate) = rate {
+            query.push(("sat_per_vbyte", rate.to_string()));
+        } else {
+            query.push(("target_conf", target.unwrap_or(6).to_string()));
+        }
+    }
+    Ok(query)
+}
+
+#[cfg(test)]
+mod close_fee_tests {
+    use super::*;
+    #[test]
+    fn close_fee_query_forwards_presets_custom_and_legacy_default() {
+        for target in [1, 3, 6, 1008] {
+            assert_eq!(
+                close_channel_fee_query(&serde_json::json!({"target_conf":target})).unwrap(),
+                vec![
+                    ("force", "false".into()),
+                    ("target_conf", target.to_string())
+                ]
+            );
+        }
+        for rate in [1, 25, 5000] {
+            let query =
+                close_channel_fee_query(&serde_json::json!({"sat_per_vbyte":rate})).unwrap();
+            let request = reqwest::Client::new()
+                .delete("http://localhost/v1/channels/test/0")
+                .query(&query)
+                .build()
+                .unwrap();
+            assert_eq!(request.method(), reqwest::Method::DELETE);
+            assert_eq!(
+                request.url().query(),
+                Some(format!("force=false&sat_per_vbyte={rate}").as_str())
+            );
+        }
+        assert_eq!(
+            close_channel_fee_query(&serde_json::json!({})).unwrap(),
+            vec![("force", "false".into()), ("target_conf", "6".into())]
+        );
+        assert_eq!(
+            close_channel_fee_query(&serde_json::json!({"force":true})).unwrap(),
+            vec![("force", "true".into())]
+        );
+    }
+    #[test]
+    fn malformed_or_conflicting_close_fees_fail_before_wallet_access() {
+        for params in [
+            serde_json::json!({"target_conf":1,"sat_per_vbyte":2}),
+            serde_json::json!({"force":true,"target_conf":1}),
+            serde_json::json!({"force":"false"}),
+            serde_json::json!({"target_conf":0}),
+            serde_json::json!({"target_conf":1009}),
+            serde_json::json!({"sat_per_vbyte":5001}),
+            serde_json::json!({"sat_per_vbyte":-1}),
+            serde_json::json!({"sat_per_vbyte":1.5}),
+            serde_json::json!({"sat_per_vbyte":"25"}),
+        ] {
+            assert!(close_channel_fee_query(&params).is_err(), "{params}");
+        }
+    }
+}

core/archipelago/src/api/rpc/lnd/wallet.rs

@@ -453,6 +453,56 @@ impl RpcHandler {
         Ok(settled)
     }
 
+    /// Verify against LND at download time, rather than relying on a browser
+    /// having polled first. The memo/amount also recover pre-upgrade in-memory
+    /// entitlements after restart; unrelated invoices never unlock a file.
+    pub(crate) async fn settle_content_invoice(
+        &self,
+        hash: &str,
+        content_id: &str,
+    ) -> Result<bool> {
+        anyhow::ensure!(
+            hash.len() == 64 && hash.bytes().all(|c| c.is_ascii_hexdigit()),
+            "Invalid payment hash"
+        );
+        let hash = hash.to_ascii_lowercase();
+        let existing = crate::content_invoice::lookup(&self.config.data_dir, &hash).await?;
+        if let Some((id, _)) = &existing {
+            if id != content_id {
+                return Ok(false);
+            }
+        }
+        if crate::content_invoice::is_paid_for(&self.config.data_dir, &hash, content_id).await {
+            return Ok(true);
+        }
+        let (client, macaroon_hex) = self.lnd_client().await?;
+        let response = client
+            .get(format!("{LND_REST_BASE_URL}/v1/invoice/{hash}"))
+            .header("Grpc-Metadata-macaroon", &macaroon_hex)
+            .send()
+            .await?;
+        if response.status() == reqwest::StatusCode::NOT_FOUND {
+            return Ok(false);
+        }
+        let body: serde_json::Value = response.error_for_status()?.json().await?;
+        let Some(price) = content_invoice_amount(&body, content_id) else {
+            return Ok(false);
+        };
+        if existing
+            .as_ref()
+            .is_some_and(|(_, expected)| *expected != price)
+        {
+            return Ok(false);
+        }
+        crate::content_invoice::record_pending(&self.config.data_dir, &hash, content_id, price)
+            .await?;
+        let settled = content_invoice_fully_settled(&body, price);
+        if settled {
+            crate::content_invoice::mark_paid(&self.config.data_dir, &hash).await?;
+        }
+        Ok(settled)
+    }
+
     /// Generate a fresh on-chain receive address (seller side, #46).
     pub(crate) async fn new_onchain_address(&self) -> Result<String> {
         let (client, macaroon_hex) = self.lnd_client().await?;
@@ -1444,3 +1494,81 @@ mod tests {
         assert!(s.contains("[LND_REST_UNREACHABLE]"), "got: {s}");
     }
 }
+
+// LND REST uses decimal strings for int64 fields. Match the complete seller
+// memo, not a substring supplied by a buyer or an arbitrary settled invoice.
+fn json_u64(value: &serde_json::Value) -> Option<u64> {
+    value.as_u64().or_else(|| value.as_str()?.parse().ok())
+}
+fn content_invoice_fully_settled(body: &serde_json::Value, price: u64) -> bool {
+    let settled = match body.get("state").and_then(|v| v.as_str()) {
+        Some(state) => state == "SETTLED",
+        None => body.get("settled").and_then(|v| v.as_bool()) == Some(true),
+    };
+    settled
+        && price > 0
+        && body
+            .get("amt_paid_sat")
+            .and_then(json_u64)
+            .is_some_and(|paid| paid >= price)
+}
+fn content_invoice_amount(body: &serde_json::Value, content_id: &str) -> Option<u64> {
+    if body.get("memo")?.as_str()? != format!("Archipelago peer file {content_id}") {
+        return None;
+    }
+    body.get("value").and_then(json_u64).filter(|v| *v > 0)
+}
+
+#[cfg(test)]
+mod peer_file_invoice_tests {
+    use super::*;
+    #[test]
+    fn settlement_requires_terminal_state_and_full_amount() {
+        for state in ["OPEN", "ACCEPTED", "CANCELED", "unknown"] {
+            assert!(!content_invoice_fully_settled(
+                &serde_json::json!({"state":state,"settled":true,"amt_paid_sat":"100"}),
+                7
+            ));
+        }
+        for amount in [
+            serde_json::json!(6),
+            serde_json::json!("-1"),
+            serde_json::json!(null),
+            serde_json::json!("bad"),
+        ] {
+            assert!(!content_invoice_fully_settled(
+                &serde_json::json!({"state":"SETTLED","amt_paid_sat":amount}),
+                7
+            ));
+        }
+        for amount in [serde_json::json!(7), serde_json::json!("8")] {
+            assert!(content_invoice_fully_settled(
+                &serde_json::json!({"state":"SETTLED","amt_paid_sat":amount}),
+                7
+            ));
+        }
+        assert!(content_invoice_fully_settled(
+            &serde_json::json!({"settled":true,"amt_paid_sat":"7"}),
+            7
+        ));
+        assert!(!content_invoice_fully_settled(
+            &serde_json::json!({"state":"SETTLED","amt_paid_sat":"7"}),
+            0
+        ));
+    }
+    #[test]
+    fn legacy_recovery_requires_exact_file_memo_and_positive_amount() {
+        let invoice = serde_json::json!({"memo":"Archipelago peer file file-1", "value":"7"});
+        assert_eq!(content_invoice_amount(&invoice, "file-1"), Some(7));
+        assert_eq!(content_invoice_amount(&invoice, "file-2"), None);
+        for value in [
+            serde_json::json!("-1"),
+            serde_json::json!(0),
+            serde_json::json!("bad"),
+        ] {
+            let mut invalid = invoice.clone();
+            invalid["value"] = value;
+            assert_eq!(content_invoice_amount(&invalid, "file-1"), None);
+        }
+    }
+}

core/archipelago/src/api/rpc/package/async_lifecycle.rs

@@ -89,6 +89,15 @@ impl RpcHandler {
             match handler.handle_package_install(params).await {
                 Ok(_) => {
                     info!("package.install {}: complete", package_id_spawn);
+                    for id in [&package_id_spawn, &format!("archy-{}", package_id_spawn)] {
+                        crate::crash_recovery::clear_user_uninstalled(&handler.config.data_dir, id)
+                            .await;
+                    }
+                    crate::crash_recovery::mark_installed(
+                        &handler.config.data_dir,
+                        &package_id_spawn,
+                    )
+                    .await;
                     // The install pipeline has verified the container is up
                     // and healthy (see install.rs post-start exit check).
                     // Kick the scanner first so the fresh manifest (with
@@ -184,17 +193,20 @@ impl RpcHandler {
                     // phase is cleared (None) so no stale InstallPhase
                     // lingers on the card.
                     let err_msg = format!("Install failed: {:#}", e);
-                    let (mut data, _) = handler.state_manager.get_snapshot().await;
-                    if let Some(entry) = data.package_data.get_mut(&package_id_spawn) {
-                        entry.state = PackageState::Stopped;
-                        entry.install_progress = Some(crate::data_model::InstallProgress {
-                            size: 0,
-                            downloaded: 0,
-                            phase: None,
-                            message: Some(err_msg),
-                        });
-                        handler.state_manager.update_data(data).await;
-                    }
+                    handler
+                        .state_manager
+                        .mutate_data(|data| {
+                            if let Some(entry) = data.package_data.get_mut(&package_id_spawn) {
+                                entry.state = PackageState::Stopped;
+                                entry.install_progress = Some(crate::data_model::InstallProgress {
+                                    size: 0,
+                                    downloaded: 0,
+                                    phase: None,
+                                    message: Some(err_msg),
+                                });
+                            }
+                        })
+                        .await;
                 }
             }
         });
@@ -252,6 +264,11 @@ impl RpcHandler {
             match handler.handle_package_uninstall(params).await {
                 Ok(_) => {
                     info!("package.uninstall {}: complete", package_id_spawn);
+                    for id in [&package_id_spawn, &format!("archy-{}", package_id_spawn)] {
+                        crate::crash_recovery::mark_user_uninstalled(&handler.config.data_dir, id)
+                            .await;
+                        crate::crash_recovery::clear_installed(&handler.config.data_dir, id).await;
+                    }
                     // Inner handler already removed the package entry on
                     // success. Nothing more to do here.
                 }
@@ -382,52 +399,56 @@ impl RpcHandler {
 /// call, but fires before the spawn so the UI sees it immediately.
 async fn flip_to_installing(state_manager: &StateManager, package_id: &str) {
     use crate::data_model::{Description, Manifest, PackageDataEntry, StaticFiles};
-    let (mut data, _) = state_manager.get_snapshot().await;
-    let entry = data
-        .package_data
-        .entry(package_id.to_string())
-        .or_insert_with(|| PackageDataEntry {
-            state: PackageState::Installing,
-            health: None,
-            exit_code: None,
-            static_files: StaticFiles {
-                license: String::new(),
-                instructions: String::new(),
-                // Leave icon empty during the transient Installing window:
-                // hardcoding `<id>.png` is wrong for ~half our apps (many use
-                // `.svg` / `.webp`), producing a broken-image flicker until
-                // the scanner refreshes the entry. The frontend's `icon`
-                // computed falls through to `curatedMap.get(id)?.icon` which
-                // has the correct extensions for known apps.
-                icon: String::new(),
-            },
-            manifest: Manifest {
-                id: package_id.to_string(),
-                title: package_id.to_string(),
-                version: String::new(),
-                description: Description {
-                    short: "Installing...".to_string(),
-                    long: String::new(),
-                },
-                release_notes: String::new(),
-                license: String::new(),
-                wrapper_repo: String::new(),
-                upstream_repo: String::new(),
-                support_site: String::new(),
-                marketing_site: String::new(),
-                donation_url: None,
-                author: None,
-                website: None,
-                interfaces: None,
-                tier: None,
-            },
-            installed: None,
-            install_progress: None,
-            uninstall_stage: None,
-            available_update: None,
-        });
-    entry.state = PackageState::Installing;
-    state_manager.update_data(data).await;
+    state_manager
+        .mutate_data(|data| {
+            let entry = data
+                .package_data
+                .entry(package_id.to_string())
+                .or_insert_with(|| PackageDataEntry {
+                    ui_ready: None,
+                    state: PackageState::Installing,
+                    health: None,
+                    exit_code: None,
+                    static_files: StaticFiles {
+                        license: String::new(),
+                        instructions: String::new(),
+                        // Leave icon empty during the transient Installing window:
+                        // hardcoding `<id>.png` is wrong for ~half our apps (many use
+                        // `.svg` / `.webp`), producing a broken-image flicker until
+                        // the scanner refreshes the entry. The frontend's `icon`
+                        // computed falls through to `curatedMap.get(id)?.icon` which
+                        // has the correct extensions for known apps.
+                        icon: String::new(),
+                    },
+                    manifest: Manifest {
+                        id: package_id.to_string(),
+                        title: package_id.to_string(),
+                        version: String::new(),
+                        description: Description {
+                            short: "Installing...".to_string(),
+                            long: String::new(),
+                        },
+                        release_notes: String::new(),
+                        license: String::new(),
+                        wrapper_repo: String::new(),
+                        upstream_repo: String::new(),
+                        support_site: String::new(),
+                        marketing_site: String::new(),
+                        donation_url: None,
+                        author: None,
+                        website: None,
+                        interfaces: None,
+                        tier: None,
+                    },
+                    installed: None,
+                    install_progress: None,
+                    uninstall_stage: None,
+                    available_update: None,
+                });
+            entry.ui_ready = Some(false);
+            entry.state = PackageState::Installing;
+        })
+        .await;
 }
 
 /// True when the failed install still has a real footprint: any container
@@ -485,20 +506,23 @@ async fn remove_entry_with_notification(
     id_prefix: &str,
     message: &str,
 ) {
-    let (mut data, _) = handler.state_manager.get_snapshot().await;
-    data.package_data.remove(package_id);
-    data.notifications.push(crate::data_model::Notification {
-        id: format!("{id_prefix}-{package_id}"),
-        level: crate::data_model::NotificationLevel::Error,
-        title: format!("Could not install {package_id}"),
-        message: message.to_string(),
-        timestamp: chrono::Utc::now().to_rfc3339(),
-        app_id: Some(package_id.to_string()),
-    });
-    while data.notifications.len() > 20 {
-        data.notifications.remove(0);
-    }
-    handler.state_manager.update_data(data).await;
+    handler
+        .state_manager
+        .mutate_data(|data| {
+            data.package_data.remove(package_id);
+            data.notifications.push(crate::data_model::Notification {
+                id: format!("{id_prefix}-{package_id}"),
+                level: crate::data_model::NotificationLevel::Error,
+                title: format!("Could not install {package_id}"),
+                message: message.to_string(),
+                timestamp: chrono::Utc::now().to_rfc3339(),
+                app_id: Some(package_id.to_string()),
+            });
+            while data.notifications.len() > 20 {
+                data.notifications.remove(0);
+            }
+        })
+        .await;
 }
 
 /// Flip an existing entry's state and return the pre-flip value (or None if
@@ -508,18 +532,23 @@ async fn flip_package_state(
     package_id: &str,
     new_state: PackageState,
 ) -> Option<PackageState> {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    let prev = data.package_data.get(package_id).map(|e| e.state.clone());
-    if let Some(entry) = data.package_data.get_mut(package_id) {
-        entry.state = new_state;
-        state_manager.update_data(data).await;
-    } else {
-        warn!(
-            "flip_package_state: no entry for {} — cannot flip",
-            package_id
-        );
-    }
-    prev
+    state_manager
+        .mutate_data(|data| {
+            let prev = data.package_data.get(package_id).map(|e| e.state.clone());
+            if let Some(entry) = data.package_data.get_mut(package_id) {
+                if new_state != PackageState::Running {
+                    entry.ui_ready = Some(false);
+                }
+                entry.state = new_state;
+            } else {
+                warn!(
+                    "flip_package_state: no entry for {} — cannot flip",
+                    package_id
+                );
+            }
+            prev
+        })
+        .await
 }
 
 /// Set state unconditionally (no-op if entry no longer exists).
@@ -528,13 +557,18 @@ async fn set_package_state(
     package_id: &str,
     new_state: PackageState,
 ) {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    if let Some(entry) = data.package_data.get_mut(package_id) {
-        if entry.state != new_state {
-            entry.state = new_state;
-            state_manager.update_data(data).await;
-        }
-    }
+    state_manager
+        .mutate_data(|data| {
+            if let Some(entry) = data.package_data.get_mut(package_id) {
+                if entry.state != new_state {
+                    if new_state != PackageState::Running {
+                        entry.ui_ready = Some(false);
+                    }
+                    entry.state = new_state;
+                }
+            }
+        })
+        .await
 }
 
 /// Set state and clear the uninstall_stage label. Used when an uninstall
@@ -545,12 +579,17 @@ async fn set_package_state_and_clear_uninstall_stage(
     package_id: &str,
     new_state: PackageState,
 ) {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    if let Some(entry) = data.package_data.get_mut(package_id) {
-        entry.state = new_state;
-        entry.uninstall_stage = None;
-        state_manager.update_data(data).await;
-    }
+    state_manager
+        .mutate_data(|data| {
+            if let Some(entry) = data.package_data.get_mut(package_id) {
+                if new_state != PackageState::Running {
+                    entry.ui_ready = Some(false);
+                }
+                entry.state = new_state;
+                entry.uninstall_stage = None;
+            }
+        })
+        .await
 }
 
 /// Kick the container scanner to run immediately and wait for it to finish

core/archipelago/src/api/rpc/package/dependencies.rs

@@ -22,6 +22,18 @@ const ARCHIVAL_BITCOIN_DEPENDENCY: &str = "bitcoin:archival";
 /// hardcoded id list below — a new app just declares the dependency instead
 /// of needing a code change here.
 fn manifest_declares_archival_bitcoin(package_id: &str) -> bool {
+    // Registry-only apps need the same guard as OTA-bundled manifests. Honor
+    // the verified catalog's effective manifest before the disk fallback.
+    if let Some((_, value)) = crate::container::app_catalog::catalog_manifest_values()
+        .into_iter()
+        .find(|(id, _)| id == package_id)
+    {
+        if let Some(manifest) =
+            crate::container::app_catalog::catalog_manifest_overlay(package_id, value)
+        {
+            return dependency_list_declares_archival_bitcoin(&manifest.app.dependencies);
+        }
+    }
     for apps_dir in manifest_apps_dirs() {
         let path = apps_dir.join(package_id).join("manifest.yml");
         let Ok(contents) = std::fs::read_to_string(&path) else {
@@ -1055,6 +1067,14 @@ mod tests {
         // edit to `requires_unpruned_bitcoin`.
         assert!(manifest_declares_archival_bitcoin("electrumx"));
         assert!(manifest_declares_archival_bitcoin("mempool"));
+        let angor = archipelago_container::AppManifest::parse(include_str!(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/../../apps/angor-indexer/manifest.yml"
+        )))
+        .unwrap();
+        assert!(dependency_list_declares_archival_bitcoin(
+            &angor.app.dependencies
+        ));
         // An app whose manifest exists but never declares the marker.
         assert!(!manifest_declares_archival_bitcoin("bitcoin-knots"));
         // An id with no manifest on disk at all.

core/archipelago/src/api/rpc/package/install.rs

@@ -545,7 +545,7 @@ impl RpcHandler {
         // Keep legacy install flow as default while migration is in progress.
         if orchestrator_managed {
             let orchestrator_app_id = orchestrator_install_app_id(package_id);
-            self.set_install_phase(package_id, InstallPhase::CreatingContainer)
+            self.set_install_phase(package_id, InstallPhase::PreparingApp)
                 .await;
             install_log(&format!(
                 "INSTALL ORCH: {} — attempting orchestrator install as {}",
@@ -573,6 +573,9 @@ impl RpcHandler {
                             "message": format!("Package {} installed and started", package_id)
                         }));
                     }
+                    Err(e) if e.downcast_ref::<crate::container::prod_orchestrator::InstallPrerequisiteError>().is_some() => {
+                        return Err(super::dependencies::DependencyGateError(e.to_string()).into());
+                    }
                     Err(e) if is_unknown_app_id_error(&e) => {
                         info!(
                             "Install {}: orchestrator has no manifest mapping yet, falling back to legacy installer",
@@ -1699,32 +1702,10 @@ autopilot.active=false\n",
             patch_indeedhub_nostr_provider().await;
         }
 
-        // Gitea: keep it on its native host port (3001). The UI opens Gitea
-        // in a new tab on that direct port so absolute asset URLs must be
-        // rooted at the host port rather than Archipelago's /app/gitea/ path.
-        if package_id == "gitea" {
-            let _ = tokio::fs::remove_file("/etc/nginx/conf.d/gitea-iframe.conf").await;
-
-            // Set ROOT_URL to the direct launch route so links/assets stay
-            // anchored under the same origin Gitea is launched from.
-            let host_ip = &self.config.host_ip;
-            let _ = tokio::process::Command::new("podman")
-                .args(["exec", "gitea", "sh", "-c",
-                       &format!("grep -q ROOT_URL /data/gitea/conf/app.ini && sed -i 's|ROOT_URL.*|ROOT_URL = http://{}:3001/|' /data/gitea/conf/app.ini || true", host_ip)])
-                .output()
-                .await;
-            // Also ensure X_FRAME_OPTIONS is empty so Gitea doesn't send the header
-            let _ = tokio::process::Command::new("podman")
-                .args(["exec", "gitea", "sh", "-c",
-                       "grep -q X_FRAME_OPTIONS /data/gitea/conf/app.ini && sed -i 's|X_FRAME_OPTIONS.*|X_FRAME_OPTIONS =|' /data/gitea/conf/app.ini || sed -i '/^\\[security\\]/a X_FRAME_OPTIONS =' /data/gitea/conf/app.ini"])
-                .output()
-                .await;
-
-            info!(
-                "Gitea: ROOT_URL set to http://{}:3001/, X_FRAME_OPTIONS cleared",
-                host_ip
-            );
-        }
+        // Gitea owns its public URL and security settings in app.ini, including
+        // values chosen in its first-run setup. Do not rewrite operator values
+        // or claim success from best-effort grep/sed commands. The app gate
+        // fronts its declared HTTP port and handles frame headers separately.
 
         if package_id == "nextcloud" {
             let host_ip = &self.config.host_ip;
@@ -2053,25 +2034,8 @@ fn parse_setup_token(lines: &[&str]) -> Option<String> {
 }
 
 async fn cleanup_stale_package_ports(package_id: &str) {
-    match package_id {
-        "grafana" => cleanup_stale_pasta_port("3000").await,
-        "homeassistant" | "home-assistant" => cleanup_stale_pasta_port("8123").await,
-        "searxng" => cleanup_stale_pasta_port("8888").await,
-        "uptime-kuma" => cleanup_stale_pasta_port("3002").await,
-        "gitea" => {
-            cleanup_stale_pasta_port("3001").await;
-            cleanup_stale_pasta_port("2222").await;
-            cleanup_stale_pasta_port("3000").await;
-        }
-        "nginx-proxy-manager" => {
-            cleanup_stale_pasta_port("8081").await;
-            cleanup_stale_pasta_port("8084").await;
-            cleanup_stale_pasta_port("8444").await;
-        }
-        "nextcloud" => cleanup_stale_pasta_port("8085").await,
-        "portainer" => cleanup_stale_pasta_port("9000").await,
-        _ => {}
-    }
+    // Never kill by port: another app or the management gate may own it.
+    crate::container::ghost_reaper::reap_for_app(package_id).await;
 }
 
 fn install_command_tail(
@@ -2196,93 +2160,11 @@ async fn cleanup_start_conflict(package_id: &str, stderr: &str) -> bool {
         return true;
     }
 
-    match package_id {
-        "grafana"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("3000").await;
-            true
-        }
-        "homeassistant" | "home-assistant"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("8123").await;
-            true
-        }
-        "searxng"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("8888").await;
-            true
-        }
-        "uptime-kuma"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("3002").await;
-            true
-        }
-        "gitea" if stderr.contains("pasta failed") || stderr.contains("address already in use") => {
-            cleanup_stale_pasta_port("3001").await;
-            cleanup_stale_pasta_port("2222").await;
-            cleanup_stale_pasta_port("3000").await;
-            true
-        }
-        "nginx-proxy-manager"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("8081").await;
-            cleanup_stale_pasta_port("8084").await;
-            cleanup_stale_pasta_port("8444").await;
-            true
-        }
-        "nextcloud"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("8085").await;
-            true
-        }
-        "portainer"
-            if stderr.contains("pasta failed") || stderr.contains("address already in use") =>
-        {
-            cleanup_stale_pasta_port("9000").await;
-            true
-        }
-        _ => false,
+    if stderr.contains("pasta failed") || stderr.contains("address already in use") {
+        crate::container::ghost_reaper::reap_for_app(package_id).await;
+        return true;
     }
-}
-
-async fn cleanup_stale_pasta_port(port: &str) {
-    // NEVER kill our own process. The daemon holds catalog app ports over
-    // IPv6 (the mesh app-port relay), so a blunt `fuser -k <port>/tcp` would
-    // terminate archipelago itself mid-install — installs failed and apps
-    // vanished on a test node 2026-07-27. Kill every listener on the port
-    // EXCEPT our PID (and our process group), leaving the relay/daemon alive.
-    let self_pid = std::process::id();
-    let kill_listener = format!(
-        "ss -ltnp 'sport = :{port}' 2>/dev/null | sed -n 's/.*pid=\\([0-9]*\\).*/\\1/p' | \
-         while read p; do [ \"$p\" = \"{self_pid}\" ] || kill \"$p\" 2>/dev/null; done || true",
-    );
-    let _ = tokio::process::Command::new("sh")
-        .args(["-c", &kill_listener])
-        .output()
-        .await;
-
-    // sudo fuser -k, but exclude our own PID: fuser prints the PIDs holding
-    // the port; kill each except self. (`fuser -k` has no exclusion flag.)
-    let fuser_kill = format!(
-        "for p in $(sudo fuser {port}/tcp 2>/dev/null); do [ \"$p\" = \"{self_pid}\" ] || sudo kill \"$p\" 2>/dev/null; done || true",
-    );
-    let _ = tokio::process::Command::new("sh")
-        .args(["-c", &fuser_kill])
-        .output()
-        .await;
-
-    let pattern = format!("pasta.*{}", port);
-    let _ = tokio::process::Command::new("pkill")
-        .args(["-f", &pattern])
-        .output()
-        .await;
-    tokio::time::sleep(std::time::Duration::from_secs(1)).await;
+    false
 }
 
 async fn repair_nextcloud_permissions() {

core/archipelago/src/api/rpc/package/progress.rs

@@ -14,20 +14,23 @@ impl RpcHandler {
     /// the rare case where the pull stream actually parses, but podman
     /// almost never emits parseable progress on a piped stderr.
     pub(super) async fn set_install_progress(&self, package_id: &str, downloaded: u64, size: u64) {
-        let (mut data, _rev) = self.state_manager.get_snapshot().await;
-        let entry = data
-            .package_data
-            .entry(package_id.to_string())
-            .or_insert_with(|| create_installing_entry(package_id));
-        entry.state = PackageState::Installing;
-        let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
-        entry.install_progress = Some(InstallProgress {
-            size,
-            downloaded,
-            phase: existing_phase,
-            message: None,
-        });
-        self.state_manager.update_data(data).await;
+        self.state_manager
+            .mutate_data(|data| {
+                let entry = data
+                    .package_data
+                    .entry(package_id.to_string())
+                    .or_insert_with(|| create_installing_entry(package_id));
+                entry.ui_ready = Some(false);
+                entry.state = PackageState::Installing;
+                let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
+                entry.install_progress = Some(InstallProgress {
+                    size,
+                    downloaded,
+                    phase: existing_phase,
+                    message: None,
+                });
+            })
+            .await;
     }
 
     /// Set the install pipeline phase and broadcast. This is the
@@ -35,76 +38,86 @@ impl RpcHandler {
     /// percentage and a user-facing label. Byte counters are retained
     /// for the rare case podman emits parseable progress.
     pub(super) async fn set_install_phase(&self, package_id: &str, phase: InstallPhase) {
-        let (mut data, _rev) = self.state_manager.get_snapshot().await;
-        let entry = data
-            .package_data
-            .entry(package_id.to_string())
-            .or_insert_with(|| create_installing_entry(package_id));
-        // Preparing / PullingImage / CreatingContainer / StartingContainer /
-        // WaitingHealthy / PostInstall all map to the Installing state.
-        // Updates use Updating state — the wrapper has already flipped
-        // state to Updating, so don't clobber it.
-        if entry.state != PackageState::Updating {
-            entry.state = PackageState::Installing;
-        }
-        let (size, downloaded) = entry
-            .install_progress
-            .as_ref()
-            .map(|p| (p.size, p.downloaded))
-            .unwrap_or((0, 0));
-        entry.install_progress = Some(InstallProgress {
-            size,
-            downloaded,
-            phase: Some(phase),
-            message: None,
-        });
-        self.state_manager.update_data(data).await;
+        self.state_manager
+            .mutate_data(|data| {
+                let entry = data
+                    .package_data
+                    .entry(package_id.to_string())
+                    .or_insert_with(|| create_installing_entry(package_id));
+                // Preparing / PullingImage / CreatingContainer / StartingContainer /
+                // WaitingHealthy / PostInstall all map to the Installing state.
+                // Updates use Updating state — the wrapper has already flipped
+                // state to Updating, so don't clobber it.
+                if entry.state != PackageState::Updating {
+                    entry.ui_ready = Some(false);
+                    entry.state = PackageState::Installing;
+                }
+                let (size, downloaded) = entry
+                    .install_progress
+                    .as_ref()
+                    .map(|p| (p.size, p.downloaded))
+                    .unwrap_or((0, 0));
+                entry.install_progress = Some(InstallProgress {
+                    size,
+                    downloaded,
+                    phase: Some(phase),
+                    message: None,
+                });
+            })
+            .await;
     }
 
     /// Set a user-facing install status message (e.g. "Waiting for Bitcoin
     /// to start…") without disturbing the current phase/byte counters.
     pub(super) async fn set_install_message(&self, package_id: &str, message: &str) {
-        let (mut data, _rev) = self.state_manager.get_snapshot().await;
-        let entry = data
-            .package_data
-            .entry(package_id.to_string())
-            .or_insert_with(|| create_installing_entry(package_id));
-        if entry.state != PackageState::Updating {
-            entry.state = PackageState::Installing;
-        }
-        let (size, downloaded, phase) = entry
-            .install_progress
-            .as_ref()
-            .map(|p| (p.size, p.downloaded, p.phase))
-            .unwrap_or((0, 0, None));
-        entry.install_progress = Some(InstallProgress {
-            size,
-            downloaded,
-            phase,
-            message: Some(message.to_string()),
-        });
-        self.state_manager.update_data(data).await;
+        self.state_manager
+            .mutate_data(|data| {
+                let entry = data
+                    .package_data
+                    .entry(package_id.to_string())
+                    .or_insert_with(|| create_installing_entry(package_id));
+                if entry.state != PackageState::Updating {
+                    entry.ui_ready = Some(false);
+                    entry.state = PackageState::Installing;
+                }
+                let (size, downloaded, phase) = entry
+                    .install_progress
+                    .as_ref()
+                    .map(|p| (p.size, p.downloaded, p.phase))
+                    .unwrap_or((0, 0, None));
+                entry.install_progress = Some(InstallProgress {
+                    size,
+                    downloaded,
+                    phase,
+                    message: Some(message.to_string()),
+                });
+            })
+            .await;
     }
 
     /// Clear install progress after pull completes or fails.
     pub(super) async fn clear_install_progress(&self, package_id: &str) {
-        let (mut data, _rev) = self.state_manager.get_snapshot().await;
-        if let Some(entry) = data.package_data.get_mut(package_id) {
-            entry.install_progress = None;
-        }
-        self.state_manager.update_data(data).await;
+        self.state_manager
+            .mutate_data(|data| {
+                if let Some(entry) = data.package_data.get_mut(package_id) {
+                    entry.install_progress = None;
+                }
+            })
+            .await;
     }
 
     /// Set the uninstall stage label so the UI can show what's happening
     /// instead of a generic spinner. Each call broadcasts a state change
     /// — call sparingly (one per pipeline phase, not per container).
     pub(super) async fn set_uninstall_stage(&self, package_id: &str, stage: &str) {
-        let (mut data, _rev) = self.state_manager.get_snapshot().await;
-        if let Some(entry) = data.package_data.get_mut(package_id) {
-            entry.uninstall_stage = Some(stage.to_string());
-            entry.state = crate::data_model::PackageState::Removing;
-        }
-        self.state_manager.update_data(data).await;
+        self.state_manager
+            .mutate_data(|data| {
+                if let Some(entry) = data.package_data.get_mut(package_id) {
+                    entry.uninstall_stage = Some(stage.to_string());
+                    entry.state = crate::data_model::PackageState::Removing;
+                }
+            })
+            .await;
     }
 
     /// Update install progress (static method for use in async closures).
@@ -114,25 +127,28 @@ impl RpcHandler {
         downloaded: u64,
         total: u64,
     ) {
-        let (mut data, _rev) = state_manager.get_snapshot().await;
-        let entry = data
-            .package_data
-            .entry(package_id.to_string())
-            .or_insert_with(|| create_installing_entry(package_id));
-        let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
-        entry.install_progress = Some(InstallProgress {
-            size: total,
-            downloaded,
-            phase: existing_phase,
-            message: None,
-        });
-        state_manager.update_data(data).await;
+        state_manager
+            .mutate_data(|data| {
+                let entry = data
+                    .package_data
+                    .entry(package_id.to_string())
+                    .or_insert_with(|| create_installing_entry(package_id));
+                let existing_phase = entry.install_progress.as_ref().and_then(|p| p.phase);
+                entry.install_progress = Some(InstallProgress {
+                    size: total,
+                    downloaded,
+                    phase: existing_phase,
+                    message: None,
+                });
+            })
+            .await;
     }
 }
 
 /// Create a minimal PackageDataEntry for a package being installed.
 fn create_installing_entry(package_id: &str) -> PackageDataEntry {
     PackageDataEntry {
+        ui_ready: None,
         state: PackageState::Installing,
         health: None,
         exit_code: None,

core/archipelago/src/api/rpc/package/runtime.rs

@@ -1431,10 +1431,9 @@ async fn repair_before_package_start(container_name: &str) {
     // published port and the data-dir file locks, so the replacement either
     // fails to bind (`address already in use`) or starts and dies on the
     // lock — and `Restart=always` loops it there forever. Ordered before
-    // the port cleanup below: killing the owner is what actually frees the
-    // port, and the port sweep alone cannot tell a ghost from a live app.
+    // starting the replacement. A port sweep cannot distinguish a ghost
+    // from the dashboard gate or another live app and must never kill it.
     crate::container::ghost_reaper::reap_for_app(container_name).await;
-    cleanup_runtime_host_ports(container_name).await;
 }
 
 async fn wait_before_package_start(container_name: &str) {
@@ -1579,7 +1578,6 @@ async fn repair_netbird_network() {
 async fn repair_nginx_proxy_manager_container() {
     repair_nginx_proxy_manager_dirs().await;
     if !nginx_proxy_manager_has_legacy_admin_port().await {
-        cleanup_nginx_proxy_manager_ports().await;
         return;
     }
 
@@ -1588,7 +1586,7 @@ async fn repair_nginx_proxy_manager_container() {
     )
     .await;
     let _ = podman_control(&["rm", "-f", "nginx-proxy-manager"]).await;
-    cleanup_nginx_proxy_manager_ports().await;
+    crate::container::ghost_reaper::reap_for_app("nginx-proxy-manager").await;
     if let Err(err) = recreate_nginx_proxy_manager_container().await {
         tracing::warn!(error = %err, "failed to recreate stale nginx-proxy-manager container");
     }
@@ -1812,6 +1810,9 @@ fn manifest_host_ports(container_name: &str) -> Vec<u16> {
 
 pub(super) fn manifest_apps_dirs() -> Vec<std::path::PathBuf> {
     let mut dirs = Vec::new();
+    if let Some(root) = std::env::var_os("ARCHIPELAGO_APPS_DIR") {
+        dirs.push(root.into());
+    }
     if let Ok(manifest_dir) = std::env::var("CARGO_MANIFEST_DIR") {
         dirs.push(Path::new(&manifest_dir).join("../../apps"));
     }
@@ -2032,51 +2033,10 @@ async fn cleanup_start_conflict(container_name: &str, stderr: &str) {
         return;
     }
 
-    let ports = runtime_host_ports(container_name);
-    if !ports.is_empty() {
-        cleanup_ports(&ports).await;
-        return;
-    }
-}
-
-async fn cleanup_runtime_host_ports(container_name: &str) {
-    let ports = runtime_host_ports(container_name);
-    if !ports.is_empty() {
-        cleanup_ports(&ports).await;
-    }
-}
-
-async fn cleanup_nginx_proxy_manager_ports() {
-    cleanup_ports(&[8081, 8084, 8444]).await;
-}
-
-async fn cleanup_ports(ports: &[u16]) {
-    for port in ports {
-        cleanup_stale_pasta_port(&port.to_string()).await;
-    }
-}
-
-async fn cleanup_stale_pasta_port(port: &str) {
-    let kill_listener = format!(
-        "ss -ltnp 'sport = :{}' 2>/dev/null | sed -n 's/.*pid=\\([0-9]*\\).*/\\1/p' | xargs -r kill 2>/dev/null || true",
-        port
-    );
-    let _ = tokio::process::Command::new("sh")
-        .args(["-c", &kill_listener])
-        .output()
-        .await;
-
-    let pattern = format!("pasta.*{}", port);
-    let _ = tokio::process::Command::new("pkill")
-        .args(["-f", &pattern])
-        .output()
-        .await;
-    let pattern = format!("rootlessport.*{}", port);
-    let _ = tokio::process::Command::new("pkill")
-        .args(["-f", &pattern])
-        .output()
-        .await;
-    tokio::time::sleep(std::time::Duration::from_secs(1)).await;
+    // Only reap processes proven to belong to an absent container. The app
+    // gate shares the app's port on other addresses and lives in this daemon;
+    // killing port owners (or matching argv with pkill) kills the dashboard.
+    crate::container::ghost_reaper::reap_for_app(container_name).await;
 }
 
 pub(super) fn is_missing_companion_ok(name: &str, stderr: &str) -> bool {
@@ -2095,13 +2055,16 @@ async fn flip_package_state(
     package_id: &str,
     transitional: PackageState,
 ) -> Option<PackageState> {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    let prev = data.package_data.get(package_id).map(|e| e.state.clone());
-    if let Some(entry) = data.package_data.get_mut(package_id) {
-        entry.state = transitional;
-        state_manager.update_data(data).await;
-    }
-    prev
+    state_manager
+        .mutate_data(|data| {
+            let prev = data.package_data.get(package_id).map(|e| e.state.clone());
+            if let Some(entry) = data.package_data.get_mut(package_id) {
+                entry.ui_ready = Some(false);
+                entry.state = transitional;
+            }
+            prev
+        })
+        .await
 }
 
 /// Write the package entry's final state. No-op if the entry has since
@@ -2111,13 +2074,18 @@ async fn set_package_state(
     package_id: &str,
     new_state: PackageState,
 ) {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    if let Some(entry) = data.package_data.get_mut(package_id) {
-        if entry.state != new_state {
-            entry.state = new_state;
-            state_manager.update_data(data).await;
-        }
-    }
+    state_manager
+        .mutate_data(|data| {
+            if let Some(entry) = data.package_data.get_mut(package_id) {
+                if entry.state != new_state {
+                    if new_state != PackageState::Running {
+                        entry.ui_ready = Some(false);
+                    }
+                    entry.state = new_state;
+                }
+            }
+        })
+        .await
 }
 
 pub(super) async fn reconcile_companions_for(package_id: &str) {
@@ -2185,6 +2153,20 @@ pub(super) fn orchestrator_uninstall_app_ids(package_id: &str) -> Vec<String> {
 mod tests {
     use super::*;
 
+    #[tokio::test]
+    async fn port_conflict_cleanup_preserves_live_host_listener() {
+        // The previous ss|kill sweep terminated the daemon's app gate on a
+        // restart. Keep a real listening socket owned by this test process.
+        let listener = tokio::net::TcpListener::bind("127.0.0.2:2342")
+            .await
+            .unwrap();
+        let addr = listener.local_addr().unwrap();
+        cleanup_start_conflict("photoprism", "address already in use").await;
+        let client = tokio::net::TcpStream::connect(addr).await.unwrap();
+        let _connection = listener.accept().await.unwrap();
+        drop(client);
+    }
+
     #[test]
     fn missing_container_classifier_covers_podman5_phrasings() {
         // Regression (.228 gate 2026-07-08): podman 5.x `inspect` on a missing

core/archipelago/src/api/rpc/transitional.rs

@@ -150,23 +150,31 @@ async fn flip_to_transitional(
     app_id: &str,
     transitional: PackageState,
 ) -> Option<PackageState> {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    let prev = data.package_data.get(app_id).map(|e| e.state.clone());
-    if let Some(entry) = data.package_data.get_mut(app_id) {
-        entry.state = transitional;
-        state_manager.update_data(data).await;
-    }
-    prev
+    state_manager
+        .mutate_data(|data| {
+            let prev = data.package_data.get(app_id).map(|e| e.state.clone());
+            if let Some(entry) = data.package_data.get_mut(app_id) {
+                entry.ui_ready = Some(false);
+                entry.state = transitional;
+            }
+            prev
+        })
+        .await
 }
 
 /// Set the entry's state to `new_state`. No-ops if the entry has since been
 /// removed (e.g. uninstall ran concurrently).
 async fn set_state(state_manager: &StateManager, app_id: &str, new_state: PackageState) {
-    let (mut data, _) = state_manager.get_snapshot().await;
-    if let Some(entry) = data.package_data.get_mut(app_id) {
-        if entry.state != new_state {
-            entry.state = new_state;
-            state_manager.update_data(data).await;
-        }
-    }
+    state_manager
+        .mutate_data(|data| {
+            if let Some(entry) = data.package_data.get_mut(app_id) {
+                if entry.state != new_state {
+                    if new_state != PackageState::Running {
+                        entry.ui_ready = Some(false);
+                    }
+                    entry.state = new_state;
+                }
+            }
+        })
+        .await
 }

core/archipelago/src/appgate/identity.rs

@@ -114,6 +114,9 @@ impl PortMap {
 /// there.
 fn apps_dirs() -> Vec<PathBuf> {
     let mut dirs = Vec::new();
+    if let Some(root) = std::env::var_os("ARCHIPELAGO_APPS_DIR") {
+        dirs.push(root.into());
+    }
     if let Ok(manifest_dir) = std::env::var("CARGO_MANIFEST_DIR") {
         dirs.push(PathBuf::from(manifest_dir).join("../../apps"));
     }

core/archipelago/src/appgate/listener.rs

@@ -144,6 +144,34 @@ pub fn shared_status() -> Arc<RwLock<GateStatus>> {
         .clone()
 }
 
+static REFRESH_KICK: std::sync::LazyLock<tokio::sync::Notify> =
+    std::sync::LazyLock::new(tokio::sync::Notify::new);
+static REFRESH_REV: std::sync::LazyLock<tokio::sync::watch::Sender<u64>> =
+    std::sync::LazyLock::new(|| tokio::sync::watch::channel(0).0);
+
+/// Installation must not wait for the minute sweep before becoming reachable.
+/// Wait for a completed sweep, bounded if shutdown/startup prevents one.
+pub async fn refresh_now() {
+    let mut completed = REFRESH_REV.subscribe();
+    REFRESH_KICK.notify_one();
+    let _ = tokio::time::timeout(std::time::Duration::from_secs(3), completed.changed()).await;
+}
+
+pub fn port_claimed(status: &GateStatus, port: u16) -> bool {
+    let mut external = false;
+    let mut tor = false;
+    for (claimed_port, address) in &status.claimed {
+        if *claimed_port != port {
+            continue;
+        }
+        if let Ok(ip) = address.parse::<IpAddr>() {
+            tor |= ip == GATE_TOR_UPSTREAM;
+            external |= !ip.is_loopback();
+        }
+    }
+    external && tor
+}
+
 /// Run the gate. Returns only on shutdown.
 pub async fn run(
     gate: Arc<AppGate>,
@@ -162,11 +190,12 @@ pub async fn run(
 
     loop {
         tokio::select! {
-            _ = interval.tick() => {
-                sweep(&gate, &status, &mut held, &shutdown_rx).await;
-            }
+            _ = interval.tick() => {}
+            _ = REFRESH_KICK.notified() => {}
             _ = shutdown_rx.changed() => return,
         }
+        sweep(&gate, &status, &mut held, &shutdown_rx).await;
+        REFRESH_REV.send_modify(|revision| *revision = revision.wrapping_add(1));
     }
 }
 
@@ -461,3 +490,19 @@ mod tests {
         assert!(!status.is_fully_enforced());
     }
 }
+
+#[cfg(test)]
+mod readiness_tests {
+    use super::*;
+    #[test]
+    fn readiness_requires_external_and_tor_claims_for_the_same_port() {
+        let mut status = GateStatus::default();
+        assert!(!port_claimed(&status, 3001));
+        status.claimed.push((3001, "127.0.0.2".into()));
+        assert!(!port_claimed(&status, 3001));
+        status.claimed.push((3002, "192.0.2.10".into()));
+        assert!(!port_claimed(&status, 3001));
+        status.claimed.push((3001, "192.0.2.10".into()));
+        assert!(port_claimed(&status, 3001));
+    }
+}

core/archipelago/src/assistant/evals.rs

@@ -322,6 +322,7 @@ async fn eval_rpc_handler() -> (Arc<RpcHandler>, tempfile::TempDir) {
 fn installed_entry(app_id: &str) -> crate::data_model::PackageDataEntry {
     use crate::data_model::{Description, Manifest, PackageDataEntry, PackageState, StaticFiles};
     PackageDataEntry {
+        ui_ready: None,
         state: PackageState::Running,
         health: None,
         exit_code: None,

core/archipelago/src/assistant/mod.rs

@@ -1069,6 +1069,7 @@ mod tests {
             Description, Manifest, PackageDataEntry, PackageState, StaticFiles,
         };
         PackageDataEntry {
+            ui_ready: None,
             state: PackageState::Running,
             health: None,
             exit_code: None,

core/archipelago/src/bootstrap.rs

@@ -138,6 +138,44 @@ const NGINX_FEDIMINT_NEW: &str = "        sub_filter_types text/css application/
 const NGINX_FEDIMINT_SNIPPET_ANCHOR: &str = "proxy_pass http://127.0.0.1:8175/;";
 const NGINX_FEDIMINT_SNIPPET_INSERT: &str = "proxy_pass http://127.0.0.1:8175/;\n    proxy_set_header Accept-Encoding \"\";\n    sub_filter_types text/css application/javascript application/json;\n    sub_filter_once off;\n    sub_filter 'href=\"/' 'href=\"/app/fedimint/';\n    sub_filter 'src=\"/' 'src=\"/app/fedimint/';\n    sub_filter \"href='/\" \"href='/app/fedimint/\";\n    sub_filter \"src='/\" \"src='/app/fedimint/\";\n    sub_filter 'url(\"/' 'url(\"/app/fedimint/';\n    sub_filter \"url('/\" \"url('/app/fedimint/\";\n    sub_filter '</head>' '<script src=\"/nostr-provider.js\"></script></head>';";
 
+/// Finish manifest promotion before constructing the orchestrator or starting
+/// catalog refresh/reconciliation. Replacing the app tree in the background
+/// could let a reload observe its temporary empty state and forget disk-only apps.
+pub async fn ensure_runtime_assets_ready() {
+    match run_runtime_assets().await {
+        Ok(changed) if changed => info!("Runtime assets synchronized from OTA payload"),
+        Ok(_) => debug!("No OTA runtime payload to synchronize"),
+        Err(e) => warn!("Runtime asset bootstrap failed (non-fatal): {:#}", e),
+    }
+    // Repair the narrowly recognized legacy NPM tunnel override before app
+    // reconciliation. The embedded script ships in both OTA and ISO binaries.
+    // It preserves native wallet services and refuses unknown custom routing.
+    match tokio::process::Command::new("python3")
+        .arg("-c")
+        .arg(include_str!("../../../scripts/repair-npm-tunnel.py"))
+        .output()
+        .await
+    {
+        Ok(output) if output.status.success() => {
+            if !output.stdout.is_empty() {
+                info!("{}", String::from_utf8_lossy(&output.stdout).trim());
+            }
+        }
+        Ok(output) => warn!(
+            "NPM tunnel migration needs attention: {}",
+            String::from_utf8_lossy(&output.stderr).trim()
+        ),
+        Err(error) => warn!("NPM tunnel migration could not run: {error}"),
+    }
+    match run_apps_dir_repair().await {
+        Ok(true) => {
+            info!("Populated /opt/archipelago/apps from installer copy at /etc/archipelago/apps")
+        }
+        Ok(false) => debug!("/opt/archipelago/apps already populated (or no installer copy)"),
+        Err(e) => warn!("Apps dir repair failed (non-fatal): {:#}", e),
+    }
+}
+
 /// Entry point called from main startup. Never returns an error to the caller —
 /// failing to bootstrap host artifacts must not prevent the backend from serving.
 pub async fn ensure_doctor_installed() {
@@ -146,11 +184,6 @@ pub async fn ensure_doctor_installed() {
         Ok(false) => debug!("No stale Archipelago dev-mode service override found"),
         Err(e) => warn!("Service override repair failed (non-fatal): {:#}", e),
     }
-    match run_runtime_assets().await {
-        Ok(changed) if changed => info!("Runtime assets synchronized from OTA payload"),
-        Ok(_) => debug!("No OTA runtime payload to synchronize"),
-        Err(e) => warn!("Runtime asset bootstrap failed (non-fatal): {:#}", e),
-    }
     match run().await {
         Ok(changed) if changed => info!("Doctor artifacts synchronized with binary"),
         Ok(_) => debug!("Doctor artifacts already in sync"),
@@ -168,13 +201,6 @@ pub async fn ensure_doctor_installed() {
         Ok(false) => debug!("No stale bitcoin.conf found"),
         Err(e) => warn!("Bitcoin RPC repair failed (non-fatal): {:#}", e),
     }
-    match run_apps_dir_repair().await {
-        Ok(true) => {
-            info!("Populated /opt/archipelago/apps from installer copy at /etc/archipelago/apps")
-        }
-        Ok(false) => debug!("/opt/archipelago/apps already populated (or no installer copy)"),
-        Err(e) => warn!("Apps dir repair failed (non-fatal): {:#}", e),
-    }
     match run_tor_helper_sync().await {
         Ok(true) => info!("tor-helper.sh synchronized with binary"),
         Ok(false) => debug!("tor-helper.sh already current"),

core/archipelago/src/container/app_catalog.rs

@@ -102,6 +102,31 @@ pub struct AppCatalogEntry {
     /// `docs/registry-manifest-design.md`.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub manifest: Option<serde_json::Value>,
+    /// Backward-compatible catalog rollout: old daemons ignore these and keep
+    /// the base manifest. New daemons choose only variants they can safely apply.
+    #[serde(default, skip_serializing_if = "Vec::is_empty")]
+    pub manifest_variants: Vec<CatalogManifestVariant>,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct CatalogManifestVariant {
+    pub requires: Vec<String>,
+    pub manifest: serde_json::Value,
+}
+
+fn selected_manifest(entry: AppCatalogEntry) -> Option<serde_json::Value> {
+    // Never let an unknown future requirement become an unsafe partial match.
+    for variant in entry.manifest_variants.into_iter().rev() {
+        if !variant.requires.is_empty()
+            && variant
+                .requires
+                .iter()
+                .all(|capability| capability == "runtime-migration-backup-v1")
+        {
+            return Some(variant.manifest);
+        }
+    }
+    entry.manifest
 }
 
 /// One selectable version in an app's `versions[]` list. The catalog carries a
@@ -234,7 +259,7 @@ pub fn catalog_manifest_values() -> Vec<(String, serde_json::Value)> {
     load_catalog()
         .apps
         .into_iter()
-        .filter_map(|(id, e)| e.manifest.map(|m| (id, m)))
+        .filter_map(|(id, e)| selected_manifest(e).map(|m| (id, m)))
         .collect()
 }
 
@@ -557,6 +582,32 @@ fn write_cache(data_dir: &Path, body: &str) -> anyhow::Result<bool> {
 mod tests {
     use super::*;
 
+    #[test]
+    fn catalog_migration_variant_is_compatible_with_old_and_future_daemons() {
+        let raw = serde_json::json!({
+            "version": "2.45.0", "manifest": {"app": {"id": "portainer", "container": {}}},
+            "manifest_variants": [{"requires": ["runtime-migration-backup-v1"],
+                "manifest": {"app": {"id": "portainer", "container": {"network": "slirp4netns"}, "backup_before_runtime_change": true}}}]
+        });
+        #[derive(Deserialize)]
+        struct OldEntry {
+            manifest: serde_json::Value,
+        }
+        let old: OldEntry = serde_json::from_value(raw.clone()).unwrap();
+        assert!(old.manifest["app"]["container"].get("network").is_none());
+        let current: AppCatalogEntry = serde_json::from_value(raw.clone()).unwrap();
+        let chosen = selected_manifest(current).unwrap();
+        assert_eq!(chosen["app"]["container"]["network"], "slirp4netns");
+        assert_eq!(chosen["app"]["backup_before_runtime_change"], true);
+        let mut future = raw;
+        future["manifest_variants"][0]["requires"]
+            .as_array_mut()
+            .unwrap()
+            .push(serde_json::json!("unknown-next-capability"));
+        let chosen = selected_manifest(serde_json::from_value(future).unwrap()).unwrap();
+        assert!(chosen["app"]["container"].get("network").is_none());
+    }
+
     #[test]
     fn parses_and_ignores_unknown_fields() {
         let json = r#"{

core/archipelago/src/container/companion.rs

@@ -103,6 +103,15 @@ pub fn companions_for(package_id: &str) -> &'static [CompanionSpec] {
     }
 }
 
+/// Missing companion UIs are provisioned here, never by snapshot recovery.
+/// A stale running-container snapshot must not resurrect an orphaned UI.
+pub fn is_companion_app(app_id: &str) -> bool {
+    ALL_COMPANIONS
+        .iter()
+        .flat_map(|specs| specs.iter())
+        .any(|spec| spec.image_base == app_id)
+}
+
 /// Every companion this build knows how to provision. Kept beside
 /// `companions_for` — a new companion must be added to both, or the reaper
 /// will not recognise it as one of ours and will leave it running forever.

core/archipelago/src/container/docker_packages.rs

@@ -3,8 +3,9 @@
 
 use anyhow::Result;
 use archipelago_container::{
-    ContainerRuntime as ContainerRuntimeTrait, ContainerState, PodmanClient,
+    ContainerRuntime as ContainerRuntimeTrait, ContainerState, ContainerStatus, PodmanClient,
 };
+use futures_util::StreamExt;
 use std::collections::HashMap;
 use std::sync::Arc;
 use tracing::{debug, info};
@@ -15,6 +16,18 @@ use crate::data_model::{
     PackageDataEntry, PackageState, ServiceStatus, StaticFiles,
 };
 
+/// One displayed package for each known container/manifest alias. Keep the
+/// stopped-app restoration path in agreement with live-container discovery.
+fn canonical_package_id(name: &str) -> &str {
+    match name.strip_prefix("archy-").unwrap_or(name) {
+        "immich_server" | "immich-server" => "immich",
+        "immich-postgres" => "immich_postgres",
+        "immich-redis" => "immich_redis",
+        "mempool-web" | "mempool-frontend" => "mempool",
+        name => name,
+    }
+}
+
 pub struct DockerPackageScanner {
     runtime: Arc<dyn ContainerRuntimeTrait>,
 }
@@ -25,8 +38,15 @@ impl DockerPackageScanner {
     }
 
     /// Scan Docker containers and convert to package data
-    pub async fn scan_containers(&self) -> Result<HashMap<String, PackageDataEntry>> {
-        let containers = self.runtime.list_containers().await?;
+    pub async fn scan_containers(
+        &self,
+        data_dir: &std::path::Path,
+        cached: &HashMap<String, PackageDataEntry>,
+    ) -> Result<HashMap<String, PackageDataEntry>> {
+        let mut containers = self.runtime.list_containers().await?;
+        let installed = crate::crash_recovery::load_installed_apps(data_dir).await;
+        let uninstalled = crate::crash_recovery::load_user_uninstalled(data_dir).await;
+        restore_absent_installed(&mut containers, &installed, &uninstalled);
 
         debug!("Found {} containers", containers.len());
 
@@ -91,24 +111,8 @@ impl DockerPackageScanner {
         debug!("Found {} UI containers", ui_containers.len());
 
         for container in containers {
-            // Extract app ID from container name
-            // Support both archy-* containers (docker-compose) and plain names (manual)
-            let app_id = if container.name.starts_with("archy-") {
-                container
-                    .name
-                    .strip_prefix("archy-")
-                    .unwrap_or(&container.name)
-                    .to_string()
-            } else {
-                // Use the container name as-is for manually started containers
-                container.name.clone()
-            };
-
-            // Normalize multi-container app IDs to their canonical names
-            let app_id = match app_id.as_str() {
-                "immich_server" => "immich".to_string(),
-                _ => app_id,
-            };
+            // Use the same alias mapping as stopped-app restoration.
+            let app_id = canonical_package_id(&container.name).to_owned();
 
             // Skip backend services (databases, APIs, etc.)
             if excluded_services.contains(&app_id.as_str()) {
@@ -139,6 +143,18 @@ impl DockerPackageScanner {
                 continue;
             }
 
+            if container.id.is_empty() {
+                if let Some(previous) = cached.get(&app_id) {
+                    let mut held = previous.clone();
+                    held.state = PackageState::Stopped;
+                    held.ui_ready = Some(false);
+                    held.health = None;
+                    held.exit_code = None;
+                    packages.insert(app_id.clone(), held);
+                    continue;
+                }
+            }
+
             // Get metadata for this app
             let metadata = get_app_metadata(&app_id);
             // Manifest-owned metadata (icon) wins over the static table: the
@@ -158,13 +174,11 @@ impl DockerPackageScanner {
             } else {
                 // Prefer the known web UI port over arbitrary first binding
                 // (for example Gitea exposes SSH on 2222 before web on 3001).
-                let candidate = if uses_allocated_launch_port(&app_id) {
-                    extract_lan_address(&container.ports)
-                        .or_else(|| PodmanClient::lan_address_for(&app_id))
-                } else {
-                    PodmanClient::lan_address_for(&app_id)
-                        .or_else(|| extract_lan_address(&container.ports))
-                };
+                let candidate = package_launch_candidate(
+                    &app_id,
+                    &container.ports,
+                    PodmanClient::lan_address_for(&app_id),
+                );
                 reachable_lan_address(&app_id, candidate).await
             };
 
@@ -179,14 +193,22 @@ impl DockerPackageScanner {
             let tor_address = read_tor_address(&app_id).await;
 
             // Extract actual version from container image tag
-            let running_version = image_versions::extract_version_from_image(&container.image);
+            let running_version = if container.id.is_empty() {
+                String::new() // Absence cannot establish the installed image version.
+            } else {
+                image_versions::extract_version_from_image(&container.image)
+            };
 
             // Decoupled from the binary OTA: prefer the remote app catalog,
             // falling back to the image-versions.sh pin when uncovered/offline.
-            let available_update =
-                crate::container::app_catalog::available_update_for_app(&app_id, &container.image);
+            let available_update = if container.id.is_empty() {
+                None
+            } else {
+                crate::container::app_catalog::available_update_for_app(&app_id, &container.image)
+            };
 
             let package = PackageDataEntry {
+                ui_ready: Some(false),
                 state: package_state.clone(),
                 health: container.health.clone(),
                 exit_code: if package_state == PackageState::Exited {
@@ -283,10 +305,250 @@ impl DockerPackageScanner {
             );
         }
 
+        let probes: Vec<_> = packages
+            .iter()
+            .filter_map(|(id, pkg)| {
+                if pkg.state != PackageState::Running {
+                    return None;
+                }
+                let url = pkg
+                    .installed
+                    .as_ref()?
+                    .interface_addresses
+                    .get("main")?
+                    .lan_address
+                    .clone()?;
+                Some((id.clone(), url))
+            })
+            .collect();
+        let mut results = futures_util::stream::iter(
+            probes
+                .into_iter()
+                .map(|(id, url)| async move { (id, launch_http_ready(&url).await) }),
+        )
+        .buffer_unordered(8);
+        while let Some((id, ready)) = results.next().await {
+            if let Some(pkg) = packages.get_mut(&id) {
+                pkg.ui_ready = Some(ready);
+            }
+        }
+        // HTTP on loopback can precede the LAN/Tor listener after install.
+        let port_map = crate::appgate::identity::build_port_map();
+        let gated: Vec<_> = packages
+            .iter()
+            .filter_map(|(id, pkg)| {
+                if pkg.ui_ready != Some(true) {
+                    return None;
+                }
+                let url = pkg
+                    .installed
+                    .as_ref()?
+                    .interface_addresses
+                    .get("main")?
+                    .lan_address
+                    .as_deref()?;
+                let port = launch_url_port(url)?;
+                port_map
+                    .gated(port)
+                    .filter(|gate| gate.declared)
+                    .map(|_| (id.clone(), port))
+            })
+            .collect();
+        if !gated.is_empty() {
+            use crate::appgate::listener::{port_claimed, refresh_now, shared_status};
+            let status = shared_status();
+            let needs_refresh = {
+                let current = status.read().await;
+                gated.iter().any(|(_, port)| !port_claimed(&current, *port))
+            };
+            if needs_refresh {
+                refresh_now().await;
+            }
+            let current = status.read().await;
+            for (id, port) in gated {
+                if !port_claimed(&current, port) {
+                    packages.get_mut(&id).unwrap().ui_ready = Some(false);
+                }
+            }
+        }
         Ok(packages)
     }
 }
 
+/// Quadlet removes containers during ordinary stops/restarts. Rebuild installed
+/// entries even on the daemon's first scan; a runtime absence is not uninstall.
+fn restore_absent_installed(
+    containers: &mut Vec<ContainerStatus>,
+    installed: &std::collections::HashSet<String>,
+    uninstalled: &std::collections::HashSet<String>,
+) {
+    let mut present: std::collections::HashSet<String> = containers
+        .iter()
+        .map(|c| canonical_package_id(&c.name).to_owned())
+        .collect();
+    let removed: std::collections::HashSet<_> = uninstalled
+        .iter()
+        .map(|id| canonical_package_id(id))
+        .collect();
+    for name in installed {
+        let id = canonical_package_id(name);
+        if removed.contains(id) || !present.insert(id.to_owned()) {
+            continue;
+        }
+        containers.push(ContainerStatus {
+            id: String::new(),
+            name: id.to_owned(),
+            state: ContainerState::Stopped,
+            health: None,
+            exit_code: None,
+            started_at: None,
+            image: String::new(),
+            created: String::new(),
+            ports: Vec::new(),
+            lan_address: None,
+        });
+    }
+}
+
+/// Probe the actual loopback upstream, not the app gate's login page. A bound
+/// TCP socket alone can still reset requests or serve a startup 503.
+async fn launch_http_ready(candidate: &str) -> bool {
+    let Ok(mut url) = reqwest::Url::parse(candidate) else {
+        return false;
+    };
+    if !matches!(url.scheme(), "http" | "https") {
+        return false;
+    }
+    if url.set_host(Some("127.0.0.1")).is_err() {
+        return false;
+    }
+    static CLIENT: std::sync::OnceLock<reqwest::Client> = std::sync::OnceLock::new();
+    let client = CLIENT.get_or_init(|| {
+        reqwest::Client::builder()
+            .no_proxy()
+            .timeout(std::time::Duration::from_secs(2))
+            .redirect(reqwest::redirect::Policy::none())
+            // Self-signed local app certificates are normal. This client only
+            // contacts loopback and never sends credentials or follows redirects.
+            .danger_accept_invalid_certs(true)
+            .build()
+            .expect("local readiness client")
+    });
+    match client.get(url).send().await {
+        Ok(response) => matches!(response.status().as_u16(), 200..=399 | 401 | 403),
+        Err(_) => false,
+    }
+}
+
+#[cfg(test)]
+mod lifecycle_regression_tests {
+    use super::*;
+    use tokio::io::{AsyncReadExt, AsyncWriteExt};
+
+    #[test]
+    fn immich_dependency_aliases_share_the_hidden_component_ids() {
+        for id in [
+            "immich-postgres",
+            "immich_postgres",
+            "archy-immich-postgres",
+        ] {
+            assert_eq!(canonical_package_id(id), "immich_postgres");
+        }
+        assert_eq!(canonical_package_id("immich-redis"), "immich_redis");
+        assert_eq!(canonical_package_id("immich-server"), "immich");
+    }
+
+    #[test]
+    fn registry_survives_empty_runtime_and_deduplicates_aliases() {
+        let installed = ["archy-gitea", "gitea", "immich_server", "archy-removed"]
+            .into_iter()
+            .map(str::to_owned)
+            .collect();
+        let removed = ["removed".to_owned()].into_iter().collect();
+        let mut containers = Vec::new();
+        restore_absent_installed(&mut containers, &installed, &removed);
+        assert_eq!(containers.len(), 2);
+        assert!(containers
+            .iter()
+            .all(|c| c.state == ContainerState::Stopped));
+        containers[0].state = ContainerState::Running;
+        restore_absent_installed(&mut containers, &installed, &removed);
+        assert_eq!(containers.len(), 2);
+        assert_eq!(containers[0].state, ContainerState::Running);
+    }
+
+    #[test]
+    fn mempool_frontend_inventory_alias_does_not_create_a_second_package() {
+        let installed = ["mempool", "archy-mempool-web", "mempool-web"]
+            .into_iter()
+            .map(str::to_owned)
+            .collect();
+        let mut containers = Vec::new();
+        restore_absent_installed(&mut containers, &installed, &Default::default());
+        assert_eq!(containers.len(), 1);
+        assert_eq!(containers[0].name, "mempool");
+        containers[0].id = "live-frontend".into();
+        containers[0].state = ContainerState::Running;
+        restore_absent_installed(&mut containers, &installed, &Default::default());
+        assert_eq!(containers.len(), 1);
+        assert_eq!(containers[0].id, "live-frontend");
+        assert_eq!(containers[0].state, ContainerState::Running);
+        assert_eq!(canonical_package_id("archy-mempool-web"), "mempool");
+        assert_eq!(canonical_package_id("mempool-api"), "mempool-api");
+        containers.clear();
+        restore_absent_installed(
+            &mut containers,
+            &installed,
+            &["mempool".into()].into_iter().collect(),
+        );
+        assert!(containers.is_empty());
+    }
+
+    #[tokio::test]
+    async fn readiness_rejects_startup_errors_and_accepts_auth_and_redirects() {
+        for (status, expected) in [
+            (200, true),
+            (302, true),
+            (401, true),
+            (403, true),
+            (404, false),
+            (500, false),
+            (502, false),
+            (503, false),
+        ] {
+            let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+            let port = listener.local_addr().unwrap().port();
+            let task = tokio::spawn(async move {
+                let (mut stream, _) = listener.accept().await.unwrap();
+                let mut buf = [0; 2048];
+                let n = stream.read(&mut buf).await.unwrap();
+                assert!(String::from_utf8_lossy(&buf[..n]).starts_with("GET /start HTTP/1.1"));
+                stream.write_all(format!("HTTP/1.1 {status} Test\r\nContent-Length: 0\r\nConnection: close\r\n\r\n").as_bytes()).await.unwrap();
+            });
+            assert_eq!(
+                launch_http_ready(&format!("http://localhost:{port}/start")).await,
+                expected,
+                "status {status}"
+            );
+            task.await.unwrap();
+        }
+    }
+
+    #[tokio::test]
+    async fn readiness_rejects_tcp_accept_without_http() {
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let port = listener.local_addr().unwrap().port();
+        let task = tokio::spawn(async move {
+            let (stream, _) = listener.accept().await.unwrap();
+            drop(stream);
+        });
+        assert!(!launch_http_ready(&format!("http://localhost:{port}/")).await);
+        task.await.unwrap();
+        assert!(!launch_http_ready(&format!("http://localhost:{port}/")).await);
+        assert!(!launch_http_ready("file:///tmp/test").await);
+    }
+}
+
 struct AppMetadata {
     title: String,
     description: String,
@@ -731,14 +993,20 @@ fn extract_lan_address(ports: &[String]) -> Option<String> {
     let mut first_candidate = None;
     for port_str in ports {
         // Parse port strings like "0.0.0.0:18443->18443/tcp" or "0.0.0.0:18443-18444->18443-18444/tcp"
-        let Some(public_part) = port_str.split("->").next() else {
+        let Some((public_part, _)) = port_str.split_once("->") else {
             continue;
         };
-        let Some(port_part) = public_part.split(':').nth(1) else {
+        let Some((_, port_part)) = public_part.rsplit_once(':') else {
             continue;
         };
         // Extract just the first port if it's a range (e.g., "18443-18444" -> "18443")
         let host_port = port_part.split('-').next().unwrap_or(port_part);
+        let Ok(host_port) = host_port.parse::<u16>() else {
+            continue;
+        };
+        if host_port == 0 {
+            continue;
+        }
         let candidate = format!("http://localhost:{}", host_port);
         if first_candidate.is_none() {
             first_candidate = Some(candidate.clone());
@@ -856,6 +1124,46 @@ fn companion_lan_address(app_id: &str) -> Option<String> {
     }
 }
 
+/// Companion dashboards remain usable while their backend is syncing. Never
+/// probe a Bitcoin RPC or Electrum protocol socket as dashboard readiness.
+fn package_launch_candidate(
+    app_id: &str,
+    ports: &[String],
+    known: Option<String>,
+) -> Option<String> {
+    if let Some(companion) = companion_lan_address(app_id) {
+        return Some(companion);
+    }
+    if app_id == "nginx-proxy-manager" {
+        // 80/443 serve users' proxy hosts; only container port 81 serves the
+        // admin UI. Podman's binding order is unstable across recreation.
+        // Resolve its actual host allocation rather than guessing the first
+        // HTTP port or hardcoding the default host port 8081.
+        let admin_ports: Vec<String> = ports
+            .iter()
+            .filter(|port| {
+                port.split_once("->")
+                    .is_some_and(|(_, target)| target == "81/tcp")
+            })
+            .cloned()
+            .collect();
+        // With published bindings, a missing admin mapping is not evidence
+        // that some unrelated service on the default host port is this UI.
+        return extract_lan_address(&admin_ports).or_else(|| {
+            if ports.is_empty() {
+                known
+            } else {
+                None
+            }
+        });
+    }
+    if uses_allocated_launch_port(app_id) {
+        extract_lan_address(ports).or(known)
+    } else {
+        known.or_else(|| extract_lan_address(ports))
+    }
+}
+
 fn uses_allocated_launch_port(app_id: &str) -> bool {
     matches!(
         app_id,
@@ -940,7 +1248,134 @@ mod tor_service_name_tests {
 
 #[cfg(test)]
 mod extract_lan_address_tests {
-    use super::extract_lan_address;
+    use super::{extract_lan_address, package_launch_candidate};
+
+    #[test]
+    fn companion_dashboard_wins_over_backend_protocol_ports() {
+        for id in ["bitcoin", "bitcoin-core", "bitcoin-knots"] {
+            assert_eq!(
+                package_launch_candidate(
+                    id,
+                    &["127.0.0.1:8332->8332/tcp".into()],
+                    Some("http://localhost:8332".into())
+                )
+                .as_deref(),
+                Some("http://localhost:8334")
+            );
+        }
+        for id in ["electrumx", "electrs", "mempool-electrs"] {
+            assert_eq!(
+                package_launch_candidate(
+                    id,
+                    &["127.0.0.1:50001->50001/tcp".into()],
+                    Some("http://localhost:50001".into())
+                )
+                .as_deref(),
+                Some("http://localhost:50002")
+            );
+        }
+        assert_eq!(
+            package_launch_candidate(
+                "filebrowser",
+                &["127.0.0.1:19080->80/tcp".into()],
+                Some("http://localhost:8080".into())
+            )
+            .as_deref(),
+            Some("http://localhost:19080")
+        );
+    }
+
+    #[test]
+    fn npm_admin_launch_is_independent_of_proxy_binding_order() {
+        let mappings = [
+            "10.77.0.2:18081->80/tcp",
+            "10.77.0.2:18443->443/tcp",
+            "127.0.0.1:8081->81/tcp",
+        ];
+        for order in [
+            [0, 1, 2],
+            [0, 2, 1],
+            [1, 0, 2],
+            [1, 2, 0],
+            [2, 0, 1],
+            [2, 1, 0],
+        ] {
+            let ports: Vec<String> = order.iter().map(|&i| mappings[i].into()).collect();
+            assert_eq!(
+                package_launch_candidate(
+                    "nginx-proxy-manager",
+                    &ports,
+                    Some("http://localhost:8081/".into())
+                )
+                .as_deref(),
+                Some("http://localhost:8081")
+            );
+        }
+    }
+
+    #[test]
+    fn npm_admin_launch_respects_host_allocation_and_ipv6_bindings() {
+        for binding in ["127.0.0.1", "0.0.0.0", "[::1]", "[::]"] {
+            let ports = vec![
+                "10.77.0.2:18081->80/tcp".into(),
+                format!("{binding}:28081->81/tcp"),
+            ];
+            assert_eq!(
+                package_launch_candidate(
+                    "nginx-proxy-manager",
+                    &ports,
+                    Some("http://localhost:8081/".into())
+                )
+                .as_deref(),
+                Some("http://localhost:28081")
+            );
+        }
+        let proxies = vec![
+            "10.77.0.2:18081->80/tcp".into(),
+            "10.77.0.2:18443->443/tcp".into(),
+        ];
+        assert_eq!(
+            package_launch_candidate("nginx-proxy-manager", &proxies, None),
+            None
+        );
+        assert_eq!(
+            package_launch_candidate(
+                "nginx-proxy-manager",
+                &proxies,
+                Some("http://localhost:8081/".into())
+            ),
+            None
+        );
+    }
+
+    #[test]
+    fn npm_without_port_information_uses_declared_admin_url() {
+        assert_eq!(
+            package_launch_candidate(
+                "nginx-proxy-manager",
+                &[],
+                Some("http://localhost:8081/".into())
+            )
+            .as_deref(),
+            Some("http://localhost:8081/")
+        );
+    }
+
+    #[test]
+    fn malformed_published_ports_do_not_become_launch_urls() {
+        for port in [
+            "81/tcp",
+            "127.0.0.1:bad->81/tcp",
+            "[::1]:0->81/tcp",
+            "[::]:65536->81/tcp",
+            "127.0.0.1:8081->81/udp",
+        ] {
+            assert_eq!(
+                package_launch_candidate("nginx-proxy-manager", &[port.into()], None),
+                None
+            );
+        }
+    }
 
     #[test]
     fn skips_ssh_port_when_web_port_is_published() {

core/archipelago/src/container/filebrowser.rs

@@ -5,7 +5,7 @@
 //! starting the container with `--config /data/.filebrowser.json`.
 
 use anyhow::{Context, Result};
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
 use tokio::fs;
 
 use crate::update::host_sudo;
@@ -117,6 +117,197 @@ fn shell_quote(s: &str) -> String {
     s.replace('\'', "'\\''")
 }
 
+/// Save a complete purchase without overwriting any existing directory entry.
+/// Both host and rootless-namespace paths publish with a no-clobber hard link.
+pub async fn save_new_file(dir: &Path, name: &str, bytes: &[u8]) -> Result<PathBuf> {
+    save_new_file_with(dir, name, bytes, write_via_userns).await
+}
+
+fn validate_filename(name: &str) -> Result<()> {
+    anyhow::ensure!(
+        !name.is_empty()
+            && name != "."
+            && name != ".."
+            && !name.contains(['/', '\\', '\0'])
+            && name.len() <= 255,
+        "Invalid purchased filename"
+    );
+    Ok(())
+}
+
+async fn save_new_file_with<F, Fut>(
+    dir: &Path,
+    name: &str,
+    bytes: &[u8],
+    fallback: F,
+) -> Result<PathBuf>
+where
+    F: FnOnce(PathBuf, String, Vec<u8>) -> Fut,
+    Fut: std::future::Future<Output = Result<PathBuf>>,
+{
+    validate_filename(name)?;
+    // Never follow a user-created destination directory symlink.
+    match fs::symlink_metadata(dir).await {
+        Ok(meta) => anyhow::ensure!(meta.is_dir(), "Files destination is not a directory"),
+        Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
+        Err(error) => return Err(error.into()),
+    }
+    save_after_direct_result(
+        write_direct(dir, name, bytes).await,
+        dir,
+        name,
+        bytes,
+        fallback,
+    )
+    .await
+}
+
+async fn save_after_direct_result<F, Fut>(
+    result: std::io::Result<PathBuf>,
+    dir: &Path,
+    name: &str,
+    bytes: &[u8],
+    fallback: F,
+) -> Result<PathBuf>
+where
+    F: FnOnce(PathBuf, String, Vec<u8>) -> Fut,
+    Fut: std::future::Future<Output = Result<PathBuf>>,
+{
+    match result {
+        Ok(path) => Ok(path),
+        Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => {
+            fallback(dir.to_owned(), name.to_owned(), bytes.to_vec())
+                .await
+                .context("Saving purchase in Files user namespace")
+        }
+        Err(error) => Err(error).context("Saving purchase in Files"),
+    }
+}
+
+fn numbered_name(name: &str, attempt: usize) -> String {
+    if attempt == 1 {
+        return name.to_owned();
+    }
+    match name.rsplit_once('.') {
+        Some((stem, extension)) if !stem.is_empty() => format!("{stem} ({attempt}).{extension}"),
+        _ => format!("{name} ({attempt})"),
+    }
+}
+
+struct PendingFile(PathBuf);
+impl Drop for PendingFile {
+    fn drop(&mut self) {
+        let _ = std::fs::remove_file(&self.0);
+    }
+}
+
+async fn write_direct(dir: &Path, name: &str, bytes: &[u8]) -> std::io::Result<PathBuf> {
+    use std::os::unix::fs::PermissionsExt;
+    use tokio::io::AsyncWriteExt;
+    fs::create_dir_all(dir).await?;
+    let temp_path = dir.join(format!(".archy-saving-{}", uuid::Uuid::new_v4()));
+    let mut file = fs::OpenOptions::new()
+        .write(true)
+        .create_new(true)
+        .mode(0o600)
+        .open(&temp_path)
+        .await?;
+    let temp = PendingFile(temp_path);
+    file.write_all(bytes).await?;
+    file.set_permissions(std::fs::Permissions::from_mode(0o644))
+        .await?;
+    file.sync_all().await?;
+    for attempt in 1..=100 {
+        let target = dir.join(numbered_name(name, attempt));
+        match fs::hard_link(&temp.0, &target).await {
+            Ok(()) => return Ok(target),
+            Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => continue,
+            Err(error) => return Err(error),
+        }
+    }
+    Err(std::io::Error::new(
+        std::io::ErrorKind::AlreadyExists,
+        "Too many existing copies; purchase cache retained",
+    ))
+}
+
+// Positional arguments carry all user-controlled text. mktemp prevents temp-name
+// collisions; ln -T refuses files, symlinks and directories, including races.
+const WRITE_VIA_USERNS: &str = r#"set -eu
+dir=$1
+name=$2
+expected=$3
+[ ! -L "$dir" ] || exit 1
+if [ ! -d "$dir" ]; then
+    mkdir -p -- "$dir"
+    chown --reference="$(dirname -- "$dir")" -- "$dir"
+fi
+tmp=$(mktemp "$dir/.archy-saving.XXXXXXXXXX")
+trap 'rm -f -- "$tmp"' EXIT HUP INT TERM
+cat > "$tmp"
+[ "$(wc -c < "$tmp")" -eq "$expected" ] || exit 1
+chown --reference="$dir" -- "$tmp"
+chmod 0644 -- "$tmp"
+sync -f -- "$tmp"
+stem=$name
+ext=
+case "$name" in
+    *.*) prefix=${name%.*}; if [ -n "$prefix" ]; then stem=$prefix; ext=.${name##*.}; fi ;;
+esac
+n=1
+while [ "$n" -le 100 ]; do
+    candidate=$name
+    if [ "$n" -gt 1 ]; then candidate="$stem ($n)$ext"; fi
+    dst="$dir/$candidate"
+    if ln -T -- "$tmp" "$dst" 2>/dev/null; then
+        printf '%s' "$candidate"
+        exit 0
+    fi
+    # A conflict may be a dangling symlink; never follow it or overwrite it.
+    if [ ! -e "$dst" ] && [ ! -L "$dst" ]; then exit 1; fi
+    n=$((n + 1))
+done
+exit 1
+"#;
+
+async fn write_via_userns(dir: PathBuf, name: String, bytes: Vec<u8>) -> Result<PathBuf> {
+    use tokio::io::AsyncWriteExt;
+    let mut child = tokio::process::Command::new("podman")
+        .args(["unshare", "sh", "-c", WRITE_VIA_USERNS, "sh"])
+        .arg(&dir)
+        .arg(&name)
+        .arg(bytes.len().to_string())
+        .kill_on_drop(true)
+        .stdin(std::process::Stdio::piped())
+        .stdout(std::process::Stdio::piped())
+        .stderr(std::process::Stdio::piped())
+        .spawn()
+        .context("Starting Files namespace writer")?;
+    let mut stdin = child.stdin.take().context("Files writer stdin missing")?;
+    let operation = async {
+        let fed = stdin.write_all(&bytes).await;
+        drop(stdin);
+        let output = child.wait_with_output().await?;
+        anyhow::ensure!(
+            output.status.success(),
+            "Files namespace writer failed: {}",
+            output.status
+        );
+        fed.context("Sending purchase bytes to Files")?;
+        let chosen =
+            String::from_utf8(output.stdout).context("Files writer returned an invalid name")?;
+        validate_filename(&chosen)?;
+        anyhow::ensure!(
+            (1..=100).any(|n| numbered_name(&name, n) == chosen),
+            "Files writer returned an unexpected name"
+        );
+        Ok(dir.join(chosen))
+    };
+    tokio::time::timeout(std::time::Duration::from_secs(120), operation)
+        .await
+        .context("Files namespace writer timed out")?
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;
@@ -152,3 +343,231 @@ mod tests {
         assert_eq!(second, EnsureOutcome::Unchanged);
     }
 }
+
+#[cfg(test)]
+mod purchase_write_tests {
+    use super::*;
+    use std::{
+        collections::HashSet,
+        os::unix::fs::{symlink, PermissionsExt},
+    };
+
+    fn no_temps(dir: &Path) {
+        assert!(std::fs::read_dir(dir).unwrap().all(|e| !e
+            .unwrap()
+            .file_name()
+            .to_string_lossy()
+            .starts_with(".archy-saving")));
+    }
+
+    #[tokio::test]
+    async fn direct_write_uses_complete_bytes_and_preserves_originals() {
+        let dir = tempfile::tempdir().unwrap();
+        fs::write(dir.path().join("song.mp3"), b"original")
+            .await
+            .unwrap();
+        let target = save_new_file(dir.path(), "song.mp3", b"new").await.unwrap();
+        assert_eq!(target.file_name().unwrap(), "song (2).mp3");
+        assert_eq!(fs::read(target).await.unwrap(), b"new");
+        assert_eq!(
+            fs::read(dir.path().join("song.mp3")).await.unwrap(),
+            b"original"
+        );
+        no_temps(dir.path());
+    }
+
+    #[tokio::test]
+    async fn simultaneous_saves_publish_unique_complete_files() {
+        let dir = tempfile::tempdir().unwrap();
+        let mut tasks = Vec::new();
+        for n in 0..24u8 {
+            let dir = dir.path().to_owned();
+            tasks.push(tokio::spawn(async move {
+                let bytes = vec![n; 32768];
+                let path = save_new_file(&dir, "same.bin", &bytes).await.unwrap();
+                assert_eq!(fs::read(&path).await.unwrap(), bytes);
+                path
+            }));
+        }
+        let mut paths = HashSet::new();
+        for task in tasks {
+            assert!(paths.insert(task.await.unwrap()));
+        }
+        assert_eq!(paths.len(), 24);
+        no_temps(dir.path());
+    }
+
+    #[tokio::test]
+    async fn existing_directories_and_dangling_symlinks_are_conflicts() {
+        let dir = tempfile::tempdir().unwrap();
+        fs::create_dir(dir.path().join("name")).await.unwrap();
+        symlink("missing", dir.path().join("name (2)")).unwrap();
+        let path = save_new_file(dir.path(), "name", b"new").await.unwrap();
+        assert_eq!(path.file_name().unwrap(), "name (3)");
+        assert!(dir.path().join("name").is_dir());
+        assert!(fs::symlink_metadata(dir.path().join("name (2)"))
+            .await
+            .unwrap()
+            .is_symlink());
+        no_temps(dir.path());
+    }
+
+    #[tokio::test]
+    async fn invalid_names_and_symlink_destination_are_refused() {
+        let dir = tempfile::tempdir().unwrap();
+        for name in [
+            "",
+            ".",
+            "..",
+            "../escape",
+            "/absolute",
+            "a/b",
+            "a\\b",
+            "a\0b",
+        ] {
+            assert!(save_new_file(dir.path(), name, b"bytes").await.is_err());
+        }
+        let outside = tempfile::tempdir().unwrap();
+        symlink(outside.path(), dir.path().join("Music")).unwrap();
+        assert!(save_new_file(&dir.path().join("Music"), "song", b"bytes")
+            .await
+            .is_err());
+        assert_eq!(std::fs::read_dir(outside.path()).unwrap().count(), 0);
+    }
+
+    #[tokio::test]
+    async fn collision_limit_preserves_all_files_and_cleans_temporary_data() {
+        let dir = tempfile::tempdir().unwrap();
+        for n in 1..=100 {
+            fs::write(dir.path().join(numbered_name("a.txt", n)), b"keep")
+                .await
+                .unwrap();
+        }
+        assert!(save_new_file(dir.path(), "a.txt", b"new").await.is_err());
+        for n in 1..=100 {
+            assert_eq!(
+                fs::read(dir.path().join(numbered_name("a.txt", n)))
+                    .await
+                    .unwrap(),
+                b"keep"
+            );
+        }
+        no_temps(dir.path());
+    }
+
+    #[tokio::test]
+    async fn permission_fallback_is_exercised_without_skipping_as_root() {
+        let dir = tempfile::tempdir().unwrap();
+        let result = save_after_direct_result(
+            Err(std::io::ErrorKind::PermissionDenied.into()),
+            dir.path(),
+            "a",
+            b"abc",
+            |dir, name, bytes| async move {
+                assert_eq!(bytes, b"abc");
+                Ok(dir.join(name))
+            },
+        )
+        .await
+        .unwrap();
+        assert_eq!(result, dir.path().join("a"));
+        assert!(save_after_direct_result(
+            Err(std::io::ErrorKind::PermissionDenied.into()),
+            dir.path(),
+            "a",
+            b"abc",
+            |_, _, _| async { anyhow::bail!("namespace unavailable") }
+        )
+        .await
+        .unwrap_err()
+        .to_string()
+        .contains("namespace"));
+        assert!(save_after_direct_result(
+            Err(std::io::ErrorKind::StorageFull.into()),
+            dir.path(),
+            "a",
+            b"abc",
+            |_, _, _| async { panic!("disk full must not trigger permission fallback") }
+        )
+        .await
+        .is_err());
+    }
+
+    async fn run_script(
+        dir: &Path,
+        name: &str,
+        bytes: &[u8],
+        expected: usize,
+    ) -> std::process::Output {
+        use tokio::io::AsyncWriteExt;
+        let mut child = tokio::process::Command::new("sh")
+            .args(["-c", WRITE_VIA_USERNS, "sh"])
+            .arg(dir)
+            .arg(name)
+            .arg(expected.to_string())
+            .stdin(std::process::Stdio::piped())
+            .stdout(std::process::Stdio::piped())
+            .stderr(std::process::Stdio::piped())
+            .spawn()
+            .unwrap();
+        let mut input = child.stdin.take().unwrap();
+        input.write_all(bytes).await.unwrap();
+        drop(input);
+        child.wait_with_output().await.unwrap()
+    }
+
+    #[tokio::test]
+    async fn namespace_script_preserves_names_bytes_modes_and_existing_entries() {
+        let dir = tempfile::tempdir().unwrap();
+        let folder = dir.path().join("Music");
+        let name = "song ' $() ; #.mp3";
+        for n in 1..=2 {
+            let output = run_script(&folder, name, b"abc", 3).await;
+            assert!(
+                output.status.success(),
+                "{}",
+                String::from_utf8_lossy(&output.stderr)
+            );
+            let chosen = String::from_utf8(output.stdout).unwrap();
+            assert_eq!(chosen, numbered_name(name, n));
+            let path = folder.join(chosen);
+            assert_eq!(fs::read(&path).await.unwrap(), b"abc");
+            assert_eq!(
+                fs::metadata(path).await.unwrap().permissions().mode() & 0o777,
+                0o644
+            );
+        }
+        no_temps(&folder);
+    }
+
+    #[tokio::test]
+    async fn namespace_script_refuses_truncated_input_and_cleans_up() {
+        let dir = tempfile::tempdir().unwrap();
+        let output = run_script(dir.path(), "never.bin", b"partial", 100).await;
+        assert!(!output.status.success());
+        assert!(!dir.path().join("never.bin").exists());
+        no_temps(dir.path());
+    }
+
+    #[tokio::test]
+    async fn namespace_script_does_not_link_inside_existing_directory() {
+        let dir = tempfile::tempdir().unwrap();
+        fs::create_dir(dir.path().join("name")).await.unwrap();
+        symlink("missing", dir.path().join("name (2)")).unwrap();
+        let output = run_script(dir.path(), "name", b"abc", 3).await;
+        assert!(output.status.success());
+        assert_eq!(output.stdout, b"name (3)");
+        assert_eq!(
+            std::fs::read_dir(dir.path().join("name")).unwrap().count(),
+            0
+        );
+        no_temps(dir.path());
+    }
+
+    #[test]
+    fn names_keep_extensions_and_dotfiles() {
+        assert_eq!(numbered_name("a.tar.gz", 2), "a.tar (2).gz");
+        assert_eq!(numbered_name(".hidden", 2), ".hidden (2)");
+        assert_eq!(numbered_name("README", 2), "README (2)");
+    }
+}

core/archipelago/src/container/migration_backup.rs

@@ -0,0 +1,254 @@
+//! Consistent, private snapshots for declaratively opted-in runtime migrations.
+use anyhow::{bail, Context, Result};
+use archipelago_container::AppManifest;
+use std::os::unix::fs::PermissionsExt;
+use std::path::{Path, PathBuf};
+
+pub fn enabled(manifest: &AppManifest) -> Result<bool> {
+    match manifest.app.extensions.get("backup_before_runtime_change") {
+        None => Ok(false),
+        Some(value) => value
+            .as_bool()
+            .context("backup_before_runtime_change must be boolean"),
+    }
+}
+
+fn relative_sources(manifest: &AppManifest, data_dir: &Path) -> Result<Vec<PathBuf>> {
+    let mut sources = Vec::new();
+    for volume in &manifest.app.volumes {
+        if volume.options.iter().any(|v| v == "ro") || volume.volume_type == "tmpfs" {
+            continue;
+        }
+        // A runtime socket is a connection, not application state.
+        if volume.source == "/run/user/1000/podman/podman.sock" {
+            continue;
+        }
+        if volume.volume_type != "bind" {
+            bail!("runtime migration backup requires bind-mounted persistent state");
+        }
+        let path = Path::new(&volume.source);
+        let relative = path
+            .strip_prefix(data_dir)
+            .context("runtime migration state must be inside the node data directory")?;
+        if relative.starts_with("migration-backups") {
+            bail!("migration backup cannot include its own archive directory");
+        }
+        if relative.as_os_str().is_empty()
+            || relative
+                .components()
+                .any(|c| !matches!(c, std::path::Component::Normal(_)))
+        {
+            bail!("invalid runtime migration state path");
+        }
+        sources.push(relative.to_path_buf());
+    }
+    sources.sort();
+    sources.dedup();
+    let mut roots: Vec<PathBuf> = Vec::new();
+    for source in sources {
+        if !roots.iter().any(|root| source.starts_with(root)) {
+            roots.push(source);
+        }
+    }
+    if roots.is_empty() {
+        bail!("runtime migration backup has no persistent state mounts");
+    }
+    Ok(roots)
+}
+
+/// Caller must gracefully stop the app before this function, and resume the old
+/// service if it fails. No source files are changed or deleted by this operation.
+pub async fn snapshot(
+    manifest: &AppManifest,
+    data_dir: &Path,
+    previous_unit: Option<&[u8]>,
+) -> Result<PathBuf> {
+    let mut command = tokio::process::Command::new("podman");
+    command.args(["unshare", "tar"]);
+    snapshot_with_command(manifest, data_dir, previous_unit, command).await
+}
+
+async fn snapshot_with_command(
+    manifest: &AppManifest,
+    data_dir: &Path,
+    previous_unit: Option<&[u8]>,
+    mut command: tokio::process::Command,
+) -> Result<PathBuf> {
+    let sources = relative_sources(manifest, data_dir)?;
+    let canonical_root = tokio::fs::canonicalize(data_dir).await?;
+    for source in &sources {
+        let path = data_dir.join(source);
+        if tokio::fs::symlink_metadata(&path)
+            .await?
+            .file_type()
+            .is_symlink()
+        {
+            bail!("runtime migration state mount is a symlink; explicit backup required");
+        }
+        let canonical = tokio::fs::canonicalize(&path).await?;
+        if !canonical.starts_with(&canonical_root) {
+            bail!("runtime migration state path resolves outside node data directory");
+        }
+    }
+    let root = data_dir.join("migration-backups");
+    tokio::fs::create_dir_all(&root).await?;
+    tokio::fs::set_permissions(&root, std::fs::Permissions::from_mode(0o700)).await?;
+    let dir = root.join(uuid::Uuid::new_v4().to_string());
+    tokio::fs::create_dir(&dir).await?;
+    tokio::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o700)).await?;
+    if let Some(unit) = previous_unit {
+        let path = dir.join("previous.container");
+        tokio::fs::write(&path, unit).await?;
+        tokio::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).await?;
+        tokio::fs::File::open(&path).await?.sync_all().await?;
+    }
+    let partial = dir.join("state.tar.partial");
+    let archive = dir.join("state.tar");
+    let output = command
+        .args([
+            "--create",
+            "--numeric-owner",
+            "--acls",
+            "--xattrs",
+            "--file",
+        ])
+        .arg(&partial)
+        .arg("--directory")
+        .arg(data_dir)
+        .arg("--")
+        .args(&sources)
+        .output()
+        .await
+        .context("start rootless migration snapshot")?;
+    if !output.status.success() {
+        // No tar stderr in public logs: it can contain private filenames.
+        let _ = tokio::fs::remove_file(&partial).await;
+        bail!("persistent-state snapshot failed; original state was left intact");
+    }
+    tokio::fs::set_permissions(&partial, std::fs::Permissions::from_mode(0o600)).await?;
+    tokio::fs::File::open(&partial).await?.sync_all().await?;
+    tokio::fs::rename(&partial, &archive).await?;
+    let metadata = serde_json::json!({"app": manifest.app.id, "version": manifest.app.version,
+        "network": manifest.app.container.network, "capabilities": manifest.app.security.capabilities, "sources": sources});
+    tokio::fs::write(
+        dir.join("metadata.json"),
+        serde_json::to_vec_pretty(&metadata)?,
+    )
+    .await?;
+    tokio::fs::File::open(&dir).await?.sync_all().await?;
+    Ok(archive)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    fn portainer() -> AppManifest {
+        AppManifest::parse(include_str!("../../../../apps/portainer/manifest.yml")).unwrap()
+    }
+    #[tokio::test]
+    async fn stopped_state_archive_round_trips_database_compose_and_old_unit() {
+        let dir = tempfile::tempdir().unwrap();
+        let state = dir.path().join("portainer");
+        tokio::fs::create_dir_all(state.join("compose"))
+            .await
+            .unwrap();
+        tokio::fs::write(state.join("portainer.db"), b"fixture database")
+            .await
+            .unwrap();
+        tokio::fs::write(state.join("compose/stack.yml"), b"services: {}\n")
+            .await
+            .unwrap();
+        let mut m = portainer();
+        m.app.volumes[0].source = state.display().to_string();
+        m.app.volumes[1].source = state.join("compose").display().to_string();
+        let archive = snapshot_with_command(
+            &m,
+            dir.path(),
+            Some(b"old unit"),
+            tokio::process::Command::new("tar"),
+        )
+        .await
+        .unwrap();
+        assert_eq!(
+            std::fs::metadata(&archive).unwrap().permissions().mode() & 0o777,
+            0o600
+        );
+        assert_eq!(
+            tokio::fs::read(archive.parent().unwrap().join("previous.container"))
+                .await
+                .unwrap(),
+            b"old unit"
+        );
+        let restored = tempfile::tempdir().unwrap();
+        assert!(tokio::process::Command::new("tar")
+            .arg("-xf")
+            .arg(archive)
+            .arg("-C")
+            .arg(restored.path())
+            .status()
+            .await
+            .unwrap()
+            .success());
+        assert_eq!(
+            tokio::fs::read(restored.path().join("portainer/portainer.db"))
+                .await
+                .unwrap(),
+            b"fixture database"
+        );
+        assert_eq!(
+            tokio::fs::read(restored.path().join("portainer/compose/stack.yml"))
+                .await
+                .unwrap(),
+            b"services: {}\n"
+        );
+        assert_eq!(
+            tokio::fs::read(state.join("portainer.db")).await.unwrap(),
+            b"fixture database"
+        );
+    }
+
+    #[tokio::test]
+    async fn failed_snapshot_never_publishes_archive_or_changes_original_state() {
+        let dir = tempfile::tempdir().unwrap();
+        let state = dir.path().join("portainer");
+        tokio::fs::create_dir_all(state.join("compose"))
+            .await
+            .unwrap();
+        tokio::fs::write(state.join("portainer.db"), b"unchanged")
+            .await
+            .unwrap();
+        let mut m = portainer();
+        m.app.volumes[0].source = state.display().to_string();
+        m.app.volumes[1].source = state.join("compose").display().to_string();
+        assert!(
+            snapshot_with_command(&m, dir.path(), None, tokio::process::Command::new("false"))
+                .await
+                .is_err()
+        );
+        assert_eq!(
+            tokio::fs::read(state.join("portainer.db")).await.unwrap(),
+            b"unchanged"
+        );
+        for entry in std::fs::read_dir(dir.path().join("migration-backups")).unwrap() {
+            assert!(!entry.unwrap().path().join("state.tar").exists());
+        }
+    }
+
+    #[test]
+    fn backup_covers_all_portainer_state_once_and_excludes_runtime_socket() {
+        let m = portainer();
+        assert!(enabled(&m).unwrap());
+        assert_eq!(
+            relative_sources(&m, Path::new("/var/lib/archipelago")).unwrap(),
+            vec![PathBuf::from("portainer")]
+        );
+    }
+    #[test]
+    fn backup_refuses_unknown_state_locations_instead_of_silently_omitting_them() {
+        let mut m = portainer();
+        m.app.volumes[0].source = "/other/operator/state".into();
+        assert!(relative_sources(&m, Path::new("/var/lib/archipelago")).is_err());
+        m.app.volumes[0].source = "/var/lib/archipelago/../secret".into();
+        assert!(relative_sources(&m, Path::new("/var/lib/archipelago")).is_err());
+    }
+}

core/archipelago/src/container/mod.rs

@@ -12,6 +12,7 @@ pub mod hooks;
 pub mod image_policy;
 pub mod image_versions;
 pub mod lnd;
+pub mod migration_backup;
 pub mod prod_orchestrator;
 pub mod quadlet;
 pub mod registry;

core/archipelago/src/container/prod_orchestrator.rs

@@ -36,6 +36,11 @@ use std::sync::Arc;
 use tokio::io::{AsyncReadExt, AsyncWriteExt};
 use tokio::sync::{Mutex, RwLock};
 
+/// Refusal before installation has created state or changed any dependency.
+#[derive(Debug, thiserror::Error)]
+#[error("{0}")]
+pub struct InstallPrerequisiteError(pub String);
+
 use crate::config::{Config, ContainerRuntime as ConfigContainerRuntime};
 use crate::container::bitcoin_ui;
 use crate::container::quadlet;
@@ -91,6 +96,23 @@ fn is_builtin_network_mode(network: &str) -> bool {
     )
 }
 
+// Only an explicitly selected rootless mode establishes drift. An omitted
+// network delegates to Podman and must not recreate unrelated installed apps.
+fn rootless_network_mode_drifted(expected: Option<&str>, actual: &str) -> bool {
+    matches!(expected, Some("slirp4netns" | "pasta"))
+        && !actual.trim().is_empty()
+        && actual.trim().split(':').next() != expected
+}
+
+fn missing_declared_capability(expected: &[String], actual: &[String]) -> bool {
+    expected.iter().any(|required| {
+        let required = required.strip_prefix("CAP_").unwrap_or(required);
+        !actual
+            .iter()
+            .any(|cap| cap.strip_prefix("CAP_").unwrap_or(cap) == required)
+    })
+}
+
 fn uses_pasta_network(manifest: &AppManifest) -> bool {
     manifest.app.container.network.as_deref() == Some("pasta")
 }
@@ -2049,6 +2071,10 @@ impl ProdContainerOrchestrator {
                 Ok(ReconcileAction::Left(reason))
                     if mode == ReconcileMode::ExistingOnly
                         && reason == "absent"
+                        // companion.rs owns missing UI provisioning/removal.
+                        // Never resurrect an orphan from a stale snapshot.
+                        // Existing UIs still pass through security config repair.
+                        && !super::companion::is_companion_app(&app_id)
                         && (was_running.contains(&compute_container_name(&lm.manifest))
                             // The durable answer, and the one that does not
                             // erode. `was_running` only records what was
@@ -2464,6 +2490,8 @@ impl ProdContainerOrchestrator {
                             .await
                         {
                             tracing::info!(app_id = %app_id, container = %name, "container published-port drift detected — recreating");
+                            self.backup_runtime_change(&name, &resolved_manifest)
+                                .await?;
                             let _ = self.runtime.stop_container(&name).await;
                             let _ = self.runtime.remove_container(&name).await;
                             self.install_fresh(lm).await?;
@@ -2499,6 +2527,8 @@ impl ProdContainerOrchestrator {
                                 return Ok(ReconcileAction::NoOp);
                             }
                             tracing::info!(app_id = %app_id, container = %name, "container env drift detected — recreating");
+                            self.backup_runtime_change(&name, &resolved_manifest)
+                                .await?;
                             let _ = self.runtime.stop_container(&name).await;
                             let _ = self.runtime.remove_container(&name).await;
                             self.install_fresh(lm).await?;
@@ -2555,6 +2585,8 @@ impl ProdContainerOrchestrator {
                                 .await
                         {
                             tracing::info!(app_id = %app_id, container = %name, "stopped container env/port drift detected — recreating");
+                            self.backup_runtime_change(&name, &resolved_manifest)
+                                .await?;
                             let _ = self.runtime.remove_container(&name).await;
                             self.install_fresh(lm).await?;
                             return Ok(ReconcileAction::Installed);
@@ -2611,6 +2643,8 @@ impl ProdContainerOrchestrator {
                         self.prepare_for_start(&resolved_manifest).await?;
                         if self.container_env_drifted(&name, &resolved_manifest).await {
                             tracing::info!(app_id = %app_id, container = %name, "created container env drift detected — recreating");
+                            self.backup_runtime_change(&name, &resolved_manifest)
+                                .await?;
                             let _ = self.runtime.remove_container(&name).await;
                             self.install_fresh(lm).await?;
                             return Ok(ReconcileAction::Installed);
@@ -3080,13 +3114,9 @@ impl ProdContainerOrchestrator {
     /// app is a companion (companion.rs owns those units), or when no
     /// unit file exists yet (install_via_quadlet handles first-write).
     ///
-    /// We DON'T restart the .service when content changes — running
-    /// containers keep their current config until an operator-initiated
-    /// restart picks up the new file. That's the right tradeoff: file
-    /// updates are cheap and non-destructive; service restarts are
-    /// destructive (the SIGKILL cascade we're trying to eliminate).
-    /// systemctl --user daemon-reload runs only when content actually
-    /// changed, so steady-state reconcile ticks pay just one fs read.
+    /// Ordinary metadata changes wait for an operator restart. Runtime-affecting
+    /// changes restart the service and retain a durable pending marker until
+    /// that succeeds, including across daemon restarts and failed reloads.
     async fn sync_quadlet_unit(&self, lm: &LoadedManifest, name: &str) -> Result<()> {
         // Companions: same reasoning as migrate_to_quadlet_if_needed —
         // companion.rs renders these units with a different shape, syncing
@@ -3106,7 +3136,7 @@ impl ProdContainerOrchestrator {
         }
         let old_body = tokio::fs::read_to_string(&unit_path)
             .await
-            .unwrap_or_default();
+            .with_context(|| format!("read existing quadlet for {name}"))?;
         let restart_required = quadlet::contains_stale_health_gate(&old_body);
 
         let mut resolved = lm.manifest.clone();
@@ -3122,49 +3152,54 @@ impl ProdContainerOrchestrator {
             quadlet::network_aliases_changed(&old_body, &new_body);
         let restart_for_exec_change = quadlet::exec_changed(&old_body, &new_body);
         let restart_for_health_change = quadlet::health_cmd_changed(&old_body, &new_body);
+        let restart_for_security_change = quadlet::security_changed(&old_body, &new_body);
+        let restart_for_managed_override =
+            quadlet::redundant_managed_network_override(&unit, &unit_dir)
+                .await?
+                .is_some();
+        let needs_restart = restart_required
+            || restart_for_managed_override
+            || restart_for_port_change
+            || restart_for_network_alias_change
+            || restart_for_exec_change
+            || restart_for_health_change
+            || restart_for_security_change;
+        // Record the obligation BEFORE replacing the unit. A failed reload or
+        // restart must not become a no-op on the next tick just because the
+        // generated file already matches the manifest.
+        let pending = quadlet::RestartObligation::prepare(&unit_path, needs_restart).await?;
+        if pending.is_pending() {
+            self.ensure_resolved_source_available(lm).await?;
+        }
+        if needs_restart {
+            self.backup_runtime_change(name, &resolved).await?;
+        }
         let changed = quadlet::write_if_changed(&unit, &unit_dir)
             .await
             .with_context(|| format!("drift-sync quadlet unit for {name}"))?;
-        if changed {
+        if changed || pending.is_pending() {
             quadlet::daemon_reload_user()
                 .await
                 .context("systemctl --user daemon-reload after drift-syncing quadlet unit")?;
-            tracing::info!(
-                app_id = %lm.manifest.app.id,
-                container = %name,
-                "Quadlet unit drift-synced — file rewritten, .service NOT restarted (operator restart picks up new config)"
-            );
         }
-        if changed
-            && (restart_required
-                || restart_for_port_change
-                || restart_for_network_alias_change
-                || restart_for_exec_change
-                || restart_for_health_change)
-        {
-            self.ensure_resolved_source_available(lm).await?;
+        if pending.is_pending() {
             let service = unit.service_name();
-            let reason = if restart_required {
-                "stale health gate"
-            } else if restart_for_port_change {
-                "port binding drift"
-            } else if restart_for_network_alias_change {
-                "network alias drift"
-            } else if restart_for_health_change {
-                "health command drift"
-            } else {
-                "exec drift"
-            };
             tracing::info!(
                 app_id = %lm.manifest.app.id,
                 container = %name,
                 service = %service,
-                reason = reason,
-                "Quadlet unit rewrite requires service restart"
+                "Applying pending Quadlet runtime change"
             );
             quadlet::restart_service(&service)
                 .await
                 .with_context(|| format!("restart drifted quadlet service {service}"))?;
+            pending.complete().await?;
+        } else if changed {
+            tracing::info!(
+                app_id = %lm.manifest.app.id,
+                container = %name,
+                "Quadlet metadata updated; operator restart will apply it"
+            );
         }
         Ok(())
     }
@@ -3483,11 +3518,9 @@ impl ProdContainerOrchestrator {
     }
 
     async fn cleanup_stale_grafana_port(&self) {
-        let _ = tokio::process::Command::new("pkill")
-            .args(["-f", "pasta.*3001"])
-            .output()
-            .await;
-        tokio::time::sleep(std::time::Duration::from_secs(1)).await;
+        // Port 3001 can belong to Gitea or the daemon's gate. Reap only a
+        // Grafana container proven absent from Podman's inventory.
+        crate::container::ghost_reaper::reap_for_app("grafana").await;
     }
 
     async fn detect_host_facts(&self) -> HostFacts {
@@ -3868,6 +3901,77 @@ impl ProdContainerOrchestrator {
         Ok(())
     }
 
+    async fn backup_runtime_change(&self, name: &str, manifest: &AppManifest) -> Result<()> {
+        if !crate::container::migration_backup::enabled(manifest)? {
+            return Ok(());
+        }
+        // A persistent disk/permission failure must not repeatedly stop a
+        // working old service. Reuse the reconciler's bounded repair budget.
+        if !self.should_attempt_repair(name).await {
+            anyhow::bail!("runtime migration retry budget exhausted; original service retained, inspect backup failure before retrying");
+        }
+        // Called only before a known runtime change. No app-specific commands;
+        // opted-in manifests identify their persistent state through bind mounts.
+        let output = tokio::process::Command::new("podman")
+            .args(["inspect", name, "--format", "{{.HostConfig.NetworkMode}}"])
+            .output()
+            .await
+            .context("inspect network before migration backup")?;
+        let present = if output.status.success() {
+            true
+        } else {
+            // A crash after gracefully stopping a --rm Quadlet container can
+            // leave only its data and old unit. Prove absence before snapshotting
+            // stopped state; an inspect/Podman failure is not proof of absence.
+            let exists = tokio::process::Command::new("podman")
+                .args(["container", "exists", name])
+                .status()
+                .await?;
+            if exists.code() != Some(1) {
+                anyhow::bail!("cannot verify existing container before runtime migration backup");
+            }
+            false
+        };
+        let service = format!("{name}.service");
+        let managed = quadlet::unit_exists(name).await;
+        let previous_unit = if managed {
+            Some(
+                tokio::fs::read(quadlet::unit_dir().await?.join(format!("{name}.container")))
+                    .await?,
+            )
+        } else {
+            None
+        };
+        if managed {
+            quadlet::stop_service(&service).await?;
+        } else if present {
+            self.runtime.stop_container(name).await?;
+        }
+        match crate::container::migration_backup::snapshot(
+            manifest,
+            &self.data_dir,
+            previous_unit.as_deref(),
+        )
+        .await
+        {
+            Ok(archive) => {
+                tracing::info!(container = %name, backup = %archive.display(), "Persistent state saved before runtime migration");
+                Ok(())
+            }
+            Err(error) => {
+                // The unit has not been rewritten yet. Restore its previous
+                // service on backup failure and report the migration failure.
+                let restored = if managed {
+                    quadlet::enable_now(&service).await
+                } else {
+                    self.runtime.start_container(name).await
+                };
+                restored.context("restore original app after failed migration snapshot")?;
+                Err(error)
+            }
+        }
+    }
+
     async fn container_env_drifted(&self, name: &str, manifest: &AppManifest) -> bool {
         if cfg!(test) {
             return false;
@@ -3877,6 +3981,52 @@ impl ProdContainerOrchestrator {
             return true;
         }
 
+        // Generated-unit drift handles managed services; preserve deliberate
+        // systemd drop-in overrides instead of recreating them every tick.
+        let unmanaged = !quadlet::unit_exists(name).await;
+        // Podman's effective bounding set, not Docker-compatible CapAdd (which
+        // can be empty even when Quadlet supplied capabilities).
+        if unmanaged && !manifest.app.security.capabilities.is_empty() {
+            if let Ok(output) = tokio::process::Command::new("podman")
+                .args(["inspect", name, "--format", "{{json .BoundingCaps}}"])
+                .output()
+                .await
+            {
+                if output.status.success() {
+                    if let Ok(actual) = serde_json::from_slice::<Vec<String>>(&output.stdout) {
+                        if missing_declared_capability(&manifest.app.security.capabilities, &actual)
+                        {
+                            return true;
+                        }
+                    }
+                }
+            }
+        }
+
+        // Quadlet handles declarative Network= drift above. Legacy rootless
+        // Podman containers need the same convergence when no unit owns them.
+        if unmanaged
+            && matches!(
+                manifest.app.container.network.as_deref(),
+                Some("slirp4netns" | "pasta")
+            )
+        {
+            if let Ok(output) = tokio::process::Command::new("podman")
+                .args(["inspect", name, "--format", "{{.HostConfig.NetworkMode}}"])
+                .output()
+                .await
+            {
+                if output.status.success()
+                    && rootless_network_mode_drifted(
+                        manifest.app.container.network.as_deref(),
+                        &String::from_utf8_lossy(&output.stdout),
+                    )
+                {
+                    return true;
+                }
+            }
+        }
+
         let inspect = tokio::process::Command::new("podman")
             .args([
                 "inspect",
@@ -4443,6 +4593,45 @@ impl ContainerOrchestrator for ProdContainerOrchestrator {
     }
 
     async fn install(&self, app_id: &str) -> Result<String> {
+        let lm = self.loaded(app_id).await?;
+        // Optional shared-service preconditions are checked before recording
+        // installation or creating anything. A headless adapter must not claim
+        // successful installation against a missing indexing stack.
+        if let Some(required) = lm
+            .manifest
+            .app
+            .extensions
+            .get("install_prerequisites")
+            .and_then(|value| value.as_sequence())
+        {
+            let present = self
+                .runtime
+                .list_containers()
+                .await
+                .context("check installed prerequisite services")?;
+            for id in required.iter().filter_map(|value| value.as_str()) {
+                let dependency = self.loaded(id).await.map_err(|_| InstallPrerequisiteError(
+                    format!("Required app {id} is unavailable. Refresh the app catalog before installing {}.",
+                        lm.manifest.app.name)))?;
+                let name = compute_container_name(&dependency.manifest);
+                if !present
+                    .iter()
+                    .any(|container| container.name.trim_start_matches('/') == name)
+                {
+                    let owner = crate::app_ops::owning_package(id);
+                    let title = self
+                        .loaded(owner)
+                        .await
+                        .map(|app| app.manifest.app.name)
+                        .unwrap_or(dependency.manifest.app.name);
+                    return Err(InstallPrerequisiteError(format!(
+                        "Install {title} first, then install {}.",
+                        lm.manifest.app.name
+                    ))
+                    .into());
+                }
+            }
+        }
         {
             let mut state = self.state.write().await;
             state.disabled.remove(app_id);
@@ -4469,7 +4658,6 @@ impl ContainerOrchestrator for ProdContainerOrchestrator {
         // health verification (the .228 "running but unreachable" failure
         // mode). Routing every install through here means the orchestrator
         // is the one source of truth for what "installed" means.
-        let lm = self.loaded(app_id).await?;
         let name = compute_container_name(&lm.manifest);
         // ensure_running takes the per-app lock itself; release the install
         // path lock first if we hold one (we don't — install is the entry
@@ -4698,6 +4886,26 @@ impl ContainerOrchestrator for ProdContainerOrchestrator {
     /// here (production volumes live under `/var/lib/archipelago` — removal is a
     /// separate operation owned by the data layer, not this orchestrator).
     async fn remove(&self, app_id: &str, _preserve_data: bool) -> Result<()> {
+        // A removed catalog entry must remain uninstallable. The RPC caller
+        // still removes legacy containers and persists uninstall intent after
+        // confirming they are gone; do not block it on a missing manifest.
+        if !self.state.read().await.manifests.contains_key(app_id) {
+            anyhow::ensure!(
+                !app_id.is_empty()
+                    && app_id.len() <= 128
+                    && app_id
+                        .bytes()
+                        .all(|c| c.is_ascii_alphanumeric() || matches!(c, b'-' | b'_')),
+                "Invalid app id"
+            );
+            let lock = self.app_lock(app_id).await;
+            let _guard = lock.lock().await;
+            for name in [app_id.to_string(), format!("archy-{app_id}")] {
+                self.remove_quadlet_unit_if_present(&name).await?;
+            }
+            self.state.write().await.disabled.insert(app_id.to_string());
+            return Ok(());
+        }
         let lm = self.loaded(app_id).await?;
         let lock = self.app_lock(app_id).await;
         let _guard = lock.lock().await;
@@ -4919,6 +5127,78 @@ mod tests {
     /// recovered when its siblings have live containers (the stack is
     /// installed), and left alone when the whole stack is gone or the app
     /// is not a stack member at all.
+    #[tokio::test]
+    async fn gitea_fresh_url_seed_preserves_operator_config_and_reports_write_failure() {
+        let manifest =
+            AppManifest::parse(include_str!("../../../../apps/gitea/manifest.yml")).unwrap();
+        let seed = &manifest.app.files[0];
+        assert!(!seed.overwrite);
+        let content = seed.content.replace("{{HOST_IP}}", "192.0.2.1");
+        assert!(content.contains("ROOT_URL = http://192.0.2.1:3001/"));
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("fresh/app.ini");
+        assert_eq!(
+            ensure_rendered_file(path.to_str().unwrap(), &content, seed.overwrite)
+                .await
+                .unwrap(),
+            HookOutcome::Rewritten
+        );
+        assert!(tokio::fs::read_to_string(&path)
+            .await
+            .unwrap()
+            .contains("ROOT_URL"));
+        let custom =
+            "[server]\nROOT_URL = https://git.example.test/\n[database]\nDB_TYPE = postgres\n";
+        tokio::fs::write(&path, custom).await.unwrap();
+        assert_eq!(
+            ensure_rendered_file(path.to_str().unwrap(), &content, seed.overwrite)
+                .await
+                .unwrap(),
+            HookOutcome::Unchanged
+        );
+        assert_eq!(tokio::fs::read_to_string(&path).await.unwrap(), custom);
+        let impossible = path.join("app.ini");
+        assert!(
+            ensure_rendered_file(impossible.to_str().unwrap(), &content, seed.overwrite)
+                .await
+                .is_err()
+        );
+    }
+
+    #[test]
+    fn ssh_sandbox_capability_repair_uses_bounding_set_and_preserves_extra_overrides() {
+        let required = vec!["CHOWN".into(), "SYS_CHROOT".into()];
+        assert!(missing_declared_capability(
+            &required,
+            &["CAP_CHOWN".into()]
+        ));
+        assert!(!missing_declared_capability(
+            &required,
+            &["CAP_CHOWN".into(), "CAP_SYS_CHROOT".into()]
+        ));
+        assert!(!missing_declared_capability(
+            &required,
+            &["CHOWN".into(), "SYS_CHROOT".into(), "CAP_KILL".into()]
+        ));
+    }
+
+    #[test]
+    fn explicit_rootless_network_change_converges_without_guessing_defaults() {
+        assert!(rootless_network_mode_drifted(Some("slirp4netns"), "pasta"));
+        assert!(rootless_network_mode_drifted(Some("slirp4netns"), "bridge"));
+        assert!(!rootless_network_mode_drifted(
+            Some("slirp4netns"),
+            "slirp4netns"
+        ));
+        assert!(!rootless_network_mode_drifted(
+            Some("slirp4netns"),
+            "slirp4netns:allow_host_loopback=true"
+        ));
+        assert!(!rootless_network_mode_drifted(None, "pasta"));
+        assert!(!rootless_network_mode_drifted(Some("slirp4netns"), ""));
+        assert!(!rootless_network_mode_drifted(Some("archy-net"), "bridge"));
+    }
+
     #[test]
     fn absent_stack_member_recovery_requires_a_live_sibling() {
         let present: HashSet<String> = ["indeedhub-redis", "indeedhub-relay", "indeedhub"]
@@ -5600,6 +5880,38 @@ app:
         orch
     }
 
+    #[tokio::test]
+    async fn missing_install_prerequisite_refuses_without_inventory_or_container_mutation() {
+        let rt = Arc::new(MockRuntime::default());
+        let orch = orch_with(rt.clone()).await;
+        let mut app = pull_manifest("indexer-adapter", "docker.io/library/alpine:3.20");
+        app.app.extensions.insert(
+            "install_prerequisites".into(),
+            serde_yaml::to_value(vec!["shared-index"]).unwrap(),
+        );
+        orch.insert_manifest_for_test(app, PathBuf::from("/tmp"))
+            .await;
+        orch.insert_manifest_for_test(
+            pull_manifest("shared-index", "index:1"),
+            PathBuf::from("/tmp"),
+        )
+        .await;
+        let error = orch.install("indexer-adapter").await.unwrap_err();
+        assert!(error.downcast_ref::<InstallPrerequisiteError>().is_some());
+        assert!(!crate::crash_recovery::load_installed_apps(&orch.data_dir)
+            .await
+            .contains("indexer-adapter"));
+        assert_eq!(rt.calls(), vec!["list_containers"]);
+        // An installed prerequisite satisfies the guard; it is never recreated
+        // or reconfigured as part of installing this adapter.
+        rt.set_state("shared-index", ContainerState::Running);
+        orch.install("indexer-adapter").await.unwrap();
+        assert!(!rt
+            .calls()
+            .iter()
+            .any(|c| c.starts_with("create_container:shared-index")));
+    }
+
     fn pull_manifest_with_dynamic_env(id: &str, image: &str) -> AppManifest {
         let yaml = format!(
             "app:\n  id: {id}\n  name: {id}\n  version: 1.0.0\n  container:\n    image: {image}\n    derived_env:\n      - key: FM_API_URL\n        template: \"ws://{{{{HOST_MDNS}}}}:8174\"\n    secret_env:\n      - key: FM_BITCOIND_PASSWORD\n        secret_file: bitcoin-rpc-password\n  environment:\n    - STATIC=1\n"
@@ -6942,6 +7254,52 @@ app:
         assert!(!calls.iter().any(|c| c.starts_with("start_container:")));
     }
 
+    #[tokio::test]
+    async fn reconcile_existing_does_not_resurrect_orphaned_companions() {
+        let rt = Arc::new(MockRuntime::default());
+        let mut orch = orch_with(rt.clone()).await;
+        orch.set_disk_gb_for_test(500);
+        let companions = [
+            "bitcoin-ui",
+            "electrs-ui",
+            "lnd-ui",
+            "fedimint-ui",
+            "cuprate-ui",
+        ];
+        let mut names = Vec::new();
+        for id in companions {
+            let manifest = pull_manifest(id, "localhost/companion:local");
+            names.push(compute_container_name(&manifest));
+            orch.insert_manifest_for_test(manifest, PathBuf::from("/tmp/companion"))
+                .await;
+        }
+        let refs: Vec<&str> = names.iter().map(String::as_str).collect();
+        crate::crash_recovery::save_container_snapshot_for_test(&orch.data_dir, &refs).await;
+        // Repeated passes must leave lifecycle ownership with companion.rs.
+        for _ in 0..3 {
+            let report = orch.reconcile_existing().await;
+            assert_eq!(report.actions.len(), companions.len());
+            assert!(report
+                .actions
+                .iter()
+                .all(|(_, action)| *action == ReconcileAction::Left("absent".into())));
+            assert!(report.failures.is_empty());
+        }
+        let calls = rt.calls();
+        for operation in [
+            "pull_image:",
+            "create_container:",
+            "start_container:",
+            "stop_container:",
+            "remove_container:",
+        ] {
+            assert!(
+                !calls.iter().any(|call| call.starts_with(operation)),
+                "{calls:?}"
+            );
+        }
+    }
+
     #[tokio::test]
     async fn reconcile_existing_self_heals_missing_optional_installed_app() {
         // A non-baseline app (gitea) self-heals ONLY with installation
@@ -7102,6 +7460,19 @@ app:
         );
     }
 
+    #[tokio::test]
+    async fn removed_catalog_entry_does_not_block_legacy_uninstall() {
+        let rt = Arc::new(MockRuntime::default());
+        let orch = orch_with(rt.clone()).await;
+        orch.remove("cryptpad", true).await.unwrap();
+        assert!(orch.state.read().await.disabled.contains("cryptpad"));
+        assert!(
+            rt.calls().is_empty(),
+            "legacy RPC teardown owns the actual containers"
+        );
+        assert!(orch.remove("../other", true).await.is_err());
+    }
+
     #[tokio::test]
     async fn remove_disables_manifest_so_reconcile_does_not_reinstall() {
         let rt = Arc::new(MockRuntime::default());

core/archipelago/src/container/quadlet.rs

@@ -390,7 +390,10 @@ fn shell_join(parts: &[String]) -> String {
         .iter()
         .map(|p| {
             let p = p.replace(['\r', '\n'], " ").replace('%', "%%");
-            if p.is_empty() || p.chars().any(|c| c.is_whitespace() || "\"\\$`".contains(c)) {
+            if p.is_empty()
+                || p.chars()
+                    .any(|c| c.is_whitespace() || "'\"\\$`".contains(c))
+            {
                 let escaped = p
                     .replace('\\', "\\\\")
                     .replace('"', "\\\"")
@@ -410,7 +413,7 @@ fn quote_environment(env: &str) -> String {
     if env.is_empty()
         || env
             .chars()
-            .any(|c| c.is_whitespace() || "\"\\$`".contains(c))
+            .any(|c| c.is_whitespace() || "'\"\\$`".contains(c))
     {
         let escaped = env
             .replace('\\', "\\\\")
@@ -710,29 +713,76 @@ pub async fn unit_dir() -> Result<PathBuf> {
     Ok(dir)
 }
 
-/// Atomically write `unit` into `dir/<name>.container` if the bytes
-/// differ from what's already there. Returns true if the file changed.
+/// The early same-node Portainer repair used a managed Quadlet drop-in. Once
+/// the manifest supplies slirp, the two Network= entries are additive and
+/// Podman rejects startup. Retire only that exact redundant managed override;
+/// arbitrary operator settings must survive reconciliation.
+pub async fn redundant_managed_network_override(
+    unit: &QuadletUnit,
+    dir: &Path,
+) -> Result<Option<PathBuf>> {
+    if unit.name != "portainer" || !matches!(unit.network, NetworkMode::Slirp4netns) {
+        return Ok(None);
+    }
+    let path = dir.join("portainer.container.d/archy-same-node-network.conf");
+    let body = match fs::read_to_string(&path).await {
+        Ok(body) => body,
+        Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
+        Err(error) => return Err(error).context("read managed Portainer network override"),
+    };
+    let lines: Vec<&str> = body
+        .lines()
+        .map(str::trim)
+        .filter(|line| !line.is_empty() && !line.starts_with(['#', ';']))
+        .collect();
+    Ok((lines == ["[Container]", "Network=slirp4netns"]).then_some(path))
+}
+
+async fn retire_managed_network_override(path: &Path) -> Result<()> {
+    let backup = path.with_extension("conf.retired");
+    match fs::hard_link(path, &backup).await {
+        Ok(()) => {}
+        Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => {
+            anyhow::ensure!(
+                fs::read(path).await? == fs::read(&backup).await?,
+                "Existing Portainer override backup differs; preserve both for operator review"
+            );
+        }
+        Err(error) => return Err(error).context("back up managed Portainer network override"),
+    }
+    fs::remove_file(path)
+        .await
+        .context("retire redundant Portainer network override")?;
+    tracing::info!("Retired redundant managed Portainer network override; backup retained");
+    Ok(())
+}
+
+/// Atomically write the manifest unit and retire known redundant managed
+/// overrides. Returns true whenever systemd needs a daemon-reload.
 pub async fn write_if_changed(unit: &QuadletUnit, dir: &Path) -> Result<bool> {
     let path = dir.join(unit.unit_filename());
     let new_bytes = unit.render();
-
-    if let Ok(old) = fs::read_to_string(&path).await {
-        if old == new_bytes {
-            return Ok(false);
-        }
+    let redundant = redundant_managed_network_override(unit, dir).await?;
+    let changed = fs::read_to_string(&path)
+        .await
+        .map(|old| old != new_bytes)
+        .unwrap_or(true);
+    if changed {
+        fs::create_dir_all(dir)
+            .await
+            .with_context(|| format!("create_dir_all {}", dir.display()))?;
+        let tmp = path.with_extension("container.tmp");
+        fs::write(&tmp, new_bytes.as_bytes())
+            .await
+            .with_context(|| format!("write tmp {}", tmp.display()))?;
+        fs::rename(&tmp, &path)
+            .await
+            .with_context(|| format!("rename {} -> {}", tmp.display(), path.display()))?;
     }
-
-    fs::create_dir_all(dir)
-        .await
-        .with_context(|| format!("create_dir_all {}", dir.display()))?;
-    let tmp = path.with_extension("container.tmp");
-    fs::write(&tmp, new_bytes.as_bytes())
-        .await
-        .with_context(|| format!("write tmp {}", tmp.display()))?;
-    fs::rename(&tmp, &path)
-        .await
-        .with_context(|| format!("rename {} -> {}", tmp.display(), path.display()))?;
-    Ok(true)
+    if let Some(override_path) = &redundant {
+        retire_managed_network_override(override_path).await?;
+    }
+    Ok(changed || redundant.is_some())
 }
 
 /// Reload the user systemd manager. Required after any quadlet write
@@ -938,12 +988,77 @@ pub fn health_cmd_changed(old_body: &str, new_body: &str) -> bool {
             != directive_values(new_body, "HealthRetries=")
 }
 
+/// A unit rewrite and a successful systemd restart are separate operations.
+/// Keep the restart obligation across errors or a management-daemon restart.
+pub struct RestartObligation {
+    marker: PathBuf,
+    pending: bool,
+}
+
+impl RestartObligation {
+    pub async fn prepare(unit_path: &Path, newly_required: bool) -> Result<Self> {
+        let marker = unit_path.with_extension("restart-pending");
+        if newly_required {
+            // Contents contain no manifest environment or credentials. sync_all
+            // makes the obligation durable before the subsequent unit rename.
+            let file = tokio::fs::OpenOptions::new()
+                .write(true)
+                .create(true)
+                .truncate(false)
+                .open(&marker)
+                .await
+                .context("record pending Quadlet restart")?;
+            file.sync_all().await?;
+            if let Some(parent) = marker.parent() {
+                tokio::fs::File::open(parent).await?.sync_all().await?;
+            }
+        }
+        let pending = tokio::fs::try_exists(&marker).await?;
+        Ok(Self { marker, pending })
+    }
+
+    pub fn is_pending(&self) -> bool {
+        self.pending
+    }
+
+    /// Call only after systemd accepted the replacement service successfully.
+    pub async fn complete(self) -> Result<()> {
+        if self.pending {
+            tokio::fs::remove_file(&self.marker)
+                .await
+                .context("clear completed Quadlet restart")?;
+            if let Some(parent) = self.marker.parent() {
+                tokio::fs::File::open(parent).await?.sync_all().await?;
+            }
+        }
+        Ok(())
+    }
+}
+
 pub fn publish_ports_changed(old_body: &str, new_body: &str) -> bool {
     let old_ports = directive_values(old_body, "PublishPort=");
     let new_ports = directive_values(new_body, "PublishPort=");
     old_ports != new_ports
 }
 
+pub fn security_changed(old_body: &str, new_body: &str) -> bool {
+    [
+        "AddCapability=",
+        "DropCapability=",
+        "NoNewPrivileges=",
+        "ReadOnly=",
+        "User=",
+    ]
+    .iter()
+    .any(|directive| {
+        let mut old = directive_values(old_body, directive);
+        let mut new = directive_values(new_body, directive);
+        old.sort();
+        new.sort();
+        old != new
+    })
+}
+
 pub fn network_aliases_changed(old_body: &str, new_body: &str) -> bool {
     let old_network = directive_values(old_body, "Network=");
     let new_network = directive_values(new_body, "Network=");
@@ -1329,6 +1444,18 @@ app:
         );
     }
 
+    #[test]
+    fn apostrophes_survive_quadlet_argument_and_environment_parsing() {
+        // A whitespace-free Node script reproduced this in a real Quadlet:
+        // unquoted apostrophes were consumed by the parser, changing JS strings
+        // into identifiers and preventing the app from starting.
+        assert_eq!(
+            shell_join(&["require('http')".into()]),
+            "\"require('http')\""
+        );
+        assert_eq!(quote_environment("NAME=O'Brien"), "\"NAME=O'Brien\"");
+    }
+
     #[test]
     fn quote_environment_quotes_values_with_spaces() {
         assert_eq!(
@@ -1541,6 +1668,26 @@ app:
         assert!(!s.contains("Network=host"));
     }
 
+    #[test]
+    fn portainer_catalog_network_repairs_same_node_routing_without_exposing_backend() {
+        let manifest = AppManifest::parse(include_str!("../../../../apps/portainer/manifest.yml"))
+            .expect("shipped Portainer manifest must parse");
+        let new = QuadletUnit::from_manifest(&manifest, "portainer").render();
+        assert!(new.contains("Network=slirp4netns\n"));
+        assert!(!new.contains("NetworkAlias="));
+        assert!(new.contains("PublishPort=127.0.0.1:9000:9000/tcp"));
+        assert!(!new.contains("PublishPort=0.0.0.0"));
+        // The upgrade changes networking only: retain both state mounts and the
+        // existing rootless socket, without an app.ini or repository rewrite.
+        assert!(new.contains("Volume=/var/lib/archipelago/portainer:/data"));
+        assert!(new.contains("Volume=/var/lib/archipelago/portainer/compose:/data/compose"));
+        assert!(new.contains("Volume=/run/user/1000/podman/podman.sock:/var/run/docker.sock"));
+        let old = new.replace("Network=slirp4netns\n", "");
+        assert!(network_aliases_changed(&old, &new));
+        assert!(!network_aliases_changed(&new, &new));
+        assert!(!publish_ports_changed(&old, &new));
+    }
+
     #[test]
     fn from_manifest_slirp4netns_omits_network_alias() {
         let yaml = r#"
@@ -1891,6 +2038,133 @@ app:
         assert!(!network_aliases_changed(new, new));
     }
 
+    #[tokio::test]
+    async fn redundant_portainer_override_is_backed_up_and_retired_even_when_base_matches() {
+        let dir = tempfile::tempdir().unwrap();
+        let manifest =
+            AppManifest::parse(include_str!("../../../../apps/portainer/manifest.yml")).unwrap();
+        let unit = QuadletUnit::from_manifest(&manifest, "portainer");
+        assert!(write_if_changed(&unit, dir.path()).await.unwrap());
+        let path = dir
+            .path()
+            .join("portainer.container.d/archy-same-node-network.conf");
+        fs::create_dir_all(path.parent().unwrap()).await.unwrap();
+        let old = "[Container]\nNetwork=slirp4netns\n";
+        fs::write(&path, old).await.unwrap();
+        assert!(redundant_managed_network_override(&unit, dir.path())
+            .await
+            .unwrap()
+            .is_some());
+        assert!(write_if_changed(&unit, dir.path()).await.unwrap());
+        assert!(!path.exists());
+        assert_eq!(
+            fs::read_to_string(path.with_extension("conf.retired"))
+                .await
+                .unwrap(),
+            old
+        );
+        assert!(!write_if_changed(&unit, dir.path()).await.unwrap());
+        assert_eq!(
+            fs::read_to_string(dir.path().join("portainer.container"))
+                .await
+                .unwrap()
+                .matches("Network=slirp4netns")
+                .count(),
+            1
+        );
+    }
+
+    #[tokio::test]
+    async fn network_override_migration_preserves_operator_customizations_and_failed_backups() {
+        let dir = tempfile::tempdir().unwrap();
+        let manifest =
+            AppManifest::parse(include_str!("../../../../apps/portainer/manifest.yml")).unwrap();
+        let mut unit = QuadletUnit::from_manifest(&manifest, "portainer");
+        let path = dir
+            .path()
+            .join("portainer.container.d/archy-same-node-network.conf");
+        fs::create_dir_all(path.parent().unwrap()).await.unwrap();
+        for custom in [
+            "[Container]\nNetwork=custom-net\n",
+            "[Container]\nNetwork=slirp4netns\nEnvironment=OPERATOR_SETTING=1\n",
+        ] {
+            fs::write(&path, custom).await.unwrap();
+            assert!(redundant_managed_network_override(&unit, dir.path())
+                .await
+                .unwrap()
+                .is_none());
+            write_if_changed(&unit, dir.path()).await.unwrap();
+            assert_eq!(fs::read_to_string(&path).await.unwrap(), custom);
+        }
+        fs::write(&path, "[Container]\nNetwork=slirp4netns\n")
+            .await
+            .unwrap();
+        unit.network = NetworkMode::Pasta;
+        assert!(redundant_managed_network_override(&unit, dir.path())
+            .await
+            .unwrap()
+            .is_none());
+        unit.network = NetworkMode::Slirp4netns;
+        fs::write(path.with_extension("conf.retired"), "different backup")
+            .await
+            .unwrap();
+        assert!(write_if_changed(&unit, dir.path()).await.is_err());
+        assert!(path.exists(), "failure must preserve the active override");
+    }
+
+    #[tokio::test]
+    async fn failed_runtime_change_remains_pending_when_unit_already_matches() {
+        let dir = tempfile::tempdir().unwrap();
+        let unit = dir.path().join("portainer.container");
+        tokio::fs::write(&unit, "[Container]\n").await.unwrap();
+        let pending = RestartObligation::prepare(&unit, true).await.unwrap();
+        assert!(pending.is_pending());
+        tokio::fs::write(&unit, "[Container]\nNetwork=slirp4netns\n")
+            .await
+            .unwrap();
+        // Simulate systemctl failure or daemon interruption after unit rewrite.
+        drop(pending);
+        let retry = RestartObligation::prepare(&unit, false).await.unwrap();
+        assert!(
+            retry.is_pending(),
+            "matching unit must not discard failed restart"
+        );
+        retry.complete().await.unwrap();
+        assert!(!RestartObligation::prepare(&unit, false)
+            .await
+            .unwrap()
+            .is_pending());
+    }
+
+    #[tokio::test]
+    async fn pending_runtime_change_errors_are_not_reported_as_success() {
+        let dir = tempfile::tempdir().unwrap();
+        let missing = dir.path().join("missing/app.container");
+        assert!(RestartObligation::prepare(&missing, true).await.is_err());
+        let unit = dir.path().join("app.container");
+        let pending = RestartObligation::prepare(&unit, true).await.unwrap();
+        tokio::fs::remove_file(unit.with_extension("restart-pending"))
+            .await
+            .unwrap();
+        assert!(pending.complete().await.is_err());
+    }
+
+    #[test]
+    fn gitea_ssh_sandbox_capability_is_applied_as_a_runtime_change() {
+        let manifest =
+            AppManifest::parse(include_str!("../../../../apps/gitea/manifest.yml")).unwrap();
+        manifest.validate().unwrap();
+        let new = QuadletUnit::from_manifest(&manifest, "gitea").render();
+        assert!(new.contains("AddCapability=SYS_CHROOT\n"));
+        let old = new.replace("AddCapability=SYS_CHROOT\n", "");
+        assert!(security_changed(&old, &new));
+        assert!(!security_changed(&new, &new));
+        assert!(!security_changed(
+            "AddCapability=CHOWN\nAddCapability=SETUID\n",
+            "AddCapability=SETUID\nAddCapability=CHOWN\n"
+        ));
+    }
+
     #[test]
     fn network_aliases_changed_detects_network_mode_drift() {
         let old = "[Container]\nNetwork=slirp4netns\n";

core/archipelago/src/content_invoice.rs

@@ -1,80 +1,155 @@
-//! Seller-side pending entitlements for Lightning-invoice peer-file sales (#46).
-//!
-//! When a buyer asks to pay for a paid catalog item with an external wallet (as
-//! opposed to the local-ecash fast path), the *selling* node mints a Lightning
-//! invoice on its own LND and records a pending entitlement here, keyed by the
-//! invoice's payment hash. The buyer pays the invoice from any wallet and polls
-//! for settlement; once the seller's LND confirms the invoice is settled we mark
-//! the entitlement paid, and the content gate (`content_server::serve_content`)
-//! then releases the file to anyone presenting that payment hash.
-//!
-//! State is in-memory and bounded by a TTL. If the seller restarts before the
-//! buyer pays, the buyer simply requests a fresh invoice — no value is lost
-//! because an unpaid invoice represents no money.
+//! Durable seller-side entitlements for peer-file invoices and on-chain sales.
+//! Payment records must outlive browser polling, process restarts and invoice
+//! expiry: an invoice can settle while the buyer is disconnected.
 
-use std::collections::HashMap;
-use std::sync::LazyLock;
-use std::time::{Duration, Instant};
-use tokio::sync::Mutex;
+use anyhow::{Context, Result};
+use serde::{Deserialize, Serialize};
+use sha2::{Digest, Sha256};
+use std::path::{Path, PathBuf};
+use tokio::{fs, io::AsyncWriteExt, sync::Mutex};
 
-/// How long a pending/paid entitlement is retained. Generous enough for a human
-/// to pay an invoice and download, short enough to keep the map small.
-const ENTITLEMENT_TTL: Duration = Duration::from_secs(3600); // 1 hour
+static WRITES: Mutex<()> = Mutex::const_new(());
 
-#[derive(Clone)]
+#[derive(Clone, Serialize, Deserialize)]
 struct Entitlement {
     content_id: String,
     price_sats: u64,
     paid: bool,
-    created_at: Instant,
 }
 
-static ENTITLEMENTS: LazyLock<Mutex<HashMap<String, Entitlement>>> =
-    LazyLock::new(|| Mutex::new(HashMap::new()));
-
-/// Drop expired entries. Caller must hold the lock.
-fn prune(map: &mut HashMap<String, Entitlement>) {
-    map.retain(|_, e| e.created_at.elapsed() < ENTITLEMENT_TTL);
+fn path(data_dir: &Path, token: &str) -> PathBuf {
+    data_dir.join("content-entitlements").join(format!(
+        "{}.json",
+        hex::encode(Sha256::digest(token.as_bytes()))
+    ))
 }
 
-/// Record a freshly-minted invoice as a pending (unpaid) entitlement.
-pub async fn record_pending(payment_hash: &str, content_id: &str, price_sats: u64) {
-    let mut map = ENTITLEMENTS.lock().await;
-    prune(&mut map);
-    map.insert(
-        payment_hash.to_string(),
-        Entitlement {
-            content_id: content_id.to_string(),
-            price_sats,
-            paid: false,
-            created_at: Instant::now(),
-        },
-    );
-}
-
-/// Mark the entitlement for `payment_hash` paid. No-op if unknown/expired.
-pub async fn mark_paid(payment_hash: &str) {
-    let mut map = ENTITLEMENTS.lock().await;
-    prune(&mut map);
-    if let Some(e) = map.get_mut(payment_hash) {
-        e.paid = true;
+async fn read(data_dir: &Path, token: &str) -> Result<Option<Entitlement>> {
+    match fs::read(path(data_dir, token)).await {
+        Ok(bytes) => Ok(Some(
+            serde_json::from_slice(&bytes).context("Invalid payment entitlement")?,
+        )),
+        Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None),
+        Err(e) => Err(e).context("Reading payment entitlement"),
     }
 }
 
-/// The content_id + price an entitlement was issued for, if still live.
-pub async fn lookup(payment_hash: &str) -> Option<(String, u64)> {
-    let mut map = ENTITLEMENTS.lock().await;
-    prune(&mut map);
-    map.get(payment_hash)
-        .map(|e| (e.content_id.clone(), e.price_sats))
+async fn write(data_dir: &Path, token: &str, entry: &Entitlement) -> Result<()> {
+    let target = path(data_dir, token);
+    let dir = target.parent().unwrap();
+    fs::create_dir_all(dir).await?;
+    let tmp = target.with_extension("tmp");
+    let mut file = fs::OpenOptions::new()
+        .write(true)
+        .create(true)
+        .truncate(true)
+        .mode(0o600)
+        .open(&tmp)
+        .await?;
+    file.write_all(&serde_json::to_vec(entry)?).await?;
+    file.sync_all().await?;
+    drop(file);
+    fs::rename(&tmp, &target).await?;
+    fs::File::open(dir).await?.sync_all().await?;
+    Ok(())
 }
 
-/// True if `payment_hash` is a paid entitlement for exactly `content_id`.
-/// This is the gate the content server consults to release a file.
-pub async fn is_paid_for(payment_hash: &str, content_id: &str) -> bool {
-    let mut map = ENTITLEMENTS.lock().await;
-    prune(&mut map);
-    map.get(payment_hash)
+/// Save before exposing an invoice/address to the buyer. Never overwrite an
+/// existing payment or silently rebind its token to another item or price.
+pub async fn record_pending(
+    data_dir: &Path,
+    token: &str,
+    content_id: &str,
+    price_sats: u64,
+) -> Result<()> {
+    let _lock = WRITES.lock().await;
+    if let Some(existing) = read(data_dir, token).await? {
+        anyhow::ensure!(
+            existing.content_id == content_id && existing.price_sats == price_sats,
+            "Payment entitlement mismatch"
+        );
+        return Ok(());
+    }
+    write(
+        data_dir,
+        token,
+        &Entitlement {
+            content_id: content_id.into(),
+            price_sats,
+            paid: false,
+        },
+    )
+    .await
+}
+
+pub async fn mark_paid(data_dir: &Path, token: &str) -> Result<()> {
+    let _lock = WRITES.lock().await;
+    let mut entry = read(data_dir, token)
+        .await?
+        .context("Unknown payment entitlement")?;
+    entry.paid = true;
+    write(data_dir, token, &entry).await
+}
+
+pub async fn lookup(data_dir: &Path, token: &str) -> Result<Option<(String, u64)>> {
+    Ok(read(data_dir, token)
+        .await?
+        .map(|e| (e.content_id, e.price_sats)))
+}
+
+pub async fn is_paid_for(data_dir: &Path, token: &str, content_id: &str) -> bool {
+    read(data_dir, token)
+        .await
+        .ok()
+        .flatten()
         .map(|e| e.paid && e.content_id == content_id)
         .unwrap_or(false)
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    #[tokio::test]
+    async fn paid_entitlement_survives_reload_and_cannot_be_rebound() {
+        let dir = tempfile::tempdir().unwrap();
+        record_pending(dir.path(), "hash", "file", 12)
+            .await
+            .unwrap();
+        assert!(!is_paid_for(dir.path(), "hash", "file").await);
+        mark_paid(dir.path(), "hash").await.unwrap();
+        // All reads reopen disk; no process-local entitlement map exists.
+        assert!(is_paid_for(dir.path(), "hash", "file").await);
+        assert!(!is_paid_for(dir.path(), "hash", "other").await);
+        record_pending(dir.path(), "hash", "file", 12)
+            .await
+            .unwrap();
+        assert!(is_paid_for(dir.path(), "hash", "file").await);
+        assert!(record_pending(dir.path(), "hash", "other", 12)
+            .await
+            .is_err());
+        assert!(record_pending(dir.path(), "hash", "file", 13)
+            .await
+            .is_err());
+        let other = tempfile::tempdir().unwrap();
+        assert!(!is_paid_for(other.path(), "hash", "file").await);
+        assert!(mark_paid(dir.path(), "unknown").await.is_err());
+    }
+    #[tokio::test]
+    async fn corrupt_or_unwritable_records_fail_closed() {
+        let dir = tempfile::tempdir().unwrap();
+        record_pending(dir.path(), "../../token", "file", 1)
+            .await
+            .unwrap();
+        fs::write(path(dir.path(), "../../token"), b"broken")
+            .await
+            .unwrap();
+        assert!(lookup(dir.path(), "../../token").await.is_err());
+        assert!(!is_paid_for(dir.path(), "../../token", "file").await);
+        assert!(record_pending(dir.path(), "../../token", "file", 1)
+            .await
+            .is_err());
+        let file = dir.path().join("not-directory");
+        fs::write(&file, b"x").await.unwrap();
+        assert!(record_pending(&file, "hash", "file", 1).await.is_err());
+    }
+}

core/archipelago/src/content_owned.rs

@@ -12,7 +12,9 @@
 use anyhow::{Context, Result};
 use serde::{Deserialize, Serialize};
 use std::path::{Path, PathBuf};
-use tokio::fs;
+use tokio::{fs, io::AsyncWriteExt, sync::Mutex};
+
+static PURCHASE_WRITES: Mutex<()> = Mutex::const_new(());
 
 const OWNED_DIR: &str = "purchased-content";
 const OWNED_INDEX: &str = "owned.json";
@@ -66,20 +68,51 @@ fn bytes_path(data_dir: &Path, onion: &str, content_id: &str) -> PathBuf {
         .join(sanitize(content_id))
 }
 
-async fn load_index(data_dir: &Path) -> OwnedIndex {
+async fn load_index_checked(data_dir: &Path) -> Result<OwnedIndex> {
     match fs::read_to_string(index_path(data_dir)).await {
-        Ok(s) => serde_json::from_str(&s).unwrap_or_default(),
-        Err(_) => OwnedIndex::default(),
+        Ok(s) => serde_json::from_str(&s)
+            .context("Invalid purchase index; existing records were preserved"),
+        Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(OwnedIndex::default()),
+        Err(error) => Err(error).context("Reading purchase index"),
     }
 }
 
+async fn load_index(data_dir: &Path) -> OwnedIndex {
+    load_index_checked(data_dir).await.unwrap_or_default()
+}
+
+async fn atomic_write(path: &Path, bytes: &[u8]) -> Result<()> {
+    let parent = path.parent().context("Purchase path has no parent")?;
+    fs::create_dir_all(parent).await?;
+    let temp = parent.join(format!(".purchase-{}.tmp", uuid::Uuid::new_v4()));
+    let result = async {
+        let mut file = fs::OpenOptions::new()
+            .write(true)
+            .create_new(true)
+            .mode(0o600)
+            .open(&temp)
+            .await?;
+        file.write_all(bytes).await?;
+        file.sync_all().await?;
+        drop(file);
+        fs::rename(&temp, path).await?;
+        fs::File::open(parent).await?.sync_all().await?;
+        Ok::<_, anyhow::Error>(())
+    }
+    .await;
+    if result.is_err() {
+        let _ = fs::remove_file(&temp).await;
+    }
+    result
+}
+
 async fn save_index(data_dir: &Path, index: &OwnedIndex) -> Result<()> {
     let root = owned_root(data_dir);
     fs::create_dir_all(&root)
         .await
         .with_context(|| format!("creating {}", root.display()))?;
     let content = serde_json::to_string_pretty(index).context("serializing owned index")?;
-    fs::write(index_path(data_dir), content)
+    atomic_write(&index_path(data_dir), content.as_bytes())
         .await
         .context("writing owned index")
 }
@@ -98,17 +131,15 @@ pub async fn record_purchase(
     ecash_backend: &str,
     purchased_at: &str,
 ) -> Result<()> {
+    // Read-modify-write must be one serialized transaction. Never replace a
+    // damaged index with an empty one, and never expose partially written bytes.
+    let _lock = PURCHASE_WRITES.lock().await;
+    let mut index = load_index_checked(data_dir).await?;
     let path = bytes_path(data_dir, onion, content_id);
-    if let Some(parent) = path.parent() {
-        fs::create_dir_all(parent)
-            .await
-            .with_context(|| format!("creating {}", parent.display()))?;
-    }
-    fs::write(&path, bytes)
+    atomic_write(&path, bytes)
         .await
         .with_context(|| format!("writing purchased bytes to {}", path.display()))?;
 
-    let mut index = load_index(data_dir).await;
     let entry = OwnedItem {
         onion: onion.to_string(),
         content_id: content_id.to_string(),
@@ -165,3 +196,90 @@ pub async fn read_owned(
         .unwrap_or_else(|| "application/octet-stream".to_string());
     Some((mime, bytes))
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    #[tokio::test]
+    async fn concurrent_purchases_preserve_every_item_and_exact_bytes() {
+        let dir = tempfile::tempdir().unwrap();
+        let mut jobs = tokio::task::JoinSet::new();
+        for n in 0..24 {
+            let root = dir.path().to_path_buf();
+            jobs.spawn(async move {
+                let id = format!("file-{n}");
+                record_purchase(
+                    &root,
+                    "seller.onion",
+                    &id,
+                    &id,
+                    "text/plain",
+                    id.as_bytes(),
+                    5,
+                    "lightning",
+                    "now",
+                )
+                .await
+                .unwrap();
+            });
+        }
+        while let Some(result) = jobs.join_next().await {
+            result.unwrap();
+        }
+        assert_eq!(list_owned(dir.path()).await.len(), 24);
+        for n in 0..24 {
+            let id = format!("file-{n}");
+            assert!(is_owned(dir.path(), "seller.onion", &id).await);
+            let (mime, bytes) = read_owned(dir.path(), "seller.onion", &id).await.unwrap();
+            assert_eq!(mime, "text/plain");
+            assert_eq!(bytes, id.as_bytes());
+        }
+        record_purchase(
+            dir.path(),
+            "seller.onion",
+            "file-0",
+            "file-0",
+            "text/plain",
+            b"updated",
+            5,
+            "lightning",
+            "later",
+        )
+        .await
+        .unwrap();
+        assert_eq!(list_owned(dir.path()).await.len(), 24);
+        assert_eq!(
+            read_owned(dir.path(), "seller.onion", "file-0")
+                .await
+                .unwrap()
+                .1,
+            b"updated"
+        );
+    }
+    #[tokio::test]
+    async fn damaged_index_is_preserved_instead_of_erasing_prior_ownership() {
+        let dir = tempfile::tempdir().unwrap();
+        fs::create_dir_all(owned_root(dir.path())).await.unwrap();
+        fs::write(index_path(dir.path()), b"damaged but preserve me")
+            .await
+            .unwrap();
+        assert!(record_purchase(
+            dir.path(),
+            "seller.onion",
+            "new",
+            "new",
+            "text/plain",
+            b"bytes",
+            5,
+            "lightning",
+            "now"
+        )
+        .await
+        .is_err());
+        assert_eq!(
+            fs::read(index_path(dir.path())).await.unwrap(),
+            b"damaged but preserve me"
+        );
+        assert!(!bytes_path(dir.path(), "seller.onion", "new").exists());
+    }
+}

core/archipelago/src/content_server.rs

@@ -400,7 +400,7 @@ where
                 if !authorized {
                     if let Some(hash) = invoice_hash {
                         if method_accepted(&item.access, "lightning")
-                            && crate::content_invoice::is_paid_for(hash, id).await
+                            && crate::content_invoice::is_paid_for(data_dir, hash, id).await
                         {
                             authorized = true;
                         }

core/archipelago/src/crash_recovery.rs

@@ -194,6 +194,7 @@ pub async fn clear_user_stopped(data_dir: &Path, name: &str) {
 // Installation is a decision, not a runtime observation, so it gets a record
 // of its own that no amount of downtime erodes.
 const INSTALLED_APPS_FILE: &str = "installed-apps.json";
+static INSTALLED_APPS_LOCK: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(());
 
 /// Load the durable set of installed app ids / container names.
 pub async fn load_installed_apps(data_dir: &Path) -> std::collections::HashSet<String> {
@@ -220,12 +221,23 @@ pub async fn load_installed_apps_if_recorded(
 async fn save_installed_apps(data_dir: &Path, installed: &std::collections::HashSet<String>) {
     let path = data_dir.join(INSTALLED_APPS_FILE);
     if let Ok(json) = serde_json::to_string_pretty(installed) {
-        let _ = fs::write(&path, json).await;
+        let tmp = path.with_extension("json.tmp");
+        let result = async {
+            fs::write(&tmp, json).await?;
+            fs::File::open(&tmp).await?.sync_all().await?;
+            fs::rename(&tmp, &path).await?;
+            fs::File::open(data_dir).await?.sync_all().await
+        }
+        .await;
+        if let Err(error) = result {
+            warn!(%error, "could not persist installed apps");
+        }
     }
 }
 
 /// Record that an app is installed. Called when an install succeeds.
 pub async fn mark_installed(data_dir: &Path, name: &str) {
+    let _guard = INSTALLED_APPS_LOCK.lock().await;
     let mut installed = load_installed_apps(data_dir).await;
     if installed.insert(name.to_string()) {
         save_installed_apps(data_dir, &installed).await;
@@ -235,6 +247,7 @@ pub async fn mark_installed(data_dir: &Path, name: &str) {
 /// Forget an app. Called on uninstall, beside `mark_user_uninstalled` — the
 /// two must move together or a reinstall-after-uninstall leaves a stale claim.
 pub async fn clear_installed(data_dir: &Path, name: &str) {
+    let _guard = INSTALLED_APPS_LOCK.lock().await;
     let mut installed = load_installed_apps(data_dir).await;
     if installed.remove(name) {
         save_installed_apps(data_dir, &installed).await;
@@ -252,6 +265,7 @@ pub async fn clear_installed(data_dir: &Path, name: &str) {
 /// need it. Runs on every boot, so an app installed before the upgrade is
 /// still picked up whenever it is next seen alive.
 pub async fn backfill_installed_apps(data_dir: &Path, present_container_names: &[String]) {
+    let _guard = INSTALLED_APPS_LOCK.lock().await;
     if present_container_names.is_empty() {
         return;
     }
@@ -650,6 +664,10 @@ pub async fn start_stopped_stack_containers(data_dir: &Path) -> RecoveryReport {
     start_stopped_app_stacks(data_dir).await
 }
 
+fn stack_member_needs_recovery(state: Option<&str>, user_stopped: bool) -> bool {
+    !user_stopped && matches!(state, Some("exited" | "stopped" | "created" | "configured"))
+}
+
 async fn start_stopped_app_stacks(data_dir: &Path) -> RecoveryReport {
     let user_stopped = load_user_stopped(data_dir).await;
     let mut report = RecoveryReport {
@@ -663,24 +681,27 @@ async fn start_stopped_app_stacks(data_dir: &Path) -> RecoveryReport {
             continue;
         }
 
-        info!(
-            "Recovering stopped {} stack containers after boot",
-            stack.name
-        );
+        // Healthy members must never acquire a restarting overlay merely
+        // because the periodic recovery scan ran. Queue existing stopped
+        // members only; recheck each immediately before starting below.
+        let mut pending = Vec::new();
+        for container in stack.containers {
+            let state = container_state(container).await;
+            if stack_member_needs_recovery(state.as_deref(), user_stopped.contains(*container)) {
+                pending.push((*container).to_string());
+            }
+        }
+        if pending.is_empty() {
+            continue;
+        }
+        info!("Recovering stopped {} stack containers", stack.name);
         repair_stack_network_aliases(stack).await;
-
-        // Register the whole stack up front: the per-member dependency waits
-        // below can take minutes, and the UI should say "Restarting", not
-        // "Stopped", for members still queued behind them.
-        pending_boot_starts_add(
-            stack
-                .containers
-                .iter()
-                .filter(|c| !user_stopped.contains(**c))
-                .map(|c| (*c).to_string()),
-        );
+        pending_boot_starts_add(pending.iter().cloned());
 
         for container in stack.containers {
+            if !pending.iter().any(|name| name.as_str() == *container) {
+                continue;
+            }
             if user_stopped.contains(*container) {
                 info!("Skipping user-stopped container: {}", container);
                 continue;
@@ -692,8 +713,8 @@ async fn start_stopped_app_stacks(data_dir: &Path) -> RecoveryReport {
                     pending_boot_start_done(container);
                     continue;
                 }
-                Some(_) => {}
-                None => {
+                Some(state) if stack_member_needs_recovery(Some(&state), false) => {}
+                _ => {
                     pending_boot_start_done(container);
                     continue;
                 }
@@ -1497,3 +1518,47 @@ mod tests {
         );
     }
 }
+
+#[cfg(test)]
+mod installed_concurrency_tests {
+    use super::*;
+    #[tokio::test]
+    async fn concurrent_install_records_are_not_lost() {
+        let dir = tempfile::tempdir().unwrap();
+        let mut tasks = Vec::new();
+        for i in 0..24 {
+            let path = dir.path().to_owned();
+            tasks.push(tokio::spawn(async move {
+                mark_installed(&path, &format!("app-{i}")).await;
+            }));
+        }
+        for task in tasks {
+            task.await.unwrap();
+        }
+        assert_eq!(load_installed_apps(dir.path()).await.len(), 24);
+        clear_installed(dir.path(), "app-3").await;
+        assert_eq!(load_installed_apps(dir.path()).await.len(), 23);
+        assert!(!dir.path().join("installed-apps.json.tmp").exists());
+    }
+}
+
+#[cfg(test)]
+mod stack_recovery_overlay_tests {
+    use super::stack_member_needs_recovery;
+    #[test]
+    fn only_existing_stopped_members_receive_recovery_overlay() {
+        for state in [
+            None,
+            Some("running"),
+            Some("paused"),
+            Some("restarting"),
+            Some("removing"),
+        ] {
+            assert!(!stack_member_needs_recovery(state, false));
+        }
+        for state in ["exited", "stopped", "created", "configured"] {
+            assert!(stack_member_needs_recovery(Some(state), false));
+            assert!(!stack_member_needs_recovery(Some(state), true));
+        }
+    }
+}

core/archipelago/src/data_model.rs

@@ -146,6 +146,10 @@ pub enum PackageState {
 
 #[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
 pub struct PackageDataEntry {
+    /// Whether the app's HTTP upstream answered this scan (independent of
+    /// container health and blockchain sync). Missing on older nodes.
+    #[serde(rename = "ui-ready", default, skip_serializing_if = "Option::is_none")]
+    pub ui_ready: Option<bool>,
     pub state: PackageState,
     /// Container health: "healthy", "unhealthy", "starting", or null
     #[serde(skip_serializing_if = "Option::is_none")]
@@ -297,6 +301,8 @@ pub enum InstallPhase {
     /// `podman pull` in progress (the longest phase — up to several
     /// minutes for large images on slow networks).
     PullingImage,
+    /// Orchestrator owns download/build and startup as one operation.
+    PreparingApp,
     /// Creating data directories, writing app-specific configs
     /// (bitcoin.conf, lnd.conf, searxng settings.yml, chown).
     CreatingContainer,

core/archipelago/src/fips/app_ports.rs

@@ -5,8 +5,45 @@
 //! are reachable over the mesh; ports of apps that aren't installed have
 //! no listener, so allowing them is inert.
 
+#[rustfmt::skip]
 pub const APP_LAUNCH_PORTS: &[u16] = &[
-    2283, 2342, 3000, 3001, 3002, 4080, 5180, 7778, 8080, 8081, 8082, 8083, 8084, 8085, 8087, 8090,
-    8096, 8123, 8175, 8176, 8187, 8240, 8334, 8336, 8337, 8888, 8999, 9000, 9100, 10380, 11434,
-    18081, 18083, 18091, 23000, 32838, 50002,
+    2283,
+    2342,
+    3000,
+    3001,
+    3002,
+    4080,
+    5180,
+    7778,
+    8080,
+    8081,
+    8082,
+    8083,
+    8084,
+    8085,
+    8087,
+    8090,
+    8091,
+    8096,
+    8123,
+    8175,
+    8176,
+    8187,
+    8240,
+    8334,
+    8336,
+    8337,
+    8888,
+    8998,
+    8999,
+    9000,
+    9100,
+    10380,
+    11434,
+    18081,
+    18083,
+    18091,
+    23000,
+    32838,
+    50002,
 ];

core/archipelago/src/health_monitor.rs

@@ -501,12 +501,12 @@ async fn check_containers() -> Vec<ContainerHealth> {
     out
 }
 
-fn host_tcp_ports_from_container(c: &serde_json::Value) -> Vec<u16> {
+fn host_tcp_ports_from_container(c: &serde_json::Value) -> Vec<std::net::SocketAddr> {
     let Some(ports) = c.get("Ports").and_then(|v| v.as_array()) else {
         return Vec::new();
     };
 
-    let mut out: Vec<u16> = ports
+    let mut out: Vec<std::net::SocketAddr> = ports
         .iter()
         .filter(|p| {
             p.get("protocol")
@@ -515,9 +515,19 @@ fn host_tcp_ports_from_container(c: &serde_json::Value) -> Vec<u16> {
                 .eq_ignore_ascii_case("tcp")
         })
         .filter_map(|p| {
-            p.get("host_port")
-                .and_then(|v| v.as_u64())
-                .and_then(|port| u16::try_from(port).ok())
+            let port = p.get("host_port")?.as_u64()?;
+            let port = u16::try_from(port).ok().filter(|port| *port != 0)?;
+            let bind = p.get("host_ip").and_then(|v| v.as_str()).unwrap_or("");
+            // Wildcard listeners are reachable through the corresponding
+            // loopback family. Explicit binds must be probed at that address:
+            // probing a WireGuard-only port on 127.0.0.1 creates false failures
+            // and endlessly restarts an otherwise healthy app.
+            let address: std::net::IpAddr = match bind {
+                "" | "0.0.0.0" => "127.0.0.1".parse().ok()?,
+                "::" => "::1".parse().ok()?,
+                explicit => explicit.parse().ok()?,
+            };
+            Some(std::net::SocketAddr::new(address, port))
         })
         .collect();
     out.sort_unstable();
@@ -525,11 +535,11 @@ fn host_tcp_ports_from_container(c: &serde_json::Value) -> Vec<u16> {
     out
 }
 
-async fn host_ports_ready(ports: &[u16]) -> bool {
+async fn host_ports_ready(ports: &[std::net::SocketAddr]) -> bool {
     for port in ports {
         let ready = tokio::time::timeout(
             std::time::Duration::from_secs(2),
-            tokio::net::TcpStream::connect(("127.0.0.1", *port)),
+            tokio::net::TcpStream::connect(*port),
         )
         .await
         .is_ok_and(|r| r.is_ok());
@@ -1662,4 +1672,51 @@ mod tests {
             "Prefetcher:catching up to daemon height 953,480"
         ));
     }
+    #[test]
+    fn published_port_probes_preserve_explicit_bind_addresses() {
+        let c = serde_json::json!({"Ports": [
+            {"host_ip":"127.0.0.1","host_port":8081,"protocol":"tcp"},
+            {"host_ip":"10.77.0.2","host_port":18081,"protocol":"tcp"},
+            {"host_ip":"10.77.0.2","host_port":18443,"protocol":"tcp"},
+            {"host_ip":"::1","host_port":8082,"protocol":"tcp"}
+        ]});
+        let targets = host_tcp_ports_from_container(&c);
+        for target in [
+            "127.0.0.1:8081",
+            "10.77.0.2:18081",
+            "10.77.0.2:18443",
+            "[::1]:8082",
+        ] {
+            assert!(targets.contains(&target.parse().unwrap()));
+        }
+        assert!(!targets.contains(&"127.0.0.1:18081".parse().unwrap()));
+    }
+
+    #[test]
+    fn published_port_probes_normalize_wildcards_and_ignore_invalid_entries() {
+        let c = serde_json::json!({"Ports": [
+            {"host_ip":"0.0.0.0","host_port":8080},
+            {"host_ip":"","host_port":8080},
+            {"host_ip":"::","host_port":8080},
+            {"host_ip":"10.0.0.1","host_port":53,"protocol":"udp"},
+            {"host_ip":"bad","host_port":8080},
+            {"host_port":0}, {"host_port":65536}, {"container_port":80}
+        ]});
+        let targets = host_tcp_ports_from_container(&c);
+        assert_eq!(targets.len(), 2);
+        assert!(targets.contains(&"127.0.0.1:8080".parse().unwrap()));
+        assert!(targets.contains(&"[::1]:8080".parse().unwrap()));
+    }
+
+    #[tokio::test]
+    async fn health_probe_reaches_non_default_loopback_and_detects_closed_port() {
+        let listener = tokio::net::TcpListener::bind("127.0.0.2:0").await.unwrap();
+        let address = listener.local_addr().unwrap();
+        assert!(host_ports_ready(&[address]).await);
+        // Same port, wrong local address reproduces the former false failure.
+        let wrong = std::net::SocketAddr::new("127.0.0.1".parse().unwrap(), address.port());
+        assert!(!host_ports_ready(&[wrong]).await);
+        drop(listener);
+        assert!(!host_ports_ready(&[address]).await);
+    }
 }

core/archipelago/src/main.rs

@@ -256,6 +256,10 @@ async fn main() -> Result<()> {
             boot_report.recovered, boot_report.total, boot_report.failed
         );
     }
+    // Disk manifests must be stable before the initial load and all later
+    // catalog reloads. Do not move this into the background doctor bootstrap.
+    bootstrap::ensure_runtime_assets_ready().await;
+
     // Construct the container orchestrator once. In prod mode we load the
     // on-disk app manifests, do an initial adoption pass, and spawn the
     // BootReconciler loop (Step 5/6 of the rust-orchestrator migration).

core/archipelago/src/server.rs

@@ -1765,12 +1765,17 @@ fn merge_preserving_transitional(
     };
 
     crate::data_model::PackageDataEntry {
-        state,
+        state: state.clone(),
         // install_progress and uninstall_stage are also owned by the
         // initiating op (same reason as state) — keep them.
         install_progress: existing.install_progress.clone(),
         uninstall_stage: existing.uninstall_stage.clone(),
         // Everything else comes from the fresh scan.
+        ui_ready: if state == crate::data_model::PackageState::Running {
+            fresh.ui_ready
+        } else {
+            Some(false)
+        },
         health: fresh.health.clone(),
         exit_code: fresh.exit_code,
         static_files: fresh.static_files.clone(),
@@ -1809,7 +1814,10 @@ async fn scan_and_update_packages(
     absence_tracker: &mut HashMap<String, u32>,
     transitional_since: &mut HashMap<String, Instant>,
 ) -> Result<()> {
-    let mut packages = scanner.scan_containers().await?;
+    let (before_scan, _) = state.get_snapshot().await;
+    let mut packages = scanner
+        .scan_containers(data_dir, &before_scan.package_data)
+        .await?;
     let user_stopped = crate::crash_recovery::load_user_stopped(data_dir).await;
     for (id, pkg) in packages.iter_mut() {
         if pkg.state == crate::data_model::PackageState::Exited && user_stopped.contains(id) {
@@ -1870,11 +1878,14 @@ async fn scan_and_update_packages(
     // once at load ~2). Better to keep saying "scanning…" than to say "empty".
     if packages.is_empty() && (!first_scan || !installed_registry.is_empty()) {
         if tor_changed || update_changed {
-            let mut data = current_data;
-            data.server_info.tor_address = tor_addr.clone();
-            data.server_info.node_address = tor_addr.as_ref().map(|t| identity.node_address(t));
-            data.server_info.status_info.updated = update_available;
-            state.update_data(data).await;
+            state
+                .mutate_data(|data| {
+                    data.server_info.tor_address = tor_addr.clone();
+                    data.server_info.node_address =
+                        tor_addr.as_ref().map(|t| identity.node_address(t));
+                    data.server_info.status_info.updated = update_available;
+                })
+                .await;
         }
         return Ok(());
     }
@@ -1899,6 +1910,13 @@ async fn scan_and_update_packages(
     // died without cleanup and let the scan override it.
     let now = Instant::now();
     for (id, pkg) in &packages {
+        if user_uninstalled.contains(id)
+            || user_uninstalled.contains(&format!("archy-{id}"))
+            || (before_scan.package_data.contains_key(id)
+                && !current_data.package_data.contains_key(id))
+        {
+            continue;
+        }
         absence_tracker.remove(id);
         let existing = merged.get(id);
         let overwrite = match existing {
@@ -2054,22 +2072,40 @@ async fn scan_and_update_packages(
     }
 
     if changed || tor_changed || first_scan || update_changed {
-        let mut data = current_data;
-        data.package_data = merged;
-        data.server_info.tor_address = tor_addr.clone();
-        data.server_info.node_address = tor_addr.as_ref().map(|t| identity.node_address(t));
-        data.server_info.status_info.containers_scanned = true;
-        data.server_info.status_info.updated = update_available;
-        state.update_data(data).await;
-        debug!(
-            "📦 State changed (packages={}, tor={}, first_scan={}, update={}), broadcasting update",
-            changed, tor_changed, first_scan, update_changed
-        );
+        state
+            .mutate_data(|data| {
+                // A lifecycle operation may have started/finished while this scan
+                // awaited probes or disk I/O. Never overwrite that newer entry or
+                // resurrect one that an uninstall removed in the meantime.
+                apply_scanned_packages(&mut data.package_data, &current_data.package_data, &merged);
+                data.server_info.tor_address = tor_addr.clone();
+                data.server_info.node_address = tor_addr.as_ref().map(|t| identity.node_address(t));
+                data.server_info.status_info.containers_scanned = true;
+                data.server_info.status_info.updated = update_available;
+            })
+            .await;
     }
 
     Ok(())
 }
 
+fn apply_scanned_packages(
+    latest: &mut HashMap<String, crate::data_model::PackageDataEntry>,
+    base: &HashMap<String, crate::data_model::PackageDataEntry>,
+    scanned: &HashMap<String, crate::data_model::PackageDataEntry>,
+) {
+    for (id, fresh) in scanned {
+        if latest.get(id) == base.get(id) {
+            latest.insert(id.clone(), fresh.clone());
+        }
+    }
+    for id in base.keys() {
+        if !scanned.contains_key(id) && latest.get(id) == base.get(id) {
+            latest.remove(id);
+        }
+    }
+}
+
 async fn normalize_reachable_package_health(
     packages: &mut HashMap<String, crate::data_model::PackageDataEntry>,
 ) {
@@ -2268,6 +2304,7 @@ mod merge_tests {
 
     fn make_entry(state: PackageState, health: Option<&str>) -> PackageDataEntry {
         PackageDataEntry {
+            ui_ready: None,
             state,
             health: health.map(|s| s.to_string()),
             exit_code: None,
@@ -2280,6 +2317,37 @@ mod merge_tests {
         }
     }
 
+    #[test]
+    fn stale_scan_cannot_remove_new_installs_or_overwrite_lifecycle_changes() {
+        let running = make_entry(PackageState::Running, Some("healthy"));
+        let restarting = make_entry(PackageState::Restarting, None);
+        let base = [
+            ("restart".into(), running.clone()),
+            ("uninstalled".into(), running.clone()),
+        ]
+        .into_iter()
+        .collect();
+        let mut latest = [
+            ("restart".into(), restarting.clone()),
+            ("new".into(), running.clone()),
+        ]
+        .into_iter()
+        .collect();
+        let scanned = [
+            ("restart".into(), running.clone()),
+            ("uninstalled".into(), running.clone()),
+        ]
+        .into_iter()
+        .collect();
+        apply_scanned_packages(&mut latest, &base, &scanned);
+        assert_eq!(latest.get("restart"), Some(&restarting));
+        assert_eq!(latest.get("new"), Some(&running));
+        assert!(!latest.contains_key("uninstalled"));
+        apply_scanned_packages(&mut latest, &base, &HashMap::new());
+        assert_eq!(latest.get("restart"), Some(&restarting));
+        assert!(latest.contains_key("new"));
+    }
+
     #[test]
     fn peer_path_filter_allows_content_catalog_and_items() {
         // Regression: the content *catalog* is exactly "/content" (no trailing

core/archipelago/src/state.rs

@@ -54,6 +54,21 @@ impl StateManager {
         let _ = self.broadcast_tx.send(message);
     }
 
+    /// Apply a small state change while holding the write lock. A lifecycle
+    /// task must not replace the entire model from an earlier snapshot.
+    pub async fn mutate_data<T>(&self, change: impl FnOnce(&mut DataModel) -> T) -> T {
+        let mut data = self.data.write().await;
+        let result = change(&mut data);
+        let mut rev = self.revision.write().await;
+        *rev += 1;
+        let _ = self.broadcast_tx.send(WebSocketMessage {
+            rev: *rev,
+            data: Some(data.clone()),
+            patch: None,
+        });
+        result
+    }
+
     /// Get a WebSocket message with the current state
     pub async fn get_initial_message(&self) -> WebSocketMessage {
         let (data, rev) = self.get_snapshot().await;
@@ -190,3 +205,29 @@ mod tests {
         assert_eq!(rev, 1);
     }
 }
+
+#[cfg(test)]
+mod atomic_mutation_tests {
+    use super::*;
+    #[tokio::test]
+    async fn concurrent_updates_preserve_independent_entries() {
+        let state = Arc::new(StateManager::new());
+        let mut tasks = Vec::new();
+        for i in 0..24 {
+            let state = state.clone();
+            tasks.push(tokio::spawn(async move {
+                state
+                    .mutate_data(|data| {
+                        data.peer_health.insert(format!("peer-{i}"), true);
+                    })
+                    .await;
+            }));
+        }
+        for task in tasks {
+            task.await.unwrap();
+        }
+        let (data, revision) = state.get_snapshot().await;
+        assert_eq!(data.peer_health.len(), 24);
+        assert_eq!(revision, 24);
+    }
+}

core/container/src/manifest.rs

@@ -989,6 +989,24 @@ impl AppManifest {
         validate_security(&self.app.security)?;
         validate_ports(&self.app.ports)?;
         validate_interfaces(&self.app.interfaces)?;
+        if let Some(value) = self.app.extensions.get("install_prerequisites") {
+            let items = value.as_sequence().ok_or_else(|| {
+                ManifestError::Invalid("install_prerequisites must be a list of app ids".into())
+            })?;
+            for item in items {
+                let id = item.as_str().unwrap_or_default();
+                if id.is_empty()
+                    || id == self.app.id
+                    || !id
+                        .bytes()
+                        .all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
+                {
+                    return Err(ManifestError::Invalid(
+                        "install_prerequisites must contain valid other app ids".into(),
+                    ));
+                }
+            }
+        }
         validate_environment(&self.app.environment)?;
         validate_devices(&self.app.devices)?;
 
@@ -1074,6 +1092,14 @@ impl AppManifest {
         // `..` copy sources). See docs/manifest-hooks-design.md.
         self.app.hooks.validate()?;
 
+        if let Some(value) = self.app.extensions.get("backup_before_runtime_change") {
+            if value.as_bool().is_none() {
+                return Err(ManifestError::Invalid(
+                    "backup_before_runtime_change must be boolean".into(),
+                ));
+            }
+        }
+
         Ok(())
     }
 }
@@ -1111,6 +1137,7 @@ fn validate_security(policy: &SecurityPolicy) -> Result<(), ManifestError> {
         "SETGID",
         "SETUID",
         "SYS_ADMIN",
+        "SYS_CHROOT",
     ];
     let mut seen = HashSet::new();
     for cap in &policy.capabilities {
@@ -1746,40 +1773,48 @@ app:
             }
         }
         exempt.sort();
-        // 28 as of 2026-08-23: the 26 below plus cuprate's two exemptions —
-        // 18183 (Monero p2p gossip, same reasoning as bitcoin's 8333) and
-        // 18090 (host mapping for Monero's canonical 18089 restricted RPC,
-        // upstream's own safe-for-public
-        // subset that wallets connect to directly as a "remote node" over
-        // plain HTTP JSON-RPC — same reasoning as electrumx's 50001).
-        // cuprate's unrestricted RPC (full node control) stays loopback-only
-        // (auth: local), not in this set.
-        //
-        // 26 as of 2026-08-16: the 25 below plus phoenixd 9740, a
-        // loopback-only JSON API whose own generated http password
-        // authenticates every request (added with the phoenixd onboarding,
-        // which did not update this count — exactly the drift this test
-        // exists to catch).
-        //
-        // 25 as of the v1.7.123 port-policy round: bitcoin p2p (8333 ×2),
-        // core-lightning 9736/9835, electrumx 50001, fedimint 8173/8174,
-        // fedimint-gateway 8176/9737, gitea ssh 2222, lightning-stack
-        // 8091/9738/10010, lnd 9735/10009/18080, netbird 3478/8086/8087,
-        // pine TLS 10381 + the three voice ports (10200/10300/10400 — the
-        // disclosed known gap), router SSDP/mDNS 1900/5353. Every one is a
-        // deliberate, rationale-carrying exemption; the release-gate test
-        // stage timed out that cycle, so the count here lagged at 17.
+        // Reviewed 2026-09-30: lightning-stack's three retired endpoints
+        // disappeared; Cuprate restricted RPC moved from none to gate-open.
+        // Compare exact endpoints, not just a count that can hide substitutions.
+        let expected = [
+            ("bitcoin-core", 8333),
+            ("bitcoin-knots", 8333),
+            ("core-lightning", 9736),
+            ("core-lightning", 9835),
+            ("cuprate", 18183),
+            ("electrumx", 50001),
+            ("fedimint", 8173),
+            ("fedimint", 8174),
+            ("fedimint-gateway", 8176),
+            ("fedimint-gateway", 9737),
+            ("gitea", 2222),
+            ("lnd", 9735),
+            ("lnd", 10009),
+            ("lnd", 18080),
+            ("netbird", 8087),
+            ("netbird-server", 3478),
+            ("netbird-server", 8086),
+            ("phoenixd", 9740),
+            ("pine", 10381),
+            ("pine-openwakeword", 10400),
+            ("pine-piper", 10200),
+            ("pine-whisper", 10300),
+            ("router", 1900),
+            ("router", 5353),
+        ]
+        .into_iter()
+        .map(|(id, port)| (id.to_owned(), port))
+        .collect::<Vec<_>>();
         assert_eq!(
-            exempt.len(),
-            28,
-            "unauthenticated port set changed — review before updating this count: {exempt:?}"
+            exempt, expected,
+            "unauthenticated endpoint set changed; review each exemption"
         );
     }
 
     /// `auth: open` ports are served by the gate WITHOUT its login challenge,
     /// so they are the second unauthenticated-by-the-gate surface and get the
-    /// same review guard as `auth: none`. Each one must be an app that
-    /// enforces a real login of its own.
+    /// same review guard as `auth: none`. Each must enforce its own login or
+    /// have an explicitly reviewed public protocol purpose.
     #[test]
     fn gate_open_ports_are_all_accounted_for() {
         let apps = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../apps");
@@ -1801,6 +1836,8 @@ app:
             }
         }
         open.sort();
+        // Cuprate 18090 is its deliberately public restricted RPC subset;
+        // unrestricted node-control RPC remains container-loopback-only.
         // Gitea 3001 (git clients speak basic-auth, not browser cookies),
         // BTCPay 23000 (checkout/invoice/webhook endpoints must be reachable
         // by anonymous payers), and — since the v1.8.7 platform round — the
@@ -1808,18 +1845,35 @@ app:
         // nginx-proxy-manager 8081 (NPM admin accounts), tailscale 8240
         // (tailnet login on the web console). Both enforce their own login,
         // and an operator can re-gate either from Settings → Access control.
+        // Angor's indexer exposes public chain data/transaction broadcast;
+        // its optional standalone relay accepts signed public Nostr events.
+        // Neither mounts credentials or the node's internal relay database.
         assert_eq!(
             open,
             vec![
+                ("angor-indexer".to_string(), 8998u16),
+                ("angor-relay".to_string(), 8091u16),
                 ("btcpay-server".to_string(), 23000u16),
+                ("cuprate".to_string(), 18090u16),
                 ("gitea".to_string(), 3001u16),
                 ("nginx-proxy-manager".to_string(), 8081u16),
                 ("tailscale".to_string(), 8240u16),
             ],
-            "gate-open port set changed — every entry must be an app with its own login"
+            "gate-open port set changed — review login or intentional public protocol purpose"
         );
     }
 
+    #[test]
+    fn invalid_install_prerequisites_are_rejected() {
+        for value in ["not-a-list", "[demo]", "['../other']", "[false]", "['']"] {
+            let yaml = format!("app:\n  id: demo\n  name: Demo\n  version: 1.0.0\n  container:\n    image: docker.io/library/alpine:3.20\n  install_prerequisites: {value}\n");
+            assert!(AppManifest::parse(&yaml)
+                .unwrap_err()
+                .to_string()
+                .contains("install_prerequisites"));
+        }
+    }
+
     #[test]
     fn an_undeclared_port_classifies_as_session_but_is_not_declared() {
         // Two different questions, and conflating them caused both gate

core/container/src/podman_client.rs

@@ -310,59 +310,7 @@ impl PodmanClient {
                 );
                 continue;
             }
-            // Honour the manifest's protocol (default tcp). netbird's STUN port
-            // is 3478/udp; forcing tcp here would publish the wrong protocol and
-            // silently break relay discovery.
-            let protocol = match port.protocol.to_ascii_lowercase().as_str() {
-                "udp" => "udp",
-                "sctp" => "sctp",
-                _ => "tcp",
-            };
-            // Effective bind. A gated port with no declared bind would
-            // publish 0.0.0.0 — the app would own every host address, which
-            // is both the exposure itself and the reason the daemon's app
-            // gate cannot bind those addresses to authenticate them. Pin it
-            // to loopback so the gate can take the external addresses.
-            //
-            // Doing it HERE, at container creation, is the point: the pin and
-            // the gate's takeover then both come from the daemon and cannot
-            // disagree. The earlier attempt put this decision in manifest
-            // data instead, and a node whose manifests lagged the binary
-            // published Bitcoin's loopback-only RPC across the LAN
-            // (test node, 2026-08-03).
-            //
-            // A port that already declares a bind is never overridden — that
-            // is exactly what keeps `bind: 127.0.0.1` ports host-local and
-            // leaves `auth: none` protocol ports (LND gRPC/REST, electrum)
-            // published as they are, so remote wallets keep working.
-            // NOTE: the daemon deliberately does NOT rewrite this. Pinning a
-            // published port to loopback is how an app hands its external
-            // addresses to the gate, but it belongs in the manifest, not in
-            // daemon-side inference:
-            //
-            //   * `bind` is already honoured by every publish path (here and
-            //     in package::install), so a manifest edit needs no code.
-            //   * inference here would cover only THIS path — proven on
-            //     a test node, where a recreate went through another one and
-            //     the pin never applied.
-            //   * and inferring from an ABSENT field is what republished
-            //     Bitcoin's loopback RPC across the LAN, and came within one
-            //     container-recreate of pinning LND's gRPC/REST and breaking
-            //     every remote wallet.
-            //
-            // So the migration ships as `bind: 127.0.0.1` in the signed
-            // catalog. Verified 2026-08-03 that a disk-only manifest edit is
-            // overridden by the catalog, which is precisely why the catalog is
-            // the right and only place to carry it.
-            let mut mapping = serde_json::json!({
-                "container_port": port.container,
-                "host_port": port.host,
-                "protocol": protocol,
-            });
-            if !port.bind.is_empty() {
-                mapping["host_ip"] = serde_json::json!(port.bind);
-            }
-            port_mappings.push(mapping);
+            port_mappings.push(podman_publish_mapping(port));
         }
 
         let mut mounts = Vec::new();
@@ -751,6 +699,25 @@ pub fn image_uses_insecure_registry(image: &str) -> bool {
         .is_some_and(|host| INSECURE_REGISTRY_HOSTS.contains(&host))
 }
 
+// Keep the explicitly declared bind and transport identical to Quadlet. The
+// app gate owns external listeners; container publication must not bypass it.
+fn podman_publish_mapping(port: &crate::manifest::PortMapping) -> serde_json::Value {
+    let protocol = match port.protocol.to_ascii_lowercase().as_str() {
+        "udp" => "udp",
+        "sctp" => "sctp",
+        _ => "tcp",
+    };
+    let mut mapping = serde_json::json!({
+        "container_port": port.container,
+        "host_port": port.host,
+        "protocol": protocol,
+    });
+    if !port.bind.is_empty() {
+        mapping["host_ip"] = serde_json::json!(port.bind);
+    }
+    mapping
+}
+
 fn podman_network_settings(
     network: Option<&str>,
     network_policy: &str,
@@ -1110,6 +1077,24 @@ mod tests {
         ));
     }
 
+    #[test]
+    fn portainer_manifest_keeps_private_network_and_loopback_api_publication() {
+        let m = AppManifest::parse(include_str!("../../../apps/portainer/manifest.yml")).unwrap();
+        assert_eq!(
+            podman_network_settings(
+                m.app.container.network.as_deref(),
+                &m.app.security.network_policy
+            ),
+            ("slirp4netns", None)
+        );
+        assert_eq!(
+            podman_publish_mapping(&m.app.ports[0]),
+            serde_json::json!({
+                "container_port": 9000, "host_port": 9000, "protocol": "tcp", "host_ip": "127.0.0.1"
+            })
+        );
+    }
+
     #[test]
     fn podman_network_settings_uses_networks_map_for_custom_networks() {
         assert_eq!(

core/container/src/runtime.rs

@@ -618,6 +618,46 @@ impl DockerRuntime {
     }
 }
 
+// Docker is a development fallback. Refuse Podman-only network modes instead
+// of silently installing a different topology; still honor binds for other apps.
+fn docker_network_and_ports(manifest: &AppManifest, offset: u16) -> Result<Vec<String>> {
+    let network = manifest
+        .app
+        .container
+        .network
+        .as_deref()
+        .filter(|v| !v.is_empty())
+        .unwrap_or(&manifest.app.security.network_policy);
+    if matches!(network, "slirp4netns" | "pasta") {
+        anyhow::bail!("this app requires rootless Podman networking ({network})");
+    }
+    let mut args = Vec::new();
+    if !network.is_empty() && network != "isolated" {
+        args.extend(["--network".to_owned(), network.to_owned()]);
+    }
+    for port in &manifest.app.ports {
+        let host = port
+            .host
+            .checked_add(offset)
+            .context("published port offset overflow")?;
+        let bind = if port.bind.is_empty() {
+            String::new()
+        } else {
+            format!("{}:", port.bind)
+        };
+        let protocol = if port.protocol.is_empty() {
+            "tcp"
+        } else {
+            &port.protocol
+        };
+        args.extend([
+            "-p".to_owned(),
+            format!("{bind}{host}:{}/{protocol}", port.container),
+        ]);
+    }
+    Ok(args)
+}
+
 #[async_trait]
 impl ContainerRuntime for DockerRuntime {
     async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()> {
@@ -657,25 +697,7 @@ impl ContainerRuntime for DockerRuntime {
             cmd.arg("--read-only");
         }
 
-        match manifest.app.security.network_policy.as_str() {
-            "host" => {
-                cmd.arg("--network").arg("host");
-            }
-            "isolated" => {
-                // Docker uses bridge network by default
-            }
-            _ => {
-                cmd.arg("--network")
-                    .arg(&manifest.app.security.network_policy);
-            }
-        }
-
-        // Port mappings with offset
-        for port in &manifest.app.ports {
-            let host_port = port.host + port_offset;
-            cmd.arg("-p")
-                .arg(format!("{}:{}", host_port, port.container));
-        }
+        cmd.args(docker_network_and_ports(manifest, port_offset)?);
 
         // Volumes
         for volume in &manifest.app.volumes {
@@ -1035,6 +1057,21 @@ mod tests {
     use super::*;
     use std::collections::HashMap;
 
+    #[test]
+    fn docker_fallback_rejects_rootless_only_topology_and_preserves_bind_protocol() {
+        let mut m =
+            AppManifest::parse(include_str!("../../../apps/portainer/manifest.yml")).unwrap();
+        assert!(docker_network_and_ports(&m, 0).is_err());
+        m.app.container.network = Some("bridge".into());
+        m.app.ports[0].protocol = "udp".into();
+        let args = docker_network_and_ports(&m, 1).unwrap();
+        assert_eq!(
+            args,
+            vec!["--network", "bridge", "-p", "127.0.0.1:9001:9000/udp"]
+        );
+        assert!(docker_network_and_ports(&m, u16::MAX).is_err());
+    }
+
     #[test]
     fn missing_container_classifier_covers_podman5_phrasings() {
         // podman 5.x `inspect` phrasing for a missing container.

docker/archipelago-source/UPSTREAM.md

@@ -16,9 +16,11 @@ lookup relays from the defaults. It does not replace GitWorkshop's NIP-34,
 GRASP, repository browser, issue, pull-request, or review interfaces.
 
 The separate dependency patch refreshes the npm lockfile and moves `fflate` to
-0.8.3, `react-router-dom` to 7.18.3, and Vitest to 5.0.0. The resulting clean
-install reports zero npm advisories; its type-check, 152 unit tests, and
-Archipelago subpath production build pass. Keeping this mechanical security
+0.8.3, `react-router-dom` to 7.18.3, and Vitest to 5.0.0. On 2026-09-30 the lockfile was refreshed again for `brace-expansion`
+1.1.21/5.0.12, `fast-uri` 3.1.8 and `ip-address` 10.7.2 after fresh node
+installs failed the retained dependency audit. The resulting clean install
+reports zero npm advisories; its type-check, 152 unit tests, and Archipelago
+subpath production build pass. The complete image also builds on the X250. Keeping this mechanical security
 update separate makes both the upstream integration and future dependency
 refreshes auditable.
 

docker/archipelago-source/gitworkshop-dependencies.patch

@@ -1,5 +1,5 @@
 diff --git a/package-lock.json b/package-lock.json
-index 20631bb..0933917 100644
+index 20631bb..86b6f86 100644
 --- a/package-lock.json
 +++ b/package-lock.json
 @@ -63,7 +63,7 @@
@@ -495,9 +495,9 @@ index 20631bb..0933917 100644
 -      "version": "5.0.7",
 -      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz",
 -      "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==",
-+      "version": "5.0.9",
-+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
-+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
++      "version": "5.0.12",
++      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.12.tgz",
++      "integrity": "sha512-YovQ3rzhaLMIrDjNDMkNS01tea93qhEhG5xy8f6+R0l+dw3Ki+5sCoIoI942iuLZTHWogWktgwVDhU09iNEimQ==",
        "dev": true,
        "license": "MIT",
        "dependencies": {
@@ -678,9 +678,9 @@ index 20631bb..0933917 100644
 -      "version": "1.1.15",
 -      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.15.tgz",
 -      "integrity": "sha512-EwOCDEex4quD37XhqM3omwtMoJjr//isUZz1JopUNWms+4Z2ViyM/k1YIRePpoVNnQhENnxtFjLaxNHrT7xIUg==",
-+      "version": "1.1.18",
-+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
-+      "integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
++      "version": "1.1.21",
++      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
++      "integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
        "dev": true,
        "license": "MIT",
        "dependencies": {
@@ -833,9 +833,9 @@ index 20631bb..0933917 100644
 -      "version": "3.1.3",
 -      "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.3.tgz",
 -      "integrity": "sha512-i70LwGWUduXqzicKXWshooq+sWL1K3WUU5rKZNG/0i3a1OSoX3HqhH5WbWwTmqWfor4urUakGPiRQcleRZTwOg==",
-+      "version": "3.1.7",
-+      "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.7.tgz",
-+      "integrity": "sha512-dOvZVzjdZdz7phd9v6jCbwxrBW3fK6n8Rc0CtdmM4bumzMnxywBYhuph6J819RRw/ku+rLbelwfMunktuzVVHg==",
++      "version": "3.1.8",
++      "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.8.tgz",
++      "integrity": "sha512-GZMtZUTNRpOVIECoXwLNZS5xUGE+mVNbTB8h/7Rwh2TFWcBQiPzTgyZi05BF9UMZKkLJv8XBRJTlU7zg8+ZfMg==",
        "funding": [
          {
            "type": "github",
@@ -859,9 +859,9 @@ index 20631bb..0933917 100644
 -      "version": "5.0.7",
 -      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.7.tgz",
 -      "integrity": "sha512-7oFy703dxfY3/NLxC1fh2SUCQ0H9rmAY+5EpDVfXjUTTs+HEwR2nYaqLv+GWcTsumwxPfiz6CzCNkwXwBUwqCA==",
-+      "version": "5.0.9",
-+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
-+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
++      "version": "5.0.12",
++      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.12.tgz",
++      "integrity": "sha512-YovQ3rzhaLMIrDjNDMkNS01tea93qhEhG5xy8f6+R0l+dw3Ki+5sCoIoI942iuLZTHWogWktgwVDhU09iNEimQ==",
        "dev": true,
        "license": "MIT",
        "dependencies": {
@@ -893,9 +893,9 @@ index 20631bb..0933917 100644
 -      "version": "10.2.0",
 -      "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
 -      "integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
-+      "version": "10.7.0",
-+      "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.7.0.tgz",
-+      "integrity": "sha512-BGFsyJd5mpXp3rK6jIdADLNgpJUK1jnjzvYF8lK+VyDab9JAmqN0YOKDdP17HlgKb2+ehPgDc8EtnRLbGCAMhA==",
++      "version": "10.7.2",
++      "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.7.2.tgz",
++      "integrity": "sha512-7H/2gFSIitxc0hG3nOI1glS8QLo/EHBFFLk8vEUjXY/xu0AdL8jZ9U1IzO2PUm0d2D/ofQcAifb0g6OBkt8U7w==",
        "license": "MIT",
        "engines": {
          "node": ">= 12"
@@ -1445,4 +1445,3 @@ index bd7190c..6aa5a6f 100644
  import { vi } from "vitest";
  
  // Mock window.matchMedia
-

docker/bitcoin-ui/Dockerfile

@@ -1,4 +1,4 @@
-FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+FROM source.archipelago-foundation.org/lfg2025/nginx:1.27.4-alpine
 # Static site content.
 COPY index.html /usr/share/nginx/html/
 COPY tailwind.css /usr/share/nginx/html/

docker/cuprate-ui/Dockerfile

@@ -1,4 +1,4 @@
-FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+FROM source.archipelago-foundation.org/lfg2025/nginx:1.27.4-alpine
 # Static site content.
 COPY index.html /usr/share/nginx/html/
 COPY 50x.html /usr/share/nginx/html/

docker/electrs-ui/Dockerfile

@@ -1,4 +1,4 @@
-FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+FROM source.archipelago-foundation.org/lfg2025/nginx:1.27.4-alpine
 COPY index.html /usr/share/nginx/html/
 COPY 50x.html /usr/share/nginx/html/
 COPY qrcode.js /usr/share/nginx/html/

docker/fedimint-ui/Dockerfile

@@ -1,4 +1,4 @@
-FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+FROM source.archipelago-foundation.org/lfg2025/nginx:1.27.4-alpine
 
 COPY index.html /usr/share/nginx/html/index.html
 COPY nginx.conf /etc/nginx/conf.d/default.conf

docker/fips-ui/Dockerfile

@@ -1,4 +1,4 @@
-FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+FROM source.archipelago-foundation.org/lfg2025/nginx:1.27.4-alpine
 # Static site content.
 COPY index.html /usr/share/nginx/html/
 #

docker/lnd-ui/Dockerfile

@@ -1,4 +1,4 @@
-FROM git.tx1138.com/lfg2025/nginx:1.27.4-alpine
+FROM source.archipelago-foundation.org/lfg2025/nginx:1.27.4-alpine
 
 # Copy the HTML file
 COPY index.html /usr/share/nginx/html/

docs/TODO.md

@@ -14,34 +14,81 @@ doc. See [`ROADMAP.md`](ROADMAP.md) for the curated, public-facing direction.
 
 ## Next release after 1.8.21 — reported 2026-09-30
 
-- [ ] **ThinkPad X250 kiosk: Bitcoin installation version selector is unreadable
-  and appears underneath the pruning information.** Operator reports white
-  styling with invisible text on the actual kiosk; the same flow works in remote
-  Brave. Reproduce on the X250's kiosk engine and record its version, display
-  scale and resolution. Inspect the native `<select>` in
-  `neode-ui/src/components/InstallVersionModal.vue`, its option colors, and the
-  scroll/stacking behavior in `BaseModal.vue`; these are investigation leads,
-  not a confirmed cause. Fix contrast and popup visibility without changing
-  version selection or pruning behavior. Validate Core and Knots, open/closed
-  and scrolled dropdowns, keyboard/touch selection, and pruning on/off on the
-  actual kiosk, with remote Brave and mobile regression checks. Browser mocks
-  alone do not establish that the kiosk rendering is fixed. Track for the next
-  release; the signed 1.8.21 artifacts remain unchanged.
+Release status and acceptance gates: [execution checklist](next-release-20260930.md).
 
-## Current repair and release tasks — 2026-09-29
+- [ ] **Release blocker: Gitea → Portainer repository integration.** Diagnose
+  smart-HTTP reachability from Portainer's actual request namespace, then provide
+  one declarative topology and idempotent migration for fresh installs and
+  existing nodes. Preserve gate/auth boundaries, operator configuration,
+  repository/key/database mounts and Portainer stacks. Cover install order,
+  lifecycle/reboot/update convergence, clone/push and source-branch/Compose-file
+  acceptance with a disposable integration setup. Ship in both OTA and ISO;
+  a healthy Gitea root page is insufficient. Operator supplied a private handover;
+  deployment addresses and credentials must not be committed.
 
-Release is blocked until these pass; see [execution record](repair-release-20260929.md).
+- [ ] **New X250: GitWorkshop failed at 70%; slow Nginx installation.** Missing
+  ISO build contexts restored on-node; package staging/smoke checks added.
+  GitWorkshop dependency audit refreshed and build/HTTP recovery verified;
+  Nginx was a slow successful image pull. Aggregate progress label corrected.
+  Include the validated repair in the next OTA/ISO. See lifecycle evidence.
 
-- [ ] Fix Cashu paid-file redemption between dev and Shorty; test keyset IDs,
+- [ ] **Angor indexer service in the app store**, requested after the other
+  current repair/review work (2026-09-30). Follow the repository's app-development
+  and packaging documentation; treat it as a headless service unless upstream
+  documentation establishes a UI. Verify Bitcoin/Mempool requirements, decide
+  whether an existing first-class relay meets Angor's requirements or a relay
+  must be packaged with the indexer, and use the Angor logo from angor.io for its
+  service icon. Official current deployment documentation located and reviewed: stock Mempool
+  plus an optional strfry relay. Both headless services and the dependency guard
+  are implemented; API outage/recovery and five relay lifecycle cycles passed.
+  Final candidate install/lifecycle checks and signed delivery remain pending.
+  Install on the development box; Bitcoin must finish syncing for indexed queries.
+
+- [ ] **App lifecycle: keep installed apps visible through restart and hard
+  refresh; gate embedded/browser launches on actual web and listener readiness.**
+  Source repair and scoped live acceptance passed; full release gate pending.
+  Includes durable inventory reconstruction,
+  concurrent inventory writes, stale scan/lifecycle updates, delayed HTTP startup,
+  and the app gate's post-install listener delay. See
+  [app lifecycle repair evidence](app-lifecycle-repair-20260930.md).
+
+- [x] Review and repair open paid-download PRs #161 and #162, refresh both
+  branches from main, run independent and combined isolated suites, and verify
+  rootless file permissions in disposable scratch storage. Combined result:
+  1,585 passed, zero failed, four existing tests ignored. See the
+  [review evidence and remaining acceptance work](pr-review-20260930.md).
+- [x] Integrate the reviewed PR branches into the next release and run funded
+  candidate acceptance, including Tor-only transport and payments with change.
+  Operator authorized completing the normal merge/closure workflow on
+  2026-09-30. Both PRs are now merged and closed through Gitea; integrate
+  local repair commits and sync git/ngit before release. The combined candidate
+  is deployed on both test endpoints. Funded Tor-only purchase with change,
+  confirmed refund, exact Files bytes and zero-cost repeat delivery passed.
+  The updated source still needs inclusion in signed OTA/ISO artifacts.
+- [ ] Design durable recovery for an accepted payment whose response is lost.
+  Preserve the truthful unconfirmed-refund warning and prevent automatic
+  duplicate payment while that recovery work is outstanding.
+- [x] **ThinkPad X250 kiosk: Bitcoin version choices readable above pruning.**
+  Replaced the native popup with inline radio choices. Actual Chromium 152 kiosk
+  assertions and screenshot verify white-on-dark choices, selection changes and
+  layout above pruning controls. Focused component tests pass. Included in the
+  next-release source; published 1.8.21 artifacts remain unchanged.
+
+## 1.8.21 repair and release tasks — completed 2026-09-30
+
+See the [execution record](repair-release-20260929.md) for evidence and limits.
+
+- [x] Fix Cashu paid-file redemption between dev and Shorty; test keyset IDs,
   mint errors, fees, and refund reporting before live validation.
-- [ ] Complete the remaining Framework incident verification and evidence.
-- [ ] Replace the unavailable tx1138.com explorer default with mempool.space;
+- [x] Record Framework verification and the operator's acceptance of the
+  remaining display check before release.
+- [x] Replace the unavailable tx1138.com explorer default with mempool.space;
   migrate the old default with fresh consent and preserve custom/local explorers.
-- [ ] Offer pruning in the Bitcoin installation version modal, using the same
+- [x] Offer pruning in the Bitcoin installation version modal, using the same
   pruning settings as automatic pruning even on large disks.
-- [ ] Explain Bitcoin warmup without raw RPC errors; gate LND unlock on Bitcoin
+- [x] Explain Bitcoin warmup without raw RPC errors; gate LND unlock on Bitcoin
   RPC readiness and show install/start/sync waiting states with automatic recovery.
-- [ ] Test the completed changes on this development box, then publish a new
+- [x] Test the completed changes on this development box, then publish a new
   signed OTA and raw ISO release. Record any remaining verification gaps.
 
 ## Dev & build process (priority)

docs/app-developer-guide.md

@@ -765,3 +765,13 @@ Every supported app must satisfy the lifecycle contract:
 For apps with special dependencies, launch must explain dependency wait states instead of showing a dead iframe. Examples include Bitcoin sync/IBD, Lightning wallet readiness, Nostr signer bridge injection, Tailscale login/auth, and app-specific setup screens.
 
 Runtime changes should be validated with focused tests first, then the release lifecycle harness on the validation host when host access is intentionally resumed.
+
+### Adapters for shared services
+
+A service that reuses an installed stack can declare `install_prerequisites`
+with the required component app ids and keep the runtime relationship in
+`dependencies`. This refuses an incomplete installation before creating the
+adapter instead of reporting a successful installation with no usable backend.
+For example, Angor Indexer requires `mempool-api` (shown to users as its owning
+Mempool app), shares that index and declares only an `api` interface. API-only
+interfaces belong in Services and do not generate browser launch buttons.

docs/app-lifecycle-repair-20260930.md

@@ -0,0 +1,111 @@
+# App lifecycle repair — 2026-09-30
+
+Status: source repairs, optimized build, new-node recovery and scoped live
+lifecycle acceptance verified. Full release gate remains pending.
+These are next-release changes. Published 1.8.21 artifacts remain unchanged.
+
+## Report
+
+The operator reports that restarting an app can make it disappear, and a hard
+refresh offers installation again. Newly installed apps sometimes fail to
+connect in both embedded views and browser tabs. The new X250 additionally reproduced GitWorkshop disappearing during install
+and Nginx Proxy Manager spending approximately 14 minutes at 70%. A disposable
+app on the dev box exposed a separate restart failure.
+
+## Findings and repairs
+
+- Quadlet removes containers during stop/restart. The scanner protected existing
+  in-memory entries but did not reconstruct an absent app on a fresh daemon.
+  It now synthesizes stopped entries from the durable installed set, respecting
+  uninstall records, normalizing container prefixes, and preserving cached
+  metadata. Absence does not establish an image version or available update.
+- Concurrent read/modify/write operations could lose installed-app records;
+  in-place writes could expose truncated JSON to readers. Serialize writers,
+  publish by atomic rename, and sync the file and parent directory. Legacy
+  package install/uninstall success paths update the durable record too.
+- Scans and lifecycle/progress operations could replace a newer model from an
+  older snapshot. Use locked mutations for lifecycle/progress, and merge scan
+  results only into entries unchanged since the scan's merge snapshot.
+- Container running state and TCP accept alone did not establish HTTP readiness.
+  Add explicit `ui-ready` based on bounded HTTP probes of the loopback upstream;
+  reject connection failures and server errors, accept normal redirects and
+  authentication challenges, and do not follow redirects or send credentials.
+  Self-signed HTTPS apps are probed locally without certificate validation.
+- The app gate swept new listeners only every 60 seconds. Wake that sweep
+  immediately for a ready upstream whose declared gate port is not yet claimed,
+  and withhold readiness until external and Tor listener claims exist.
+- Fixed launch URLs could bypass suppressed runtime URLs. Enforce readiness in
+  app cards, details, centralized embedded/browser launchers, and session frames.
+  Starting/restarting clears readiness immediately. A waiting frame does not
+  load an iframe and resumes when the backend reports readiness.
+
+### New X250 findings
+
+- The published ISO copied only `bitcoin-ui`, `lnd-ui` and `electrs-ui` build
+  directories. GitWorkshop failed because `/opt/archipelago/docker/archipelago-source`
+  was missing. Copy the complete docker source tree for bundled and unbundled
+  ISOs, matching OTA packaging. Validate every manifest build context and
+  Dockerfile in OTA staging, ISO staging and the mounted ISO smoke test.
+- After restoring the omitted contexts, GitWorkshop's retained npm audit rejected
+  newly reported brace-expansion, fast-uri and ip-address vulnerabilities.
+  Refresh the existing pinned dependency patch, keeping the audit enabled.
+  Clean install/audit (zero advisories), type-check, 152 upstream tests and
+  subpath production build pass. The image builds on the X250 and `/healthz`
+  returns 200. No wallet or Bitcoin container restart was needed.
+- Nginx was receiving data, not frozen: over 1 GB read during the pull. It
+  completed at 12:40:46 UTC after starting at 12:26:27; its web endpoint returns
+  200. The orchestrated path previously labelled the entire download/build/start
+  operation "Creating container" at 70%. Give that aggregate operation its own
+  truthful label and earlier phase; no byte-level download estimate is claimed.
+- Restore install progress immediately from an already-loaded server snapshot,
+  so a new store created after hard refresh does not wait for another mutation.
+- Replace the install modal's native version popup with inline radio choices.
+  On this actual X250's Chromium 152 kiosk renderer, selection changes work,
+  options have white text on dark backgrounds, and remain above pruning controls.
+  Screenshot and browser assertions captured; no install confirmation was clicked.
+
+### Restart safety
+
+The disposable fixture restart at 12:38:05 UTC stopped its container, then
+`ss | kill` in runtime port cleanup sent SIGTERM to the management daemon at
+12:38:35. The daemon owned the gate listener on the same port at other addresses.
+Systemd restarted management; Bitcoin and LND container IDs/start times were
+unchanged. Remove port-owner kills and broad `pkill` patterns from restart,
+install recovery and Grafana preparation. Recovery now uses the existing
+container-ID-aware ghost reaper: absent container ownership must be established
+before a process is terminated. A real listening-socket regression checks that
+conflict cleanup preserves the host listener. App-gate manifest lookup now honors
+`ARCHIPELAGO_APPS_DIR`, matching the orchestrator's configured manifest root.
+
+## Validation
+
+- Full frontend suite: 139 files, 1,126 tests passed; final focused kiosk/store
+  checks: nine passed. Production frontend build passed.
+- Final isolated backend suite: 1,567 passed, zero failed, four existing ignored
+  tests. Optimized backend build passed and was deployed to the development node.
+- Tests cover empty runtime inventory, alias deduplication, uninstall exclusion,
+  concurrent durable writes, concurrent state changes, stale scan publication,
+  TCP-without-HTTP, HTTP statuses including 502/503, and gate listener claims.
+- Live disposable Node fixture delayed HTTP startup by 25 seconds. Desktop and
+  mobile retained the waiting screen through hard refresh without mounting an
+  iframe, then opened the exact fixture page automatically when ready.
+- Restart retained the app in both state APIs throughout and returned to ready;
+  the management PID did not change. Stopping removed the Quadlet container;
+  restarting management reconstructed its installed/stopped entry without a
+  false update offer. Starting it again succeeded. Desktop and mobile continued
+  to show the installed app after hard refresh.
+- LAN access required node authentication and returned exact fixture bytes after
+  authentication. The fixture was uninstalled through the package lifecycle API;
+  its temporary manifest root and service override were removed.
+- Bitcoin and LND container IDs and start times stayed unchanged through all
+  scoped checks and management restarts. No wallet data was used by the fixture.
+- X250 kiosk checks also opened the repaired GitWorkshop and Nginx Proxy Manager
+  pages successfully, with no failed local resource loads.
+
+## Limits
+
+This prevents the identified lifecycle/readiness failures; it cannot guarantee
+that an app or network never fails after a successful readiness check. Actual
+application failures must remain visible rather than being labelled successful.
+The full lifecycle/reboot release gate and funded acceptance of the reviewed
+paid-download PRs remain pending. The X250 kiosk fix has live rendering evidence.

docs/app-manifest-spec.md

@@ -290,3 +290,35 @@ app:
 Validate with `scripts/validate-app-manifest.sh` and regenerate the catalog
 with `scripts/generate-app-catalog.py` (drift-checked in CI by
 `scripts/check-app-catalog-drift.py`).
+
+### Persistent-state backup for runtime repairs
+
+`app.backup_before_runtime_change: true` opts an app into a stopped-state snapshot
+before reconciliation changes a service’s network, ports, security settings,
+command or health configuration. Image-upgrade backup policy remains separate. The orchestrator
+archives writable persistent bind mounts under the node data directory, collapses
+nested mounts, excludes the runtime Podman socket, and preserves the previous
+Quadlet definition for rollback. Named volumes, outside-data-root state and
+symlinked mount roots fail closed rather than silently producing an incomplete
+backup. A failed snapshot resumes the original service and leaves migration
+pending. Private archives are retained under `migration-backups/`; fresh installs
+and unchanged runtime configurations do not create migration snapshots.
+
+Catalog generation preserves the previously published base manifest for older
+daemons and puts opted-in network changes in a signed `manifest_variants` entry
+requiring `runtime-migration-backup-v1`. New runtimes select only variants whose
+complete requirement list they support. Supply `BASE_CATALOG` when generating
+against a different reviewed pre-migration catalog. This keeps catalog refresh
+from applying a migration before the matching OTA code is installed.
+
+### Existing shared-service prerequisites
+
+`app.install_prerequisites` is an optional list of existing app ids, for example
+`[mempool-api]` for a headless indexer adapter. The runtime checks their manifest
+container names before recording installation or changing any dependency. If one
+is missing, installation refuses with its owning app's title and removes the
+optimistic install tile. Runtime observation errors fail closed. This does not
+automatically install dependencies, alter Bitcoin pruning, or require a synced
+backend merely to recognize an already-installed service. Declare ongoing
+relationships separately in `dependencies`; use the app health check for actual
+API readiness. Self-dependencies and malformed ids are invalid.

docs/gitea-portainer-repair-20260930.md

@@ -0,0 +1,167 @@
+# Same-node Gitea sources in Portainer
+
+Status: root cause reproduced and network repair verified in disposable and actual
+production Portainer instances; final migration integration and release acceptance remain in progress.
+This change belongs to the next signed catalog, OTA and ISO. It does not modify
+published 1.8.21 artifacts.
+
+## Confirmed cause
+
+On the affected X250, Gitea 1.27.3 and Portainer 2.45.0 run in rootless Podman
+5.4.2, managed by user Quadlet services. Gitea publishes HTTP on loopback and the
+Archipelago app gate serves its public port. Gitea's public ROOT_URL already
+matches that gate URL.
+
+Portainer had no explicit network selection and Podman selected pasta. Its
+network namespace contained the host's LAN address. A Git request to that same
+LAN address therefore reached Portainer's namespace rather than the host gate:
+connection refused before authentication. The exact smart-HTTP request from the
+host returned 200 with `application/x-git-upload-pack-advertisement`. From
+Portainer's actual namespace the LAN request was refused, while its host mapping
+returned a Git advertisement and the expected branch tip. Direct container-IP
+requests timed out. Container health and host-only HTTP checks missed the defect.
+
+A disposable Portainer using `slirp4netns` successfully created a Source through
+Portainer's own API, using the original LAN clone URL. Returning that fixture to
+pasta reproduced the refusal; recreating with slirp repaired it while preserving
+its account and saved Source. Restart also passed. The requested branch tip and
+Compose file were read from that actual Portainer network namespace. No user
+stack was deployed. Deployment addresses and repository details are kept outside
+this public record.
+
+## Source changes
+
+- Declare Portainer's rootless `slirp4netns` mode in its manifest. No shared static
+  container IP, host networking, all-interface backend publication or auth bypass.
+- Keep Gitea's loopback HTTP backend and gate port; machine Git uses Gitea's
+  authentication. Remove obsolete port-3000 nginx metadata/template and the old
+  best-effort installer commands which silently rewrote app.ini and falsely
+  claimed success. Gitea owns first-run setup and operator configuration.
+- Gitea SSH also failed before authentication: OpenSSH logged a denied
+  `chroot("/var/empty")` because the manifest dropped `SYS_CHROOT`. Add that
+  specific sandbox capability and reconcile security-directive changes. A
+  disposable fixture then passed SSH clone/push with host-key checking enabled.
+- Existing Quadlet reconciliation applies Network= drift. Record a durable
+  pending restart before updating the unit and clear it only after a successful
+  restart, so failed reloads/restarts and management interruptions retry.
+- Detect explicit rootless network-mode drift in the older Podman runtime too.
+  Unspecified networks do not trigger inferred changes to unrelated apps.
+- Portainer and Gitea opt into `backup_before_runtime_change`. Before recreation, gracefully
+  stop the app and archive its writable persistent bind mounts, including nested
+  Compose state, once each. Runtime sockets are excluded. Save the previous
+  Quadlet definition, where present. Archives live under the node data directory's
+  private `migration-backups/<id>/` directory; state is never deleted. Backup
+  failures resume the original service and fail the migration visibly.
+- Keep Podman API and Quadlet bind/network behavior covered by actual-manifest
+  tests. Docker remains a development fallback: it now preserves bind/protocol
+  declarations and rejects Podman-only networking instead of silently changing it.
+
+## Operator use and diagnostics
+
+Use Gitea's advertised HTTP(S) clone URL in Portainer Sources, with the Gitea
+username and token in the credential fields. On first-run Gitea setup, the public
+base URL must match the origin opened through Archipelago (including its port).
+Keep a deliberately configured HTTPS/domain origin when one exists. Do not use a
+container IP or put a token into the URL. A private repository requires repository
+read permission. A successful Source check fetches Git refs; it does not deploy
+a stack or establish that a Compose build uses a desired application revision.
+
+`scripts/check-portainer-git-source.py` calls Portainer's own read-only Source
+connection test. Supply a private mode-600 JSON credential file containing
+`api_key` or `jwt`, and optionally `git: {username, password}`. Pass
+`--portainer-url`, `--repository-url` and `--credentials-file`. It does not create
+Sources or stacks and prints no credentials or raw server errors. It distinguishes
+Portainer login/API failures from Git connection refusal, timeout, DNS/TLS
+failure, HTML/login interception and repository authentication failure. TLS
+verification stays enabled and API redirects are refused.
+
+## Upgrade and rollback
+
+The signed catalog embeds manifests and overrides installed disk copies.
+Capability-gated manifest variants keep the previous Portainer manifest as the
+base for older daemons; only daemons supporting `runtime-migration-backup-v1`
+select the network repair. This prevents catalog refresh from triggering an
+unbacked recreation before the OTA is installed. A disk
+edit alone cannot deliver this fix. Publish the matching catalog with the tested
+runtime, then verify the generated unit, actual network mode and Source API.
+Expect a Portainer interruption while the snapshot and recreation run; duration
+depends on its saved state size.
+The Portainer routing repair does not require a Gitea configuration change.
+The separate SSH capability repair does recreate Gitea, preserving and snapshotting
+both data/config mounts first. Supported systemd drop-in overrides remain intact.
+
+Keep the previous trusted catalog/runtime for rollback. Restore that catalog
+before restoring the saved `previous.container`, reloading user systemd and
+starting Portainer; otherwise reconciliation will correctly reapply the new
+manifest. The archive is a stopped-state emergency backup, not an instruction to
+roll back a live database automatically. Restore it only with Portainer stopped
+and after preserving any newer state. Do not replace Gitea data/config, keys,
+repositories or the production Portainer database with disposable test data.
+
+## Validation and remaining gates
+
+- Disposable X250 Portainer Source API: old mode refuses; repaired mode succeeds;
+  saved account/Source survive recreation; restart succeeds.
+- Invalid Git credentials produce a repository-authentication error, distinct
+  from TCP refusal. Requested branch and Compose file read from Portainer context.
+- Combined backend suite including the reviewed paid-download PRs and catalog
+  rollout guard: 1,605 passed, zero failed, four existing ignored
+  tests, including stopped-state archive round trips and failure preservation. Container runtime suite: 78 passed.
+  Five diagnostic regression tests passed; catalog regeneration is idempotent
+  and the generated catalog has zero manifest metadata drift.
+- Fresh managed Gitea and Portainer fixtures: authenticated private Source
+  creation, invalid-token rejection, workstation clone/push and exact branch
+  lookup from Portainer namespace passed. LFS batch/upload/download and OCI
+  registry authentication/blob/manifest round trips passed. Desktop and mobile
+  login/private-repository/assets/hard-refresh checks passed.
+- Still required before release: live automatic migration with the new runtime,
+  snapshot/rollback verification and reversed install-order acceptance,
+  lifecycle/reboot convergence, and signed-catalog delivery to the existing app.
+  Record LFS/registry/SSH/browser checks and actual hardware/runtime coverage.
+
+### Affected X250: production routing repair verified
+
+Applied the tested rootless network setting to the actual installed Portainer
+through a persistent Quadlet drop-in, after gracefully stopping it and creating a
+private archive of its database and Compose directory. Compared the archive
+against the stopped original before changing configuration; retained the original
+unit and a rollback path. A verification helper initially compared mount list
+order rather than mount identity and safely rolled back; the corrected check
+compares sorted source/destination/write-mode tuples and passed.
+
+The actual production Portainer namespace reproduced connection refusal before
+repair. After repair it received a Git smart-HTTP advertisement, fetched the
+requested branch at its current tip and read its Compose file. Repeating these
+checks after restarting the managed Portainer service passed. All original data
+and socket mounts and the loopback-only HTTP binding are retained. Gitea,
+Bitcoin and the wallet container IDs and start times were unchanged. No stack
+was deployed and no repository credential was changed.
+
+This establishes the routing repair on the affected hardware. A logged-in
+production Portainer Source UI/API acceptance has not yet been recorded; the
+corresponding API checks passed on disposable instances as documented above.
+The installed-node drop-in persists through service restart/reboot but is not the
+fleet delivery mechanism. Automatic migration and signed catalog/OTA/ISO release
+validation remain pending; the source manifest declares the same network mode.
+Private deployment addresses, branch details and state archives are not committed.
+
+### Managed automatic migration and archive restore
+
+The new runtime candidate migrated an existing managed fixture from pasta to
+slirp without a manual unit edit. It preserved the account, saved Source and
+mount set, saved a private stopped-state archive plus the previous unit, restored
+Source API access, and cleared the pending restart marker. A management-service
+restart preserved the new container identity/start time and did not create
+another archive. The archive extracted into an isolated scratch directory and
+compared cleanly, including the database and Compose directory. Rootless archive
+ownership required scratch cleanup inside `podman unshare`; no production data
+was overwritten. Native Bitcoin and LND IDs/start times remained unchanged.
+
+This optimized candidate predates the final bounded backup-retry guard; that
+latest source passed the isolated 1,605-test suite and must also be exercised in
+the final release build. A fixture-only systemd start failure was then injected during a security
+directive migration. The failure retained the durable restart marker. After
+removing the injected failure, the reconciler restarted the service without a
+manual container start, restored Source API access and cleared the marker.
+Reverse install order, final-build retry-budget coverage, full reboot and
+signed delivery remain open.

docs/next-release-20260930.md

@@ -0,0 +1,459 @@
+# Next OTA and raw ISO after 1.8.21
+
+**Status: COMPLETE — 1.8.22-alpha OTA, compatible signed app catalog and raw ISO published on Git and ngit on 2026-10-01; artifact signatures, public downloads and fleet feed verified. Angor full-chain indexing still awaits dev Bitcoin synchronization.**
+
+Current acceptance evidence: [1.8.22 release acceptance](release-1.8.22-acceptance.md).
+The chronological notes below retain earlier failures and superseded candidates;
+the final tested source is `6d5f3ffb`.
+
+This is the consolidated execution checklist for the operator's chat requests.
+Release acceptance and publication are complete, with live wallet and app data
+preservation checks documented below. No universal absence of future failures
+is claimed.
+
+## Changes already shipped in 1.8.21 or earlier
+
+Keep these fixes in the next build and include relevant regressions:
+
+- Mempool image/catalog version agreement and update-button behavior.
+- Minibits integration; Framework automatic LND startup and safe unavailable
+  balances. Framework incident closed with operator acceptance.
+- Shorter, single-column ecash backup messaging.
+- AIUI transparent background on desktop/mobile.
+- Cashu paid-file keyset/mint/error/refund corrections, with live purchases.
+- mempool.space explorer fallback, preserving local/custom explorer settings.
+- Bitcoin install pruning choice and matching automatic-pruning behavior.
+- Friendly Bitcoin warmup and LND install/start/sync waiting states.
+- Raw ISO publishing and upload support.
+
+The Primal automatic LNURL comment problem was traced to sender behavior and
+Minibits metadata. The user accepted clearing the sender's automatic comment;
+no unsupported local metadata rewrite or wallet-identity replacement is planned.
+See the Framework incident and 1.8.21 execution records for evidence/limits.
+
+## New release scope and gates
+
+| Task | Implemented/verified | Remaining before release |
+| --- | --- | --- |
+| X250 Bitcoin picker | Inline choices; actual Chromium kiosk selection, readability and pruning layout passed | Final UI/build checks passed |
+| App disappearance/readiness | Durable inventory and safe lifecycle repair; delayed HTTP and desktop/mobile hard-refresh checks passed | Final candidate lifecycle, hard-refresh and stability checks passed |
+| X250 GitWorkshop/Nginx | Missing build contexts restored, dependency/build checks and live UI passed; Nginx slow pull diagnosed; truthful progress label | OTA and ISO build-context/content checks passed |
+| PRs 161/162 | Reviewed, repaired, merged/closed normally; combined regression suite passed | Funded Tor-only candidate purchase, retained change, refund, Files bytes and cached repeat passed; included in signed artifacts |
+| Gitea/Portainer | Root cause confirmed; source network/backup/retry/catalog changes; real X250 routing repair and restart verified; private Git, SSH, LFS, registry and browser fixture checks passed | Automatic migration, scratch restore, failed-start recovery and reverse installation order passed. Operator confirms production site works through Portainer; production host reboot also preserved network/Git/Compose access; final candidate delivery and integration checks passed |
+| Angor headless store service | Implemented standard Mempool adapter and separate optional relay, official logo, headless store entries and declarative dependency guard. API security/outage/DNS tests and five relay lifecycle cycles passed | Final candidate prerequisite/API, lifecycle and catalog checks passed; real indexing on dev waits for Bitcoin sync |
+
+Durable payment receipts after a lost seller response remain a separately
+recorded design follow-up. Preserve the truthful unconfirmed-refund warning and
+prevent duplicate automatic payment; do not describe an unconfirmed refund as
+completed. See PR review for the accepted scope and coverage limits.
+
+## Final release checklist
+
+- [x] Finish new-scope implementation and release acceptance; full-chain Angor
+      indexing still depends on the dev node finishing initial sync.
+- [x] Remove disposable fixtures and temporary test overrides; verify native
+      Bitcoin/LND identity and start-state baselines remain protected.
+- [x] Commit and push completed source changes to git and ngit.
+- [x] Run final backend/UI/regression/release gates on the final source; inspect
+      skipped tests and report actual hardware/runtime coverage.
+- [x] Prepare compatible signed app catalog; old runtimes must not apply a
+      migration before they have backup/recovery support.
+- [x] Version/changelog and OTA payload prepared, validated and signed by user.
+- [x] Raw ISO built; payload hashes/content verified; full installation and
+      installed-system boot tested in QEMU/KVM without network.
+- [x] User signs ISO checksums; publish OTA and ISO plus verification files on
+      git and ngit; independently read back hashes and update discovery.
+- [x] Provide LAN scp command for the new raw ISO.
+
+Latest backend source verification: 1,617 passed, zero failed, four existing
+ignored tests. This is one layer of evidence, not a substitute for live gates.
+
+## Angor verification — 2026-09-30
+
+- Isolated backend suite: 1,606 passed, four existing ignored; container suite:
+  79 passed. Frontend: 140 files / 1,130 tests passed; production build passed.
+- Disposable rootless API gateway: versioned and legacy API paths, query/body
+  forwarding, transaction-only POST, method/body limits, CORS, removal of
+  dashboard credentials, read-only non-root operation, truthful backend outage
+  and DNS recovery after backend recreation passed. No real transaction broadcast.
+- Dedicated relay: NIP-11, signed event publish/read, invalid signature rejection
+  and event/config persistence across five managed stop/start/restart cycles
+  passed. Internal relay identity and start time stayed unchanged. Follow-up
+  acknowledgement samples were 2–9 ms through both backend and app gate.
+- Published adapter 1.0.1 and relay 1.1.2 to the authenticated maintainer namespace.
+  Anonymous registry readback succeeded. Adapter digest:
+  `sha256:997be611700b55c521ad801fa92daaca2ae6951ac71407434c85eb9603f77c38`;
+  relay mirror digest:
+  `sha256:80444ad1304a0e504948b48ea1550c091b18b9f10757f07ce9a68fc261b8f6c1`.
+- Delivery target is the development box, as clarified by the operator. Do not
+  install Angor on the separate Portainer node. Full indexer availability still
+  requires the dev box's Bitcoin sync and Mempool/Electrum indexing to finish.
+- Funded PR acceptance passed after the operator funded the dev Cashu wallet
+  with 16 sats. Exact net payment was 1 sat; underpayment refunded in full;
+  repeat delivery cost zero. Both endpoints ran the combined candidate.
+  No spent proofs were reactivated and no native Bitcoin/LND funds were moved.
+
+## Development candidate and cleanup
+
+The combined optimized backend and production UI are deployed on the development
+box with a private rollback copy. Native Bitcoin/LND containers were unchanged
+during deployment. The operator separately uninstalled/reinstalled Bitcoin Core
+to select an unpruned node; RPC confirmed `pruned=false`, and a separate baseline
+was recorded after that operator action. Do not compare subsequent checks with
+the pre-reinstall container start times.
+
+Completed Gitea setup/private-repository and Portainer integration fixtures were
+uninstalled through the supported lifecycle and removed from installed inventory.
+Their private evidence/data were retained outside the active manifests. The old
+Cuprate UI review container was also removed. Active Angor acceptance fixtures
+must be removed on completion; the requested Angor services remain installed.
+
+Funded acceptance used Tor-only peer-file transport, verified exact delivery
+bytes and compatibility response fields, and read the result back through
+FileBrowser. The original transport preference was restored, and temporary
+seller catalog entries/files and the exact buyer test document were removed.
+Financial receipt history was retained.
+
+The final managed-install fixture exposed a separate Quadlet quoting defect:
+whitespace-free command arguments containing apostrophes lost those characters
+in the generated service. The renderer now quotes these arguments and
+environment values; the updated isolated backend suite passed (1,607 passed, four opt-in tests
+ignored), and the final candidate rebuild is in progress. Do not tag a release before this live regression is verified.
+
+The production Portainer host subsequently rebooted after the routing repair.
+A post-boot probe from the actual Portainer namespace again verified the Git
+smart-HTTP response type, current branch ref and Compose contents. Its
+slirp4netns route and all production app containers survived. The temporary
+Portainer fixture was absent. This verifies the repaired production route
+across reboot; it does not substitute for final new-runtime delivery checks.
+
+## Follow-up acceptance: app cards and Angor icon
+
+- Mempool duplicate traced to `archy-mempool-web` durable inventory alias being
+  restored beside the real `mempool` frontend. Shared scanner canonicalization
+  fixes live and absent-container paths without deleting installed markers.
+  Frontend suppresses aliases only while a canonical tile exists.
+- Readiness text names the app and condition: “Web UI not ready: Gitea”. It shares the status row,
+  with full text available through its title; card actions use bottom alignment.
+- Angor uses the operator-supplied dark-mode icon with green outer corners.
+  Built-in imagegen prompt: fill transparent/white corners with the existing
+  flat green, preserve the black symbol, square opaque PNG, no added details.
+- Backend alias suite: 1608 passed, 4 ignored. Focused readiness/frame UI tests:
+  30 passed. Production UI build passed and is live on dev. Browser checks at
+  1440 and 1024 pixels verified named waiting text, bottom-aligned actions,
+  equal row heights, no overflow and one Mempool card after hard refresh.
+  The 390-pixel mobile icon layout also passed hard refresh. Final-source
+  isolated Mempool alias regression passed after the scanner simplification.
+- Managed Angor adapter acceptance: five stop/start/restart cycles, missing
+  prerequisite refusal, management restart and cleanup all passed. Both temporary
+  fixtures and their network were removed. Actual dev API verification remains
+  pending after removing an incomplete legacy-created adapter.
+
+## Startup manifest reload race
+
+Live Angor acceptance exposed a separate startup race: runtime asset bootstrap
+cleared and copied `/opt/archipelago/apps` in the background while the startup
+catalog refresh reloaded it. The daemon logged 62 loaded manifests followed by
+54 and then rejected the new disk-only app as unknown. A stable manifest snapshot
+confirmed the diagnosis: supported uninstall/reinstall produced the correct
+rootless Quadlet service with its declared port and network.
+
+Runtime promotion and the legacy installer-directory repair now finish before
+orchestrator construction. The background doctor no longer changes that tree.
+The final source backend suite passed 1,608 tests (four existing opt-in tests
+ignored). Optimized build and normal-path live startup/restart verification have now passed (see final follow-up below).
+
+Actual dev Angor acceptance passed managed service identity, no capabilities,
+UID 101:101, archy-net, public block height, CORS and both fee URL forms. During
+Bitcoin initial sync, the real Mempool fee API returns 503; the adapter faithfully
+returns the same status and body. Full-sync fee availability remains unverified;
+ready-backend API and failure/recovery behavior passed the isolated live fixture.
+The temporary `/run/archy-candidate-manifests` snapshot override and snapshot
+are now removed; normal startup/reload verification passed.
+
+The latest complete UI suite passed 140 files / 1,132 tests. Release preflight
+passed all static, manifest, catalog, type and UI gates. The requested named
+waiting message, compact card layout and green Angor icon are deployed to dev;
+desktop 1440/1024 and mobile 390 browser checks passed after hard refresh.
+The startup-order optimized build and normal-path startup checks are complete.
+The later dashboard-address candidate is now deployed with rollback; see the
+final live follow-up below. Do not rerun the earlier deployment helper: its
+temporary override has already been removed. No new release version/tag, OTA
+or ISO has been created.
+
+## Mempool and dashboard follow-up
+
+- Deployed the Mempool alias and runtime-promotion-order backend to dev. Real
+  server state contains one healthy `mempool`; the stale `mempool-web` record
+  is gone. Real-data browser checks at 1440/390 pixels found exactly one tile
+  before and after hard refresh. Bitcoin/LND identities/start times unchanged.
+- Removed the candidate manifest override. Normal management startup passed
+  two full cycles with 62 manifests retained through both initial catalog
+  refreshes. The next cycle hit a single readiness assertion; a subsequent
+  read-only check found Angor healthy and the manifest count intact. Remaining
+  repeat coverage should use bounded polling to distinguish transient request
+  failures from loss of app definitions; do not report five cycles passed yet.
+- Bitcoin Core's dashboard was serving HTTP 200 on 8334 while readiness checked
+  RPC 8332. Companion URL selection now takes priority over protocol sockets
+  for Core/Knots and Electrum aliases, with a regression preserving allocated
+  UI ports for other apps. Backend suite: 1,609 passed, four opt-in ignored.
+  Optimized build is `/tmp/archy-dashboard-address-build.log`; deployment and
+  live IBD verification helper: `/tmp/archy-dashboard-address-deploy.py`.
+- Phoenixd has no browser UI. Headless services now omit web-readiness messages;
+  actual browser apps name their web interface rather than waiting for
+  themselves. Focused 23 UI tests and production build passed; deployed to dev.
+
+## Final live follow-up: all three reported readiness/display defects fixed
+
+- Final optimized backend is deployed on dev. Bitcoin Core's launch address is
+  `http://localhost:8334` and `ui-ready` is true during initial block download.
+  Live verification recorded height 293,855 with `initialblockdownload=true`.
+  Chromium at 1440 and 390 pixels opened the embedded dashboard, read a numeric
+  current block height, and repeated that check after hard refresh.
+- One healthy Mempool remains in server state and in desktop/mobile My Apps
+  after hard refresh. Its durable install markers were preserved.
+- Phoenixd remains a running headless service without a web launcher or false
+  web-readiness message. Desktop/mobile browser checks passed. For actual web
+  apps, the compact copy is “Web UI not ready: [app]”; the reason comes first so
+  narrower cards do not truncate it into a misleading self-dependency.
+- Five normal management startup and managed Angor restart cycles passed
+  across the two acceptance logs. The retry harness uses bounded readiness
+  polling; it does not accept a running container alone as API readiness.
+  Disk + catalog manifest count remained 62 across startup refreshes, replacing
+  the previous 62-to-54 failure. Temporary override and snapshot are removed.
+- Final backend tests: 1,609 passed, zero failed, four existing opt-in ignored.
+  Final UI tests: 140 files / 1,133 passed. Production UI build passed and is live.
+  Bitcoin/LND container identities and start timestamps stayed unchanged.
+- Evidence: `/tmp/archy-dashboard-address-deploy.log`,
+  `/tmp/archy-bitcoin-ibd-browser.log`, `/tmp/archy-mempool-live-browser.log`,
+  `/tmp/archy-service-readiness-browser.log`,
+  `/tmp/archy-runtime-order-remaining-cycles.log`, and
+  `/tmp/archy-readiness-final-ui-tests.log`.
+- These are live development fixes. The new signed catalog, versioned OTA and
+  raw ISO still need preparation, artifact verification, signing and publication.
+
+### X250 Nginx Proxy Manager tunnel repair (2026-09-30)
+
+A further live report was a real startup failure, separate from the earlier slow
+image pull. An operator-specific Quadlet `web-tunnel.conf` published NPM's HTTP
+listener on tunnel port 18080. LND subsequently occupied 18080 on all addresses;
+pasta failed before NPM could start, with more than 1,400 systemd retries. The
+standard NPM manifest only publishes admin port 8081 and did not introduce this
+extra mapping. Changing the standard manifest would not repair this override.
+
+The node's override now uses free tunnel-local port 18081. Its persistent nftables
+configuration redirects only HTTP arriving from the configured WireGuard peer on
+the original tunnel destination to that port. The peer/public routing is unchanged;
+the input rule accepts the translated port and retains the existing interface,
+peer and forwarding restrictions. LND's REST port and native processes were not
+changed. This deployment-specific topology must not be copied into global app
+manifests or applied indiscriminately to other nodes.
+
+An abandoned certificate request also left an unreferenced database record and
+a temporary nginx challenge server for the same hostname. After backing up the
+entire NPM data directory and both configuration files, the unused failed record
+was soft-deleted and the stale challenge file archived. The referenced, valid
+certificate, proxy host, keys and user accounts were preserved.
+
+Live checks: NPM admin and API HTTP 200; nginx configuration validation with no
+duplicate-host warning; public HTTP redirects to HTTPS; valid public TLS reaches
+the site's existing authentication response, matching its direct upstream. NPM
+starts with zero automatic restarts and no missing-certificate renewal error.
+Bitcoin, LND and the production site container identities/start times were
+unchanged by the port repair. Rollback copies and the data archive are retained
+in the node's private support directory. No global OTA or ISO was published by
+this repair; the remaining release gates above still apply.
+
+#### Follow-up: fleet delivery and false health failures
+
+A longer observation exposed a second, generic defect after the port conflict
+was repaired: the health monitor probed all published ports at `127.0.0.1`,
+including NPM's tunnel-only listeners. Every monitor interval could therefore
+restart a healthy app. The short initial restart check did not catch this.
+
+The next backend now probes the actual `host_ip` from Podman; only wildcard
+addresses map to the corresponding loopback family. Regression tests cover
+explicit IPv4/IPv6 binds, wildcards, UDP/unpublished/invalid entries, and a real
+listener on a different loopback address. NPM's manifest now checks its internal
+admin HTTP API. The same check is deployed as a persistent Quadlet drop-in on
+the affected node so its older backend stops making false recovery attempts.
+
+The backend embeds `scripts/repair-npm-tunnel.py` and runs it before app
+reconciliation, after runtime asset promotion. This makes the targeted legacy
+port migration available to both OTA and ISO installations without relying on
+an independently installed script. Standard fresh installs are a no-op. Only
+the recognized legacy tunnel/firewall profile is migrated; unknown operator
+routing, occupied replacement ports and live-only firewall changes fail closed
+with a startup warning. Configuration backups, an interrupted-migration journal,
+atomic nft transactions and rollback protect the existing routing. Native wallet
+services and certificate databases are never modified by this fleet migration.
+
+The Python migration tests run in the release gate. The unsigned next catalog
+was regenerated successfully with the new NPM HTTP health check. These changes
+are prepared for the next release; existing published OTA/ISO artifacts remain
+unchanged and the new signed artifacts still require the release gates above.
+
+Verification for this follow-up: 18 migration tests passed; 43 health-monitor
+backend tests passed through the isolated runner. A disposable network-namespace
+regression exercised actual peer traffic through the nft redirect while a
+separate simulated LND listener retained port 18080. The generated rules also
+passed nft validation and atomic replacement. Run that regression with
+`sudo unshare --net python3 tests/regression/npm-tunnel-network.py`; it refuses
+to run in the host network namespace. The migration is a verified no-op on the
+already repaired node and on a standard development install without the override.
+
+After deploying the API health check, a 270-second live observation crossed
+multiple health-monitor intervals: NPM stayed healthy with the same container
+ID/start time, every API probe returned success, and Bitcoin/LND/production-site
+container IDs/start times were unchanged. This supersedes the initial short
+restart-only acceptance recorded above. The generic backend fix is committed
+for release, while the live node uses the equivalent internal NPM health check.
+
+### Final-gate Angor health-check correction
+
+Final release observation found the adapter healthy over IPv4 but marked
+unhealthy by its in-container BusyBox wget: `localhost` resolved to `::1`, where
+nginx does not listen. Its manifest now explicitly probes `127.0.0.1`. The live
+managed service was refreshed and its real Podman health check passed. The
+rootless gateway integration now runs the manifest's health check inside the
+actual image, in addition to endpoint/security/outage/DNS recovery assertions;
+all passed. A metadata regression covers the address-family requirement. Test
+containers and their network were removed by the fixture cleanup.
+
+## 1.8.22-alpha release preparation
+
+Final implementation gate passed: 1,612 isolated backend tests, zero failures,
+four existing opt-in tests ignored; 140 frontend files / 1,133 tests; frontend
+type check and production build; static/catalog/trust/build-context checks.
+The four exclusions require external AI backends, Reticulum subprocess/live
+transport, physical RNode hardware, or creation of a live Minibits profile.
+They are not claimed as executed by the isolated suite. Existing funded
+Cashu/Minibits and Framework acceptance remains recorded above.
+
+The real dev Angor stack now returns HTTP 200 fee estimates through both API
+forms; Bitcoin is still in initial sync, so full-chain completion remains an
+operational prerequisite rather than a completed test. The image-level health
+probe, outage/recovery and live native-state preservation checks passed after
+reloading the corrected manifest. Production Portainer again fetched the exact
+repository branch and Compose content from its own network namespace.
+
+Version preparation is 1.8.22-alpha. No new release tag or fleet-visible update
+manifest is published by the version commit. Optimized candidate deployment,
+artifact inspection, ISO smoke/boot checks and offline signatures follow.
+
+### Release blocker discovered during candidate observation: scheduled doctor
+
+The initial `02b840f2` 1.8.22 candidate is rejected for release. On the X250,
+2026-09-30 21:10–21:11 UTC, the scheduled `archipelago-doctor.service` explicitly
+ran `podman stop --all --time 30`, killed rootless network helpers and ran
+`podman system migrate` after a two-attempt external network probe failed.
+The journal attributes the stop to that unit, not the app health monitor or a
+host reboot. All apps restarted, including Bitcoin, LND and the production site.
+The earlier unchanged-container acceptance applies only to immediate deployment;
+the later observation failed and must not be represented as a stability pass.
+No persistent-data loss has been established. Keep this distinct from the closed
+Framework incident; do not wipe or recreate any wallet as a recovery action.
+
+Containment: stopped doctor timers on both test boxes, installed a safe diagnostic
+script into both the executable and runtime payload, and rejected/stopped the
+old ISO build. Network failure now produces a warning without stopping apps,
+killing network processes, migrating Podman or deleting network state. Repeated
+failures remain warnings, never a successful repair/check. Regression cases cover
+healthy, absent network, non-root invocation, host failure, transient recovery,
+repeated endpoint failure and namespace access failure, with mutation tripwires.
+Live scheduled-cycle observation and final rebuilt-artifact acceptance are pending.
+
+Recovery also exposed retired `git.tx1138.com` nginx base references in six
+companion UI Dockerfiles. They now use the existing primary registry at the same
+pinned version. All six images built successfully against that registry; payload
+validation rejects the retired host before OTA/ISO packaging.
+
+The post-recovery X250 check passes: Bitcoin authenticated RPC responds and IBD
+advances; NPM/Gitea/Portainer APIs respond; Portainer's real namespace fetches
+`demo-portainer` at `3ae171d6b0c728665a860520fe393c0abb772798` and its Compose
+file; Portainer's original persistent mounts match the earlier backup evidence;
+LND wallet/channel databases remain present on their persistent mount. No new
+pre-incident cryptographic wallet-identity baseline was available, so these checks
+must not be described as an exact identity/balance comparison.
+
+The dev all-container observation also caught a separate Cuprate UI orphan loop:
+`companion.rs` removed it because Cuprate was not installed, while generic desired-
+state recovery resurrected it from an old running snapshot, using a unit without
+nginx's required capabilities. Generic desired-state recovery now excludes missing companions
+owned by `companion.rs`; existing companion provisioning/reaping remains the
+single owner. Running UIs still receive the existing security configuration repairs. Regression runs repeated reconciliation against stale companion
+snapshots and checks that no image/container lifecycle operations occur.
+
+Safe-doctor live acceptance: the X250 completed a 12-minute observation with all
+running container IDs, start times and data mounts unchanged. Its journal records
+successful doctor runs at 21:22:06, 21:27:51 and 21:33:10 UTC. Both doctor timers
+are restored with the safe script. Dev's native Bitcoin/LND stayed running;
+all-container dev acceptance remains pending the companion-loop backend fix.
+Final-source UI suite: 1,133 passed. Heavy backend compilation is serialized with
+remaining build steps to reduce memory/IO pressure on the syncing dev node.
+
+Final source release gates at `96fb5a4f`: 1,613 backend tests passed, zero failed,
+four explicitly ignored; 1,133 UI tests passed; type-check, production UI build,
+catalog/trust, shell, pruning, LND readiness, NPM migration and doctor regressions
+passed. The isolated companion-loop regression passed independently as well.
+
+ISO cache hardening: the installer now carries the current doctor script and
+service/timer separately from rootfs.tar and overwrites both historical and active
+script locations before first boot. This prevents a cached base image restoring
+the old recovery code. A regression executes the actual installer block against
+stale disposable files twice and confirms a missing safety payload fails closed.
+The mounted-ISO smoke test also compares all three overlay files to source.
+The final ISO build captures the exact newly deployed OTA UI/runtime payload.
+
+### Final kiosk acceptance found nondeterministic NPM launch selection
+
+Do not publish the staged `d1bc1273` candidate. NPM itself remains healthy and its
+API, Portainer integration, site, and native services passed stability checks.
+However, final kiosk acceptance found its card stuck at "Web UI not ready".
+The runtime reported bindings in proxy-HTTP, proxy-HTTPS, admin order. The scanner
+chose the first non-database/SSH binding, then rejected its tunnel-only host port
+as unreachable on loopback, leaving the launch address empty. Earlier tests had
+passed with admin first. This is a confirmed order-dependent scanner defect.
+
+The candidate fix explicitly resolves NPM container port 81 to its actual host
+allocation. Proxy ports never become the admin URL. Missing/malformed admin
+bindings do not fall back to another service when published bindings are present.
+Port parsing handles IPv6 authorities and rejects invalid ports. Regressions
+cover all six three-port permutations, allocated admin ports, IPv4/IPv6 binding
+strings, missing admin mappings, missing runtime port information, and malformed
+or UDP bindings. Full backend regression execution is pending for this change.
+The ISO build is frozen at installer-environment creation; no release was signed
+or published. Rebuild/revalidate the OTA and ISO with this correction.
+
+The companion orphan fix worked live: Cuprate UI was automatically removed and
+all installed app container IDs remained unchanged. One observation helper raced
+that expected removal between `podman ps` and `inspect`; it now excludes that
+known orphan before inspection and repeats the stability check. This was a test
+snapshot race, not another installed-app restart.
+
+NPM selector final backend gate passed: 1,617 tests, zero failures, four explicitly
+ignored, through the isolated runner. This includes all new port-selection cases
+and the existing companion security/configuration and lifecycle regressions.
+Rebuild the release binary and UI metadata, deploy those exact OTA bytes to both
+boxes, and require actual kiosk hard-refresh/Launch acceptance before ISO assembly.
+
+
+## Final accepted artifacts — 1.8.22-alpha
+
+Source `6d5f3ffb` passed 1,617 backend tests (four explicit opt-in exclusions),
+1,133 frontend tests and final release gates. Exact OTA bytes were deployed to
+both boxes. Actual X250 kiosk NPM Launch, version/pruning, desktop/mobile
+readiness/AIUI, production Portainer Git/Compose and 12-minute stability checks
+on both boxes passed. No installed app was restarted by the safe diagnostics,
+and the Cuprate orphan stayed absent. Native Bitcoin/LND and the production site
+were preserved during final management deployment.
+
+The raw ISO passed mounted payload checks and matches all 653 OTA frontend/runtime
+files plus the backend. Full offline installation and installed UEFI boot to the
+visible setup screen passed in a disposable QEMU/KVM VM. Both installed doctor
+paths and the installed backend have the expected hashes. No VM wallet was set up.
+
+See `release-1.8.22-acceptance.md` for exact artifact hashes, hardware/runtime
+coverage and limits. Draft upload verification, offline signatures, publication
+and public readback remain; the fleet still advertises 1.8.21 until those gates
+finish. Do not confuse a draft asset or source push with completed publication.

docs/post-1.8.22-regressions-20261001.md

@@ -0,0 +1,258 @@
+# Post-1.8.22 regressions and retained release checklist
+
+Status: OPEN. New regressions reported after publication on 2026-10-01.
+Do not mark complete from source changes alone. Preserve wallets, app state and
+operator uninstall decisions. Never send a second payment to recover delivery.
+The earlier Framework startup incident remains separately closed with operator
+acceptance; this is a new paid-file incident.
+
+## Current tasks
+
+- [ ] Recover the Framework's Lightning paid-file purchase without another payment;
+      inspect buyer/seller evidence and verify delivered bytes.
+- [ ] Correct seller settlement verification when local-node payment skips polling.
+- [ ] Durable seller entitlements and safe buyer retry after navigation/restart;
+      do not issue another payment on an uncertain or successful attempt.
+- [ ] Cache Lightning purchases, preserve ownership, optional Files copy, free repeat.
+- [ ] Diagnose mobile companion uploads on the affected route/device.
+- [ ] Real progress in the existing compact upload bar; no increased height.
+- [ ] Preserve uploads/progress across screens and original batch destination.
+- [ ] Cancel active transfer and queued files; truthful partial/error/server-save status.
+- [ ] Transparent transaction-filter container; single horizontal scrolling mobile row.
+- [ ] Immich displayed as one app, internal components hidden; diagnose restarting services.
+- [ ] Diagnose unwanted CryptPad after upgrade, failed uninstall, and persistent removal.
+- [ ] Identify the other removed unexpected service from affected-node records.
+- [ ] Upgrade regression matrix: installed/stopped/restarting/removed/legacy apps,
+      aliases, dependencies, inventory, desired-state reconciliation and data preservation.
+
+- [ ] Portainer duplicate-network migration: retire the redundant managed repair
+      override, preserve operator settings/state, verify generated command,
+      actual request namespace, dashboard readiness and repeated reconciliation.
+
+- [ ] Lightning cooperative-close fees: Standard/Medium/Fast/Custom selection,
+      explicit default target, strict backend validation and forwarding, error
+      handling, mobile layout and no real channel closure during tests.
+
+- [x] Apps search clear control: My Apps, Services and App Store, desktop/mobile,
+      existing design tokens, right-aligned icon, no size change, keyboard focus.
+
+## Retained release work (previous acceptance is not new-regression acceptance)
+
+- Mempool patched image/catalog version agreement, update-button clearing, one card.
+- Minibits PR160, Lightning address availability, concise single-column backup copy.
+- Framework LND startup/Receive and unknown-vs-zero balance behavior.
+- Friendly Bitcoin warmup; LND waiting for install/sync; Bitcoin UI during IBD;
+  headless Phoenixd without self-waiting or bogus launch action.
+- Cashu same-mint paid files, exact amounts/change/refund, errors, stored bytes,
+  Files copy and repeat access without re-payment.
+- mempool.space public explorer fallback, preserving local/custom configuration.
+- Optional install pruning and consistent automatic-pruning policy.
+- X250 kiosk version picker layering/contrast and pruning layout.
+- AIUI single desktop/mobile background, transparent embedded layers,
+  preserved standalone wallpaper.
+- PR review/fixes/tests and normal merge/closure (160 previously shipped;
+  161/162 merged and included in 1.8.22).
+- Installed inventory retained during app restart/hard-refresh.
+- Correct iframe/browser launch readiness, useful errors and delayed startup.
+- GitWorkshop payload/build contexts, progress and persistence after refresh.
+- Gitea/Portainer same-server Git from actual request namespace; URLs, auth,
+  fresh installation in either order, migration/rollback, restart/reboot,
+  Git/SSH/LFS/registry/browser compatibility and data/stack preservation.
+- NPM correct admin port/URL, malformed URL behavior, bind-aware readiness,
+  persistent backed-up tunnel/LND port-conflict repair on OTA and ISO.
+- Angor headless indexer on DEV BOX only, full unpruned Bitcoin/Mempool/ElectrumX
+  prerequisites, optional separate relay, official icon with green white areas.
+- Compact named readiness messages and bottom-aligned app-card actions.
+- Remove unused integration/build fixtures from Apps/Services, preserve app data.
+- Safe network doctor, no all-app stop/reset on failed egress probe.
+- No orphan companion resurrection; retain existing companion security repairs.
+- Current companion image registry, build contexts, runtime asset promotion order,
+  generated-service argument quoting and graceful Bitcoin/LND shutdown.
+- OTA + RAW ISO, root signatures/catalog compatibility/checksums, independently
+  verified public files, Git/ngit source/releases and fleet discovery.
+- Correct LAN SCP command for the new ISO.
+
+## Explicit boundaries/follow-ups
+
+- Full-chain Angor indexing awaits development Bitcoin IBD.
+- Primal automatic comment exceeding Minibits metadata limit: previously accepted
+  upstream limitation, no unsupported local identity/metadata rewrite.
+- Lost-response ecash seller receipt redesign is a separately accepted follow-up;
+  do not claim an uncertain refund completed or automatically pay twice.
+- Optional external-provider/hardware tests must be labelled if not exercised.
+
+## Initial source evidence
+
+`PeerFiles.vue::payWithLightning` immediately downloaded after buyer payment,
+while only seller `handle_content_invoice_status` marked a pending invoice paid.
+Seller download checked only that cached flag. This matches the reported error
+and is supported by source inspection. An earlier diagnostic's HTTP 404 is not
+valid confirmation: it incorrectly base64-decoded lncli's already-hex payment
+hash. The corrected live diagnostic recognizes the settled invoice on the
+candidate; do not cite the earlier 404 as proof of the original failure sequence.
+`content_invoice.rs` stored all entitlements only in process memory with a
+one-hour TTL, losing both pending and paid access on restart/expiry.
+Lightning download returned transient base64 without the Cashu ownership cache.
+CloudFolder's view-local spinner had no byte progress/cancel; batch upload read
+`currentPath` independently for each file, allowing navigation to move destinations.
+Immich's underscore dependencies are scanner-excluded; hyphen manifest IDs are
+not. Live inventory confirmed both hyphenated synthetic entries while the
+actual underscore-named containers had remained running for nine days.
+
+## Access / acceptance
+
+Operator provided updated Framework SSH authentication privately in chat.
+Do not put credentials or deployment addresses in this public document.
+Framework was reached over SSH. Native Bitcoin, LND and all three Immich
+container identities/start times were recorded before candidate deployment.
+The kiosk is at its login page. Dashboard password authentication succeeds but
+requires the operator's second factor; normal uninstall acceptance remains pending.
+
+Confirmed live evidence:
+
+- A 10,000-sat peer-file invoice settled at 12:19:29 UTC. The original status
+  diagnostic used an incorrectly decoded hash; see the correction above. Buyer
+  identity and confirmation that this is the reported sale remain pending.
+- The matching item currently allows free access; preserve that operator setting.
+- CryptPad has no container but remains in installed-apps metadata. Uninstall
+  repeatedly aborts because the removed catalog ID has no manifest.
+- Immich server/database/cache are running; synthetic hyphenated dependencies
+  appear stopped and the recovery overlay briefly advertises restarting.
+- The other removed service was Core Lightning; uninstall tombstones exist.
+- No Android resource-upload POST appears in the inspected recent nginx log.
+  This does not establish why the affected companion failed.
+
+## Candidate implementation and validation
+
+Source changes persist seller entitlements with atomic writes, verify settlement
+at delivery, recover older Lightning entitlements from the seller's LND invoice,
+perform the status handshake for older sellers, and cache delivered Lightning
+files. Buyer purchase bytes and the shared ownership index now use atomic,
+synced writes and a serialized read/modify/write transaction; a corrupt index
+fails the write instead of silently replacing existing ownership. The browser saves the invoice before payment and retries delivery without
+another payment. Browser receipts are not yet a node-wide recovery store.
+
+The upload queue now belongs to the shared Cloud store, captures its original
+folder, reports actual sent bytes and server completion, and cancels its active
+XHR and remaining queue. The fixed-height bar remains available across routes.
+Transaction filters use a transparent container and one scrollable row. Immich
+aliases normalize to their real component names and internal cards are hidden.
+Unknown catalog entries no longer prevent the regular uninstall flow.
+
+Validation so far (additional acceptance still pending):
+
+- Final isolated backend suite: **1,631 passed**, zero failed, four optional
+  tests ignored. This includes invoice settlement/amount boundaries, durable
+  seller records, concurrent buyer ownership, damaged-index preservation,
+  Portainer override retirement/idempotence/customization/backup failures,
+  recovery overlays and channel-close fee forwarding/validation.
+- Final frontend suite: **1,157 passed** across 142 files. Production build
+  passed. Six payment-recovery and twelve channel-close tests are included.
+- Real FileBrowser uploads at 1440px and 390px: exact bytes and original folder
+  verified after navigation, 44px bar, cancellation and queue stop passed.
+- Mobile viewport acceptance is not physical Android companion acceptance.
+- Final release backend build passed. Candidate backend and dashboard are now
+  deployed on dev and Framework. Another OTA/ISO remains pending; published
+  1.8.22 artifacts remain unchanged.
+
+Release gates still include actual-node payment recovery/delivery, durable
+CryptPad removal through normal controls, Immich inventory after refresh/restart,
+physical companion diagnosis, and remaining upgrade regression acceptance.
+No new payments, native-service restarts or wallet changes were used in testing.
+
+## Additional live Portainer regression
+
+The X250 user service exited 125 because the generated command supplied
+`--network slirp4netns` twice. The manifest already supplies the network, while
+an older Archipelago-created `archy-same-node-network.conf` drop-in adds it
+again. Quadlet's Network directives accumulate; they do not override each other.
+This repair artifact should have been retired when the declarative fix shipped.
+
+The live repair backed up the override and Portainer state, removed only the
+exact redundant override, reloaded user systemd and restarted Portainer. API
+status returned HTTP 200 with version 2.45.0; the actual kiosk's package state
+reported running and UI-ready. Bitcoin/LND and the production site's container
+identities/start times remained unchanged. The source migration now detects
+this exact managed override before preparing the persistent restart obligation,
+backs up app state, retires the redundant file with a retained copy, and reloads
+and restarts through normal reconciliation. Custom overrides are preserved.
+Automated migration coverage passed; candidate is now deployed on dev and
+Framework. The X250 retains its verified live repair pending the next OTA.
+
+## Channel-close fee selection
+
+The existing close UI sent only the channel point, and the backend forwarded
+only `force=false`. LND therefore used its lax default confirmation target.
+The candidate reuses the channel-opening fee choices (six/three/one block target,
+or custom target/rate), explicitly sends six blocks for legacy clients that omit
+fees, and validates query parameters before accessing the wallet. Cooperative
+fees are never silently applied to force closes. Close RPC retries are disabled
+so a timeout cannot silently repeat this mutation.
+
+Protocol reference: [LND CloseChannel](https://lightning.engineering/api-docs/api/lnd/lightning/close-channel/).
+Fee targets are estimates, not guaranteed confirmation times. Tests use mocked
+requests; no production channel is closed to verify the feature.
+
+Additional browser acceptance:
+
+- Transaction filters at 390px: computed transparent background, one row and
+  horizontal overflow verified.
+- Close-channel selector at 1440px and 390px: preset/custom controls visible,
+  no overflow, custom 25 sat/vB forwarded. The close request was intercepted;
+  no real channel closure or wallet mutation occurred.
+- All three Apps search screens at both widths: clear icon stays inside the
+  field; click/Escape clear; input retains focus; desktop 40px/mobile 52px heights
+  stay unchanged. Shared design-system search-field classes are retained.
+- Portainer remained active with zero service restarts and no pending marker.
+  Its real network namespace read smart HTTP Git refs and the Compose file.
+  Original persistent mounts were unchanged. The old integration test containers
+  are absent from dev, Framework and X250. One leftover upload-test folder was
+  removed after checking it contained only this task's test files.
+
+## Build resource observation
+
+The final optimized compile coincided with heavy memory/disk pressure and local
+Bitcoin/LND RPC timeouts on the development node. After pausing the compiler,
+both authenticated RPCs responded again; Bitcoin reported height 506400 and
+19.6% verification progress, with LND waiting for chain sync. No native service
+was restarted. Compilation resumed in a separate user scope limited to one CPU,
+with nice 19 and idle I/O priority. This is evidence of resource contention,
+not proof of a new wallet or startup defect. Verify native RPC health again
+before candidate deployment.
+
+## Candidate deployment and live acceptance — 2026-10-01
+
+Source: `f4d34554` (later commits update this checklist only).
+Backend SHA-256:
+`8fb6249d1869bb8c9aea26d0f846de5b3eec328113a5c7f573628306e26e652e`.
+
+- Backed up backend, dashboard and app metadata on both nodes under the root-only
+  support directory `post1822-regressions-20261001`. Deployed assets before
+  promoting the dashboard entry point; manager health passed first.
+- Only the Archipelago manager restarted. Bitcoin/LND IDs and start times stayed
+  unchanged; Framework's three Immich containers also stayed unchanged.
+- Dev authenticated Bitcoin/LND RPCs responded after deployment. Bitcoin IBD
+  continued above height 513000; LND correctly reported not yet chain-synced.
+- Both nodes serve dashboard entry bytes identical to the production build.
+  All six search-clear cases passed against the live dev dashboard (three
+  screens, desktop/mobile), including focus, Escape and unchanged field size.
+- Framework's stale CryptPad installed claim was removed while the manager was
+  stopped; both legacy IDs were recorded as user-uninstalled. Data was preserved.
+  Removal remained after another management restart. Dashboard uninstall-flow
+  acceptance still awaits authentication; this repair was performed over SSH.
+- Corrected invoice diagnostic recovered the settled seller entitlement, returned
+  HTTP 200 with `paid: true`, and saved a mode-0600 record. A different item was
+  rejected. Paid status survived another manager restart without native restarts
+  or a second payment.
+- Delivery acceptance remains OPEN: the catalog's file is absent from both its
+  dedicated content path and FileBrowser path; a privileged filename search of
+  those trees found no copy. Its free-access setting was preserved. Buyer and
+  reported-purchase identity still need confirmation; do not claim the actual
+  buyer received the file.
+- The malformed-hash diagnostic also exposed that invoice-status currently
+  propagates validation errors as a closed connection. Before release, return a
+  structured 400 for malformed hashes and an explicit retryable response for
+  settlement-service errors, with endpoint coverage.
+
+Physical companion upload diagnosis and remaining release acceptance stay OPEN.
+This is a candidate deployment, not a newly signed OTA or ISO.

docs/pr-review-20260930.md

@@ -0,0 +1,126 @@
+# Paid-download PR review — 2026-09-30
+
+## Scope and result
+
+Reviewed both open PRs from the repository pull-request list: [#161](https://source.archipelago-foundation.org/lfg2025/archy/pulls/161)
+and [#162](https://source.archipelago-foundation.org/lfg2025/archy/pulls/162).
+Both branches were updated from main, repaired and tested independently and
+together. Their existing remote branches were advanced without rewriting the
+contributors' history. Both were subsequently merged and closed and are now
+integrated on main. Candidate live-wallet acceptance remains pending.
+The signed 1.8.21 artifacts are unchanged.
+
+| Candidate | Tested commit | Isolated backend result |
+| --- | --- | --- |
+| PR #161 | `971d4777` | 1,576 passed, 0 failed, 4 existing tests ignored |
+| PR #162 | `0677924a` | 1,568 passed, 0 failed, 4 existing tests ignored |
+| Both together | `4bf4bf1a` | 1,585 passed, 0 failed, 4 existing tests ignored |
+
+Both individual branches also passed production `cargo check`, with the
+repository's existing 16 warnings. The combined merge required no conflict
+resolution. Backend tests ran through `scripts/test-backend-isolated.sh` so they
+could not access host wallets, native services or production container storage.
+
+## Findings and repairs
+
+### #161 — payment delivery and file readability
+
+- The branch conflicted with newer mint-fee, keyset-ID and truthful refund
+  reporting fixes. Preserve those implementations from main; do not reintroduce
+  its older unconditional “refunded” messages or duplicate keyset resolution.
+- Opening a file before charging, then reopening/reading it afterward, still
+  permits a read failure after payment. Prepare the complete requested bytes
+  before redemption, including ranged reads. Tests delete or alter the backing
+  file during payment verification and still receive the prepared original data.
+- Empty/out-of-bounds/reversed ranges could fail after redemption, and empty
+  files could underflow the range calculation. Validate ranges before charging
+  and return HTTP 416 when unsatisfiable.
+- `chmod a+r` unnecessarily changed the permissions of shared paid/private
+  files. Read restricted FileBrowser files through the rootless namespace while
+  retaining their mode. Scope the fallback to regular files canonically inside
+  FileBrowser storage, and reject unauthorized peers before reading.
+- A single-delivery flag must also prevent redirects and ambiguous transport
+  retries. Payment-bearing GET and POST requests now retain the first HTTP
+  response and do not retry after timeouts or disconnects that might follow
+  delivery. Refused connections and normal nonpayment browsing retain the
+  appropriate retry behavior.
+- Interrupted response bodies now report the outcome using the actual local
+  refund result. Seller explanations are bounded, stripped of control
+  characters and explicitly identified as peer text.
+- Original permission tests silently returned when run as root. Replacement
+  tests inject read/payment boundary failures, exercise them under the isolated
+  runner, and assert that read failures never invoke redemption.
+
+### #162 — saving purchases in Files
+
+- Its host-permission repair overlapped 1.8.21's authenticated Files API path.
+  Review of [FileBrowser v2.63.23's resource handler](https://github.com/filebrowser/filebrowser/blob/v2.63.23/http/resource.go)
+  showed that `override=false` checks for existence separately from opening
+  with truncation. It does not guarantee no overwrites under concurrent saves.
+- The proposed direct path exposed the final filename before the write
+  completed. Both direct and namespace paths now finish a private temporary
+  file and publish it through a no-clobber hard link, retrying numbered names.
+- Plain `ln` could place a temporary file inside an existing directory instead
+  of treating the destination as a collision. Use `ln -T`; existing directories
+  and dangling symlinks are conflicts, never replacement targets.
+- Add filename and destination checks, unique temporary names, bounded name
+  retries, synchronization before publication, and exact input-length checks.
+  Truncated pipe input cannot become a completed purchased file.
+- Files storage remains optional. An unavailable copy destination does not
+  undo the purchase or create a fake FileBrowser installation; the durable
+  purchased-content cache remains primary.
+
+## Additional verification on the development node
+
+Used disposable scratch directories only, then removed them:
+
+- Reproduced a FileBrowser-style rootless-owned 0640 upload. The host backend
+  UID could not read it. `podman unshare cat` returned identical bytes without
+  changing its 0640 mode.
+- Ran the exact namespace writer script with four concurrent writers against
+  a directory owned by the container UID range. Every file had unique naming,
+  exact bytes, the expected owner and mode, and no remaining temporary file.
+- Sent truncated input to the namespace writer and verified refusal, no final
+  file and temporary-file cleanup.
+
+The isolated tests additionally exercised 24 simultaneous direct writes,
+existing-file preservation, symlink/directory conflicts, collision exhaustion,
+root-independent permission failures, read-before-redemption ordering,
+authorization, redirects and peer disconnects.
+
+Logs on the development box:
+`/tmp/archy-pr161-tests.log`, `/tmp/archy-pr162-tests.log`,
+`/tmp/archy-pr-integration-tests.log`, `/tmp/archy-pr161-check.log`,
+`/tmp/archy-pr162-check.log`, `/tmp/archy-pr-userns-scratch-test.log`.
+
+## Next-release acceptance and limits
+
+- Both reviewed branches are integrated on main alongside the lifecycle fixes.
+  Repeat release gates against the final release commit after remaining changes.
+- Perform funded peer-to-peer acceptance on the candidate build, including a
+  Tor-only purchase and a purchase requiring change, before the next release.
+  The new review branches were not deployed to funded live wallets here.
+- These PRs do not implement durable payment receipts. If a seller redeems a
+  payment and the connection subsequently loses the response, the buyer may
+  receive an unconfirmed-refund warning. Do not represent that warning as proof
+  of a refund or automatically charge the buyer again. Receipt-based recovery
+  remains separate follow-up work.
+- Abrupt process termination can leave a hidden namespace temporary file;
+  ordinary write failures and truncated input are tested to clean up. The final
+  filename is published only after complete input, and existing files remain
+  protected.
+- The separately reported X250 kiosk selector is fixed and verified on the
+  actual kiosk; see the lifecycle evidence and consolidated release checklist.
+
+## Authorized merge — 2026-09-30
+
+The operator explicitly requested normal merged/closed PR status after review.
+Re-read both PRs and verified their heads still exactly matched the reviewed
+commits. Changes from the integration-test base to main were documentation only.
+Gitea normal merges completed and read-back confirmed `merged=true`, `state=closed`:
+
+- #161: `3daea6623be3e2c7222101b8e6ac411423c7e16c`.
+- #162: `b02ba4100d922dd1b75c6a78121ef446c2159a54`.
+
+Local next-release lifecycle work was integrated with main at `d69e8452`. Funded release acceptance and the documented delivery-receipt
+limitation remain as recorded above; merging does not claim a new release.

docs/release-1.8.22-acceptance.md

@@ -0,0 +1,101 @@
+# Archipelago 1.8.22-alpha acceptance
+
+Source: `6d5f3ffb850bfd3dcd396bac986ba770935d1daa`.
+
+## Verified application and runtime changes
+
+- Full isolated backend suite: 1,617 passed, zero failed, four explicit opt-in exclusions.
+- Frontend suite: 1,133 passed. Final frontend and AIUI production builds succeeded.
+- Container suite: 79 passed. Catalog compatibility/trust, release manifest, build contexts, pruning, Lightning readiness, NPM migration, safe doctor, companion recovery and ISO doctor-overlay regressions passed.
+- Six companion dashboard images built using the current registry.
+- Final OTA backend SHA-256: `e108b78bbbd21cb7d5d47c8d0b7b9b19b63fb0c44678773603202440ec7d6f5b`.
+- Final OTA frontend SHA-256: `2da485a2da75ff2fbe4aba52d6f217150e303be43a031723480c9c4ff9d43f41`.
+
+## Live acceptance
+
+The exact OTA bytes were deployed to the development box and ThinkPad X250.
+Native Bitcoin/LND and the X250 production site retained their container identity
+and start time during these final management deployments. Both boxes completed
+12-minute observations including scheduled diagnostics with running containers
+and persistent mounts unchanged. The orphaned Cuprate dashboard stayed absent.
+
+Actual X250 Chromium kiosk: hard refresh, NPM Launch to the correct admin URL,
+visible login/admin page, readable inline Bitcoin version choices and pruning
+checkbox passed. No Bitcoin installation was triggered by this test.
+
+Final desktop/mobile checks passed for Bitcoin's IBD dashboard, one Mempool card,
+headless Phoenixd, LND waiting/unknown-balance behavior and all five transparent
+AIUI embedding layers. Standalone AIUI retains its wallpaper.
+
+Portainer's actual production network namespace fetched Git refs and the Compose
+file after final deployment. Original mounts were preserved. Earlier disposable
+fresh/reverse-install and migration/rollback tests, and the production host's
+operator-initiated reboot check, passed.
+
+Live Tor-only Cashu paid-file acceptance verified a one-satoshi net purchase,
+change, rejected-payment refund, exact file bytes, Files access and free repeat
+delivery. No native Bitcoin/LND funds were moved. PRs 161/162 are merged and
+closed; the open pull-request list is empty.
+
+## Boundaries
+
+- Angor's real dev API, fees, block tip, CORS and rootless/headless configuration
+  passed. Full-chain indexing remains dependent on initial Bitcoin sync finishing.
+- Optional live AI providers, physical RNode hardware, the opt-in Reticulum TCP
+  subprocess test and creation of a production Minibits profile were not run.
+- The previously recorded unsafe-doctor incident changed X250 container start
+  times before the final fix. Persistent databases were present after recovery,
+  but no pre-incident cryptographic wallet-identity baseline was available.
+  Do not describe recovery evidence as an exact pre-incident balance comparison.
+- No claim of perfect behavior on every device, network or future failure is made.
+
+## Raw ISO acceptance
+
+The raw ISO is 2,755,072,000 bytes. SHA-256:
+`cf7be6378dcd52f6f62774523341fa75dd453fa73a9cadff5390846f483e0140`.
+
+Mounted-artifact smoke checks passed, including BIOS/UEFI boot files, live-boot
+hooks, build contexts, current doctor overlay, crash-capture configuration,
+version and frontend payload. The ISO backend and all 653 OTA frontend/runtime
+files match exactly. AIUI metadata names the tested source commit.
+
+A disposable QEMU/KVM x86_64 VM with UEFI firmware, 3 GiB RAM, two vCPUs, a fresh
+64 GiB NVMe virtual disk and no network completed the full installation. This
+covered partitioning, LUKS2 data encryption, swap, system configuration, UEFI
+bootloader and initramfs generation. Cold boot with the ISO detached reached the
+visible Welcome to Archipelago setup screen. The installed backend and both
+historical/current doctor paths matched source hashes. Backend/nginx were active;
+health reported RPC/sessions ready, crash recovery complete and version 1.8.22.
+No wallet was initialized in this disposable VM.
+
+The first automatic VM reboot selected the still-attached installer ISO. That
+was corrected in the test configuration by detaching the ISO and explicitly
+booting NVMe. It was not accepted as an installed-system boot. The subsequent
+cold boot above is the successful acceptance run.
+
+The dev native Bitcoin/LND identity/start-time baseline also remained unchanged
+after the ISO build and VM acceptance.
+
+## Publication verification
+
+All three operator signatures verify against the pinned release root. The five
+Gitea assets match independent server-side SHA-256 checks. Both OTA components
+also passed complete public HTTPS downloads with exact hashes and sizes; the
+raw ISO passed public size/range checks and both checksum sidecars read back
+exactly. Only after these checks were the signed OTA manifest and compatible
+app catalog promoted. The release tag identifies the tested source above.
+
+Git and ngit publication completed on 2026-10-01. Both repository relays
+acknowledged the ngit release; independent `release view` resolved all five
+assets with exact hashes and sizes and no unresolved asset IDs. Main and the
+release tag were pushed to both remotes.
+
+Public main-branch OTA manifests and the app catalog read back byte-for-byte
+and verified cryptographically. Live `update.check` on the accepted dev node
+reported 1.8.22-alpha with no further update, as expected for the installed
+release. Discovery on an older production node was not repeated during this
+publication step.
+
+- [Release and verification files](https://source.archipelago-foundation.org/lfg2025/archy/releases/tag/v1.8.22-alpha)
+- [Raw ISO](https://source.archipelago-foundation.org/lfg2025/archy/releases/download/v1.8.22-alpha/archipelago-installer-1.8.22-alpha-unbundled-x86_64_RC1.iso)
+

docs/repair-release-20260929.md

@@ -1,6 +1,9 @@
 # Repair and release execution — 2026-09-29
 
-**Status: IN PROGRESS. Do not publish an OTA or ISO until the release gates pass.**
+**Status: 1.8.21 PUBLISHED — see the completion record at the end.**
+
+The next release is tracked in [the current execution checklist](next-release-20260930.md).
+The dated entries below preserve the investigation history.
 
 User requires all tasks completed and tested on the development box before the
 next OTA and raw ISO. Passing unit tests alone does not establish live correctness.
@@ -52,8 +55,8 @@ next OTA and raw ISO. Passing unit tests alone does not establish live correctne
 - [x] Live waiting/UI verified on dev; recovery covered by deterministic tests.
 - [x] Framework operator acceptance and authorization to release recorded.
 - [x] Release version/changelog, catalog/image implications, signing prepared.
-- [ ] Signed OTA built, tested, published to git and ngit.
-- [ ] Raw ISO built, boot-tested, signed and published; download command supplied.
+- [x] Signed OTA built, tested, published to git and ngit.
+- [x] Raw ISO built, boot-tested, signed and published; download command supplied.
 
 Tests must not wipe/recreate wallets, prune the operator's existing full chain,
 or claim that arbitrary failures can never happen. Record material gaps before
@@ -356,3 +359,42 @@ Bitcoin and LND IDs/start times remained unchanged, with generated stop settings
 still verified. No temporary graceful-stop overrides remain. Catalog signature
 verifies against the pinned release root; final 1.8.21 artifact validator passes.
 The candidate is ready for the user's local OTA signing ceremony.
+
+### 1.8.21 publication completed — 2026-09-30
+
+The operator signed the OTA manifest and subsequently the ISO checksum JSON.
+Both signatures verified against the pinned release root. The signed OTA was
+published on git/ngit, and the operator confirmed that Framework could see the
+update. Source main and the annotated `v1.8.21-alpha` tag were published.
+
+Raw ISO:
+`archipelago-installer-1.8.21-alpha-unbundled-x86_64_RC1.iso`
+
+- Size: 2,682,419,200 bytes.
+- SHA256: `8667b5522c476a40e29abba19df4180086191527a194a88765aa70ed527f9406`.
+- Build and ISO smoke checks passed. The mounted backend matched the staged
+  OTA backend hash, and the full dashboard/AIUI tree matched the fresh build.
+- An isolated UEFI QEMU guest, with no network or host disks attached, booted to
+  the installer prompt. The VM was stopped and the ISO unmounted afterward.
+  This was an installer boot check, not a full installation onto hardware.
+- Uploaded the raw ISO, plain SHA256 sidecar and signed checksum JSON to the
+  [1.8.21 release](https://source.archipelago-foundation.org/lfg2025/archy/releases/tag/v1.8.21-alpha).
+  The stored server file hashes matched, the public ISO headers and first/last
+  byte samples matched, and both public checksum files matched byte-for-byte.
+- The ngit downloader's full-ISO acquisition exceeded its fixed 30-minute
+  deadline on the available connection. Published Nostr asset records using
+  the already verified hashes, sizes and public URLs with the existing ngit
+  signer; both repository relays acknowledged them. Ngit then accepted those
+  records and final readback resolved all five release assets with the expected
+  hashes and sizes. No new release-root signing was performed by the assistant.
+
+Final publication evidence: `/tmp/archy-1821-finish-events.log`,
+`/tmp/archy-ngit-1821-complete-view.json`, and
+`/tmp/archy-1821-verified-asset-events.log` on the development box.
+
+Subsequent review of PRs #161/#162 found additional delivery and concurrent
+file-save edge cases. Their repaired, tested branches are recorded in
+[the next-release review](pr-review-20260930.md); those changes are not in the
+signed 1.8.21 artifacts. The X250 kiosk selector report is also tracked for the
+next release. No claim of exhaustive hardware or network-failure coverage is
+made for this release.

image-recipe/_archived/build-auto-installer-iso.sh

@@ -2607,21 +2607,19 @@ if [ -f "$SCRIPT_DIR/../../scripts/image-versions.sh" ]; then
     echo "   ✅ Bundled image-versions.sh"
 fi
 
-# Bundle docker UI source files for building custom UIs on first boot
-# Always bundle — these are tiny HTML/CSS files, not container images
-if true; then
-    DOCKER_UI_DIR="$SCRIPT_DIR/../../docker"
-    if [ -d "$DOCKER_UI_DIR" ]; then
-        echo "   Bundling docker UI source files..."
-        mkdir -p "$ARCH_DIR/docker"
-        for ui_dir in bitcoin-ui lnd-ui electrs-ui; do
-            if [ -d "$DOCKER_UI_DIR/$ui_dir" ]; then
-                cp -r "$DOCKER_UI_DIR/$ui_dir" "$ARCH_DIR/docker/"
-                echo "   ✅ Bundled $ui_dir source"
-            fi
-        done
-    fi
-fi
+# Always overlay the current doctor, including when rootfs.tar is cached.
+cp "$SCRIPT_DIR/../../scripts/container-doctor.sh" "$ARCH_DIR/scripts/"
+cp "$SCRIPT_DIR/../configs/archipelago-doctor.service" "$ARCH_DIR/scripts/"
+cp "$SCRIPT_DIR/../configs/archipelago-doctor.timer" "$ARCH_DIR/scripts/"
+
+# Build-source apps need their complete contexts even on unbundled ISOs.
+# Keep this identical to the OTA runtime payload; a per-app allowlist silently
+# omitted GitWorkshop, FIPS and Cuprate and made fresh installs fail at 70%.
+DOCKER_UI_DIR="$SCRIPT_DIR/../../docker"
+[ -d "$DOCKER_UI_DIR" ] || { echo "Missing docker build sources" >&2; exit 1; }
+mkdir -p "$ARCH_DIR/docker"
+cp -a "$DOCKER_UI_DIR/." "$ARCH_DIR/docker/"
+python3 "$SCRIPT_DIR/../../scripts/check-app-build-contexts.py" "$ARCH_DIR"
 
 if [ "$UNBUNDLED" = "1" ]; then
     echo "   ✅ Unbundled build ready (Tor setup included, no container images)"
@@ -3237,6 +3235,18 @@ for test_script in run-e2e-tests.sh run-post-install-tests.sh; do
     fi
 done
 
+# BEGIN DOCTOR OVERLAY
+# Replace both the active and historical script locations before first boot.
+# A cached rootfs can contain the unsafe network recovery implementation.
+mkdir -p /mnt/target/opt/archipelago/scripts /mnt/target/home/archipelago/archy/scripts
+for doctor_dir in /mnt/target/opt/archipelago/scripts /mnt/target/home/archipelago/archy/scripts; do
+    install -m 755 "$BOOT_MEDIA/archipelago/scripts/container-doctor.sh" "$doctor_dir/container-doctor.sh" || exit 1
+done
+for doctor_unit in archipelago-doctor.service archipelago-doctor.timer; do
+    install -m 644 "$BOOT_MEDIA/archipelago/scripts/$doctor_unit" "/mnt/target/etc/systemd/system/$doctor_unit" || exit 1
+done
+# END DOCTOR OVERLAY
+
 # Copy self-update script
 if [ -f "$BOOT_MEDIA/archipelago/scripts/self-update.sh" ]; then
     cp "$BOOT_MEDIA/archipelago/scripts/self-update.sh" /mnt/target/opt/archipelago/scripts/

image-recipe/configs/nginx-gitea-iframe.conf

@@ -1,21 +0,0 @@
-# Gitea iframe proxy — strips X-Frame-Options so Gitea works in Archipelago iframe.
-# Gitea container binds to port 3001, this proxy listens on port 3000 (the public port).
-# Deployed to /etc/nginx/conf.d/gitea-iframe.conf
-server {
-    listen 3000;
-    server_name _;
-    client_max_body_size 1G;
-
-    location / {
-        proxy_pass http://127.0.0.1:3001;
-        proxy_set_header Host $http_host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-        proxy_http_version 1.1;
-        proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
-        proxy_hide_header X-Frame-Options;
-        proxy_hide_header Content-Security-Policy;
-    }
-}

neode-ui/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "neode-ui",
-  "version": "1.8.21-alpha",
+  "version": "1.8.22-alpha",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "neode-ui",
-      "version": "1.8.21-alpha",
+      "version": "1.8.22-alpha",
       "dependencies": {
         "@scure/bip39": "^2.2.0",
         "@types/dompurify": "^3.0.5",

neode-ui/package.json

@@ -1,7 +1,7 @@
 {
   "name": "neode-ui",
   "private": true,
-  "version": "1.8.21-alpha",
+  "version": "1.8.22-alpha",
   "type": "module",
   "scripts": {
     "start": "./start-dev.sh",

neode-ui/public/catalog.json

@@ -644,6 +644,35 @@
           "/var/lib/archipelago/vaultwarden:/data"
         ]
       }
+    },
+    {
+      "id": "angor-indexer",
+      "title": "Angor Indexer",
+      "version": "1.0.1",
+      "description": "Headless Bitcoin indexer endpoint for Angor. Reuses this node’s Mempool and Electrum index; requires a synced, unpruned Bitcoin node. Add this service’s address as the custom indexer in Angor settings. A relay is optional and installed separately.",
+      "dockerImage": "source.archipelago-foundation.org/chaum/angor-indexer:1.0.1",
+      "author": "Angor / Archipelago",
+      "requires": [
+        "Mempool API",
+        "Unpruned Bitcoin"
+      ],
+      "category": "money",
+      "tier": "optional",
+      "icon": "/assets/img/app-icons/angor-green.png",
+      "repoUrl": "https://github.com/block-core/angor"
+    },
+    {
+      "id": "angor-relay",
+      "title": "Angor Relay",
+      "version": "1.1.2",
+      "description": "Optional dedicated Nostr relay for Angor project metadata. Separate storage and access settings keep the node’s internal relay private. Add this service’s address to Angor’s relay settings; use WSS for browser clients.",
+      "dockerImage": "source.archipelago-foundation.org/chaum/angor-relay:1.1.2",
+      "author": "Angor / Archipelago",
+      "requires": [],
+      "category": "nostr",
+      "tier": "optional",
+      "icon": "/assets/img/app-icons/angor-green.png",
+      "repoUrl": "https://github.com/hoytech/strfry"
     }
   ]
 }

neode-ui/src/App.vue

@@ -19,6 +19,8 @@
     <AppLauncherOverlay />
     <AppCredentialInterstitial />
 
+    <UploadProgress v-if="route.name !== 'cloud-folder'" floating />
+
     <!-- Global toast notifications -->
     <ToastStack />
 
@@ -96,6 +98,7 @@
 </template>
 
 <script setup lang="ts">
+import UploadProgress from '@/components/cloud/UploadProgress.vue'
 import { computed, ref, onMounted, onBeforeUnmount, watch } from 'vue'
 import { useRouter, useRoute } from 'vue-router'
 import SplashScreen from './components/SplashScreen.vue'

neode-ui/src/api/__tests__/filebrowser-client.test.ts

@@ -336,3 +336,45 @@ describe('sanitizePath', () => {
     expect(sanitizePath('/photos//image.jpg')).toBe('/photos/image.jpg')
   })
 })
+
+describe('upload transport progress and cancellation', () => {
+  class UploadXHR {
+    static instances: UploadXHR[] = []
+    upload = { onprogress: null as ((e: { loaded: number }) => void) | null }
+    onload: (() => void) | null = null
+    onabort: (() => void) | null = null
+    onerror: (() => void) | null = null
+    status = 200
+    withCredentials = false
+    contentType = 'application/json'
+    open = vi.fn(); setRequestHeader = vi.fn(); send = vi.fn()
+    abort = vi.fn(() => this.onabort?.())
+    getResponseHeader() { return this.contentType }
+    constructor() { UploadXHR.instances.push(this) }
+  }
+  beforeEach(() => { setAuthenticated(); UploadXHR.instances = []; vi.stubGlobal('XMLHttpRequest', UploadXHR) })
+  it('sends the file bytes, escapes folder names, and waits for server acceptance after 100%', async () => {
+    const controller = new AbortController(); const onProgress = vi.fn()
+    const file = new File(['data'], 'a #.txt')
+    let complete = false
+    const job = fileBrowserClient.upload('/folder #1', file, { signal: controller.signal, onProgress }).then(() => { complete = true })
+    await Promise.resolve(); await Promise.resolve()
+    const xhr = UploadXHR.instances[0]!
+    expect(xhr.open).toHaveBeenCalledWith('POST', expect.stringMatching(/\/app\/filebrowser\/api\/resources\/folder%20%231\/a%20%23.txt\?override=true$/))
+    expect(xhr.send).toHaveBeenCalledWith(file)
+    xhr.upload.onprogress?.({ loaded: 4 }); expect(onProgress).toHaveBeenCalledWith(4)
+    expect(complete).toBe(false)
+    xhr.onload?.(); await job; expect(complete).toBe(true)
+  })
+  it('cancels the actual request and refuses HTML masquerading as upload success', async () => {
+    const controller = new AbortController()
+    const job = fileBrowserClient.upload('/', new File(['x'], 'f'), { signal: controller.signal, onProgress: vi.fn() })
+    await Promise.resolve(); await Promise.resolve()
+    controller.abort(); await expect(job).rejects.toMatchObject({ name: 'AbortError' })
+    expect(UploadXHR.instances[0]!.abort).toHaveBeenCalled()
+    const next = fileBrowserClient.upload('/', new File(['x'], 'f'), { signal: new AbortController().signal, onProgress: vi.fn() })
+    await Promise.resolve(); await Promise.resolve()
+    const xhr = UploadXHR.instances[1]!; xhr.contentType = 'text/html'; xhr.onload?.()
+    await expect(next).rejects.toThrow('login page')
+  })
+})

neode-ui/src/api/filebrowser-client.ts

@@ -201,7 +201,43 @@ class FileBrowserClient {
     URL.revokeObjectURL(blobUrl)
   }
 
-  async upload(dirPath: string, file: File): Promise<void> {
+  async upload(dirPath: string, file: File, options?: { signal: AbortSignal; onProgress: (sent: number) => void }): Promise<void> {
+    if (options) {
+      await this.ensureAuth()
+      if (options.signal.aborted) throw new DOMException('Upload cancelled', 'AbortError')
+      const folder = sanitizePath(dirPath).split('/').map(encodeURIComponent).join('/').replace(/\/$/, '')
+      const url = `${this.baseUrl}/api/resources${folder}/${encodeURIComponent(file.name)}?override=true`
+      const send = () => new Promise<number>((resolve, reject) => {
+        const xhr = new XMLHttpRequest()
+        const cleanup = () => options.signal.removeEventListener('abort', abort)
+        const abort = () => { xhr.abort(); cleanup(); reject(new DOMException('Upload cancelled', 'AbortError')) }
+        xhr.open('POST', url)
+        xhr.withCredentials = true
+        for (const [key, value] of Object.entries(this.headers())) xhr.setRequestHeader(key, value)
+        xhr.upload.onprogress = (event) => options.onProgress(Math.min(file.size, event.loaded))
+        xhr.onerror = () => { cleanup(); reject(new Error('Upload connection lost. Keep the companion open and check the server connection.')) }
+        xhr.onabort = () => { cleanup(); reject(new DOMException('Upload cancelled', 'AbortError')) }
+        xhr.onload = () => {
+          cleanup()
+          if (xhr.status === 401) { resolve(401); return }
+          if (xhr.status < 200 || xhr.status >= 300) { reject(new Error(`Upload failed (HTTP ${xhr.status})`)); return }
+          if ((xhr.getResponseHeader('Content-Type') || '').includes('text/html')) {
+            reject(new Error('File Browser returned a login page instead of accepting the upload.')); return
+          }
+          resolve(xhr.status)
+        }
+        options.signal.addEventListener('abort', abort, { once: true })
+        if (options.signal.aborted) { abort(); return }
+        xhr.send(file)
+      })
+      if (await send() === 401) {
+        this._authenticated = false
+        await this.ensureAuth()
+        if (options.signal.aborted) throw new DOMException('Upload cancelled', 'AbortError')
+        if (await send() === 401) throw new Error('Upload authentication expired. Sign in again.')
+      }
+      return
+    }
     const sanitized = sanitizePath(dirPath)
     const safePath = sanitized.endsWith('/') ? sanitized : `${sanitized}/`
     const encodedName = encodeURIComponent(file.name)

neode-ui/src/components/AppSearchField.vue

@@ -0,0 +1,37 @@
+<template>
+  <div class="relative min-w-0 flex-1">
+    <input
+      ref="input"
+      v-model="query"
+      type="text"
+      :placeholder="placeholder"
+      :aria-label="label"
+      data-controller-no-submit
+      class="app-header-search w-full pr-10 text-white placeholder-white/50 focus:outline-none transition-colors"
+      @keydown.esc.prevent="clear"
+    />
+    <button
+      v-if="query.length"
+      type="button"
+      aria-label="Clear search"
+      title="Clear search"
+      class="absolute right-1 top-1/2 -translate-y-1/2 w-8 h-8 flex items-center justify-center rounded-lg text-white/50 hover:text-white hover:bg-white/10 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-orange-400/60 transition-colors"
+      @click="clear"
+    >
+      <svg class="w-4 h-4" viewBox="0 0 24 24" fill="none" stroke="currentColor" aria-hidden="true">
+        <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="m6 6 12 12M6 18 18 6" />
+      </svg>
+    </button>
+  </div>
+</template>
+<script setup lang="ts">
+import { nextTick, ref } from 'vue'
+defineProps<{ placeholder: string; label: string }>()
+const query = defineModel<string>({ default: '' })
+const input = ref<HTMLInputElement | null>(null)
+async function clear() {
+  query.value = ''
+  await nextTick()
+  input.value?.focus()
+}
+</script>

neode-ui/src/components/InstallVersionModal.vue

@@ -25,13 +25,22 @@
     <div v-if="loading" class="py-6 text-center text-white/60 text-sm">{{ t('common.loading') }}</div>
 
     <div v-else class="space-y-2">
-      <label class="block text-white/60 text-sm">{{ t('appDetails.selectVersion') }}</label>
-      <select
-        v-model="selected"
-        class="w-full rounded-lg bg-white/[0.06] border border-white/10 text-white pl-3 pr-9 py-2 text-sm focus:outline-none focus:border-blue-400/60"
-      >
-        <option v-for="v in versions" :key="v.version" :value="v.version">{{ optionLabel(v) }}</option>
-      </select>
+      <fieldset class="space-y-2">
+        <legend class="text-white/60 text-sm mb-2">{{ t('appDetails.selectVersion') }}</legend>
+        <!-- Inline options avoid native popup rendering in the kiosk WebView.
+             They stay in document flow above the pruning explanation. -->
+        <div class="max-h-40 overflow-y-auto space-y-2 rounded-lg">
+          <label
+            v-for="v in versions"
+            :key="v.version"
+            class="flex items-center gap-3 rounded-lg border px-3 py-2.5 text-sm text-white cursor-pointer"
+            :class="selected === v.version ? 'border-blue-400/60 bg-slate-800' : 'border-white/10 bg-slate-900'"
+          >
+            <input v-model="selected" type="radio" :name="`install-version-${appId}`" :value="v.version" class="shrink-0 accent-blue-400" />
+            <span>{{ optionLabel(v) }}</span>
+          </label>
+        </div>
+      </fieldset>
       <p class="text-white/40 text-xs">{{ t('marketplace.installModalHint') }}</p>
     </div>
 
@@ -113,6 +122,7 @@ async function load() {
   } catch (err) {
     if (import.meta.env.DEV) console.warn('[InstallVersionModal] getPackageVersions failed:', err)
     // Fall back to the floating "latest" so the install can still proceed.
+    versions.value = [{ version: 'latest' } as CatalogVersionInfo]
     selected.value = 'latest'
   } finally {
     loading.value = false

neode-ui/src/components/LightningChannelsPanel.vue

@@ -353,15 +353,57 @@
 
     <!-- Close Confirmation Modal -->
     <Teleport to="body">
-    <div v-if="closeTarget" class="fixed inset-0 z-[3100] flex items-center justify-center bg-black/60 backdrop-blur-md" @click.self="closeTarget = null">
+    <div v-if="closeTarget" class="fixed inset-0 z-[3100] flex items-center justify-center bg-black/60 backdrop-blur-md" @click.self="!closingChannel && (closeTarget = null)">
       <div class="glass-card p-6 w-full max-w-sm mx-4">
         <h2 class="text-lg font-bold text-white mb-2">Close Channel?</h2>
         <p class="text-white/60 text-sm mb-4">This will cooperatively close the channel with peer {{ closeTarget.remote_pubkey.slice(0, 16) }}...</p>
+          <!-- Fee selection -->
+          <div class="mb-4">
+            <label class="text-white/60 text-sm block mb-1">Fee</label>
+            <div class="flex gap-1 p-1 bg-white/5 rounded-lg">
+              <button
+                v-for="preset in feePresets"
+                :key="preset.key"
+                @click="closeForm.feePreset = preset.key"
+                class="flex-1 px-2 py-1.5 rounded text-xs font-medium transition-colors"
+                :class="closeForm.feePreset === preset.key ? 'bg-white/15 text-white' : 'text-white/50 hover:text-white/80'"
+              >{{ preset.label }}</button>
+            </div>
+            <p v-if="closeForm.feePreset !== 'custom'" class="text-white/40 text-xs mt-1">
+              {{ feePresets.find(p => p.key === closeForm.feePreset)?.hint }}
+            </p>
+            <div v-else class="grid grid-cols-2 gap-3 mt-2">
+              <div>
+                <label class="text-white/60 text-xs block mb-1">Target confirmations</label>
+                <input
+                  v-model.number="closeForm.customConfTarget"
+                  type="number"
+                  min="1"
+                  max="1008"
+                  placeholder="6"
+                  class="w-full input-glass"
+                />
+              </div>
+              <div>
+                <label class="text-white/60 text-xs block mb-1">Sats per vByte</label>
+                <input
+                  v-model.number="closeForm.customSatPerVbyte"
+                  type="number"
+                  min="1"
+                  max="5000"
+                  placeholder="—"
+                  class="w-full input-glass"
+                />
+              </div>
+              <p class="text-white/40 text-xs col-span-2">Set one — sats per vByte takes precedence when both are set</p>
+            </div>
+          </div>
+
         <div v-if="closeError" class="mb-3 alert-error">
           <p class="text-xs">{{ closeError }}</p>
         </div>
         <div class="flex gap-3">
-          <button @click="closeTarget = null" class="flex-1 glass-button px-4 py-2 rounded-lg text-sm">Cancel</button>
+          <button @click="closeTarget = null" :disabled="closingChannel" class="flex-1 glass-button px-4 py-2 rounded-lg text-sm">Cancel</button>
           <button
             @click="closeChannel"
             :disabled="closingChannel"
@@ -457,8 +499,8 @@ function closeTypeLabel(ch: ClosedChannel): string {
 type FeePreset = 'standard' | 'medium' | 'fast' | 'custom'
 
 const feePresets: { key: FeePreset; label: string; hint?: string; confTarget?: number }[] = [
-  { key: 'standard', label: 'Standard', hint: 'Confirms within ~6 blocks (about an hour)', confTarget: 6 },
-  { key: 'medium', label: 'Medium', hint: 'Confirms within ~3 blocks (about 30 minutes)', confTarget: 3 },
+  { key: 'standard', label: 'Standard', hint: 'Targets ~6 blocks (about an hour)', confTarget: 6 },
+  { key: 'medium', label: 'Medium', hint: 'Targets ~3 blocks (about 30 minutes)', confTarget: 3 },
   { key: 'fast', label: 'Fast', hint: 'Targets the next block', confTarget: 1 },
   { key: 'custom', label: 'Custom' },
 ]
@@ -577,22 +619,24 @@ function loadChannels(): Promise<void> {
   return main
 }
 
-function feeParams(): { target_conf?: number; sat_per_vbyte?: number } | null {
-  const form = openForm.value
+function feeParams(
+  form: { feePreset: FeePreset; customSatPerVbyte: number | null; customConfTarget: number | null } = openForm.value,
+  setError: (message: string) => void = message => { openError.value = message },
+): { target_conf?: number; sat_per_vbyte?: number } | null {
   if (form.feePreset !== 'custom') {
     return { target_conf: feePresets.find(p => p.key === form.feePreset)?.confTarget ?? 6 }
   }
   const rate = form.customSatPerVbyte
   const conf = form.customConfTarget
   if (rate != null && rate !== 0) {
-    if (rate < 1 || rate > 5000) { openError.value = 'Sats per vByte must be between 1 and 5000'; return null }
+    if (!Number.isInteger(rate) || rate < 1 || rate > 5000) { setError('Sats per vByte must be a whole number between 1 and 5000'); return null }
     return { sat_per_vbyte: Math.floor(rate) }
   }
   if (conf != null && conf !== 0) {
-    if (conf < 1 || conf > 1008) { openError.value = 'Target confirmations must be between 1 and 1008'; return null }
+    if (!Number.isInteger(conf) || conf < 1 || conf > 1008) { setError('Target confirmations must be a whole number between 1 and 1008'); return null }
     return { target_conf: Math.floor(conf) }
   }
-  openError.value = 'Custom fee requires target confirmations or sats per vByte'
+  setError('Custom fee requires target confirmations or sats per vByte')
   return null
 }
 
@@ -629,7 +673,12 @@ async function openChannel() {
   }
 }
 
+const defaultCloseForm = () => ({ feePreset: 'standard' as FeePreset, customConfTarget: null as number | null, customSatPerVbyte: null as number | null })
+const closeForm = ref(defaultCloseForm())
+
 function confirmClose(ch: Channel) {
+  if (closingChannel.value) return
+  closeForm.value = defaultCloseForm()
   closeTarget.value = ch
   closeError.value = null
 }
@@ -637,12 +686,15 @@ function confirmClose(ch: Channel) {
 async function closeChannel() {
   if (closingChannel.value || !closeTarget.value) return
   closeError.value = null
+  const fee = feeParams(closeForm.value, message => { closeError.value = message })
+  if (!fee) return
   closingChannel.value = true
   try {
     await rpcClient.call({
       method: 'lnd.closechannel',
-      params: { channel_point: closeTarget.value.channel_point },
-      timeout: 30000,
+      params: { channel_point: closeTarget.value.channel_point, ...fee },
+      timeout: 45000,
+      maxRetries: 1,
     })
     closeTarget.value = null
     await loadChannels()

neode-ui/src/components/TransactionsModal.vue

@@ -9,11 +9,11 @@
         <!-- Transparent glass, not a black slab (operator, 2026-08-09): the
              backdrop blur alone keeps the pinned tabs legible over scrolling
              rows without painting an opaque container onto the modal. -->
-        <div v-if="transactions.length > 0" class="sticky top-0 z-10 -mx-2 px-2 pb-2 mb-1 flex gap-1.5 flex-wrap bg-white/5 backdrop-blur-md">
+        <div v-if="transactions.length > 0" class="sticky top-0 z-10 -mx-2 px-2 pb-2 mb-1 flex gap-1.5 flex-nowrap overflow-x-auto bg-transparent backdrop-blur-md">
           <button
             v-for="f in filters"
             :key="f.key"
-            class="px-2.5 py-1 rounded-full text-xs transition-colors"
+            class="shrink-0 whitespace-nowrap px-2.5 py-1 rounded-full text-xs transition-colors"
             :class="activeFilter === f.key
               ? 'bg-orange-500/25 text-orange-200 border border-orange-400/40'
               : 'bg-white/5 text-white/50 border border-white/10 hover:text-white/80'"

neode-ui/src/components/__tests__/InstallVersionModal.test.ts

@@ -16,12 +16,14 @@ describe('Bitcoin install storage choice', () => {
     versions.mockResolvedValue({ bitcoinPrune: false, default: 'latest', versions: [{ version: 'latest' }, { version: '28.4' }] })
     const wrapper = modal(id)
     await flushPromises()
-    await wrapper.get('select').setValue('28.4')
+    await wrapper.get('input[type=radio][value="28.4"]').setValue(true)
     await wrapper.get('input[type=checkbox]').setValue(true)
     await wrapper.get('button').trigger('click')
     expect(wrapper.emitted('confirm')).toEqual([['28.4', true]])
     expect(wrapper.text()).toContain('automatic pruning')
     expect(wrapper.text()).toContain('Mempool')
+    expect(wrapper.find('select').exists()).toBe(false)
+    expect(wrapper.findAll('input[type=radio]')).toHaveLength(2)
   })
   it('keeps automatic disk selection by default and resets on reopening', async () => {
     versions.mockResolvedValue({ bitcoinPrune: false, versions: [{ version: 'latest' }] })
@@ -29,17 +31,17 @@ describe('Bitcoin install storage choice', () => {
     await flushPromises()
     await wrapper.get('button').trigger('click')
     expect(wrapper.emitted('confirm')).toEqual([['latest', false]])
-    await wrapper.get('input').setValue(true)
+    await wrapper.get('input[type=checkbox]').setValue(true)
     await wrapper.setProps({ show: false })
     await wrapper.setProps({ show: true })
     await flushPromises()
-    expect((wrapper.get('input').element as HTMLInputElement).checked).toBe(false)
+    expect((wrapper.get('input[type=checkbox]').element as HTMLInputElement).checked).toBe(false)
   })
   it('still allows choosing pruning when version lookup fails', async () => {
     versions.mockRejectedValue(new Error('offline'))
     const wrapper = modal()
     await flushPromises()
-    await wrapper.get('input').setValue(true)
+    await wrapper.get('input[type=checkbox]').setValue(true)
     await wrapper.get('button').trigger('click')
     expect(wrapper.emitted('confirm')).toEqual([['latest', true]])
   })
@@ -47,7 +49,7 @@ describe('Bitcoin install storage choice', () => {
     versions.mockResolvedValue({ bitcoinPrune: true, versions: [{ version: 'latest' }] })
     const wrapper = modal('bitcoin-knots')
     await flushPromises()
-    expect((wrapper.get('input').element as HTMLInputElement).checked).toBe(true)
+    expect((wrapper.get('input[type=checkbox]').element as HTMLInputElement).checked).toBe(true)
     await wrapper.get('button').trigger('click')
     expect(wrapper.emitted('confirm')).toEqual([['latest', true]])
   })
@@ -62,6 +64,6 @@ describe('Bitcoin install storage choice', () => {
     versions.mockResolvedValue({ bitcoinPrune: false, versions: [{ version: 'latest' }] })
     const wrapper = modal('other')
     await flushPromises()
-    expect(wrapper.find('input').exists()).toBe(false)
+    expect(wrapper.find('input[type=checkbox]').exists()).toBe(false)
   })
 })

neode-ui/src/components/__tests__/LightningChannelsPanel.test.ts

@@ -0,0 +1,53 @@
+import { flushPromises, mount } from '@vue/test-utils'
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+import LightningChannelsPanel from '../LightningChannelsPanel.vue'
+import { rpcClient } from '@/api/rpc-client'
+vi.mock('@/api/rpc-client', () => ({ rpcClient: { call: vi.fn() } }))
+vi.mock('@/composables/useTxExplorer', () => ({ useTxExplorer: () => ({ openTx: vi.fn() }) }))
+vi.mock('@/composables/useCachedResource', async () => {
+  const { ref } = await import('vue')
+  return { useCachedResource: () => ({ data: ref(null), loadState: ref('ready'), error: ref(null), refresh: vi.fn().mockResolvedValue(undefined) }) }
+})
+const channel = { chan_id: 'test', remote_pubkey: '02' + 'a'.repeat(64), channel_point: 'b'.repeat(64) + ':0', capacity: 100000, local_balance: 50000, remote_balance: 50000, active: true }
+function open() {
+  const wrapper = mount(LightningChannelsPanel, { global: { stubs: { Teleport: true } } })
+  const vm = (wrapper.vm as any).$.setupState
+  vm.confirmClose(channel)
+  return { wrapper, vm }
+}
+beforeEach(() => { vi.clearAllMocks(); vi.mocked(rpcClient.call).mockResolvedValue({ success: true } as never) })
+describe('channel closing fee choice', () => {
+  it.each([['standard', 6], ['medium', 3], ['fast', 1]])('forwards %s target and never automatically retries the mutation', async (preset, target) => {
+    const { wrapper, vm } = open(); vm.closeForm.feePreset = preset
+    await vm.closeChannel()
+    expect(rpcClient.call).toHaveBeenCalledWith(expect.objectContaining({ method: 'lnd.closechannel', params: { channel_point: channel.channel_point, target_conf: target }, maxRetries: 1 }))
+    expect(vm.closeTarget).toBeNull(); wrapper.unmount()
+  })
+  it('sends the custom rate instead of a confirmation target', async () => {
+    const { wrapper, vm } = open(); vm.closeForm.feePreset = 'custom'; vm.closeForm.customSatPerVbyte = 25; vm.closeForm.customConfTarget = 3
+    await vm.closeChannel()
+    expect(rpcClient.call).toHaveBeenCalledWith(expect.objectContaining({ params: { channel_point: channel.channel_point, sat_per_vbyte: 25 } }))
+    wrapper.unmount()
+  })
+  it.each([0.5, -1, 5001, NaN, Infinity])('rejects invalid custom rate %s before RPC', async rate => {
+    const { wrapper, vm } = open(); vm.closeForm.feePreset = 'custom'; vm.closeForm.customSatPerVbyte = rate
+    await vm.closeChannel(); expect(rpcClient.call).not.toHaveBeenCalled(); expect(vm.closeError).toBeTruthy(); wrapper.unmount()
+  })
+  it('requires a custom value and accepts a custom confirmation target', async () => {
+    const { wrapper, vm } = open(); vm.closeForm.feePreset = 'custom'
+    await vm.closeChannel(); expect(rpcClient.call).not.toHaveBeenCalled()
+    vm.closeForm.customConfTarget = 2; await vm.closeChannel()
+    expect(rpcClient.call).toHaveBeenCalledWith(expect.objectContaining({ params: { channel_point: channel.channel_point, target_conf: 2 } })); wrapper.unmount()
+  })
+  it('keeps the chosen fee and error visible when LND rejects a close', async () => {
+    vi.mocked(rpcClient.call).mockRejectedValue(new Error('Peer is offline'))
+    const { wrapper, vm } = open(); vm.closeForm.feePreset = 'fast'
+    await vm.closeChannel(); expect(vm.closeTarget).not.toBeNull(); expect(vm.closeError).toBe('Peer is offline'); expect(vm.closeForm.feePreset).toBe('fast'); wrapper.unmount()
+  })
+  it('prevents duplicate submits while a close is pending', async () => {
+    let finish!: (value: unknown) => void
+    vi.mocked(rpcClient.call).mockImplementation(() => new Promise(resolve => { finish = resolve }) as never)
+    const { wrapper, vm } = open(); const pending = vm.closeChannel(); await vm.closeChannel()
+    expect(rpcClient.call).toHaveBeenCalledTimes(1); finish({ success: true }); await pending; await flushPromises(); wrapper.unmount()
+  })
+})

neode-ui/src/components/cloud/CloudToolbar.vue

@@ -45,7 +45,7 @@
         </button>
       </div>
 
-      <button class="glass-button cloud-toolbar-btn" title="Upload file" @click="triggerUpload">
+      <button class="glass-button cloud-toolbar-btn" title="Upload file" :disabled="uploading" @click="triggerUpload">
         <svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
           <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-8l-4-4m0 0L8 8m4-4v12" />
         </svg>
@@ -61,6 +61,7 @@
     <input
       ref="fileInput"
       type="file"
+      :disabled="uploading"
       class="hidden"
       multiple
       @change="handleFileSelect"
@@ -74,6 +75,7 @@ import { ref } from 'vue'
 defineProps<{
   breadcrumbs: { name: string; path: string }[]
   viewMode: 'list' | 'grid'
+  uploading?: boolean
 }>()
 
 const emit = defineEmits<{

neode-ui/src/components/cloud/UploadProgress.vue

@@ -0,0 +1,29 @@
+<template>
+  <div v-if="task" :class="floating ? 'fixed z-50 top-20 left-4 right-4 md:left-auto md:w-96' : 'mb-3 shrink-0'">
+    <div class="glass-card relative overflow-hidden h-11 px-3 flex items-center gap-2" role="status" aria-live="polite">
+      <div v-if="task.active" class="absolute inset-y-0 left-0 bg-emerald-400/10 transition-[width] duration-200 pointer-events-none" :style="{ width: `${percent}%` }" />
+      <div v-if="task.active" class="absolute bottom-0 left-0 h-0.5 bg-emerald-400 transition-[width] duration-200" :style="{ width: `${percent}%` }" role="progressbar" :aria-valuenow="percent" aria-valuemin="0" aria-valuemax="100" :aria-label="`Uploading ${task.filename}`" />
+      <span class="relative min-w-0 flex-1 text-sm truncate" :class="task.error ? 'text-red-300' : 'text-white/80'" :title="label">{{ label }}</span>
+      <span v-if="task.active" class="relative shrink-0 text-xs tabular-nums text-white/60">{{ percent }}%</span>
+      <button class="relative shrink-0 w-8 h-8 flex items-center justify-center rounded-lg text-white/60 hover:text-white hover:bg-white/10" :aria-label="task.active ? 'Cancel upload' : 'Dismiss upload'" @click="task.active ? store.cancelUpload() : store.dismissUpload()">
+        <svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24" aria-hidden="true"><path stroke-linecap="round" stroke-width="2" d="m6 6 12 12M6 18 18 6" /></svg>
+      </button>
+    </div>
+  </div>
+</template>
+<script setup lang="ts">
+import { computed } from 'vue'
+import { useCloudStore } from '@/stores/cloud'
+defineProps<{ floating?: boolean }>()
+const store = useCloudStore()
+const task = computed(() => store.upload)
+const percent = computed(() => !task.value ? 0 : task.value.total ? Math.min(100, Math.floor(task.value.sent * 100 / task.value.total)) : task.value.active ? 0 : 100)
+const label = computed(() => {
+  const t = task.value
+  if (!t) return ''
+  if (t.error) return t.error
+  if (t.cancelled) return `Upload stopped · ${t.completed}/${t.count} saved`
+  if (!t.active) return `${t.count === 1 ? t.filename : `${t.count} files`} uploaded`
+  return `${percent.value === 100 ? 'Saving' : 'Uploading'} ${t.filename}${t.count > 1 ? ` · ${t.completed + 1}/${t.count}` : ''}`
+})
+</script>

neode-ui/src/stores/__tests__/appLauncher.test.ts

@@ -34,6 +34,7 @@ vi.mock('@/api/rpc-client', () => ({
 vi.stubGlobal('open', mockWindowOpen)
 
 import { useAppLauncherStore, senderMatchesApp } from '../appLauncher'
+import { useAppStore } from '../app'
 
 describe('useAppLauncherStore', () => {
   beforeEach(() => {
@@ -54,6 +55,25 @@ describe('useAppLauncherStore', () => {
     })
   })
 
+  it('blocks both browser and embedded launch while HTTP is unready', () => {
+    const app = useAppStore()
+    app.data = { 'package-data': { gitea: { state: 'running', 'ui-ready': false, health: 'healthy', manifest: { id: 'gitea', title: 'Gitea' } } } } as never
+    const launcher = useAppLauncherStore()
+    launcher.openSession('gitea')
+    expect(launcher.panelAppId).toBeNull()
+    launcher.open({ url: 'http://192.0.2.10:3001/', title: 'Gitea', openInNewTab: true })
+    expect(mockWindowOpen).not.toHaveBeenCalled()
+    expect(launcher.isOpen).toBe(false)
+  })
+
+  it('also gates a dynamic app resolved through its runtime URL', () => {
+    useAppStore().data = { 'package-data': { custom: { state: 'running', 'ui-ready': false, manifest: { id: 'custom', title: 'Custom' }, installed: { 'interface-addresses': { main: { 'lan-address': 'http://localhost:18993/' } } } } } } as never
+    const launcher = useAppLauncherStore()
+    launcher.open({ url: 'http://192.0.2.10:18993/', title: 'Custom', openInNewTab: true })
+    expect(mockWindowOpen).not.toHaveBeenCalled()
+    expect(launcher.isOpen).toBe(false)
+  })
+
   it('starts closed with empty state', () => {
     const store = useAppLauncherStore()
     expect(store.isOpen).toBe(false)

neode-ui/src/stores/__tests__/cloud.test.ts

@@ -231,3 +231,50 @@ describe('useCloudStore', () => {
     expect(store.error).toBeNull()
   })
 })
+
+describe('persistent upload batch', () => {
+  beforeEach(() => { setActivePinia(createPinia()); vi.clearAllMocks() })
+  it('retains the destination and byte progress across folder navigation', async () => {
+    const store = useCloudStore(); store.authenticated = true; store.currentPath = '/original'
+    let release!: () => void
+    mockedClient.upload.mockImplementationOnce(async (_path, _file, options) => {
+      options!.onProgress(2)
+      await new Promise<void>(resolve => { release = resolve })
+    }).mockResolvedValueOnce(undefined)
+    mockedClient.listDirectory.mockResolvedValue([])
+    const job = store.uploadFiles([new File(['abcd'], 'one'), new File(['ef'], 'two')])
+    expect(store.upload?.sent).toBe(2)
+    expect(store.upload?.total).toBe(6)
+    await store.navigate('/elsewhere')
+    expect(useCloudStore().upload?.active).toBe(true)
+    release(); await job
+    expect(mockedClient.upload.mock.calls.map(call => call[0])).toEqual(['/original', '/original'])
+    expect(store.currentPath).toBe('/elsewhere')
+    expect(store.upload).toMatchObject({ active: false, sent: 6, completed: 2, error: null })
+  })
+  it('aborts the active request, stops the queue and preserves completed files', async () => {
+    const store = useCloudStore(); store.authenticated = true
+    mockedClient.listDirectory.mockResolvedValue([])
+    mockedClient.upload.mockResolvedValueOnce(undefined).mockImplementationOnce((_path, _file, options) => new Promise((_resolve, reject) => {
+      options!.signal.addEventListener('abort', () => reject(new DOMException('Cancelled', 'AbortError')))
+    }))
+    const files = ['one', 'two', 'three'].map(name => new File(['abc'], name))
+    const job = store.uploadFiles(files)
+    await Promise.resolve(); await Promise.resolve()
+    store.cancelUpload(); await job
+    expect(mockedClient.upload).toHaveBeenCalledTimes(2)
+    expect(store.upload).toMatchObject({ active: false, completed: 1, cancelled: true, error: null })
+    store.dismissUpload(); expect(store.upload).toBeNull()
+  })
+  it('keeps failures visible and does not start a second overlapping batch', async () => {
+    const store = useCloudStore(); store.authenticated = true
+    mockedClient.listDirectory.mockResolvedValue([])
+    let fail!: (error: Error) => void
+    mockedClient.upload.mockImplementationOnce(() => new Promise((_resolve, reject) => { fail = reject }))
+    const file = new File(['x'], 'one')
+    const job = store.uploadFiles([file]); await store.uploadFiles([file])
+    expect(mockedClient.upload).toHaveBeenCalledTimes(1)
+    fail(new Error('No space')); await job
+    expect(store.upload).toMatchObject({ active: false, error: 'No space', completed: 0 })
+  })
+})

neode-ui/src/stores/__tests__/server.test.ts

@@ -0,0 +1,42 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+import { createPinia, setActivePinia } from 'pinia'
+import { reactive, nextTick } from 'vue'
+
+const fake = reactive<{ packages: Record<string, unknown> }>({ packages: {} })
+vi.mock('../sync', () => ({ useSyncStore: () => fake }))
+vi.mock('../../api/rpc-client', () => ({ rpcClient: {} }))
+import { useServerStore } from '../server'
+
+function installing(phase = 'preparing-app') {
+  return { state: 'installing', manifest: { title: 'Git Workshop' }, 'install-progress': { phase, size: 0, downloaded: 0 } }
+}
+
+describe('installation state after hard refresh', () => {
+  beforeEach(() => {
+    setActivePinia(createPinia())
+    fake.packages = {}
+  })
+
+  it('restores an in-flight install from an already-loaded server snapshot', () => {
+    fake.packages = { 'archipelago-source': installing() }
+    const store = useServerStore()
+    expect(store.isInstalling('archipelago-source')).toBe(true)
+    expect(store.installingApps.get('archipelago-source')).toMatchObject({
+      progress: 20,
+      message: 'Downloading, building and starting app…',
+    })
+  })
+
+  it('keeps a long download visible and clears it on terminal success', async () => {
+    const store = useServerStore()
+    fake.packages = { 'nginx-proxy-manager': installing() }
+    await nextTick()
+    expect(store.isInstalling('nginx-proxy-manager')).toBe(true)
+    fake.packages = { 'nginx-proxy-manager': installing() }
+    await nextTick()
+    expect(store.installingApps.get('nginx-proxy-manager')?.progress).toBe(20)
+    fake.packages = { 'nginx-proxy-manager': { state: 'running' } }
+    await nextTick()
+    expect(store.isInstalling('nginx-proxy-manager')).toBe(false)
+  })
+})

neode-ui/src/stores/appLauncher.ts

@@ -239,6 +239,11 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
   const panelPath = ref<string | null>(null)
 
   function openSessionNow(appId: string, opts: LaunchOptions = {}) {
+    const pkg = useAppStore().data?.['package-data']?.[appId]
+    if (pkg?.['ui-ready'] === false) {
+      useToast().info(`${pkg.manifest?.title || appId} is not ready to open yet`)
+      return
+    }
     recordAppLaunch(appId)
     const mobile = isMobileViewport()
 
@@ -295,7 +300,7 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
     // Apply the same readiness gate here so a container that has just entered
     // `running` cannot race nginx and show a transient 502 to the user.
     const pkg = useAppStore().data?.['package-data']?.[appId]
-    if (pkg && pkg.state === 'running' && !isAppReadyForLaunch(pkg)) {
+    if (pkg && (pkg['ui-ready'] === false || (pkg.state === 'running' && !isAppReadyForLaunch(pkg)))) {
       useToast().info(`${pkg.manifest?.title || appId} is still starting — try again in a moment`)
       return
     }
@@ -394,6 +399,12 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
     let launchUrl = normalizeLaunchUrl(payload.url, titleHintId)
     const resolvedId = resolveAppIdFromUrl(launchUrl) || titleHintId
 
+    const pkg = resolvedId ? useAppStore().data?.['package-data']?.[resolvedId] : undefined
+    if (pkg?.['ui-ready'] === false) {
+      useToast().info(`${pkg.manifest?.title || resolvedId} is not ready to open yet`)
+      return
+    }
+
     // Scheme discipline for everything launched on this host. Ports fronted
     // by the node's app gate (manifest auth gated/open) serve TLS on the same
     // port — on an HTTPS connection those must open over https. Ports that
@@ -472,6 +483,17 @@ export const useAppLauncherStore = defineStore('appLauncher', () => {
       // Check /app/{id}/ path-style routes first (HTTPS proxy mode)
       const m = u.pathname.match(/^\/app\/([a-z0-9._-]+)(?:\/|$)/i)
       if (m?.[1]) return m[1].toLowerCase()
+      // Dynamic/sideloaded apps have no entry in the static port map.
+      if (u.hostname === window.location.hostname && u.port) {
+        for (const [id, pkg] of Object.entries(useAppStore().data?.['package-data'] || {})) {
+          const address = pkg.installed?.['interface-addresses']?.main?.['lan-address']
+          if (!address) continue
+          try {
+            const runtime = new URL(address)
+            if (runtime.port === u.port && ['localhost', '127.0.0.1', window.location.hostname].includes(runtime.hostname)) return id
+          } catch { /* malformed runtime address is not a launch target */ }
+        }
+      }
       // Check port-based apps
       const appId = PORT_TO_APP_ID[u.port]
       if (appId) return appId

neode-ui/src/stores/cloud.ts

@@ -8,6 +8,41 @@ export const useCloudStore = defineStore('cloud', () => {
   const loading = ref(false)
   const error = ref<string | null>(null)
   const authenticated = ref(false)
+  const upload = ref<{ active: boolean; filename: string; destination: string; sent: number; total: number; completed: number; count: number; error: string | null; cancelled: boolean } | null>(null)
+  let uploadController: AbortController | null = null
+  function cancelUpload() { uploadController?.abort() }
+  function dismissUpload() { if (!upload.value?.active) upload.value = null }
+  async function uploadFiles(files: File[]) {
+    if (!files.length || upload.value?.active) return
+    const destination = currentPath.value
+    const controller = new AbortController()
+    uploadController = controller
+    const task = { active: true, filename: files[0]!.name, destination, sent: 0, total: files.reduce((n, f) => n + f.size, 0), completed: 0, count: files.length, error: null as string | null, cancelled: false }
+    upload.value = task
+    let completedBytes = 0
+    try {
+      for (const file of files) {
+        if (controller.signal.aborted) throw new DOMException('Upload cancelled', 'AbortError')
+        upload.value.filename = file.name
+        await fileBrowserClient.upload(destination, file, { signal: controller.signal, onProgress: (sent) => {
+          if (upload.value?.active) upload.value.sent = completedBytes + sent
+        } })
+        completedBytes += file.size
+        upload.value.sent = completedBytes
+        upload.value.completed++
+      }
+    } catch (error) {
+      upload.value.cancelled = controller.signal.aborted
+      if (!upload.value.cancelled) upload.value.error = error instanceof Error ? error.message : 'Upload failed'
+    } finally {
+      upload.value.active = false
+      uploadController = null
+      // Navigation must never redirect subsequent files into the new folder.
+      pathCache.delete(destination)
+      if (currentPath.value === destination) await refresh()
+    }
+  }
+
   // Per-path listing cache: re-entering a folder paints the last listing
   // immediately (no spinner) while the fresh listing loads behind it.
   const pathCache = new Map<string, FileBrowserItem[]>()
@@ -131,6 +166,7 @@ export const useCloudStore = defineStore('cloud', () => {
   }
 
   return {
+    upload, uploadFiles, cancelUpload, dismissUpload,
     currentPath,
     items,
     loading,

neode-ui/src/stores/server.ts

@@ -25,6 +25,7 @@ import type { InstallPhase } from '../types/api'
 const PHASE_INFO: Record<InstallPhase, { progress: number; message: string; status: InstallProgress['status'] }> = {
   'preparing':          { progress: 5,  message: 'Preparing…',                status: 'downloading' },
   'pulling-image':      { progress: 20, message: 'Downloading image…',        status: 'downloading' },
+  'preparing-app':      { progress: 20, message: 'Downloading, building and starting app…', status: 'downloading' },
   'creating-container': { progress: 70, message: 'Creating container…',       status: 'installing' },
   'starting-container': { progress: 80, message: 'Starting container…',       status: 'starting' },
   'waiting-healthy':    { progress: 88, message: 'Finalizing first start…',   status: 'starting' },
@@ -149,7 +150,7 @@ export const useServerStore = defineStore('server', () => {
         uninstallingApps.value.delete(appId)
       }
     }
-  }, { deep: true })
+  }, { deep: true, immediate: true })
 
   function setInstallProgress(appId: string, progress: Partial<InstallProgress> & { id: string; title: string }) {
     const existing = installingApps.value.get(appId)

neode-ui/src/types/api.ts

@@ -88,6 +88,7 @@ export const PackageState = {
 export type PackageState = typeof PackageState[keyof typeof PackageState]
 
 export interface PackageDataEntry {
+  'ui-ready'?: boolean // HTTP upstream readiness, separate from container health
   state: PackageState
   health?: string | null // "healthy", "unhealthy", "starting", or null
   'exit-code'?: number | null // container exit code: 0 = clean stop, non-zero = crash
@@ -180,6 +181,7 @@ export type ServiceStatus = typeof ServiceStatus[keyof typeof ServiceStatus]
 export type InstallPhase =
   | 'preparing'
   | 'pulling-image'
+  | 'preparing-app'
   | 'creating-container'
   | 'starting-container'
   | 'waiting-healthy'

neode-ui/src/views/AppDetails.vue

@@ -259,7 +259,7 @@ const canLaunch = computed(() => {
   const hasRuntimeAddress = !!pkg.value.installed?.['interface-addresses']?.main?.['lan-address']
   const hasKnownLaunchUrl = typeof window !== 'undefined' && !!resolveAppUrl(pkg.value.manifest.id)
   const hasUI = !!(pkg.value.manifest.interfaces?.main?.ui || hasRuntimeAddress || hasKnownLaunchUrl)
-  return hasUI && pkg.value.state === 'running' && pkg.value.health !== 'starting' && pkg.value.health !== 'unhealthy'
+  return hasUI && pkg.value['ui-ready'] !== false && pkg.value.state === 'running' && pkg.value.health !== 'starting' && pkg.value.health !== 'unhealthy'
 })
 
 const features = computed(() => {

neode-ui/src/views/AppSession.vue

@@ -39,6 +39,7 @@
         :must-open-new-tab="mustOpenNewTab"
         :auto-retry-count="autoRetryCount"
         :refresh-key="refreshKey"
+        :ui-ready-blocked="packageEntry?.['ui-ready'] === false"
         :blocked-reason="blockedReason"
         :blocked-title="blockedTitle"
         :warming-up="warmingUp"
@@ -375,6 +376,20 @@ const panelClasses = computed(() => {
   return `${base} app-session-overlay`
 })
 
+// A cold/restarting upstream is held outside the iframe. Start one fresh
+// load when the scanner observes HTTP readiness; no manual refresh required.
+watch(() => packageEntry.value?.['ui-ready'], (ready, previous) => {
+  if (ready === false) {
+    if (loadTimeoutId) clearTimeout(loadTimeoutId)
+    if (autoRetryId) clearTimeout(autoRetryId)
+    if (iframeCheckId) clearTimeout(iframeCheckId)
+    loading.value = false
+  } else if (previous === false && ready === true) {
+    autoRetryCount.value = 0
+    refresh()
+  }
+})
+
 // --- Lifecycle handlers ---
 
 function onLoad() {
@@ -433,6 +448,7 @@ function refresh() {
 
 function startLoadTimeout() {
   if (loadTimeoutId) clearTimeout(loadTimeoutId)
+  if (packageEntry.value?.['ui-ready'] === false) return
   loadTimeoutId = setTimeout(() => {
     if (loading.value) {
       loading.value = false
@@ -442,11 +458,13 @@ function startLoadTimeout() {
 }
 
 function openNewTabAndBack() {
+  if (packageEntry.value?.['ui-ready'] === false) return
   if (appUrl.value) openExternalUrl(appUrl.value)
   closeSession()
 }
 
 function openNewTab() {
+  if (packageEntry.value?.['ui-ready'] === false) return
   if (appUrl.value) openExternalUrl(appUrl.value)
 }
 

neode-ui/src/views/Apps.vue

@@ -56,14 +56,7 @@
           </button>
         </div>
         <div class="app-header-search-wrap flex items-center gap-2">
-          <input
-            v-model="searchQuery"
-            type="text"
-            :placeholder="t('apps.searchPlaceholder')"
-            :aria-label="t('apps.searchLabel')"
-            data-controller-no-submit
-            class="app-header-search min-w-0 flex-1 text-white placeholder-white/50 focus:outline-none transition-colors"
-          />
+          <AppSearchField v-model="searchQuery" :placeholder="t('apps.searchPlaceholder')" :label="t('apps.searchLabel')" />
           <button
             type="button"
             class="sideload-icon-btn"
@@ -106,14 +99,7 @@
           >{{ category.name }}</button>
         </div>
         <div class="flex items-center gap-2">
-          <input
-            v-model="searchQuery"
-            type="text"
-            :placeholder="t('apps.searchPlaceholder')"
-            :aria-label="t('apps.searchLabel')"
-            data-controller-no-submit
-            class="app-header-search min-w-0 flex-1 text-white placeholder-white/50 focus:outline-none transition-colors"
-          />
+          <AppSearchField v-model="searchQuery" :placeholder="t('apps.searchPlaceholder')" :label="t('apps.searchLabel')" />
           <button
             type="button"
             class="sideload-icon-btn sideload-icon-btn-mobile"
@@ -374,6 +360,7 @@ let appsAnimationDone = false
 </script>
 
 <script setup lang="ts">
+import AppSearchField from '@/components/AppSearchField.vue'
 import { computed, ref, watch, onActivated, onBeforeUnmount, onDeactivated, onMounted } from 'vue'
 import { useRouter, useRoute, RouterLink } from 'vue-router'
 import { useI18n } from 'vue-i18n'

neode-ui/src/views/CloudFolder.vue

@@ -39,6 +39,8 @@
       </div>
     </div>
 
+    <UploadProgress />
+
     <!-- App Not Installed -->
     <div v-if="!appRunning" class="glass-card p-12 text-center flex-1 flex flex-col items-center justify-center">
       <svg class="w-20 h-20 text-white/15 mb-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -75,19 +77,11 @@
           <p class="text-sm text-white/50">Files will be added to the current folder</p>
         </div>
       </div>
-      <!-- Upload progress -->
-      <div v-if="uploading" class="glass-card p-3 mb-3 flex items-center gap-3">
-        <div class="w-5 h-5 border-2 border-white/20 border-t-white/80 rounded-full animate-spin"></div>
-        <span class="text-sm text-white/70">Uploading...</span>
-      </div>
-      <div v-if="uploadError" class="glass-card p-3 mb-3 flex items-center gap-3 border border-red-500/30">
-        <span class="text-sm text-red-400">{{ uploadError }}</span>
-        <button class="text-xs text-white/50 hover:text-white ml-auto" @click="uploadError = null">Dismiss</button>
-      </div>
 
       <CloudToolbar
         :breadcrumbs="cloudStore.breadcrumbs"
         :view-mode="viewMode"
+        :uploading="!!cloudStore.upload?.active"
         @navigate="navigateCloudPath"
         @refresh="cloudStore.refresh()"
         @upload="handleUpload"
@@ -105,6 +99,7 @@
           :items="cloudStore.sortedItems"
           :loading="cloudStore.loading"
           :view-mode="viewMode"
+        :uploading="!!cloudStore.upload?.active"
           @navigate="navigateCloudPath"
           @delete="handleDelete"
           @play="handlePlay"
@@ -159,6 +154,7 @@
 </template>
 
 <script setup lang="ts">
+import UploadProgress from '@/components/cloud/UploadProgress.vue'
 import { ref, computed, watch } from 'vue'
 import { useRouter, useRoute, RouterLink } from 'vue-router'
 import { useAppStore } from '../stores/app'
@@ -193,7 +189,6 @@ watch(() => cloudStore.currentPath, (path) => {
 })
 
 const iframeLoaded = ref(false)
-const uploading = ref(false)
 const folderId = computed(() => route.params.folderId as string)
 const routeFolderPath = computed(() => normalizeCloudPath(route.query.path, section.value?.initialPath || '/'))
 
@@ -348,7 +343,6 @@ function handleShare(path: string, name: string, isDir: boolean) {
   shareTarget.value = { path, name, isDir }
 }
 
-const uploadError = ref<string | null>(null)
 const draggingOver = ref(false)
 let dragLeaveTimer: ReturnType<typeof setTimeout> | null = null
 
@@ -372,17 +366,7 @@ function onDrop(e: DragEvent) {
 }
 
 async function handleUpload(files: File[]) {
-  uploading.value = true
-  uploadError.value = null
-  try {
-    for (const file of files) {
-      await cloudStore.uploadFile(file)
-    }
-  } catch (e) {
-    uploadError.value = e instanceof Error ? e.message : 'Upload failed'
-  } finally {
-    uploading.value = false
-  }
+  await cloudStore.uploadFiles(files)
 }
 
 async function handleDelete(path: string) {

neode-ui/src/views/Discover.vue

@@ -49,14 +49,7 @@
             {{ section.name }}
           </button>
         </div>
-        <input
-          v-model="searchQuery"
-          type="text"
-          placeholder="Search apps..."
-          aria-label="Search apps"
-          data-controller-no-submit
-          class="app-header-search text-white placeholder-white/50 focus:outline-none transition-colors"
-        />
+        <AppSearchField v-model="searchQuery" placeholder="Search apps..." label="Search apps" />
         <RefreshIndicator :state="catalogResource.entry.loadState" label="Refreshing app store catalog" />
       </div>
 
@@ -82,14 +75,7 @@
             type="button"
           >{{ section.name }}</button>
         </div>
-        <input
-          v-model="searchQuery"
-          type="text"
-          placeholder="Search apps..."
-          aria-label="Search apps"
-          data-controller-no-submit
-          class="app-header-search w-full text-white placeholder-white/50 focus:outline-none transition-colors"
-        />
+        <AppSearchField v-model="searchQuery" placeholder="Search apps..." label="Search apps" />
       </div>
     </div>
 
@@ -345,6 +331,7 @@ let discoverAnimationDone = false
 </script>
 
 <script setup lang="ts">
+import AppSearchField from '@/components/AppSearchField.vue'
 import { ref, computed, onBeforeUnmount, onMounted } from 'vue'
 import { useRouter, RouterLink } from 'vue-router'
 import { useAppStore } from '@/stores/app'