373 lines
22 KiB
Markdown
373 lines
22 KiB
Markdown
# Release 1.7.121 — task list
|
||
|
||
Opened 2026-08-03, immediately after v1.7.120-alpha shipped. Everything the operator has
|
||
asked for since, plus the items v1.7.120 deliberately left open. Ordered by severity.
|
||
|
||
Status key: **DONE** (committed) · **READY** (written, not yet committed/tested) ·
|
||
**OPEN** (not started) · **BLOCKED** (needs an operator decision)
|
||
|
||
---
|
||
|
||
## P0 — Security
|
||
|
||
### 1. App ports are reachable with no login, on every transport — **OPEN**
|
||
> "if I'm logged out I can reach every app port on tailscale and LAN, this can not be
|
||
> allowed… it must present the login to access the app with an app icon of what you're
|
||
> accessing to confirm, and 2FA if present" — operator, 2026-08-03
|
||
|
||
- Applies to **Tailscale, LAN, Tor, FIPS** alike, and to "ssh access to that port or whatever".
|
||
- Required behaviour: an unauthenticated request to any app port serves a **login page
|
||
naming and showing the icon of the app being accessed**, then honours **2FA when set**.
|
||
- **Research first:** how umbrelOS and StartOS gate app access (operator asked explicitly).
|
||
Both are open source — `getumbrel/umbrel` and `Start9Labs/start-os`. Do not guess at
|
||
their model; read it.
|
||
- This is the same class as the v1.7.120 `/lnd-connect-info` + `/bitcoin-rpc/` leaks, but
|
||
**fleet-wide across every app port** rather than two endpoints. Those two were closed by
|
||
moving authorisation to the resource; this needs a general gate.
|
||
- Scope note: `fips/app_ports.rs` holds the mesh allowlist; `is_peer_allowed_path` in
|
||
`server.rs` holds the peer HTTP allowlist. Neither currently authenticates app ports.
|
||
|
||
#### Research — umbrelOS (verified from their docs/source, 2026-08-03)
|
||
|
||
umbrelOS solves this **architecturally, not per-app**: the app's own port is never
|
||
published. Each app gets a sidecar `app_proxy` container that owns the published port and
|
||
forwards to the app on the internal network.
|
||
|
||
- `containers/app-proxy` is described as *"a transparent HTTP proxy to add authentication
|
||
to Umbrel apps"* — **every** HTTP request and WebSocket upgrade passes through it and
|
||
has its session token checked.
|
||
- Tokens come from a separate `app-auth` service; the proxy talks to it over a local port
|
||
(default 2000) with a shared secret (`UMBREL_AUTH_SECRET`). Two JWTs exist: an **API
|
||
token** in localStorage (`{loggedIn: true}`) for the dashboard's own API, and a
|
||
**proxy token** in an **HttpOnly cookie** (`{proxyToken: true}`) for app access. Both
|
||
HS256, 7-day expiry.
|
||
- Unauthenticated requests are redirected to the login screen.
|
||
- Per-app escape hatches, all env vars on the proxy: `PROXY_AUTH_ADD` (bool, **default
|
||
true** — so apps are protected unless opted out), `PROXY_AUTH_WHITELIST` (paths exempt,
|
||
e.g. `/public/*`), `PROXY_AUTH_BLACKLIST` (paths that must be authed, e.g. `/admin/*`).
|
||
- Known friction worth designing around: apps with their own login (Frigate, and the
|
||
`PROXY_AUTH_ADD=false` tracker issue) end up double-authenticating, and non-browser API
|
||
clients (Home Assistant hitting an app's API) break because they have no cookie. Any
|
||
gate we build needs a story for machine clients, not just browsers.
|
||
|
||
**The lesson for us:** the reason umbrel doesn't have this bug class is that there is no
|
||
unauthenticated path to bind to in the first place. Our apps publish their own ports
|
||
directly, so a gate bolted onto one transport leaves the others open — which is exactly
|
||
the shape of the `/lnd-connect-info` + `/bitcoin-rpc/` leaks. The fix likely has to move
|
||
the port binding, not just add a check.
|
||
|
||
#### Reproduced ON archi-dev-box, 2026-08-03 — baseline before the fix
|
||
|
||
No session cookie, over the Tailscale IP `100.69.68.39`:
|
||
|
||
```
|
||
port 18083 HTTP 200 LND - Archipelago
|
||
port 8334 HTTP 200
|
||
port 8175 HTTP 200 Fedimint Guardian - Archipelago
|
||
port 8336 HTTP 200 FIPS Mesh
|
||
port 8090 HTTP 200
|
||
port 7777 HTTP 200
|
||
```
|
||
|
||
`ss -tlnp` confirms these are bound `0.0.0.0`, so the same responses are served on the LAN
|
||
IP and every other host address. Re-run this exact loop after the fix: every one must
|
||
become the login page, and the ports listed as protocol exemptions (item 1b) must be the
|
||
*only* ones still answering.
|
||
|
||
#### Exposure map — how app ports are reachable TODAY (verified in source, 2026-08-03)
|
||
|
||
All four transports converge on `127.0.0.1:<app_port>`. This is the whole reason the fix
|
||
is tractable: it is **one gate, not four**.
|
||
|
||
| Transport | Path to the app | Code |
|
||
|---|---|---|
|
||
| LAN / Tailscale | container publishes the port on the host (`--network host`, so `0.0.0.0:<port>`) — reachable on *every* host IP | `scripts/container-specs.sh`, `first-boot-containers.sh` |
|
||
| FIPS mesh | daemon binds `[fips0-ULA]:<port>` and raw-TCP-forwards to `127.0.0.1:<port>` | `server.rs:1130` `app_port_v6_relay_loop` |
|
||
| FIPS firewall | `tcp dport { …APP_LAUNCH_PORTS… } accept` drop-in opens them all | `fips/config.rs:274`, `fips/app_ports.rs` |
|
||
| Tor | `HiddenServicePort 80 127.0.0.1:<local_port>` per service | `api/rpc/tor/mod.rs:243` |
|
||
|
||
#### Design decision (operator, 2026-08-03)
|
||
|
||
**Gate app UIs + bearer tokens; protocol ports exempt.** HTTP app UIs get the login gate
|
||
(app name + icon, 2FA honoured). Protocol ports (LND gRPC 10009 + REST, electrum 50002,
|
||
bitcoin p2p 8333) stay open but MUST be declared `auth: none` with a rationale in the
|
||
manifest, so the exceptions are a grep rather than a discovery — see item 1b. Per-app
|
||
long-lived bearer tokens cover machine clients that speak HTTP (Home Assistant). **Zeus
|
||
and electrum wallets keep working untouched** — that was the deciding constraint.
|
||
|
||
The gate lives in the **daemon**, not a per-app sidecar container (umbrel's `app_proxy`
|
||
model): rootless, no extra containers per app, one place to update, and it can reuse the
|
||
existing `app_port_v6_relay_loop` rather than fight it.
|
||
|
||
#### ⚠️ Trap found while designing — an nft-only gate FAILS OPEN
|
||
|
||
The obvious implementation is an nft redirect of inbound app-port traffic to the gate.
|
||
But `/etc/fips/fips.nft` is **provisioned out-of-band** and `fips/config.rs:290` treats its
|
||
absence as a no-op (`if try_exists("/etc/fips/fips.nft")`). A gate shipped as a `fips.d`
|
||
drop-in would therefore be **silently absent on every node without the hardening
|
||
baseline** — i.e. it fails open, which is exactly the failure class this item exists to
|
||
close.
|
||
|
||
Two viable shapes, both fail-closed:
|
||
- **(a) Apps bind loopback only**, daemon owns every external bind. Airtight, the true
|
||
umbrel model, but requires touching each app's own listen config (nginx.conf etc.).
|
||
Note you *cannot* half-do this: while an app holds `0.0.0.0:<port>`, the daemon cannot
|
||
bind `<lan-ip>:<port>` at all.
|
||
- **(b) Daemon owns a dedicated `archipelago-appgate` nft table** with its own
|
||
default-deny + redirect, independent of whether `fips.nft` exists, and refuses to start
|
||
/ alarms loudly if it cannot install it. Non-invasive to apps.
|
||
|
||
#### Enabler found — `PortMapping.bind` already does half of (a)
|
||
|
||
`core/container/src/manifest.rs:518` — `PortMapping` has a `bind` field, documented as
|
||
*"Host address to bind the publish to. Empty = all interfaces (0.0.0.0). Set `127.0.0.1`
|
||
to keep a port host-local"*. So for **bridge apps that declare `ports:`**, going
|
||
loopback-only is a **manifest edit, not app surgery**, and the daemon can then own the
|
||
external bind. That is most of the catalog.
|
||
|
||
The exception is **host-networked apps** (`security.network_policy: host` — `lnd-ui`,
|
||
`bitcoin-ui`, `electrs-ui`): host networking bypasses port mapping entirely, so `bind` has
|
||
no effect and `ports:` is deliberately empty. Those bind whatever their internal nginx
|
||
binds. We build those images ourselves, so the fix is a `listen 127.0.0.1:<port>;` change
|
||
in each `docker/*-ui/nginx.conf` — still no third-party surgery.
|
||
|
||
Watch the rootless trap documented at `manifest.rs:532`: a publish bound to an address the
|
||
host cannot actually bind crash-loops the whole unit (took bitcoin down fleet-wide on .228,
|
||
2026-07-09). Loopback binds are explicitly always accepted without probing, so this
|
||
direction is safe.
|
||
|
||
**Tor needs separate handling either way**: the onion connects *from* localhost, so a
|
||
redirect that exempts loopback will not catch it. `HiddenServicePort` must be repointed at
|
||
the gate, and since that mapping loses the original destination port, each app needs its
|
||
own gate port (or an HTTP-level Host mapping).
|
||
|
||
#### Primitives that already exist — do NOT build these from scratch
|
||
|
||
The gate is mostly assembly, not invention:
|
||
|
||
| Need | Existing API |
|
||
|---|---|
|
||
| Read the session cookie off a request | `session::extract_session_cookie(&HeaderMap) -> Option<String>` (`session.rs:479`) |
|
||
| Validate a session | `SessionStore::validate(&token) -> bool` (`session.rs:194`) |
|
||
| **Honour 2FA** | Already modelled: `create_pending(totp_secret)` (`:176`) + `upgrade_to_full` (`:247`). A session still pending 2FA **fails `validate()`**, so the gate gets 2FA for free by calling `validate` — no TOTP code in the gate itself |
|
||
| **Machine-client bearer tokens** | `device_tokens::create/verify` (`device_tokens.rs:63/:90`) — long-lived, minted from an authenticated session, only the SHA-256 persisted, plaintext returned once. Built for the companion pairing QR; needs **per-app scoping** added for this use |
|
||
| Rate limiting | `device_tokens` verification already rides `auth.login`'s limiter |
|
||
|
||
So the new code is: the listener/redirect, the app-identification step (which app is this port?),
|
||
the login page render (app name + icon), and per-app scoping on `device_tokens`.
|
||
|
||
#### Research — StartOS: **NOT YET VERIFIED**
|
||
|
||
Their public docs cover the *addressing* model (per-service `.onion` and `.local`
|
||
addresses, an explicit "make public" opt-in for clearnet) but do not state whether a
|
||
universal auth layer sits in front of service interfaces, and the source could not be
|
||
read from this box (`gh` is not installed, and raw GitHub paths 404'd). **Do not assume
|
||
they delegate auth to each service — read `Start9Labs/start-os` before designing.**
|
||
|
||
### 2. Filebrowser ships an insecure default login — **OPEN**
|
||
- Change the default credential **without breaking the dashboard's Cloud view**, which
|
||
authenticates to filebrowser on the user's behalf.
|
||
- Related prior art: FED-07 rotated the shipped Fedimint gateway credential and had to
|
||
recreate the running container for it to take effect (`06e0e695`) — the same trap
|
||
applies here.
|
||
|
||
### 3. Federation trust escalation — **DONE** (`c0cfc72a`, pushed)
|
||
Two independent fail-open paths granted `Trusted` without any operator decision:
|
||
|
||
- `federation.peer-joined` is **unauthenticated** (middleware no-session list) and
|
||
peer-reachable on `/rpc/v1`. Its ed25519 check verifies the caller against **the pubkey
|
||
the caller supplied**, so it proves key possession, never authorisation. A join with no
|
||
`invite_token` fell through to `TrustLevel::Trusted.min(claimed_trust)`, and
|
||
`claimed_trust` defaults to `Trusted` — so anyone able to reach the node could
|
||
self-grant Trusted. **Now capped at `Observer`.**
|
||
- `merge_transitive_peers` added every peer advertised by a Trusted source as `Trusted`,
|
||
making trust viral across the whole federation graph. **Now `Observer`** — which is what
|
||
`NodeStateSnapshot.federated_peers`' own doc comment always said it should be
|
||
("adds them as Observers on her side… doesn't auto-promote to Trusted"). The code
|
||
contradicted its own spec.
|
||
- Added `FederatedNode.trust_source` (`invite` | `uninvited-join` | `transitive-merge` |
|
||
`manual`, `None` = pre-existing/unknown) so existing grants are **auditable**. Per
|
||
operator decision: existing peers are **left alone, not auto-demoted**.
|
||
- `trust_source` is now **surfaced** in `federation.list-nodes` (as an explicit `null`
|
||
when unknown, not omitted — "recorded before this was tracked" is the population that
|
||
needs review, so the UI must be able to tell it apart from a field it didn't read) and
|
||
rendered under the trust dropdown in the node detail modal as "Granted via:".
|
||
|
||
### 3b. Granting Trusted must require the node password — **DONE** (uncommitted at time of writing)
|
||
> "to make someone trusted must require the node password to generate the code or change
|
||
> in the modal dropdown when you click a node" — operator, 2026-08-03
|
||
|
||
Re-authentication on privilege escalation. Both entry points are covered:
|
||
|
||
- **Minting a Trusted invite** (`federation.invite`) — gated on the **resolved** level,
|
||
which matters because "Link Your Nodes" sends no `trust_level` at all and falls through
|
||
to the `Trusted` default. The invite is a bearer grant of Trusted to whoever redeems
|
||
it, so minting it *is* the escalation. Observer invites are untouched.
|
||
- **Changing a node's level in the UI dropdown** (`federation.set-trust`) — gated only
|
||
when the peer is **not already** Trusted, so the dropdown re-emitting its own value
|
||
doesn't demand a password for a no-op.
|
||
|
||
Demotion is NOT gated: making something less privileged must never be harder than leaving
|
||
it, or the safe action becomes the inconvenient one. The operator path stamps
|
||
`TrustSource::Manual`; `set_trust_level` grew an `Option<TrustSource>` so automatic
|
||
adjustments (the discovery-handshake demotion safety net) pass `None` and leave the
|
||
recorded provenance alone rather than laundering an `uninvited-join` peer into looking
|
||
operator-approved.
|
||
|
||
Wiring: the backend is the sole authority on what counts as an escalation — it returns a
|
||
`PASSWORD_REQUIRED:` prefixed error, and the UI prompts and retries only on that. The
|
||
frontend never pre-judges, so the rule lives in exactly one place.
|
||
`TrustPasswordModal.vue` (modelled on `RotateDidModal.vue`) serves both flows.
|
||
`NodeDetailModal`'s select now snaps back to the node's real level on change, because a
|
||
cancelled or failed promotion would otherwise leave the dropdown displaying a level the
|
||
node never accepted.
|
||
|
||
**Follow-up, deliberately not done here:** `federation.join` also grants Trusted (when
|
||
redeeming someone else's Trusted invite) with no re-auth. It is an explicit operator
|
||
paste rather than a UI toggle, and was outside the two entry points specified — but it is
|
||
the third way a node reaches Trusted and should be reviewed.
|
||
|
||
---
|
||
|
||
## P1 — Correctness the operator hit directly
|
||
|
||
### 4. LND UI never updates over OTA — **DONE** (`5088aef5`, pushed)
|
||
- `LND_UI_IMAGE` was `lnd-ui:latest` while `BITCOIN_UI_IMAGE` was pinned to
|
||
`1.7.119-alpha`. Podman will not re-pull a tag it already holds locally, so nodes kept a
|
||
stale lnd-ui forever. **Now pinned to `1.7.119-alpha`.**
|
||
- `scripts/first-boot-containers.sh` declared lnd-ui as bridge `-p 18083:80`. That is the
|
||
**third copy** of the declaration the UI agent already corrected in
|
||
`scripts/container-specs.sh` and `apps/lnd-ui/manifest.yml` — so **fresh installs** still
|
||
produced the reproduced `HTTP 000`. **Now `--network host`, ports empty.**
|
||
- Root cause worth fixing separately: the same container spec is declared in three places.
|
||
|
||
### 5. Federated/peered nodes must message without a LoRa hop first — **OPEN**
|
||
> "make it so federated/peered nodes can message without needing to connect on Lora first
|
||
> once connected"
|
||
|
||
- Investigate the split contact model (radio contact vs federation peer) — there is prior
|
||
art in memory: `project_archy_lora_e2e_rootcause` ("split contact model; don't touch
|
||
federation") and `mesh::seed_federation_peers_into_mesh` /
|
||
`upsert_federation_peer`, which already mirror federation peers into the mesh table.
|
||
- Likely the gap is addressing/route selection rather than transport availability.
|
||
|
||
### 6. In-app app updates, independent of OTA — **OPEN**
|
||
> "we need app update to see updates in the registry, whether UI or not… show the update
|
||
> mechanism in the app… a modal and update now / cancel… same in the detail page… the
|
||
> update button should show 'see update' and a different graphic for just ui, app, or both
|
||
> together. All pushed through the signed-catalog flow." … "This has to show independent of
|
||
> OTA updates as a separate pipeline, I think we've done a lot of work on it."
|
||
|
||
- **Operator says much of this already exists — research the codebase before building.**
|
||
Known groundwork: the signed catalog (`releases/app-catalog.json`, `sign-catalog.sh`),
|
||
catalog→manifest runtime reload, `package.update` RPC, `check-app-catalog-drift.py`,
|
||
and `scripts/image-versions.sh` pinning.
|
||
|
||
#### What already exists (verified in source, 2026-08-03) — the operator was right
|
||
|
||
The whole update *pipeline* is built and is already independent of OTA:
|
||
|
||
- `package.check-updates` (`api/rpc/package/update.rs:180`) refreshes the signed catalog
|
||
and hot-reloads manifests when it changed — no daemon restart, no OTA involved.
|
||
- `package.update` (`spawn_package_update`), `package.versions`, `package.set-config`
|
||
version pinning, and `execute_update` (stop → pull → remove → recreate → verify).
|
||
- Version awareness: `app_catalog::catalog_versions(app_id)` vs `installed_version()`
|
||
(`api/rpc/package/set_config.rs:46`).
|
||
- Frontend: `AppCard.vue` already renders an Update button off `pkg['available-update']`
|
||
(`:48`, `:128`) and emits `update`.
|
||
|
||
#### What is actually MISSING (this is the real scope of item 6)
|
||
|
||
1. **The UI-vs-app-vs-both distinction does not exist.** `available-update` is a single
|
||
version string — nothing classifies whether the change is the app image, its `*-ui`
|
||
image, or both. This is the core of the operator's ask ("a different graphic for just
|
||
ui, app, or both together") and needs a backend change, not just an icon.
|
||
⚠️ Compounding factor: per `reference_app_ui_delivery_model`, `*-ui` apps are **not in
|
||
the signed catalog** at all — so "is there a UI update" cannot be answered from the
|
||
catalog today. That gap has to be closed first or the UI half is unanswerable.
|
||
2. **The modal** (Update now / Cancel) — the card currently updates on click, no confirm.
|
||
3. **The detail-page affordance** — same treatment as the card.
|
||
4. **Button copy**: "See update" rather than "Update".
|
||
|
||
---
|
||
|
||
## P2 — Carried over from v1.7.120
|
||
|
||
### 7. `create-release.sh` commits the manifest BEFORE signing — **OPEN**
|
||
Release commit always carries an **unsigned** manifest; nodes fetch it from branch `main`
|
||
and refuse to auto-apply. Caught manually this cycle. Fix the ordering so it cannot ship.
|
||
|
||
### 8. `gitea-vps2` remote is dead, and is the same server as `gitea-ai` — **OPEN**
|
||
Stored token fails auth. `source.archipelago-foundation.org` == `146.59.87.168`, so
|
||
`git push gitea-ai` already publishes to the "primary" OTA host. Ties into the existing
|
||
"migrate VPS2 IP to domain" todo.
|
||
|
||
### 9. Fleet SSH host-key rotation — **BLOCKED** (operator decision)
|
||
`archipelago-1`, `archy-x250-beta`, `archipelago` share all three SSH host keys; two also
|
||
share a TLS private key. Detection shipped; rotation deliberately not performed.
|
||
|
||
### 10. 5× lifecycle gate — **OPEN**
|
||
Not run for v1.7.120 (disclosed in its changelog). Needs repeated reboots of a live node.
|
||
|
||
### 11. `prod_orchestrator.rs:3181` unreachable code — **OPEN**
|
||
`bitcoin_host()` returns unconditionally at :3171, so the podman container-name lookup
|
||
below is dead on every path. Pre-existing; spotted in the v1.7.120 build warnings.
|
||
|
||
---
|
||
|
||
## Notes for whoever picks this up
|
||
|
||
- A separate agent is doing **AIUI planning with GSD** — do not touch AIUI.
|
||
- AIUI must always be built `VITE_BASE_PATH=/aiui/` (see the memory note); a hand-built
|
||
bundle renders a black page.
|
||
- Verify security claims on the node, not from the source. v1.7.120's headline bug was a
|
||
fix that shipped in the binary and silently never reached the running container.
|
||
|
||
---
|
||
|
||
## RESUME HERE — next session
|
||
|
||
**Landed this session (both pushed):**
|
||
- `c0cfc72a` federation trust escalation (items 3) — 42/42 federation tests green
|
||
- `5088aef5` lnd-ui OTA pin + host networking (item 4), and this task file
|
||
|
||
**v1.7.120-alpha is SHIPPED** — signed, published, assets verified live. Do not re-cut it.
|
||
|
||
### Start with item 3b (password gate) — groundwork already located
|
||
|
||
Everything needed to implement it, so the next session does not re-search:
|
||
|
||
- **The helper to use:** `self.auth_manager.verify_password(password).await?` — returns
|
||
`bool`. Existing callers to copy the shape from: `api/rpc/node.rs:176`,
|
||
`api/rpc/totp.rs:18` / `:66` / `:121`.
|
||
- **Entry point A — minting a Trusted invite:** `handle_federation_invite`,
|
||
`api/rpc/federation/handlers.rs:58`. It reads `trust_level` from params and
|
||
**defaults to `TrustLevel::Trusted` at :72**. Gate only when the resolved level is
|
||
`Trusted`; leave Observer invites unchanged.
|
||
- **Entry point B — the UI dropdown:** `handle_federation_set_trust`,
|
||
`api/rpc/federation/handlers.rs:326`, dispatched as `"federation.set-trust"`
|
||
(`api/rpc/dispatcher.rs:353`). Its parse is at `:342`.
|
||
- **Rule:** gate PROMOTION to Trusted only. Demotion must stay ungated — making something
|
||
less privileged must never be harder than leaving it.
|
||
- Set `TrustSource::Manual` on the operator path so the audit trail distinguishes a
|
||
deliberate grant from the capped automatic ones.
|
||
- Frontend will need the password prompt in both places (invite modal, node dropdown).
|
||
|
||
### Then item 1 (app ports unauthenticated) — the big one
|
||
|
||
Start with the research the operator explicitly asked for: how **umbrelOS**
|
||
(`getumbrel/umbrel`) and **StartOS** (`Start9Labs/start-os`) gate app access. Read their
|
||
model rather than inventing one. Only then design the gate.
|
||
|
||
Give this a fresh session with real context — it is the largest item here and is the same
|
||
bug class as the `/lnd-connect-info` + `/bitcoin-rpc/` leaks fixed in v1.7.120, but across
|
||
every app port and every transport.
|
||
|
||
### Working notes
|
||
- A separate agent is doing **AIUI planning with GSD** — do not touch AIUI.
|
||
- The shared tree has concurrent agents: stage by explicit path, never `git add -A`.
|
||
- Verify security claims **on the node**, not from source. v1.7.120's headline bug was a
|
||
fix that shipped in the binary and silently never reached the running container.
|
||
- A piped command's exit code is the pipe's, not the script's — redirect to a log file and
|
||
read the content.
|