A node already running a gateway on the shipped default credential rotates itself onto a unique one without an operator having to know it was affected (FED-07 migration)
Rotation preserves the gateway's data: /var/lib/archipelago/fedimint-gateway survives, and so do the container name, its ports, its volumes and its adoption identity (CLAUDE.md — migrations never destroy data)
A node already carrying a unique credential is left completely alone — detection matches the known defaults only, never 'anything I did not generate this run' (FED-07 adjacency edge)
Rotation runs at most once per affected node: after it completes, later reconcile ticks detect nothing and change nothing (FED-07 idempotence)
A rotation the operator can see: it is announced in the node's logs naming the app and that credentials changed, and it never prints the credential itself
After rotation the operator has a supported way to obtain the new gateway credential, so rotating does not lock them out of their own gateway
A rotation that cannot complete leaves the previous working state intact and reports an error rather than leaving a gateway configured against a credential nobody holds (FED-07 failure-surfacing)
statement
category
Rotation MUST NOT delete, move, reinitialise or chown the gateway's data directory, its Lightning backend credentials, or any other app's secrets — it replaces one credential file and lets the existing recreate path rebuild the container around unchanged data
safety
statement
category
The rotated credential MUST NOT be written to a log line, a status RPC response, a deploy transcript, or any file outside the 0600 rootless secrets directory
privacy
statement
category
Detection MUST NOT rotate a credential merely because it is unrecognised — only an exact match against the known-default denylist triggers rotation, so an operator who set their own credential deliberately keeps it
safety
path
provides
contains
core/archipelago/src/container/secrets.rs
Denylist-driven detection and rotation of a compromised gateway credential
the reconcile path that already materialises generated secrets also asks for compromised-credential rotation, so an existing node heals on its next tick
rotate_compromised_gateway_credential
Get the nodes that are already running on the shipped gateway credential off it, without touching
their data.
Purpose: plan 01-11 stops new installs from ever taking a shipped credential, but it does nothing for
the nodes that already did. Those gateways answer to a credential published in this repository, so
until they rotate, FED-07 is only half closed — and the requirement is explicit that existing installs
carrying the default get a migration path. The repo's standing rule bounds how: migrations never
destroy data — preserve /var/lib/archipelago/<app>, secrets, credentials, ports and adoption
container names, and keep a rollback path.
Output: detection against the denylist plan 01-11 established, rotation through the recreate machinery
that already preserves data, an operator-visible announcement, and a sign-off on a real node.
Task 1: End-to-end — a node carrying the default rotates itself and keeps its data
core/archipelago/src/container/secrets.rs, core/archipelago/src/container/prod_orchestrator.rs
- `core/archipelago/src/container/secrets.rs` as left by plan 01-11 — the
`KNOWN_DEFAULT_GATEWAY_HASHES` denylist, `ensure_gateway_credential`, `gateway_bcrypt_hash`, the
shared bcrypt generation helper, and the atomic 0600 `write_secret`. Rotation reuses all of it;
write no new generation or file-writing code.
- `core/archipelago/src/container/prod_orchestrator.rs` around line 3227 and line 3245 — the
comment naming the per-app generated secrets (`fmcd-password`, `fedimint-gateway-hash`, …) and
the `crate::container::secrets::ensure_generated_secrets(&self.secrets_dir, manifest)?` call.
This is the tick that runs on every reconcile and the natural place to hang detection.
- `core/container/src/manifest.rs` — grep for `secret_env_hash` and read its definition and every
use. This is the existing mechanism by which a changed secret drives a container recreate, and
it is what makes rotation preserve data: the recreate path it feeds already keeps the data
directory, ports, volumes and container name. Reuse it rather than stopping and removing the
container by hand.
- `apps/bitcoin-ui/manifest.yml` lines 8-45 — the in-repo precedent for "the password rotated, so
the rendered bytes changed, so the container is recreated". Read it for how a rotation is
expected to propagate on this platform.
- `core/archipelago/src/container/boot_reconciler.rs` — enough to determine whether boot has its
own separate path that also needs the call, or whether it funnels through the same reconcile
step. Record the finding; if it needs the call too, add that file to `files_modified` in the
SUMMARY.
- Given a secrets dir whose gateway hash file contains a denylist entry, rotation replaces it with
a freshly generated pair and reports that it rotated.
- Given a secrets dir whose gateway hash is not on the denylist, rotation changes nothing and
reports that it did not rotate — including when the value is one nobody recognises.
- Given a secrets dir with no gateway hash at all, rotation changes nothing and reports that it did
not rotate; generation is `ensure_gateway_credential`'s job, not rotation's.
- Running rotation twice on the same affected dir rotates once; the second run is a no-op.
- After rotation the new hash is not on the denylist and its `.pw` sibling verifies against it.
- Rotation touches no file other than the gateway credential pair — every other file in the
secrets dir is byte-identical afterwards.
Write the tests in `secrets.rs`'s `mod tests` first and confirm they fail. Use `tempfile::tempdir`
the way the existing tests in that module do, and seed the affected case by writing a denylist
entry into the hash file. Include a case that seeds several unrelated secret files alongside it and
asserts they are untouched.
Add `pub fn rotate_compromised_gateway_credential(secrets_dir: &Path) -> Result<bool>` to
`secrets.rs`. It reads the gateway hash file; if it is absent or unreadable it returns `Ok(false)`
without writing; if its trimmed value is not an exact match for a denylist entry it returns
`Ok(false)`; only on an exact match does it generate a replacement pair through the same helper
`ensure_gateway_credential` uses and return `Ok(true)`. Because the underlying write is the
existing atomic temp-file-plus-rename, a failure mid-rotation leaves the previous file in place —
that is the rollback path, and it should be stated in the function's doc comment so nobody later
"improves" it into a truncate-in-place.
Wire it into `prod_orchestrator.rs` immediately alongside the existing `ensure_generated_secrets`
call. When it returns `true`, log at info level that the Fedimint gateway credential was rotated
because the node was carrying a publicly known default, that the gateway will be recreated, and
where the operator can obtain the new one — and never log the value. Then make the recreate happen
through the existing `secret_env_hash` change-detection path rather than by stopping or removing
the container directly: the hash file changed, so the resolved secret env changes, so the platform's
own recreate machinery fires with the data directory, ports, volumes and container name all
preserved. If that path does not fire for this app for some reason you discover, do not hand-roll a
remove-and-run; stop and record what you found, because a hand-rolled recreate is the exact
anti-pattern CLAUDE.md names.
Settle the operator-recovery question and record the answer. The plaintext already lands at
`fedimint-gateway-hash.pw`, 0600, rootless. Determine whether the app-credentials surface in the UI
(`neode-ui/src/views/Credentials.vue` and whatever RPC feeds it) already exposes per-app generated
credentials. If it does, confirm the rotated value appears there and say so. If it does not, the
log line must name the exact path an operator reads, and the SUMMARY must record that a UI surface
is a gap with the file that would own it. Do not leave "how does the operator get the new password"
unanswered — rotating a credential the user cannot retrieve is a lockout, not a fix.
Do not change the gateway's ports, volumes, data directory, network, capabilities, health check or
any other manifest-driven property.
cd core && cargo test -p archipelago secrets 2>&1 | tail -20
- `cd core && cargo test -p archipelago secrets` exits 0 and its output names cases for: rotates-on-denylisted, no-op-on-unique, no-op-on-absent, idempotent-second-run, and other-secrets-untouched.
- `grep -v '^\s*//' core/archipelago/src/container/secrets.rs | grep -c 'rotate_compromised_gateway_credential'` is at least 2 (definition plus test use).
- `grep -v '^\s*//' core/archipelago/src/container/prod_orchestrator.rs | grep -c 'rotate_compromised_gateway_credential'` equals 1.
- `git diff -- core/archipelago/src/container/prod_orchestrator.rs | grep -ciE '^\+.*(rm -f|remove_dir_all|podman rm|chown)'` equals 0 — no hand-rolled teardown was introduced.
- `cd core && cargo build -p archipelago` exits 0 and `cd core && cargo test -p archipelago` exits 0.
- The SUMMARY records the boot-reconciler finding, whether the recreate fired through `secret_env_hash`, and the operator-recovery answer with its evidence.
An affected node heals itself on its next reconcile tick, once, without losing data, and the operator can still get into their gateway.
Task 2: Confirm the rotation on a real node
FED-07 in full, ready to run on a node:
- No code path in the tree can configure a Fedimint gateway with a credential that shipped with
the repository. The five fallback sites (two in the Rust orchestrator, three in the install and
deploy scripts) are gone, along with the plaintext password fallback in the Tailscale deploy
path. The one surviving copy of the old hash is a denylist used only to detect it.
- Every install now takes its credential from the per-install secret the manifest already
declared, generated at 0600 by `container::secrets`.
- A node that is already carrying the old default rotates itself on its next reconcile tick and
is recreated around its existing data directory, ports and container name.
Run this on archi-dev-box. Note that archy-x250-dev has been offline since phase 2 — do not wait
for it; single-node verification with the second-node gap recorded honestly is the expected
pattern here.
1. **Before you change anything, record the current state.** On the node, check whether the
gateway credential file currently holds the old shipped value, and whether a gateway container
is running. Note both. This is what tells you whether you are testing the rotation path or the
already-clean path — say which one you got.
2. **Deploy this phase's build to archi-dev-box only.** Use the dev-pair deploy path, not a
release, not an OTA, and not the Tailscale alpha-tester path. Record the exact command.
3. **Watch the rotation.** Follow the node's logs across a reconcile tick. Expected if the node was
affected: one info line saying the gateway credential was rotated because a publicly known
default was in use, naming where to get the new one — and no credential value anywhere in the
log. Expected if the node was already clean: no rotation line at all.
4. **Confirm the credential is now unique.** Read the gateway hash file on the node and confirm it
is not the old shipped value, and that its file mode is 0600 and it is owned by the rootless
service user, not root.
5. **Confirm the data survived.** List `/var/lib/archipelago/fedimint-gateway` and confirm its
contents are the same ones that were there in step 1 — the gateway's own state must not have been
reinitialised. Confirm the container came back with the same name and the same published ports.
6. **Confirm the gateway actually works.** Check the container is running and healthy, and that its
admin endpoint answers. Then authenticate to it with the new credential from the path the log
line named. Expected: the new credential works. Then try the old shipped one. Expected: rejected.
7. **Confirm a fresh install is unique too.** If practical, uninstall and reinstall the gateway on
the node and confirm the credential it comes up with differs from the one from step 4 — that is
the per-install property, and it is the whole point of the requirement.
8. **Confirm nothing else moved.** Run `tests/lifecycle/run-gate.sh` on the node (the gate runs
on-node, never over RPC) and confirm it is still green. This plan changed orchestrator reconcile
behaviour, which is exactly the case CLAUDE.md says to re-run the gate for. A single clean pass
is enough here; the 5× run is Phase 3's criterion.
If any step fails, say which numbered step and what you saw — that becomes the gap list rather than
a re-run of the whole plan.
Type "approved" to sign off FED-07, or describe the issues by step number.
Planner Assumptions (flagged, unresolved)
Whether archi-dev-box is actually affected is unknown to the planner. Its gateway may have been
provisioned by a path that generated a unique credential. Step 1 makes the executor establish which
case they are in and say so, rather than reporting a green run that never exercised the rotation. If
the node is clean, the rotation path still needs proving — seed the old value into the credential
file on the node deliberately, then re-run steps 3 to 6, and record that you did.
Whether the UI already exposes per-app generated credentials was not verified by the planner.
Task 1 makes it an explicit finding with a named owning file if it turns out to be a gap.
<threat_model>
Trust Boundaries
Boundary
Description
gateway admin API → network
The credential being rotated is the only gate on Lightning gateway administration
reconcile tick → running container
An automated rotation recreates a live, funded service without asking
node logs → operator and anyone who can read them
The rotation announcement crosses this boundary
STRIDE Threat Register
Threat ID
Category
Component
Severity
Disposition
Mitigation Plan
T-01-72
Elevation of Privilege
a node continuing to answer to the published default after the code fix ships
critical
mitigate
Detection and rotation run on the same reconcile tick that already materialises secrets, so an affected node heals without operator action; step 6 proves the old credential is rejected afterwards
T-01-73
Denial of Service
rotation recreating the gateway repeatedly, or in a loop, on every tick
high
mitigate
Rotation is denylist-exact and therefore self-terminating — the rotated value is not on the denylist, so the next tick is a no-op; an idempotence test and step 3's log observation both cover it
T-01-74
Information Disclosure
the new credential appearing in a log line, status output or deploy transcript
high
mitigate
An explicit prohibition, the log line is specified to name a path rather than a value, and step 3 requires confirming no value appears in the log
T-01-75
Tampering
a hand-rolled remove-and-recreate losing the gateway's data directory
critical
mitigate
The action forbids hand-rolled teardown, routes the recreate through the existing secret_env_hash path, and an acceptance criterion greps the diff for teardown primitives; step 5 verifies the data on the node
T-01-76
Repudiation
signing off without ever exercising the rotation because the node happened to be clean
high
mitigate
Step 1 forces the executor to declare which case they are in, and the planner assumption requires deliberately seeding the affected state if the node is clean
T-01-77
Denial of Service
an operator locked out of their own gateway by a rotation they cannot recover from
high
mitigate
Task 1 requires the recovery path to be settled and named in the log line before this plan is done; step 6 proves the new credential actually authenticates
T-01-SC
Tampering
npm/pip/cargo installs
high
mitigate
This plan installs nothing — two Rust edits. If an implementation choice would add a crate, stop: RESEARCH.md's Package Legitimacy Audit must cover it first, with a blocking human checkpoint for any [ASSUMED]/[SUS] entry
</threat_model>
- `cd core && cargo test -p archipelago` — green.
- The blocking checkpoint's eight steps, run on archi-dev-box, with the affected-or-clean case declared.
- `tests/lifecycle/run-gate.sh` green on-node after the change.
<success_criteria>
An affected node rotates itself once, keeps its data, ports and container name, and comes back healthy.
The old shipped credential no longer authenticates; the new one does.
A fresh install produces a different credential again.
The rotation is announced without ever printing the value, and the operator has a named way to retrieve it.
The second dev-pair node's absence is recorded as a gap rather than glossed over.
</success_criteria>
Create `.planning/phases/01-federation-mesh-hardening/01-16-SUMMARY.md` when done, recording the
affected-or-clean verdict for archi-dev-box, the deploy command used, the gate result, the operator
recovery path, and any issue text verbatim.
Stage by explicit path, commit, and `git push gitea-ai main`.
