A !ai (or any typed message) from a trusted, federated node was denied when
it arrived over the radio. The radio half of a node that is also a federation
peer carried no archipelago identity (identity adverts are no longer broadcast
on the public channel), so the trusted_only gate and signature verification
had no key to check the asker against — and the same node showed up as two
contacts (a radio twin + a federation twin).
- bind_federation_twins(): correlate a radio contact with its federation twin
by exact, case-insensitive advert_name and copy the federation peer's
arch_pubkey_hex/did/x25519 onto the radio record. Called from
upsert_federation_peer and refresh_contacts. Ambiguous names (held by >1
federation peer) are skipped. This is only a CANDIDATE key — security is
unchanged: the inbound envelope signature must still verify against it.
- send_message now signs the typed Text envelope (new_signed) so a radio !ai
authenticates against the bound key. A meshcore node merely named like a
trusted node cannot forge the signature, so it is still denied.
Receiver-side verification (handle_typed_envelope_direct) and federation-trust
matching (is_sender_allowed) already existed; this supplies the missing key
binding and signature. Also resolves the radio/federation duplicate-contact
display for same-named nodes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Messages to a federated peer that is out of LoRa range (e.g. on another
continent) were dropped into the radio with no fallback, or hung on a dead
FIPS path before reaching Tor — so they never arrived.
- Route a radio contact over the federation transport (FIPS->Tor) when it is
the same node as a federated peer (known archipelago identity -> onion) AND
it is not currently reachable over the radio. Reachable radio peers stay on
the mesh (preferred); oversized/file envelopes still always take federation.
- Resolve the onion via the archipelago identity key (arch_pubkey_hex), not
the firmware routing key, so a radio contact maps to its nodes.json onion.
- Add .fips_timeout(8s) to the federation message POST so an unreachable FIPS
overlay fast-fails to Tor (~3-5s) instead of burning the 120s budget.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Federated nodes failed to converge to full-mesh across the LAN<->Tailscale
boundary: nodes were invisible to peers, sync 'took ages'/timed out, and
names only updated on a manual sync. Onions were healthy in both directions
(~3-5s); the failures were app-layer.
- B: federation dials fast-fail a dead FIPS path via .fips_timeout(6s) in
sync_with_peer + notify_join, so the Tor fallback isn't stuck behind the
full 30s FIPS budget when LAN and remote peers share no FIPS path.
- A: notify_join (peer-joined) now spawns with retries+backoff instead of a
single awaited best-effort POST, so the join RPC returns instantly (no
'Request timeout') and the inviter reliably learns the joiner (was
asymmetric).
- C: new 90s periodic federation auto-sync (none existed) so renamed nodes
and roster changes propagate without a manual Sync click.
- self-heal: each auto-sync re-asserts membership to any peer that doesn't
list us back, converging the fleet to full-mesh and healing pre-existing
asymmetry with no manual re-joins.
Validated live across 7 nodes: a previously fleet-invisible node became
fully meshed automatically (logs: 'auto-sync ... reasserted=1',
'peer-joined ... delivered').
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
UI (this session):
- Global audio player now scales the whole interface into the space above it
on desktop (sidebar + main) and docks directly above the tab bar on mobile;
it stays visible while navigating.
- Mesh mobile redesign: floating Chat / BTC / Dead Man / AI / Map tab strip
with a single fixed, internally-scrolling pane (page no longer scrolls);
tabs hide while a conversation is open; floating back button; collapsible
Device panel (starts collapsed); keyboard-aware conversation sizing via
VisualViewport so the chat sits just above the keyboard.
- Cloud file grid: uniform 4/3 card heights (folders + images match).
- Swipe left/right switches tabs on the Apps and Web5 screens.
- Map tool fills its pane (no bottom gap); fix skewed Share Location toggle
on mobile (global min-height rule was deforming the switch).
- Trim redundant helper copy from the mesh AI tab.
Also bundles pre-existing in-progress work that was already in the tree:
mesh listener/session + wallet + container + bitcoin-status backend changes,
docker UI updates, and assorted other UI tweaks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fedimint never appeared in Wallet > Settings > Fedimint because the
fmcd (fedimint-clientd) sidecar was never installed: ensure_default_
federation() needs the fmcd password to reach the daemon, found none,
and silently no-oped, leaving the registry empty.
- prod_orchestrator: add fedimint-clientd to the baseline auto-install
set so it self-heals onto every node and auto-joins the default
federation; generate the fmcd-password secret before secret_env
resolves.
- fedimint_client: ensure_fmcd_password (random hex, 0600) shared with
the container's secret_env; from_node reads the same secret (legacy
fmcd/password kept as fallback); reissue_into_any redeems received
notes into the first joined federation that accepts them.
- wallet.ecash-receive: dual-token — cashu* tokens redeem at the mint,
anything else is reissued via fmcd; returns the kind + federation_id.
- UI: receive box advertises "Cashu or Fedimint" and reports which kind.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Meshtastic DMs were falling back to a channel broadcast, so every node
on the LoRa channel saw a "direct" message. Send a directed MeshPacket
(to = node num, decoded from the synthetic pubkey's node-id bytes)
instead — the Meshtastic analog of the meshcore CMD_SEND_TXT_MSG fix.
DMs now reach only the recipient; firmware auto-PKC-encrypts them
end-to-end once NodeInfo keys are exchanged.
Capture E2E status at the driver level (no shared-type/UI change):
- learn each peer's real Curve25519 key from User.public_key (field 8)
and inbound MeshPacket.public_key (16), kept in a side-map separate
from the synthetic routing key so unicast routing is untouched
- detect inbound MeshPacket.pki_encrypted (17) to tell a true E2E DM
from a channel-PSK fallback
- peer_is_pkc_capable() seam for a future mesh-tab E2E badge
Hot-swap preserved: no dispatched MeshRadioDevice signature or the
shared ParsedContact changed, so meshcore and meshtastic stay
interchangeable behind the listener.
Adds tests/multinode/meshtastic.sh, a two/three-radio on-air parity
harness (detect, discover, DM round-trip, DM privacy, channel
broadcast, typed envelope, reachability).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Cashu default mint was the local Fedimint guardian (:8175), wrongly surfacing
a Fedimint URL in the Cashu mints list. Default is now Minibits
(https://mint.minibits.cash/Bitcoin) — Cashu and Fedimint are distinct
protocols (Fedimint lives under its own tab).
- Peer-file (buy) invoice creation: retry the LND REST call (3× / 400ms) so a
transient LND-REST blip (swap pressure / just-restarted / TLS race) no longer
hard-fails as an opaque 503, and surface the real error chain ({:#}) in the
response + logs instead of a generic "Failed to create invoice".
- Autojoined default federation now shows a friendly name ("Archipelago
Federation") in the Fedimint tab instead of a bare federation id.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sizes bitcoind -dbcache to host RAM (~1/16, floor 300MB, cap 4096) instead of a
fixed 2048/4096. A multi-GB UTXO cache on an 8GB node running the full app stack
pushed memory past physical RAM and triggered system-wide swap thrash: the disk
saturated, bitcoind could not answer its own RPC, and the dashboard backend's
sqlite reads stalled — surfacing as fleet-wide /rpc/v1 502s and a blank Bitcoin
UI. Applied in scripts/container-specs.sh (reconciler path) and the config.rs
bitcoin-core path.
Bitcoin status cache now polls every 5s (was 10/15) with an 8s timeout (was 20s)
and fetches the four RPCs concurrently, so the cached snapshot tracks bitcoind's
responsive windows during IBD and the UI stops dwelling on "reconnecting...".
Unifies the divergent discover AppGrid/FeaturedApps image-error handlers onto the
canonical placeholder fallback so missing app icons render the placeholder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- DMs now use native meshcore unicast (CMD_SEND_TXT_MSG) instead of @DM2 channel
broadcasts: private (E2E-encrypted to the recipient pubkey by firmware), off the
public channel, and decodable by stock clients. Plain text (split, not MC-chunked)
to non-archipelago contacts; typed envelopes to archy peers.
- !ai replies now DM the asker privately (RadioDm) instead of broadcasting on ch0.
- Auto contact-import: a heard advert (PUSH_CONTACT_ADVERT/0x80, 32-byte pubkey) is
added via CMD_ADD_UPDATE_CONTACT (0x09) so contacts appear without a flood advert.
- clear-all now DELETES firmware contacts via CMD_REMOVE_CONTACT (0x0F) instead of
blocklisting; blocking filter removed entirely. Wiped contacts return when reachable.
- Contact reachability: MeshPeer carries last_advert + reachable (path-based); UI shows
a reachability dot.
- Peers list: contact search box (filter by name/DID/npub/pubkey) with a clear button.
- send_message routes stock contacts as plain native text (fixes garbled envelopes).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- mesh: stop broadcasting ARCHY:2 identity on the public channel (startup + every advert tick); receive path still parses inbound. No more public-channel spam.
- mesh assistant: trigger on !ai/!ask typed in 1:1 chat (was only the dead AssistQuery path + bare channel text); route the reply transport-aware via MeshService::send_message (Tor for federation peers, LoRa for radio) through a new AssistChatReply event consumed at the server layer — fixes replies never reaching federation askers.
- mesh assistant: per-contact !ai allowlist (allowed_contacts) bypassing trusted_only; config + RPC + is_sender_allowed.
- fedimint-clientd manifest: network_policy open -> bridge (invalid value made the loader skip the whole manifest, so fmcd never ran and federations never joined/listed).
- ui: AI panel — Claude model dropdown (Haiku/Sonnet/Opus presets) + allowlist contact picker.
- ui: Settings — App Updates + App Registry moved under Account.
- ui: mesh chat — overscroll-behavior: contain so chat scroll no longer bleeds to the contacts panel.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the assistant scheduler, MeshAssistantPanel UI, and the remaining
config-RPC / live-toggle / Ollama-detect wiring on top of Phase 1.x.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2 backend. AssistantConfig is now live-updatable (RwLock) so the UI
toggle applies without a listener restart. New RPCs:
- mesh.assistant-status -> {enabled, model, trusted_only, default_model,
ollama_detected, models[]} (probes local Ollama :11434/api/tags)
- mesh.assistant-configure -> set enabled/model/trusted_only live + persist
MeshService::assistant_config / configure_assistant. Compiles clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ARCHY:2 identity broadcast (DID + ed25519 + x25519) was unwired dead
code on both send and receive. Wiring it lets a radio peer prove its
archipelago identity, so the assistant's trusted-only gate (and encrypted
DMs) work over meshcore AND Meshtastic — the latter otherwise only exposes
synthetic node keys.
- session.rs: broadcast ARCHY:2 as channel text at startup + each advert tick
- frames.rs: parse inbound ARCHY:2 on the channel path, dedupe-keyed by
archipelago pubkey (federation_peer_contact_id) so it MERGES with the
federation-seeded peer instead of duplicating; self-echo guarded
- threads our_x25519_secret into handle_channel_payload (was reserved)
Reuses the existing handle_identity_received verifier (ed/x25519 consistency
check + shared-secret derivation). Compiles clean. Needs a live 2-radio test
before trusting trusted-only over radio.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A plain '!ai <q>' / '!ask <q>' on the channel is now answered by the node's
local model and broadcast back as plain text, so ANY client (bare meshcore
or Meshtastic) can ask. Generalised run_assist with an AssistReply target:
Typed chunks to a peer (archipelago UI path) vs plain channel-text (bare
clients). Trust/rate gate unchanged; asker identity is separate from reply
mode. Works over both radios.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The IndeeHub API needs MinIO (object storage) up to serve, but the
health monitor's dependency map listed only postgres + redis, so it
would restart the API while MinIO was still starting — the "recovers
only after 1-2 container restarts" symptom. Add indeedhub-minio to the
API's deps; MinIO has no deps of its own so the monitor restarts it
first, no deadlock. (First-start ordering in the stack definition is a
deeper, separate follow-up.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
federation.remove-node only edited nodes.json, so a removed/renamed node
(e.g. a stale "Arch HP") lingered in the mesh chat list with its old
thread. Capture the node's pubkey before removal, then purge its
synthetic mesh peer, shared secret, messages, presence, and persisted
contact entry via the new mesh::purge_federation_peer. Combined with the
#42 name refresh, stale federation contacts can now be fully cleaned from
a node.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Settings shows the node-level Nostr key (HKDF derive_node_nostr_key,
read via node.nostr-pubkey) while Web5 > Identities showed the identity
record's own key — the mirrored "Node" identity stores nostr=None and
seed identities use a different BIP-32 NIP-06 key, so the two surfaces
disagreed.
Resolve the node-level Nostr key once in identity.list and override it
onto whichever identity record is the node's own (ed25519 == server_info
.pubkey). Display-only — no stored key is rewritten, so it self-applies
to existing nodes with no migration and the discovery identity is
unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A peer accepted via invite is seeded into the mesh peer table with
name=None, so it shows as "Archipelago <pubkey8>" in chat. Federation
sync later learns the real name (update_node_state writes it to
nodes.json) and discovers transitive peers (merge_transitive_peers),
but nothing pushed those into the live mesh peer table — the chat list
stayed stale until the next mesh restart, and transitive peers never
appeared as contacts at all.
Add RpcHandler::refresh_federation_mesh_peers() (re-runs the idempotent,
onion-deduped seed_federation_peers_into_mesh) and call it after every
periodic sync cycle (server.rs) and after the manual federation.sync-all
RPC. Names now correct themselves and the full roster meshes within a
sync cycle, no restart needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 3 wiring (task #12):
- NostrSeedDiscovery: async ProviderDiscovery that queries relays for signed
seed adverts and parses endpoint ids (swarm/iroh_provider.rs, seed_advert.rs).
- seed_and_advertise publish path; dep-free fetch/publish helpers reuse the
node's Nostr identity (build_nostr_client/load_or_create_nostr_keys made
pub(crate)).
- swarm::init builds the IrohProvider once into a OnceLock runtime; providers()
returns it; announce_held_blob() is called from update.rs after a release
component passes both hash gates.
- config swarm_enabled (ARCHIPELAGO_SWARM_ENABLED, default off); server.rs init.
Paid swarm serving (Phase 4 step F):
- swarm/paid.rs gates the iroh-blobs provider through streaming::gate,
intercepting connect + GET (peer push hard-disabled). Free by default
(content-download service disabled); denies unpaid peers when enabled;
fails open on internal error so a payment fault never blocks distribution.
Wired into IrohProvider::new.
All iroh code behind the iroh-swarm feature; the default build is inert.
Default build clean; --features iroh-swarm: 11/11 swarm tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Messaging a federation-only peer (e.g. 'Arch Dev') failed with 'Missing
contact_id'. The UI gave federation-only rows a *negative* placeholder
contact_id derived from a DID hash, but the backend parses contact_id as u64,
so a negative value deserialized to None. The negative id also never matched
the positive federation-synthetic id that federation-routed messages are stored
under, so those threads looked empty.
- Frontend: derive the SAME positive federation-synthetic id the backend uses
(federationContactId mirrors federation_peer_contact_id) so mesh.send accepts
it and messages thread correctly.
- Backend: send_typed_wire now resolves a federation-synthetic contact_id from
nodes.json when it isn't in the live mesh peer table (radio-less node),
instead of bailing 'Unknown federation peer'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Large peer downloads (~178MB) failed with a generic 'Operation failed', and
the download path had three stacked problems:
- The FIPS reqwest client used a hard-coded 20s total timeout regardless of the
caller's .timeout(), so a big transfer over the mesh aborted at 20s before
the Tor fallback could help. Honor the per-request timeout (client_with_timeout).
- The peer-content proxy buffered the whole file into node memory via
resp.bytes() before sending a byte, and capped the transfer at 60s. Stream
the body through with hyper::Body::wrap_stream (constant memory) and raise the
timeout to 900s; bump the nginx peer-content read timeout to match.
- Free downloads pulled the file as base64 over RPC, doubling it in node memory
and the browser — fatal for large files. Download free files by streaming
from /api/peer-content straight to disk, after a 1-byte Range probe that
surfaces the real reason (peer offline on mesh and Tor) instead of a generic
failure. Paid downloads now return the real error through the {error} channel
the UI already displays.
Adds the reqwest 'stream' feature for bytes_stream().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The prod orchestrator only checked whether a build-image tag was *present*
before deciding to skip the build. The local UI images (bitcoin-ui, lnd-ui,
electrs-ui) COPY a built neode-ui dist, so a UI update changed the source but
left the old tag in place and the new UI never shipped.
Gate the build on a content fingerprint of the build context (sorted relative
path + length + mtime, SHA-256) recorded in a per-tag stamp under data_dir.
Rebuild whenever the fingerprint differs from the one that produced the
existing image; podman's own COPY-layer cache keeps a no-op rebuild cheap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apps could fail install when a stack member exited on its first start
because a dependency (db/redis/the bitcoin node) was not ready yet — a
transient crash, not a broken install. wait_for_stack_containers now
restarts each exited/dead container up to 3 times before declaring the
install failed; the runtime supervisor keeps it alive afterwards.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The discovery wire format that feeds the swarm's ProviderDiscovery seam: a
node announces 'I seed blake3 H from iroh endpoint E' as a signed NIP-33
addressable Nostr event. Scope is releases/catalog content ONLY (decided
2026-06-16) — never private user blobs.
- swarm/seed_advert.rs: kind 30081, d-tag = blake3 hex (one current advert
per author+hash, latest-replaces), content {"v":1,"endpoint_id":...}.
advertisement_builder / advertisement_filter / parse_endpoint_id /
endpoint_ids_from_events (dedup). Endpoint ids stay opaque strings so the
protocol is dep-light + unit-testable on the default build.
4/4 tests pass (sign->parse roundtrip, filter targeting, reject wrong-kind/
empty, dedup across nodes).
Next (task #12): gated NostrSeedDiscovery glue (query relays, parse ids ->
iroh::EndpointId), publish path, wire swarm::providers().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pulls iroh 1.0 + iroh-blobs 0.103 as OPTIONAL deps under the iroh-swarm
feature and implements a real BlobProvider over them. Verified: the full
iroh QUIC dep tree (260 pkgs) resolves and compiles against the pinned
bitcoin/nostr-sdk/reqwest-rustls stack; the provider compiles against the
0.103/1.0 API.
- swarm/iroh_provider.rs: IrohProvider::new binds a QUIC Endpoint, opens a
persistent FsStore (data_dir/iroh-blobs), and serves blobs via the
iroh-blobs protocol/Router — a node that fetches also SEEDS. try_fetch
maps ContentDigest -> iroh Hash, asks discovery for seed EndpointIds, then
downloader.download(hash, providers) (range-verified) + export to staging.
- ProviderDiscovery trait: the seam Phase 3 (signed Nostr advertisement
events) fills. discovery=None -> no seeds -> origin-only, so enabling the
feature is never worse than today.
- Default build untouched: iroh is optional, the module is cfg-gated, and
providers() stays empty until Phase 3 wires discovery in.
Build: cargo build --features iroh-swarm succeeds (dev). Default build +
44 swarm/update/content_hash/blobs tests unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lands the transport/swarm orchestration layer (the iroh engine attaches
later, behind a flag). The seam is fully exercised today with the origin
HTTP path; with no swarm providers registered the behaviour is byte-for-byte
identical to before.
- swarm/mod.rs: BlobProvider trait + fetch_content_addressed() — tries each
provider in order, VERIFIES peer-sourced bytes against the content digest
before accepting (untrusted seeds can't inject tampered bytes), falls back
to the origin closure if none serve. Returns Swarm|Origin.
- Cargo: iroh-swarm feature (off by default; heavy QUIC dep tree attaches
here). providers() is empty until enabled → every fetch hits origin.
- update.rs: components with a BLAKE3 digest route through the seam, using
the existing resumable HTTP downloader as the origin fallback; a swarm hit
is re-checked against the mandatory SHA-256 manifest gate (re-fetch from
origin on any disagreement). Components without blake3 take the original
path untouched.
44/44 swarm/update/content_hash/blobs tests pass (incl. swarm hit/miss,
tampered-bytes-rejected→origin, fall-through ordering).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the iroh-native, range-verifiable hash next to the incumbent SHA-256
so the swarm can later fetch/verify by BLAKE3 with the registry/origin as
fallback. Non-breaking: SHA-256 stays the mandatory gate; BLAKE3 is verified
only when present.
- content_hash.rs: HashAlg + ContentDigest (parse/verify '<alg>:<hex>'
multihash strings), blake3_hex/sha256_hex; BLAKE3 known-answer test
- update.rs: ComponentUpdate.blake3 (serde-default); verified ALONGSIDE
SHA-256 in the resumable download loop, re-download on mismatch
- blobs.rs: BlobMeta.blake3 computed on put (on-disk path stays
SHA-256-keyed for back-compat; advertises the future swarm address)
Drive-by: fix a pre-existing stale test (test_save_and_load_state_roundtrip)
that never wrote the .download-complete marker #26 requires, so load_state's
self-heal cleared update_in_progress. Unrelated to BLAKE3 — surfaced by
running the full update:: suite.
40/40 content_hash/update/blobs tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #26 fix makes has_staged_update require the .download-complete
marker, so the state self-heal treats a marker-less staging dir as a
partial download and clears update_in_progress. The roundtrip test
staged a binary file but not the marker, so it began failing. Write
the marker to simulate a *complete* staged update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the parked trust module and wires it into the live build:
- main.rs: register `mod trust`
- app_catalog::fetch_one: verify the release-root detached signature when
present (verify against raw JSON so forward-compat fields stay in the
signed preimage); accept unsigned during the migration window, hard-reject
a present-but-bad signature so a tampering mirror can't pass altered bytes
- seed: pin release-root Ed25519 known-answer test (priv+pub) for the
signing ceremony / pinned-anchor / external-verifier cross-check
- signed_doc: drop unused import
20/20 Phase 0 unit tests pass (trust::canonical/did/signed_doc/anchor,
seed release-root, app_catalog). Crate compiles clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moved here so main stays clean for the v1.7.98 release. Contains the trust/
module (canonical.rs, did.rs, signed_doc.rs) + seed::derive_release_root_ed25519.
Not wired into the build yet. Continue this work on this branch.
The kiosk chromium pinned ~92% of a core (software-compositing spin from
--enable-gpu-rasterization on a GPU-less/headless node), saturating the machine
and starving the backend + container builds — it caused the .198 receive timeout
and the deploy storms.
- archipelago-kiosk.service: CPUQuota=75% + MemoryMax/High + Delegate, so a
runaway kiosk can never take the whole node down.
- archipelago-kiosk-launcher.sh: detect /dev/dri — use GPU rasterization only
when a GPU exists, else --disable-gpu (avoids the headless spin).
- bootstrap::ensure_kiosk_hardened: OTA self-heal that installs the updated
unit+launcher on already-deployed nodes, daemon-reloads, and only try-restarts
a *running* kiosk (never re-enables an operator-disabled one).
cargo check clean; launcher bash -n clean; unit syntax valid.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
immich_server/redis/postgres + indeedhub-* are multi-container stack members
whose sub-container app_ids are NOT in package_data, so the health monitor skips
them as "orphans" and never restarts them when they exit — Immich/IndeedHub stay
down until the next reboot (the boot-only start_stopped_stack_containers was the
only recovery). Spawn a 120s supervisor that reuses that same recovery at
runtime. It cheaply skips already-running containers and honours the user-stopped
list (set on every container by package.stop), so it only revives genuinely
crashed members and never fights a user stop.
cargo check clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A resumable-but-failed download leaves partial component files in update-staging.
has_staged_update() treated ANY staged file as "install-ready", so the state
self-heal kept update_in_progress=true and the UI showed Install instead of
Download (no clean retry).
- update.rs: write a .download-complete marker only after EVERY component
downloads+verifies; has_staged_update() now checks that marker. Partial/failed
downloads (no marker) correctly read as not-staged → self-heal clears
update_in_progress → UI shows Download. Resume still works (partial files kept).
- SystemUpdate.vue: on a genuine download failure, reset downloaded/in_progress
and re-sync, so the user lands back on Download immediately.
cargo check + vue-tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make each peer file card a flex column filling its grid cell (flex flex-col
h-full) and pin the body row (filename + Play/Download) with mt-auto, so cards
with a media preview and cards without line their footers up across the row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
User: chat history (messages + mesh/Tor contacts) must persist and be
secure/encrypted per best practice. Root cause of the .198 loss was the B17
mount race writing empty stores over real data (B17 already fixes the trigger);
this hardens storage so it can never silently lose or expose data:
- storage_crypto: shared at-rest envelope mirroring credentials::store — key =
SHA-256(domain ‖ node identity key) (seed-derived, per-store domain
separation), ChaCha20-Poly1305 AEAD with a random 96-bit nonce, tamper-evident.
Transparent migration of legacy plaintext files. Unit-tested (round-trip,
wrong-key/tamper rejection, plaintext detection).
- messages.json: encrypted at rest + ATOMIC write (temp+rename) so a crash/
reboot mid-write cannot corrupt history; decrypt-with-migration on load; a
failed decrypt never overwrites the on-disk data.
- mesh contacts (alias/notes/pinned/blocked): were ONLY in memory and lost on
every restart — now persisted to mesh-contacts.json (encrypted, atomic),
loaded on MeshState startup, saved after contacts-save/contacts-block.
Explicit clear (mesh.clear-all) still wipes everything, as intended.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
User priority: FIPS is the main transport but it was unreliable and needed a
manual "Activate" button. Improvements (all in the FIPS dial/supervisor):
- Auto-activate: ensure_activated() installs the daemon config + starts the
service on its own once seed onboarding has materialised the key — no Activate
button needed. Idempotent; runs from the supervisor every 45s so a node that
onboards after boot still comes up automatically.
- Dial retry: try_fips_get/post now retry ONCE on a connect/timeout error. The
first dial to a peer triggers NAT hole-punching and often times out before the
path is up; the retry lands on the now-warm path — the main reason calls were
dropping to Tor despite the peer being FIPS-reachable.
- More patient connect_timeout (5s→8s) so a reachable-but-cold peer isn't
abandoned to Tor while hole-punching completes.
- Path warmer: spawn_fips_supervisor() keeps hole-punched paths to known
federation peers warm (every 45s, concurrent), so on-demand dials are fast and
land on FIPS.
- Confirmed the daemon config already enables BOTH udp + tcp transports
(render_config_yaml), so FIPS already uses TCP where UDP is blocked; the Tor
fallback was path-establishment, addressed above.
cargo check + fmt clean. Backend — needs a binary rebuild+deploy to validate on
.116/.198 (watch last_transport flip fips, and FIPS coming up with no button).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On production nodes /var/lib/archipelago (the app data dir AND podman's
graphroot=/var/lib/archipelago/containers/storage) is a separate
device-mapper volume. archipelago.service ordered only After=network-online
.target, so on cold boots it (and its ExecStartPre) could start BEFORE
var-lib-archipelago.mount, write to the bare mountpoint on rootfs, fail every
podman call, exit, and be restarted every 5s until the volume mounted — the
"~20x [FAILED] Failed to start over ~5min" boot flap. Proven live on .198:
"var-lib-archipelago.mount: Directory /var/lib/archipelago to mount over is
not empty, mounting anyway" — the service had written there pre-mount.
Fix: RequiresMountsFor=/var/lib/archipelago (adds Requires= + After= on the
mount unit).
- image-recipe/configs/archipelago.service: ships the directive on fresh ISOs.
- bootstrap::ensure_archipelago_mount_ordering(): self-heals already-deployed
nodes' installed unit + daemon-reload (boot-ordering only, effective next
reboot; never restarts the running service). Idempotent; harmless on rootfs
installs (maps to the always-mounted root).
Verified on .198: after applying, systemctl shows After=var-lib-archipelago
.mount and systemd-analyze verify is clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
