<p>Archipelago (nicknamed "Archy") is a <strong>personal server operating system</strong> focused on Bitcoin. You download an ISO file, flash it to a USB drive, install it on any computer, and it gives you:</p>
<ul>
<li>A <strong>full Bitcoin node</strong> — you verify your own transactions, no trust in anyone else</li>
<li>A <strong>web dashboard</strong> — manage everything from your phone or laptop browser</li>
<li>An <strong>app marketplace</strong> — install apps like Nextcloud, Jellyfin, Vaultwarden with one click</li>
<li><strong>Privacy by default</strong> — Tor routing, encrypted secrets, no telemetry</li>
</ul>
<divclass="analogy">
<p>Think of it like an iPhone for servers. Apple gives you a phone with an App Store where you install apps. Archipelago gives you a server with a Marketplace where you install self-hosted apps. The difference? <em>You</em> own and control everything — your data never leaves your machine.</p>
</div>
<p>Similar projects exist (Umbrel, Start9, RaspiBlitz), but Archipelago is built from scratch with production-grade security and a custom Rust backend instead of Node.js.</p>
<strong>Key Concept: Separation of Concerns</strong>
Each layer has ONE job. The browser shows things. Nginx routes traffic. Rust makes decisions. Podman runs apps. This makes the system easier to understand, test, and fix — if the UI breaks, you know the problem is in the Vue code, not the Rust code.
<divclass="flow-step"><divclass="num">2</div><divclass="content"><p><strong>systemd starts services</strong> — A program called <code>systemd</code> reads <code>archipelago.service</code> and launches the Rust backend</p></div></div>
<divclass="flow-step"><divclass="num">3</div><divclass="content"><p><strong>Rust backend initializes</strong> — Loads config, creates/loads encryption keys, starts the HTTP server on port 5678</p></div></div>
<divclass="flow-step"><divclass="num">4</div><divclass="content"><p><strong>Health monitor starts</strong> — Checks which containers are running, restarts crashed ones, reports readiness</p></div></div>
<divclass="flow-step"><divclass="num">5</div><divclass="content"><p><strong>Nginx starts</strong> — Listens on port 80 (HTTP) and routes all incoming traffic</p></div></div>
<divclass="flow-step"><divclass="num">6</div><divclass="content"><p><strong>Containers start</strong> — Bitcoin, LND, and other apps start in priority order (Bitcoin first, then things that depend on it)</p></div></div>
<divclass="flow-step"><divclass="num">7</div><divclass="content"><p><strong>Ready!</strong> — You open a browser, go to your server's IP address, and see the dashboard</p></div></div>
<divclass="analogy">
<p>It's like starting a restaurant. First the building opens (Linux). Then the manager arrives (Rust backend). They check if all kitchen stations are ready (health monitor). The front door opens (Nginx). The cooks start preparing (containers). Customers can now order (you open the web UI).</p>
<h3id="rust-backend">Layer 1: The Rust Backend (The Brain)</h3>
<p>This is the most important piece. It's written in <strong>Rust</strong> — a programming language known for speed and safety. The backend is the "brain" that controls everything.</p>
<divclass="callout callout-learn">
<strong>Why Rust?</strong>
Rust prevents entire categories of bugs (memory leaks, crashes, race conditions) at compile time. For a server that manages Bitcoin wallets and runs 24/7, this matters. A crash could mean lost money. Rust makes crashes nearly impossible.
<p>The recent refactoring split monolithic files into focused module directories. Each directory has a <code>mod.rs</code> entry point and focused sub-files:</p>
<spanclass="highlight">Step 3:</span><spanclass="blue">handler/mod.rs</span> parses the JSON, extracts the method name
<spanclass="highlight">Step 4:</span><spanclass="blue">rpc/mod.rs</span> checks the CSRF token (security check)
<spanclass="highlight">Step 5:</span><spanclass="blue">rpc/mod.rs</span> checks the session cookie (are you logged in?)
<spanclass="highlight">Step 6:</span><spanclass="blue">dispatcher.rs</span> routes to <spanclass="green">package/install.rs</span> based on method name
<spanclass="highlight">Step 7:</span><spanclass="green">package/install.rs</span> validates the app ID
<h3id="rust-deep-dive">Rust Backend Deep Dive — Should We Use Custom Code?</h3>
<divclass="callout callout-learn">
<strong>The short answer:</strong> Yes, custom Rust is the right call for Archipelago. The backend does things no off-the-shelf tool provides: it orchestrates rootless Podman containers, manages Bitcoin/LND RPC, handles encrypted secrets, runs federation/mesh networking, and serves a real-time WebSocket to the Vue frontend — all as a single binary with zero runtime dependencies. The alternatives (Node.js, Go, Python) would need dozens of third-party packages to match, and none offer Rust's memory safety guarantees for a server handling Bitcoin keys.
</div>
<h4>Why not use an existing solution?</h4>
<p>Projects like Umbrel use a <strong>Node.js + Docker Compose</strong> backend. Start9 uses <strong>Rust</strong> (like us). RaspiBlitz uses <strong>bash scripts</strong>. Here's why custom Rust wins:</p>
<p>Every action the frontend takes goes through <code>POST /rpc/v1</code> as a JSON-RPC call. The RPC layer was recently refactored from monolithic files into <strong>29 standalone modules + 8 domain subdirectories</strong>, totaling ~20,000 lines. Requests flow through <code>dispatcher.rs</code> (395 LOC) which routes to the appropriate handler:</p>
<strong>Refactoring win:</strong> The old monolithic <code>package.rs</code> (1,795 lines) was split into 7 focused files under <code>package/</code>. Similarly, <code>federation.rs</code>, <code>identity.rs</code>, <code>mesh.rs</code>, <code>system.rs</code>, and <code>tor.rs</code> were each extracted into their own subdirectories with <code>handlers.rs</code> + <code>mod.rs</code> separation. The API handler layer (<code>handler.rs</code>) was split into 6 focused files under <code>api/handler/</code>.
</div>
<h4>Container Orchestration — How Podman Is Controlled</h4>
<p>The backend talks to <strong>rootless Podman</strong> via its Unix socket REST API (not CLI). This is faster, more reliable, and avoids shell injection risks.</p>
<divclass="diagram">
<spanclass="highlight">PodmanClient</span> connects to:
/run/user/1000/podman/podman.sock (API v4.0.0)
<spanclass="highlight">Install flow:</span>
1. <spanclass="blue">package.rs</span> validates app ID + checks dependencies
2. <spanclass="blue">DependencyResolver</span> topological sort → install order
<td>Login endpoints rate-limited, IP extracted from <code>X-Real-IP</code> (loopback only)</td>
</tr>
<tr>
<td>RBAC</td>
<td>Explicit method allowlists</td>
<td>No prefix matching — each role lists exact permitted methods</td>
</tr>
<tr>
<td>Key material</td>
<td><code>zeroize::Zeroizing</code></td>
<td>All crypto keys zeroed from memory after use</td>
</tr>
</table>
<h4>WebSocket Real-Time Sync</h4>
<p>The frontend connects to <code>/ws/db</code> and receives the full <strong>DataModel</strong> on connect, then incremental updates as state changes. This is how the UI shows live container status, sync progress, and notifications without polling.</p>
<divclass="diagram">
<spanclass="highlight">DataModel</span> (broadcast to all WebSocket clients):
{
<spanclass="green">server_info</span>: { node_id, name, tor_address, lan_ip, version }
<td>Must be custom — proxies authenticated calls to local Bitcoin/LND with macaroon management</td>
</tr>
</table>
<divclass="callout callout-learn">
<strong>Bottom line:</strong> The custom code isn't reinventing the wheel — it's glue that connects Podman, Bitcoin, LND, Tor, Nostr, mesh radios, and a Vue frontend into a cohesive OS. No existing framework does this. The individual pieces (hyper, serde, tokio, ed25519-dalek, aes-gcm) are all battle-tested crates. The custom part is the orchestration logic that ties them together.
<h3id="vue-frontend">Layer 2: The Vue.js Frontend (The Face)</h3>
<p>The frontend is what you see in the browser. It's built with <strong>Vue 3</strong> — a JavaScript framework for building interactive web pages — and <strong>TypeScript</strong> — JavaScript with type safety.</p>
<divclass="callout callout-learn">
<strong>What is a Single Page Application (SPA)?</strong>
Instead of loading a new HTML page every time you click something (like old websites), an SPA loads once and then dynamically updates the page content. When you click "Marketplace" in Archipelago, it doesn't load a new page — it swaps out the content area. This makes it feel fast and smooth, like a native app.
<p>The frontend was heavily refactored — large "god components" were split into focused sub-views, and the god store was decomposed into dedicated stores:</p>
<p>A Vue component is like a LEGO brick. Each brick (component) has its own shape (template), color (styles), and moving parts (script). You snap them together to build the full UI. The <code><Dashboard></code> component contains <code><Sidebar></code>, which contains <code><NavItem></code> components — just like nesting LEGO bricks.</p>
</div>
<hr>
<h3id="containers">Layer 3: The Container System (The Apps)</h3>
<p>Containers are how Archipelago runs apps like Bitcoin Core, Lightning, Nextcloud, etc. Each app runs in its own isolated "box" called a container.</p>
<divclass="callout callout-learn">
<strong>What is a Container?</strong>
A container is like a lightweight virtual machine. It has its own filesystem, its own network, and its own processes — but it shares the host's Linux kernel, so it's much faster than a full VM. Think of it as an apartment in a building — each apartment has its own walls and locks, but they all share the same building infrastructure.
<p>Archipelago uses <strong>rootless Podman</strong> instead of Docker. Podman runs entirely without root privileges under the <code>archipelago</code> user (UID 1000) — no background daemon, no root access needed. The backend communicates with Podman via its REST API socket, not the CLI.</p>
<p><strong>Nginx</strong> (pronounced "engine-X") is a web server that sits between the internet and everything else. Every single request goes through it first. Archipelago's nginx config is ~1,100 lines — one of the most complex parts of the system.</p>
<p>Nginx is like the receptionist at a hospital. You walk in and say what you need. "I need the API" — they send you to the Rust backend. "I need the Bitcoin app" — they send you to the Bitcoin container. "I need the website" — they hand you the static files. Without the receptionist, you'd be wandering the hallways lost.</p>
<spanclass="cross">✗</span> No web UI (CLI only)<br>
<spanclass="cross">✗</span> Not beginner-friendly<br>
<spanclass="cross">✗</span> Long rebuild times</p>
</div>
</div>
<divclass="callout callout-info">
<strong>Archipelago's choice:</strong> Nginx gives the most control over security headers, rate limiting, and HTML rewriting (injecting Nostr provider scripts into app iframes). The tradeoff is a 1,100-line config instead of a 50-line Caddyfile — but for a Bitcoin node OS, that control is worth it.
<tr><td><code>/dwn</code></td><td>:5678</td><td>—</td><td>default</td><td>Decentralized Web Node</td></tr>
<tr><td><code>/electrs-status</code></td><td>:5678</td><td>—</td><td>default</td><td>Electrum sync status (CORS enabled)</td></tr>
<tr><td><code>/lnd-connect-info</code></td><td>:5678</td><td>—</td><td>default</td><td>LND connection URI (CORS enabled)</td></tr>
</table>
</div>
<divclass="card">
<h4>App Proxies — 24 Container Apps</h4>
<p>Every <code>/app/{id}/</code> route proxies into a container. All share a common pattern: strip the upstream <code>X-Frame-Options</code>, set <code>SAMEORIGIN</code>, inject the Nostr provider script, and forward real IP headers.</p>
<strong>Archipelago leads on security headers.</strong>
Most node OS projects ship with minimal or no HTTP security headers. Archipelago sets a full Content-Security-Policy, HSTS with 1-year max-age, Permissions-Policy blocking camera/microphone/geolocation/payment, and dual-zone rate limiting — defense-in-depth at the proxy layer.
</div>
<!-- ─── Unique Features ─── -->
<h4>Unique Nginx Features in Archipelago</h4>
<divclass="card-grid">
<divclass="card-sm">
<h4>Nostr NIP-07 Injection</h4>
<ulclass="item-list">
<li>Every app proxy uses <code>sub_filter</code> to inject <code>nostr-provider.js</code> into <code></head></code></li>
<li>Gives all container apps <code>window.nostr</code> for signing</li>
<li>No other node OS does this — unique to Archipelago</li>
<li><code>Accept-Encoding</code> disabled to enable text rewriting</li>
</ul>
</div>
<divclass="card-sm">
<h4>Dual Rate Limit Zones</h4>
<ulclass="item-list">
<li><strong>rpc zone:</strong> 20 req/s base, burst of 40 — for API calls</li>
<li><strong>auth zone:</strong> 3 req/s — for login/auth endpoints (brute-force protection)</li>
<li>Returns HTTP 429 on violation</li>
<li>Per-IP tracking with 10MB shared memory zone</li>
</ul>
</div>
<divclass="card-sm">
<h4>External Site Proxying</h4>
<ulclass="item-list">
<li><code>/ext/botfights/</code>, <code>/ext/484-kitchen/</code>, etc. proxy external HTTPS sites</li>
<li>Strips CORS/COEP/COOP headers for iframe embedding</li>
<li>Rewrites <code>href</code>/<code>src</code> attributes to rebase paths</li>
<li>Standalone proxy servers on ports 8901–8903</li>
<tr><td><code>image-recipe/configs/nginx-archipelago.conf</code></td><td>~1,100</td><td>Production config — HTTP + HTTPS servers, all routing</td></tr>
<tr><td><code>image-recipe/configs/snippets/archipelago-https-app-proxies.conf</code></td><td>~400</td><td>HTTPS app proxy blocks (included in main config)</td></tr>
<tr><td><code>image-recipe/configs/snippets/archipelago-pwa.conf</code></td><td>~30</td><td>PWA service worker and manifest caching</td></tr>
<tr><td><code>image-recipe/configs/external-app-proxies.conf</code></td><td>~200</td><td>External site reverse proxies (BotFights, 484 Kitchen)</td></tr>
<tr><td><code>neode-ui/docker/nginx.conf</code></td><td>~60</td><td>Dev Docker config (mock backend on :5959)</td></tr>
<tr><td><code>neode-ui/docker/nginx-demo.conf</code></td><td>~80</td><td>Demo mode config (no security, mock backend)</td></tr>
<tr><td><code>docker/bitcoin-ui/nginx.conf</code></td><td>~50</td><td>Bitcoin UI container — RPC proxy with CORS</td></tr>
<tr><td><code>docker/electrs-ui/nginx.conf</code></td><td>~30</td><td>Electrs UI container — status endpoint</td></tr>
There are three layers of nginx: (1) the <strong>main server nginx</strong> that routes all traffic, (2) <strong>per-app container nginx</strong> configs inside some containers (bitcoin-ui, electrs-ui, lnd-ui, indeedhub) that serve their own SPAs and proxy to internal services, and (3) <strong>dev/demo nginx</strong> configs for local development. Changes to app routing require updating BOTH the main config AND the relevant container config.
<h2id="data-flow">How Data Flows Through the System</h2>
<p>Let's trace what happens when you click "Install Bitcoin" in the UI:</p>
<divclass="card">
<divclass="flow-step"><divclass="num">1</div><divclass="content"><p><strong>You click the Install button</strong> in <code>Marketplace.vue</code>. Vue calls the Pinia store action <code>installPackage('bitcoin-knots')</code></p></div></div>
<divclass="flow-step"><divclass="num">2</div><divclass="content"><p><strong>The store calls the RPC client:</strong><code>rpcClient.installPackage('bitcoin-knots', 'docker.io/bitcoin/knots:28')</code></p></div></div>
<divclass="flow-step"><divclass="num">3</div><divclass="content"><p><strong>RPC client sends HTTP POST</strong> to <code>/rpc/v1</code> with a session cookie and CSRF token for security</p></div></div>
<divclass="flow-step"><divclass="num">4</div><divclass="content"><p><strong>Nginx receives the request</strong> on port 80, checks rate limits, forwards to the Rust backend on port 5678</p></div></div>
<divclass="flow-step"><divclass="num">5</div><divclass="content"><p><strong>Rust backend validates</strong> — checks your session is valid, CSRF token matches, app ID is safe (no shell injection characters)</p></div></div>
<divclass="flow-step"><divclass="num">6</div><divclass="content"><p><strong>Rust checks dependencies</strong> — if you're installing LND, it checks Bitcoin is already running</p></div></div>
<divclass="flow-step"><divclass="num">7</div><divclass="content"><p><strong>Rust tells Podman to pull the image</strong> — <code>podman pull docker.io/bitcoin/knots:28</code> (downloads the app)</p></div></div>
<divclass="flow-step"><divclass="num">8</div><divclass="content"><p><strong>Rust creates and starts the container</strong> with all security flags (cap-drop, readonly root, etc.)</p></div></div>
<divclass="flow-step"><divclass="num">9</div><divclass="content"><p><strong>Backend sends a WebSocket update</strong> — the frontend receives a "state changed" event in real time</p></div></div>
<divclass="flow-step"><divclass="num">10</div><divclass="content"><p><strong>Vue reactively updates the UI</strong> — the Marketplace card changes from "Install" to "Running" with no page reload</p></div></div>
<h2id="rpc">RPC: How Frontend Talks to Backend</h2>
<p><strong>RPC</strong> stands for Remote Procedure Call. It's a way for the frontend to tell the backend "do something" — like calling a function on a remote computer.</p>
<divclass="callout callout-learn">
<strong>RPC vs REST</strong>
Most web APIs use REST (different URLs for different things: <code>GET /users</code>, <code>POST /users</code>, <code>DELETE /users/5</code>). Archipelago uses RPC instead — every request goes to the same URL (<code>/rpc/v1</code>) and the <em>method name</em> says what to do. It's like having one phone number for a building, and you say who you want to talk to.
</div>
<p>The frontend has a class called <code>RPCClient</code> (in <code>rpc-client.ts</code>) with ~70 methods. Each method maps to a backend function:</p>
<table>
<tr><th>Frontend Method</th><th>Backend Handler</th><th>What It Does</th></tr>
<tr><td><code>rpcClient.login(password)</code></td><td><code>auth.login</code></td><td>Log in with password</td></tr>
<tr><td><code>rpcClient.getServerInfo()</code></td><td><code>system.info</code></td><td>Get server name, version, uptime</td></tr>
<tr><td><code>rpcClient.installPackage(id, image)</code></td><td><code>package.install</code></td><td>Install a container app</td></tr>
<p><strong>State</strong> is the data your app is currently working with: is the user logged in? What apps are installed? Is Bitcoin synced? This data needs to be shared between components.</p>
<divclass="callout callout-learn">
<strong>What is Pinia?</strong>
Pinia is Vue's state management library. Instead of each component keeping its own data (which leads to chaos), you put shared data in a "store" — a central place that any component can read from and write to. When the store changes, every component that uses it updates automatically.
<p>AI data access permission management (5.2 KB)</p>
</div>
</div>
<divclass="callout callout-success">
<strong>Store decomposition complete.</strong> The old "god store" (<code>app.ts</code>) that handled auth + WebSocket + server data + package management was split into three new focused stores: <code>auth.ts</code> (authentication state machine), <code>server.ts</code> (server state + RPC actions), and <code>sync.ts</code> (WebSocket + data synchronization). The login flow is now: <code>useAuthStore().login()</code> → <code>useSyncStore().initializeData()</code> + <code>connectWebSocket()</code> → views consume <code>sync.data</code> reactively.
<p>Instead of the frontend asking "has anything changed?" every second (polling), the backend <em>pushes</em> updates to the frontend through a WebSocket — a persistent, two-way connection.</p>
<p>When you log in, the backend creates a <strong>session</strong> — a temporary "you're allowed in" token. Here's how it works:</p>
<divclass="flow-step"><divclass="num">1</div><divclass="content"><p><strong>You enter your password</strong> on the login page</p></div></div>
<divclass="flow-step"><divclass="num">2</div><divclass="content"><p><strong>Backend hashes it with bcrypt</strong> — a one-way function that makes it impossible to reverse</p></div></div>
<divclass="flow-step"><divclass="num">3</div><divclass="content"><p><strong>Backend compares the hash</strong> to the stored hash (never compares raw passwords)</p></div></div>
<divclass="flow-step"><divclass="num">4</div><divclass="content"><p><strong>Backend creates a session</strong> — generates a random 256-bit token using a cryptographically secure random number generator</p></div></div>
<divclass="flow-step"><divclass="num">5</div><divclass="content"><p><strong>Session ID sent as a cookie</strong> — the browser stores it and sends it with every request</p></div></div>
<divclass="flow-step"><divclass="num">6</div><divclass="content"><p><strong>CSRF token also sent</strong> — a second token that prevents cross-site request forgery attacks</p></div></div>
<divclass="callout callout-info">
<strong>Why two tokens?</strong>
The session cookie proves you're logged in. The CSRF token proves the request came from YOUR browser tab, not a malicious website that tricked your browser into sending a request. Both must match for any request to succeed.
<p>Bitcoin is the heart of Archipelago. The backend communicates with Bitcoin Core/Knots using <strong>JSON-RPC</strong> — the same protocol Bitcoin has used since 2009.</p>
<divclass="callout callout-warn">
<strong>Critical Rule: Never Use Floating Point for Bitcoin</strong>
Bitcoin amounts are always in <strong>satoshis</strong> (1 BTC = 100,000,000 sats) as <strong>integers</strong>. Using floating point (decimals) causes rounding errors. 0.1 + 0.2 ≠ 0.3 in floating point. When you're dealing with money, that's unacceptable. Archipelago uses <code>u64</code> in Rust and <code>BigInt</code> in TypeScript for all Bitcoin amounts.
</div>
<h4>Bitcoin RPC examples</h4>
<pre><code><spanclass="code-comment">// The backend calls Bitcoin Core like this:</span>
<p>Multiple Archipelago nodes can form a <strong>federation</strong> — a trusted network of servers that sync data, share state, and communicate privately.</p>
<divclass="diagram">
<spanclass="highlight">Your Node (.228)</span> ←── Tor ──→ <spanclass="blue">Friend's Node</span>
│ │
└──── Tor ──→ <spanclass="green">Office Node</span> ←── Tor ──┘
• <spanclass="blue">DID</span> (Decentralized Identifier — like a username that can't be taken away)
• <spanclass="green">Onion address</span> (Tor hidden service — no IP address exposed)
• <spanclass="highlight">DWN</span> (Decentralized Web Node — stores and syncs data)
</div>
<p>Nodes discover each other through <strong>Nostr relays</strong> (publish presence, but never onion addresses — those are exchanged privately via encrypted DMs).</p>
<p>Archipelago can communicate over <strong>LoRa radio</strong> — no internet needed. A small radio device plugs into the server's USB port and sends messages up to 10+ km using the Meshtastic/Meshcore protocol.</p>
<divclass="analogy">
<p>Imagine walkie-talkies that can send text messages. Each radio can relay messages for others, so even if two radios can't reach each other directly, they can communicate through intermediate radios. That's mesh networking — no cell towers, no ISPs, no internet required.</p>
<p>The deploy script (<code>scripts/deploy-to-target.sh</code>) is how code gets from your development laptop to the live server. It's a ~1,790-line shell script (with shared functions from <code>lib/common.sh</code>) that automates everything:</p>
<divclass="flow-step"><divclass="num">2</div><divclass="content"><p><strong>Frontend build</strong> — runs <code>npm run build</code> to compile Vue/TypeScript into static files</p></div></div>
<divclass="flow-step"><divclass="num">3</div><divclass="content"><p><strong>Upload frontend</strong> — rsyncs built files to <code>/opt/archipelago/web-ui/</code> on the server</p></div></div>
<divclass="flow-step"><divclass="num">4</div><divclass="content"><p><strong>Upload Rust source</strong> — rsyncs <code>core/</code> to the server (builds ON the server, not macOS)</p></div></div>
<divclass="flow-step"><divclass="num">5</div><divclass="content"><p><strong>Build on server</strong> — runs <code>cargo build --release</code> on the Linux server</p></div></div>
<divclass="flow-step"><divclass="num">6</div><divclass="content"><p><strong>Sync configs</strong> — copies nginx config, systemd service from <code>image-recipe/configs/</code></p></div></div>
<divclass="flow-step"><divclass="num">7</div><divclass="content"><p><strong>Restart services</strong> — reloads nginx, restarts the Rust backend via systemd</p></div></div>
<divclass="flow-step"><divclass="num">8</div><divclass="content"><p><strong>Health check</strong> — pings <code>/health</code> endpoint to verify everything came back up</p></div></div>
<divclass="flow-step"><divclass="num">9</div><divclass="content"><p><strong>Deploy manifest</strong> — writes a JSON file recording the commit, timestamp, and deploy status</p></div></div>
<divclass="callout callout-warn">
<strong>Why build on the server?</strong>
Rust compiles to machine code specific to the CPU architecture. If you compile on macOS (ARM/x86) and copy the binary to a Linux server, it won't run — you get an "Exec format error". The deploy script sends the <em>source code</em> and compiles on the target machine.
<p>After reviewing ~45,000 lines of Rust (213 files), ~45,500 lines of TypeScript/Vue (232 files), and ~40 shell scripts, here are the quality scores. <strong>Several scores improved</strong> since the last review thanks to major refactoring:</p>
<p>Zero <code>unwrap()</code> or <code>panic!()</code> in production code. Every fallible operation uses the <code>?</code> operator to propagate errors gracefully. This is rare even in professional Rust codebases.</p>
<h4>Backend Module Architecture (Refactored)</h4>
<p>The backend was comprehensively refactored from monolithic files into domain-focused subdirectories. Previously: <code>package.rs</code> (1,795 lines), <code>federation.rs</code> (810 lines), <code>handler.rs</code> (800+ lines) were all single files. Now: each is a clean directory with focused sub-modules (e.g., <code>package/</code> has config.rs, install.rs, lifecycle.rs, runtime.rs, stacks.rs, dependencies.rs, progress.rs). The RPC layer uses a dedicated <code>dispatcher.rs</code> for routing. All 8 major domains (package, federation, identity, mesh, system, tor, handler, credentials) follow the same <code>mod.rs</code> + <code>handlers.rs</code> pattern.</p>
<p>All "god components" were split into sub-views: <code>Web5.vue</code> (3,940 lines) → 14 focused sub-views under <code>views/web5/</code>. <code>Settings.vue</code> (1,792 lines) → 13 sections. <code>Dashboard.vue</code>, <code>Apps.vue</code>, <code>AppDetails.vue</code>, <code>AppSession.vue</code>, <code>Federation.vue</code>, <code>Fleet.vue</code>, <code>Discover.vue</code> — all extracted into subdirectories with focused components. The Pinia god store was decomposed into <code>auth.ts</code>, <code>server.ts</code>, and <code>sync.ts</code>.</p>
<p>App IDs validated against a strict character whitelist. Container image names checked for shell injection characters. All external input sanitized at the boundary. Backend rate limiting on login + endpoints via new <code>rate_limit.rs</code>.</p>
<p>All 5 strictest compiler flags enabled. Zero <code>any</code> types across 45,500+ lines. Every function has proper types. This prevents entire categories of bugs.</p>
<p>Every container drops all capabilities and adds back only what's needed. Read-only root filesystems. Non-root users. No-new-privileges. This is better than most commercial container platforms.</p>
<p>11 Vue composables, each focused on one concern (toasts, audio, keyboard, onboarding, controller nav). Clean, reusable, properly scoped. 10 test files for composables.</p>
<p>Rollback backups before deployment, deploy manifests tracking what was deployed, health checks after deployment, progress bars with ETAs. Deploy locking prevents concurrent deploys. Shared script library (<code>scripts/lib/common.sh</code>) eliminates function duplication.</p>
<h4>Monitoring & Telemetry System</h4>
<p>New <code>monitoring/</code> module (1,380 LOC) with metrics collection, alert generation, persistent storage, beta telemetry reporting, and notification dispatch. Production-grade observability for the beta phase.</p>
<h4>PodmanClient Uses REST API Socket</h4>
<p>The container management layer communicates with Podman via its async REST API unix socket (<code>/run/user/{UID}/podman/podman.sock</code>), not CLI. This is faster, more reliable, and avoids shell injection risks.</p>
<h4>Full Security Audit Completed</h4>
<p>A comprehensive penetration test (33 findings) was completed in March 2026 and all findings were remediated. Security rules from findings are enforced in CLAUDE.md for all future code.</p>
<p><strong>What:</strong><code>"health"</code> is listed in <code>UNAUTHENTICATED_METHODS</code> but has no match handler — returns "Unknown method" error instead of actual health status.</p>
<p><strong>Impact:</strong> Frontend, load balancers, and orchestrators can't verify the backend is actually healthy. System appears unhealthy when it's fine.</p>
<p><strong>Fix:</strong> Add handler that checks crash recovery status, Podman responsiveness, and service readiness.</p>
<h4>P0-2. Zero container health checks across all 30 containers</h4>
<p><strong>What:</strong><code>first-boot-containers.sh</code> creates 30+ containers with <code>--restart unless-stopped</code> but zero <code>--health-cmd</code> flags. Crashed containers restart endlessly in a hammer loop.</p>
<p><strong>Impact:</strong> Silent failures — a broken app looks "running" but returns errors. No way for the backend to distinguish healthy from crashed.</p>
<p><strong>Fix:</strong> Add <code>--health-cmd</code> with appropriate checks (HTTP, TCP, CLI) to every container.</p>
<h4>P0-3. Backup restore has no pre-validation or atomic rollback</h4>
<p><strong>What:</strong><code>restore_full_backup()</code> extracts directly to the live data directory. If extraction fails halfway, the system is left in a corrupt partial state with no way to recover.</p>
<p><strong>Impact:</strong> A corrupted backup can brick a fresh install. Data loss on partial restore failure.</p>
<p><strong>Fix:</strong> Extract to staging directory, validate required files, atomic rename, rollback on failure.</p>
<p><strong>What:</strong><code>/archipelago/</code>, <code>/content</code>, <code>/dwn</code> endpoints (used for Tor P2P federation) have no timeout, body size limit, or rate limiting.</p>
<p><strong>Impact:</strong> Vulnerable to slow-loris attacks, payload flooding, and connection exhaustion via Tor.</p>
<p><strong>Fix:</strong> Add <code>proxy_connect_timeout</code>, <code>client_max_body_size 10m</code>, and <code>limit_req</code> to all three locations.</p>
<h4><spanclass="check">✓</span> RESOLVED: useAppStore was a "god store" — split into 3 focused stores</h4>
<p><strong>Before:</strong> One store handling auth, WebSocket, server data, and package management.</p>
<p><strong>After:</strong> Decomposed into <code>auth.ts</code> (login/logout/TOTP/sessions), <code>server.ts</code> (server state + RPC actions), <code>sync.ts</code> (WebSocket + JSON patch data sync). <code>app.ts</code> is now a thin data store.</p>
</div>
<divclass="card">
<h4><spanclass="check">✓</span> RESOLVED: Shell scripts had no shared library</h4>
<p><strong>Before:</strong> Duplicated functions across deploy, first-boot, and helper scripts.</p>
<p><strong>After:</strong><code>scripts/lib/common.sh</code> provides shared functions: colored logging, SSH wrappers (<code>ssh_cmd</code>, <code>scp_cmd</code>), health checks, disk checks, memory limits. Sourced by all deployment scripts.</p>
<p><strong>What:</strong> 38 frontend test files and 36+ backend test modules exist. However, coverage is uneven — critical paths like session validation, federation sync, and the app install flow lack thorough test suites.</p>
<p><strong>Fix:</strong> Add integration tests for critical paths (auth flow, container lifecycle, federation handshake). Add CI that runs <code>cargo test</code> + <code>npm test</code> on every push.</p>
<h4>P1-A. Nostr client.connect() hangs indefinitely (no timeout)</h4>
<p><strong>What:</strong> 4 calls to <code>client.connect().await</code> in <code>nostr_handshake.rs</code> have no timeout wrapper. If a relay is down, peer discovery hangs forever.</p>
<p><strong>Fix:</strong> Wrap all in <code>tokio::time::timeout(Duration::from_secs(10), ...)</code>.</p>
<p><strong>What:</strong><code>EndpointRateLimiter::cleanup()</code> and <code>LoginRateLimiter</code> cleanup methods exist but are never spawned. HashMap of (method, IP) entries grows forever.</p>
<p><strong>Fix:</strong> Spawn cleanup task every 5 minutes in <code>RpcHandler::new()</code>.</p>
</div>
<divclass="card">
<h4>P1-C. Systemd service missing resource limits</h4>
<p><strong>What:</strong> No <code>MemoryMax</code>, <code>LimitNOFILE</code>, or <code>TasksMax</code> in <code>archipelago.service</code>. A memory leak in the backend can OOM-kill the entire system.</p>
<h4>P1-D. Container images using :latest tag (7 instances)</h4>
<p><strong>What:</strong> Several containers in <code>first-boot-containers.sh</code> and the ISO build pull <code>:latest</code> — no version pinning.</p>
<p><strong>Impact:</strong> Two machines installed a week apart may have different Bitcoin node versions. Supply chain risk.</p>
<p><strong>Fix:</strong> Pin every image to a specific version tag or SHA256 digest.</p>
</div>
<divclass="card">
<h4>P1-E. WebSocket reconnect doesn't refresh full state</h4>
<p><strong>What:</strong> After a WebSocket disconnect (5+ minutes), the UI shows stale data. Reconnection applies patches to an outdated base state instead of fetching fresh data.</p>
<p><strong>Fix:</strong> On reconnect, call <code>server.get-state</code> RPC to refresh full state before accepting patches.</p>
</div>
<divclass="card">
<h4>P1-F. No global Vue error handler</h4>
<p><strong>What:</strong> No <code>app.config.errorHandler</code> in <code>main.ts</code>. Component errors silently log to console — user sees blank screen with no recovery path.</p>
<p><strong>Fix:</strong> Add error handler that shows user-visible toast and logs structured error.</p>
<p><strong>Why it's bad:</strong> A minor version bump in a crypto library could introduce a vulnerability or behavioral change. The project's own rules require exact pinning for crypto deps.</p>
<p><strong>Fix:</strong> Pin to exact versions: <code>"1.7.0"</code>, <code>"0.10.1"</code>, <code>"2.1.1"</code>.</p>
</div>
<divclass="card">
<h4>6. No frontend-backend type synchronization</h4>
<p><strong>What:</strong> TypeScript types in <code>types/api.ts</code> are manually maintained copies of Rust structs. If the backend changes a field name, the frontend doesn't know until runtime.</p>
<p><strong>Why it's bad:</strong> Types can drift apart silently. A backend developer renames <code>sync_progress</code> to <code>syncProgress</code> and the frontend breaks in production.</p>
<p><strong>Fix:</strong> Generate TypeScript types from Rust structs (using <code>ts-rs</code> or a JSON Schema).</p>
</div>
<divclass="card">
<h4>7. Container metadata duplicated in 3 places</h4>
<h4>8. Deploy and ISO build scripts are still 1,700+ lines each</h4>
<p><strong>What:</strong> Two monolithic shell scripts (deploy: ~1,790 lines, ISO build: ~1,870 lines) handle dozens of responsibilities each. Shared functions have been extracted to <code>scripts/lib/common.sh</code>, but the scripts themselves are still large.</p>
<p><strong>Improvement:</strong><code>scripts/lib/common.sh</code> now provides shared logging, SSH wrappers, health checks, and memory limits — eliminating most duplication. But the core scripts could still benefit from modular splitting post-beta.</p>
<p><strong>Next step:</strong> Split deploy into modules: <code>deploy-frontend.sh</code>, <code>deploy-backend.sh</code>, <code>sync-configs.sh</code>. Split ISO build into <code>lib/rootfs.sh</code>, <code>lib/components.sh</code>, <code>lib/installer-env.sh</code>.</p>
<p><strong>What:</strong> One GitHub Action builds macOS release binaries on tag push. No tests run in CI. No linting. No Linux build or deploy automation.</p>
<p><strong>Fix:</strong> Add CI that runs <code>cargo clippy</code>, <code>cargo test</code>, <code>npm run type-check</code>, and <code>npm run lint</code> on every push. Add Linux cross-compilation.</p>
<p><strong>What:</strong> On startup, <code>session.rs</code> reads <code>sessions.json</code> using synchronous (blocking) file I/O in an async context.</p>
<p><strong>Fix:</strong> Use <code>tokio::fs::read_to_string</code> for non-blocking I/O at startup.</p>
</div>
<divclass="card">
<h4>12. Inconsistent loading state patterns in frontend</h4>
<p><strong>What:</strong> Some components use <code>loading</code>, others <code>isLoading</code>, others <code>loadingApps</code>. No shared composable.</p>
<p><strong>Fix:</strong> Create a <code>useAsyncState</code> composable that standardizes loading/error/data patterns.</p>
<tr><td><strong>API</strong></td><td>Application Programming Interface — a defined way for two programs to talk to each other</td></tr>
<tr><td><strong>Async/Await</strong></td><td>A way to write code that waits for slow things (network, disk) without blocking other work</td></tr>
<tr><td><strong>Backend</strong></td><td>The server-side code that runs on the machine (not visible to users)</td></tr>
<tr><td><strong>Container</strong></td><td>An isolated environment for running an app, like a lightweight virtual machine</td></tr>
<tr><td><strong>Composable</strong></td><td>A reusable piece of logic in Vue (similar to React hooks)</td></tr>
<tr><td><strong>CSRF</strong></td><td>Cross-Site Request Forgery — an attack where a malicious site tricks your browser into sending requests</td></tr>
<tr><td><strong>Crate</strong></td><td>A Rust package (like npm package for JavaScript)</td></tr>
<tr><td><strong>DID</strong></td><td>Decentralized Identifier — a self-owned digital identity (no central authority controls it)</td></tr>
<tr><td><strong>DWN</strong></td><td>Decentralized Web Node — personal data storage that syncs across your devices</td></tr>
<tr><td><strong>Frontend</strong></td><td>The browser-side code that users see and interact with</td></tr>
<tr><td><strong>ISO</strong></td><td>A disk image file — like a digital copy of an installation CD</td></tr>
<tr><td><strong>JWT</strong></td><td>JSON Web Token — a compact way to pass verified identity between systems</td></tr>
<tr><td><strong>LoRa</strong></td><td>Long Range radio — low-power wireless communication over several kilometers</td></tr>
<tr><td><strong>Nginx</strong></td><td>A web server that also works as a reverse proxy (routes traffic to the right service)</td></tr>
<tr><td><strong>Nostr</strong></td><td>A decentralized messaging protocol using public/private key pairs</td></tr>
<tr><td><strong>Onion Service</strong></td><td>A Tor hidden service — a server accessible only through the Tor network (no IP address)</td></tr>
<tr><td><strong>Pinia</strong></td><td>Vue's official state management library (successor to Vuex)</td></tr>
<tr><td><strong>Podman</strong></td><td>A container runtime like Docker, but rootless (more secure)</td></tr>
<tr><td><strong>RPC</strong></td><td>Remote Procedure Call — calling a function on another computer over the network</td></tr>
<tr><td><strong>Reactive</strong></td><td>Data that automatically updates the UI when it changes (core Vue concept)</td></tr>
<tr><td><strong>Reverse Proxy</strong></td><td>A server that sits between clients and backend servers, forwarding requests</td></tr>
<tr><td><strong>Rust</strong></td><td>A systems programming language focused on safety and performance</td></tr>
<tr><td><strong>SPA</strong></td><td>Single Page Application — a web app that loads once and dynamically updates content</td></tr>
<tr><td><strong>Satoshi (sat)</strong></td><td>The smallest unit of Bitcoin. 1 BTC = 100,000,000 sats</td></tr>
<tr><td><strong>systemd</strong></td><td>Linux's service manager — starts, stops, and monitors background services</td></tr>
<tr><td><strong>Tokio</strong></td><td>Rust's async runtime — handles thousands of concurrent operations efficiently</td></tr>
<tr><td><strong>Tor</strong></td><td>The Onion Router — anonymizes internet traffic by routing through multiple relays</td></tr>
<tr><td><strong>TypeScript</strong></td><td>JavaScript with static types — catches bugs at compile time instead of runtime</td></tr>
<tr><td><strong>Vue 3</strong></td><td>A JavaScript framework for building reactive user interfaces</td></tr>
<tr><td><strong>WebSocket</strong></td><td>A persistent, two-way connection between browser and server for real-time data</td></tr>
