From fa7372dd7afae7590742ff489d2c7fb69b5c5436 Mon Sep 17 00:00:00 2001 From: Dorian Date: Sat, 14 Mar 2026 03:11:07 +0000 Subject: [PATCH] docs: update architecture and current-state for v1.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - DOC-02: architecture.md — remove StartOS refs, add identity/federation section, update networking (archy-net, UFW, Tor), data persistence paths - DOC-03: current-state.md — full rewrite reflecting pure Archipelago stack, 2-node federation, 30+ apps, test coverage matrix Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/architecture.md | 46 +++++++++++++----- docs/current-state.md | 109 ++++++++++++++++++++++++++---------------- loop/plan.md | 4 +- 3 files changed, 104 insertions(+), 55 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 159df983..244f9bde 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2,7 +2,7 @@ ## Overview -Archipelago is a next-generation Bitcoin Node OS built on Debian Linux with Podman containerization, combining the modularity of Parmanode with the security and reliability of a proven server OS. Similar to StartOS, we use Debian Live for reliable USB boot and installation. +Archipelago is a self-sovereign Bitcoin Node OS built on Debian Linux with Podman containerization. Flash to USB, install on hardware, manage via web UI. It runs 30+ containerized apps (Bitcoin, Lightning, BTCPay, Mempool, DWN, and more) with multi-node federation over Tor, W3C DID identity, and encrypted backups. ## System Architecture @@ -85,12 +85,17 @@ Standardized YAML format in `apps/`: - Security policies and health checks - Bitcoin/Lightning/Web5 integration metadata -### 6. Parmanode Compatibility +### 6. Identity & Federation -Located in `core/parmanode/`: -- **script_runner.rs**: Executes Parmanode scripts in containers -- **converter.rs**: Converts Parmanode modules to app manifests -- **parmanode-wrapper.sh**: Shell wrapper for direct script execution +Located in `core/archipelago/src/`: +- **identity.rs**: Ed25519 node identity, did:key generation, DID Documents +- **identity_manager.rs**: Multi-identity manager (Personal/Business/Anonymous) +- **credentials.rs**: W3C Verifiable Credentials (issue, verify, present, revoke) +- **federation.rs**: Multi-node federation with invite codes, trust levels, Tor sync +- **network/dwn_store.rs**: DWN message persistence (JSON files on disk) +- **network/dwn_sync.rs**: Bidirectional DWN replication over Tor +- **nostr_discovery.rs**: Nostr presence publishing (kind 30078, no onion addresses) +- **nostr_handshake.rs**: NIP-44 encrypted peer communication ### 7. Security Modules @@ -144,18 +149,33 @@ Located in `core/performance/`: ## Networking -- **Isolated Networks**: Each app on separate bridge network by default -- **Bitcoin Core**: Isolated network, explicit RPC access -- **Lightning Nodes**: Separate network, gRPC/REST exposed -- **Tor Integration**: Optional, default for privacy-sensitive apps -- **Mesh Networking**: Meshtastic and router support for decentralized communication +- **Container Networks**: `archy-net` (main), `immich-net`, `penpot-net` via Podman +- **Aardvark DNS**: Container hostname resolution within networks +- **UFW Firewall**: Podman subnets (10.88.0.0/16, 10.89.0.0/16) allowed for DNS +- **Tor Integration**: System Tor daemon, SOCKS5 proxy on port 9050 +- **Federation**: Inter-node communication over Tor hidden services +- **Nginx**: Reverse proxy on port 80/443, proxies to backend on 5678 ## Data Persistence - **App Data**: `/var/lib/archipelago/{app-id}/` +- **Identity**: `/var/lib/archipelago/identity/` (Ed25519 node key) +- **Multi-Identity**: `/var/lib/archipelago/identities/` (per-identity JSON) +- **DWN Messages**: `/var/lib/archipelago/dwn/messages/` (JSON files) +- **Credentials**: `/var/lib/archipelago/credentials/` +- **Federation**: `/var/lib/archipelago/federation/` +- **Content Catalog**: `/var/lib/archipelago/content/` +- **Backups**: `/var/lib/archipelago/backups/` (ChaCha20-Poly1305 encrypted) - **Secrets**: `/var/lib/archipelago/secrets/{app-id}/` (encrypted) -- **Logs**: `/var/lib/archipelago/logs/{app-id}/` -- **Backups**: `/var/lib/archipelago/backups/` + +## Multi-Node Federation + +Nodes form a federated network over Tor: +- **Invite-based joining**: Generate invite code, peer joins via Tor +- **Trust levels**: Trusted, Verified, Untrusted +- **State sync**: Federation state, DWN messages, file catalogs sync bidirectionally +- **DWN protocols**: 4 interoperable schemas (node-identity, file-catalog, federation, app-deploy) +- **Verifiable Credentials**: W3C VCs for inter-node trust attestation ## Future Enhancements diff --git a/docs/current-state.md b/docs/current-state.md index f33516de..ba255adb 100644 --- a/docs/current-state.md +++ b/docs/current-state.md @@ -1,54 +1,83 @@ # Current Development State -## What We're Actually Using +**Last updated**: 2026-03-14 (v1.2.0) -### ✅ NEW Archipelago Components (Following Plan) -- **Container Orchestration** (`core/container/`) - Our NEW implementation -- **Security Modules** (`core/security/`) - Our NEW implementation -- **Performance Modules** (`core/performance/`) - Our NEW implementation -- **Parmanode Compatibility** (`core/parmanode/`) - Our NEW implementation -- **App Manifests** (`apps/*/manifest.yml`) - Our NEW YAML format -- **Container RPC Endpoints** (`core/startos/src/container/`) - Our NEW API layer -- **Vue.js Container Views** - Our NEW UI components +## Architecture -### ⚠️ TEMPORARY StartOS Dependencies (To Be Replaced) -- **Backend Binary** (`startbox`) - Still using StartOS binary temporarily -- **Backend Core** (`core/startos/`) - StartOS fork, being extended with our modules -- **Some RPC Infrastructure** - StartOS RPC system, we're adding to it +Archipelago is a pure, standalone Bitcoin Node OS. All StartOS dependencies were removed in v1.0. The stack: -## Development Strategy +- **OS**: Debian 12 (Bookworm) — x86_64 and ARM64 +- **Backend**: Rust binary (`archipelago`) on port 5678 +- **Frontend**: Vue 3 + TypeScript + Vite 7 + Tailwind CSS +- **Containers**: Podman (root-level via `sudo podman`) +- **Web Server**: Nginx reverse proxy (port 80/443 → backend 5678) +- **Tor**: System Tor daemon with hidden services +- **Systemd**: `archipelago.service` with watchdog (60s) -### Phase 1: Extension (Current) -- ✅ Use our new container orchestration modules -- ✅ Add container RPC endpoints to StartOS backend -- ✅ Use our new app manifest system -- ⚠️ Still running `startbox` binary (StartOS) as base +## Active Nodes -### Phase 2: Refactoring (Next) -- 🔄 Replace StartOS-specific code with Archipelago-native -- 🔄 Create our own backend binary (`archipelago` or `arch`) -- 🔄 Remove StartOS dependencies -- 🔄 Build on Alpine Linux base +| Node | IP | Hardware | RAM | Disk | Containers | +|------|-----|----------|-----|------|------------| +| Arch 1 | 192.168.1.228 | Intel i3-8100T | 16GB | 1.8TB NVMe | 32 | +| Arch 2 | 192.168.1.198 | — | 8GB | 457GB | 34 | -### Phase 3: Complete Independence -- ✅ Pure Archipelago implementation -- ✅ No StartOS code remaining -- ✅ Alpine Linux + Podman only +Both nodes are federated over Tor with bidirectional DWN sync. -## What We Should Be Doing NOW +## Backend Modules -### For Development -1. **Use Mock Backend** - For UI development, mock backend is fine -2. **Use Our New Modules** - All container code uses `archipelago-container` -3. **Use Our App Manifests** - All apps defined in `apps/*/manifest.yml` -4. **Mark StartOS Code** - Add `// TODO: Refactor to Archipelago-native` comments +``` +core/archipelago/src/ +├── api/ — HTTP server, RPC handler (100+ endpoints) +│ ├── handler.rs — HTTP routing (/rpc, /health, /dwn, /ws) +│ └── rpc/ — RPC endpoints (auth, identity, federation, etc.) +├── backup/ — Full backup + identity backup (ChaCha20-Poly1305) +├── container/ — Container management (Podman, health checks) +├── identity.rs — Ed25519 node identity, did:key, DID Documents +├── identity_manager.rs — Multi-identity (Personal/Business/Anonymous) +├── credentials.rs — W3C Verifiable Credentials +├── federation.rs — Multi-node federation over Tor +├── network/ — DWN store, DWN sync, DNS, routing +├── nostr_discovery.rs — Nostr presence (kind 30078) +├── nostr_handshake.rs — NIP-44 encrypted peer communication +├── health_monitor.rs — Container health monitoring + auto-restart +├── crash_recovery.rs — Start stopped containers on boot +├── monitoring/ — System metrics collection +├── session.rs — Session management + rate limiting +├── update.rs — Update checking + scheduling +└── wallet/ — LND integration, ecash, profits tracking +``` -### For Backend -- Currently: Using StartOS backend as base, extending with our modules -- Future: Build our own Archipelago backend that uses ONLY our modules +## Key Features (Working) -## Key Point +- **30+ containerized apps**: Bitcoin Knots, LND, Mempool, BTCPay, Nextcloud, Immich, Jellyfin, Home Assistant, etc. +- **Multi-node federation**: Invite-based joining, trust levels, state sync over Tor +- **DWN sync**: Bidirectional message replication between federated nodes +- **W3C identity**: did:key, DID Documents, Verifiable Credentials +- **NIP-07 signing**: iframe apps can sign Nostr events with consent +- **File sharing**: Content catalog with access controls (free/peers-only/paid) +- **Encrypted backups**: Create, verify, restore with Argon2 + ChaCha20-Poly1305 +- **Health monitoring**: Auto-restart crashed containers with exponential backoff +- **Tiered startup**: Databases → Core → Applications ordering on boot +- **Web UI**: Glass morphism design, real-time WebSocket updates -**We ARE following the plan** - we built all the NEW Archipelago components. We're just using StartOS backend temporarily as a base while we extend it. The plan was to "integrate with existing Rust backend via enhanced API layer" - which is what we did. +## Planned (Not Yet Implemented) -But you're right - we should be moving toward complete independence. The mock backend is fine for now since we're developing the UI and container management features. +- **did:dht**: DHT-based discoverable identities (architecture documented) +- **DWN protocol schemas**: 4 Archipelago protocols defined, registration implemented +- **Multi-user support**: Admin/viewer/app-user roles +- **Cluster mode**: Multi-node HA with consensus +- **Mobile companion**: Read-only PWA for remote monitoring + +## Test Coverage + +Cross-node test suite (`scripts/test-cross-node.sh`) covers: +- US-01: System health (6 checks × 2 nodes) +- US-02: Container lifecycle (3 checks × 2 nodes) +- US-03: Federation join (4 checks × 2 nodes) +- US-04: Federation sync (4 checks × 2 nodes) +- US-05: Tor hidden services (2 checks bidirectional) +- US-07: File sharing (5 checks bidirectional) +- US-08: DWN sync (5 checks × 10 iterations) +- US-09: NIP-07 signing (1 check × 2 nodes) +- US-10: Backup/restore (4 checks × 10 iterations × 2 nodes) +- US-15: Boot recovery (3 checks × 3 reboots per node) diff --git a/loop/plan.md b/loop/plan.md index e99cdf31..2de26780 100644 --- a/loop/plan.md +++ b/loop/plan.md @@ -359,9 +359,9 @@ Every test must pass **10 consecutive times** from BOTH .228→.198 AND .198→. - [x] **DOC-01** — Updated CHANGELOG.md with v1.2.0 release. Covers: crash loop fixes, DWN sync performance, backup reliability, deploy script hardening, cross-node test suite (DWN/backup/boot recovery), did:dht architecture, DWN protocol definitions, deploy --dry-run, ISO swap/tiered startup, security hardening. -- [ ] **DOC-02** — Update architecture.md for current state. The current doc references StartOS, Docker, macOS. Update to reflect: Debian 12, Podman, multi-node federation, did:dht, DWN protocols. **Acceptance**: Architecture doc matches actual system. +- [x] **DOC-02** — Updated architecture.md. Removed StartOS references. Added: Identity & Federation section (identity.rs, credentials.rs, federation, DWN), container networking (archy-net, Aardvark DNS, UFW rules), Tor integration, multi-node federation overview, updated data persistence paths (DWN, identity, credentials, content, federation). -- [ ] **DOC-03** — Update current-state.md. Remove references to StartOS dependencies (already removed). Document actual current state: pure Archipelago backend, Podman, 33+ containers, 2-node federation. **Acceptance**: current-state.md reflects reality. +- [x] **DOC-03** — Rewrote current-state.md from scratch. Removed all StartOS references. Documents: pure Archipelago stack (Debian 12, Rust, Vue 3, Podman), 2 active nodes with specs, backend module layout, 10+ working features, planned features, cross-node test coverage matrix. - [x] **DOC-04** — Created `docs/operations-runbook.md` with 17 sections: health checks, container status, fix crashes, federation peers, Tor rotation, backup/restore, updates, CPU/memory/disk diagnostics, Tor connectivity, DWN sync, service restart, log viewing, network diagnostics, emergency boot recovery, cross-node tests.