diff --git a/DOCKER_DEV_SETUP.md b/DOCKER_DEV_SETUP.md new file mode 100644 index 00000000..5cfb0d2e --- /dev/null +++ b/DOCKER_DEV_SETUP.md @@ -0,0 +1,302 @@ +# Archipelago Docker Development Environment + +This setup provides a complete development environment with all apps running in Docker containers, mimicking the production environment as closely as possible. + +## Overview + +All apps from the "My Apps" page are prepackaged and run as Docker containers: + +- **Bitcoin Core** (regtest mode - no blockchain sync) +- **BTCPay Server** (Bitcoin payment processor) +- **Endurain** (placeholder) +- **Fedimint** (federated Bitcoin mint) +- **Grafana** (monitoring/analytics) +- **Home Assistant** (home automation) +- **Lightning Stack** (LND - Lightning Network) +- **Mempool** (blockchain explorer) +- **MorphOS Server** (placeholder) +- **Ollama** (local LLM) +- **OnlyOffice** (office suite) +- **Penpot** (design platform) +- **SearXNG** (privacy-respecting search) + +## Quick Start + +### Prerequisites + +- Docker Desktop installed and running +- Node.js 18+ installed +- ~4GB of free disk space for Docker images + +### Starting Development Environment + +From the `neode-ui` directory: + +```bash +npm start +``` + +This will: +1. Check and start Docker Desktop if needed +2. Start all Docker containers with apps +3. Start the mock backend API server (port 5959) +4. Start the Vite dev server (port 8100) + +### Stopping Everything + +From the `neode-ui` directory: + +```bash +npm stop +``` + +This stops all Node processes and Docker containers. + +## Manual Docker Control + +### Start all Docker apps + +```bash +./start-docker-apps.sh +``` + +### Stop all Docker apps + +```bash +./stop-docker-apps.sh +``` + +### View logs for a specific app + +```bash +docker compose logs -f [service-name] + +# Examples: +docker compose logs -f bitcoin +docker compose logs -f btcpay +docker compose logs -f lnd +``` + +### Restart a specific app + +```bash +docker compose restart [service-name] +``` + +### View all running containers + +```bash +docker compose ps +``` + +## App URLs + +All apps are accessible on localhost with unique ports: + +| App | URL | Notes | +|-----|-----|-------| +| Bitcoin Core | http://localhost:18443 | RPC only (regtest) | +| BTCPay Server | http://localhost:14142 | Full UI | +| Endurain | http://localhost:8084 | Placeholder | +| Fedimint | http://localhost:8173 | API | +| Grafana | http://localhost:3000 | UI (admin/admin) | +| Home Assistant | http://localhost:8123 | Full UI | +| Lightning (REST) | http://localhost:8080 | REST API | +| Lightning (gRPC) | localhost:10009 | gRPC | +| Mempool | http://localhost:4080 | Full UI | +| MorphOS | http://localhost:8081 | Placeholder | +| Ollama | http://localhost:11434 | API | +| OnlyOffice | http://localhost:8083 | Full UI | +| Penpot | http://localhost:9001 | Full UI | +| SearXNG | http://localhost:8082 | Full UI | + +## Architecture + +### Docker Compose Structure + +The `docker-compose.yml` file orchestrates all services: + +- **Networking**: All services run on a shared `archy-net` bridge network +- **Volumes**: Persistent data stored in Docker volumes +- **Dependencies**: Services with dependencies (e.g., BTCPay depends on Bitcoin Core) start in the correct order + +### Bitcoin & Lightning + +- **Bitcoin Core** runs in regtest mode (no mainnet/testnet sync) +- **LND** connects to Bitcoin Core for Lightning functionality +- Both are pre-configured to work together without blockchain sync +- Generate test blocks with: `docker exec archy-bitcoin bitcoin-cli -regtest -rpcuser=bitcoin -rpcpassword=bitcoinpass generatetoaddress 101
` + +### Database Services + +Some apps require databases: +- BTCPay Server โ PostgreSQL +- Mempool โ MariaDB +- Penpot โ PostgreSQL + Redis + +These are automatically started as supporting services. + +## Development Workflow + +### Starting Development + +```bash +cd neode-ui +npm start +``` + +The UI will be available at http://localhost:8100 + +### Testing App Launch + +1. Open http://localhost:8100 in your browser +2. Navigate to "My Apps" +3. Apps appear alphabetically and stay in order regardless of state +4. Click "Launch" on any running app to open it in a new tab + +### Stop/Start Apps + +Use the UI buttons or manually: + +```bash +# Stop an app +docker compose stop bitcoin + +# Start an app +docker compose start bitcoin + +# Restart an app +docker compose restart bitcoin +``` + +### Viewing Logs + +Monitor app logs: + +```bash +# All logs +docker compose logs -f + +# Specific app +docker compose logs -f bitcoin + +# Last 50 lines +docker compose logs --tail=50 bitcoin +``` + +### Resetting Data + +Remove all app data and start fresh: + +```bash +./stop-docker-apps.sh +docker compose down -v # Remove volumes +./start-docker-apps.sh +``` + +## Configuration + +### Bitcoin Core (Regtest) + +Configuration is in `docker-compose.yml`: +- RPC user: `bitcoin` +- RPC password: `bitcoinpass` +- Ports: 18443 (RPC), 18444 (P2P) + +### Grafana + +Default credentials: +- Username: `admin` +- Password: `admin` + +### App Icons + +Icons are loaded from `/assets/img/app-icons/` in the UI. + +## Troubleshooting + +### Docker not starting + +```bash +# Check if Docker is running +docker ps + +# Start Docker Desktop manually +open -a Docker +``` + +### Port conflicts + +If a port is already in use, modify `docker-compose.yml`: + +```yaml +services: + bitcoin: + ports: + - "18443:18443" # Change left side to a free port +``` + +### App won't start + +Check logs: + +```bash +docker compose logs [service-name] +``` + +Restart the container: + +```bash +docker compose restart [service-name] +``` + +### Out of disk space + +Clean up unused Docker data: + +```bash +docker system prune -a --volumes +``` + +### Reset everything + +```bash +npm stop +docker compose down -v +docker system prune -a +npm start +``` + +## Production vs Development + +### Development + +- Bitcoin Core in regtest (no blockchain) +- LND without real channels +- All services on localhost +- Placeholder apps for unimplemented services + +### Production (Future) + +- Bitcoin Core syncing mainnet +- LND with real Lightning channels +- Services behind reverse proxy +- All apps fully functional +- Tor integration +- Hardware security + +## Next Steps + +- Implement real start/stop functionality in the UI +- Connect backend API to Docker API +- Add health checks and status monitoring +- Implement backup/restore for Docker volumes +- Add app configuration UI +- Integrate with Tor for privacy + +## Resources + +- [Docker Compose Documentation](https://docs.docker.com/compose/) +- [Bitcoin Core Documentation](https://bitcoin.org/en/bitcoin-core/) +- [LND Documentation](https://docs.lightning.engineering/) +- [BTCPay Server Documentation](https://docs.btcpayserver.org/) diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index 258b1508..414c7226 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -1,384 +1,335 @@ -# ๐ Archipelago - Getting Started +# Getting Started with Archipelago Docker Development -Welcome to Archipelago! This guide will get you up and running in minutes. +This guide will help you get the full Archipelago development environment running with all apps containerized. -## โก Quick Start +## What You'll Get -### Step 1: Install Dependencies +A complete Bitcoin node OS with 13+ apps running in Docker: +- Bitcoin Core (regtest - no blockchain download) +- Lightning Network (LND) +- BTCPay Server +- Mempool Explorer +- Home Assistant +- Grafana +- Ollama (local AI) +- SearXNG +- OnlyOffice +- Penpot +- And more... -Run the automated installation script: +## Prerequisites + +1. **Docker Desktop** - [Download here](https://www.docker.com/products/docker-desktop/) + - Mac: 4GB RAM minimum, 8GB recommended + - Disk space: ~5GB for all images + +2. **Node.js 18+** - [Download here](https://nodejs.org/) + +3. **Git** (you probably have this) + +## Installation Steps + +### Step 1: Start Docker Desktop + +Make sure Docker Desktop is running: +- Mac: Look for the Docker icon in your menu bar +- If not running, open Docker Desktop from Applications + +### Step 2: Clone and Setup ```bash -./INSTALL.sh +cd ~/Projects +git clone [your-repo] archy +cd archy ``` -This installs everything you need: -- โ Rust (latest stable) -- โ Node.js 18+ -- โ Podman (container runtime) -- โ PostgreSQL 15 -- โ All project dependencies - -**Note:** The script requires administrator privileges for Homebrew and system tools. - -### Step 2: Verify Installation +### Step 3: Install UI Dependencies ```bash -./verify-install.sh -``` - -This checks that all dependencies are properly installed and configured. - -### Step 3: Configure (Optional) - -Copy environment files and adjust if needed: - -```bash -# Backend configuration -cp core/.env.example core/.env - -# Frontend configuration -cp neode-ui/.env.example neode-ui/.env -``` - -### Step 4: Start Development - -Open two terminal windows: - -**Terminal 1 - Backend:** -```bash -cd core -cargo run --bin archipelago -``` - -**Terminal 2 - Frontend:** -```bash -cd neode-ui -npm run dev -``` - -### Step 5: Open in Browser - -Navigate to: **http://localhost:8100** - ---- - -## ๐ What Gets Installed - -| Tool | Purpose | Version Required | -|------|---------|-----------------| -| **Rust** | Backend development | Latest stable (1.93+) | -| **Node.js** | Frontend & custom apps | v18 or higher | -| **Podman** | Container runtime | Latest | -| **PostgreSQL** | Database | v15 | -| **Homebrew** | Package manager (macOS) | Latest | - ---- - -## ๐ฏ Alternative: Manual Installation - -If you prefer to install manually or the script fails, follow these steps: - -### 1. Install Homebrew (if not installed) - -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -``` - -### 2. Install Rust - -```bash -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -source "$HOME/.cargo/env" -``` - -### 3. Install Node.js, Podman, and PostgreSQL - -```bash -brew install node podman postgresql@15 -``` - -### 4. Initialize Podman - -```bash -podman machine init -podman machine start -``` - -### 5. Start PostgreSQL - -```bash -brew services start postgresql@15 -createdb archipelago_dev -``` - -### 6. Install Project Dependencies - -```bash -# Frontend cd neode-ui npm install - -# Custom apps -cd ../apps/did-wallet && npm install -cd ../endurain && npm install -cd ../morphos-server && npm install -cd ../router && npm install -cd ../web5-dwn && npm install - -# Backend (this may take a while) -cd ../../core -cargo build ``` ---- - -## ๐ Verify Everything Works - -Run the verification script: +### Step 4: Start Everything ```bash -./verify-install.sh +npm start ``` -You should see all checkmarks (โ). If you see warnings (โ ) or errors (โ), follow the suggested commands to fix them. +This will: +1. Check if Docker is running +2. Download all Docker images (first time only, ~10-30 minutes) +3. Start all containers +4. Start the development UI server ---- +**First Time?** You'll be prompted to confirm the image download. Type `y` and press Enter. -## ๐ Development Modes +### Step 5: Open the UI -### Mode 1: Full Stack (Recommended) +Once everything starts, open your browser to: -Best for full-stack development with real backend API. +**http://localhost:8100** -**Terminal 1:** -```bash -cd core && cargo run --bin archipelago -``` +Navigate to "My Apps" to see all your apps running! -**Terminal 2:** -```bash -cd neode-ui && npm run dev -``` +## What's Running? -**URLs:** -- Frontend: http://localhost:8100 -- Backend: http://localhost:5959 +After startup, you'll have these services: -### Mode 2: Mock Backend (Fastest) +### Bitcoin & Lightning Network +- **Bitcoin Core** - Full node in regtest mode (no mainnet sync) +- **LND** - Lightning Network daemon +- **BTCPay Server** - Accept Bitcoin payments +- **Mempool** - Blockchain explorer +- **Fedimint** - Federated Bitcoin mint -Best for frontend-only development without backend setup. +### Self-Hosted Apps +- **Home Assistant** - Home automation +- **Grafana** - Monitoring and analytics +- **SearXNG** - Private search engine +- **Ollama** - Run AI models locally -**Single Terminal:** -```bash -cd neode-ui && npm run dev:mock -``` +### Collaboration Tools +- **OnlyOffice** - Office suite (docs, spreadsheets, presentations) +- **Penpot** - Design and prototyping (like Figma) -**URL:** -- Frontend: http://localhost:8100 -- Mock API: http://localhost:3000 +### Placeholders +- **Endurain** - Coming soon +- **MorphOS** - Coming soon -### Mode 3: Quick Dev Script +## Using the Apps -Uses convenience scripts for easy startup. +### From the UI + +1. Go to http://localhost:8100 +2. Click "My Apps" in the sidebar +3. All apps are listed alphabetically +4. Click "Launch" on any running app + +### Direct Access + +All apps have unique ports: + +| App | URL | +|-----|-----| +| Grafana | http://localhost:3000 | +| Home Assistant | http://localhost:8123 | +| SearXNG | http://localhost:8082 | +| Mempool | http://localhost:4080 | +| Penpot | http://localhost:9001 | +| OnlyOffice | http://localhost:8083 | +| BTCPay Server | http://localhost:14142 | + +See [DOCKER_DEV_SETUP.md](DOCKER_DEV_SETUP.md) for the complete list. + +## Managing Apps + +### Start/Stop Individual Apps ```bash -./scripts/dev.sh # Mock backend mode -# or -./scripts/dev-start.sh # Interactive mode +# Stop an app +docker compose stop bitcoin + +# Start an app +docker compose start bitcoin + +# Restart an app +docker compose restart bitcoin ``` ---- +### View Logs -## ๐ Project Structure +```bash +# All logs +docker compose logs -f + +# Specific app +docker compose logs -f bitcoin +docker compose logs -f btcpay +docker compose logs -f lnd +``` + +### Check Status + +```bash +docker compose ps +``` + +## Stopping Everything + +### Stop All Apps + +```bash +cd neode-ui +npm stop +``` + +This stops the UI and all Docker containers. + +### Stop Just Docker Apps + +```bash +cd .. # Go to project root +./stop-docker-apps.sh +``` + +### Remove All Data (Fresh Start) + +```bash +docker compose down -v +``` + +**Warning:** This deletes all app data! + +## Troubleshooting + +### Docker not starting + +```bash +# Check if Docker is running +docker ps + +# If not, start Docker Desktop +open -a Docker +``` + +### Port conflicts + +If you see port errors: +1. Check what's using the port: `lsof -i :PORT` +2. Stop that service or change the port in `docker-compose.yml` + +### App won't load + +1. Check if container is running: `docker compose ps` +2. View logs: `docker compose logs [app-name]` +3. Restart the container: `docker compose restart [app-name]` + +### Out of disk space + +Clean up Docker: + +```bash +docker system prune -a +``` + +### Complete reset + +```bash +cd neode-ui +npm stop +cd .. +docker compose down -v +docker system prune -a +cd neode-ui +npm start +``` + +## Development Tips + +### Making Changes + +The UI hot-reloads automatically. Just edit files in `neode-ui/src/` and refresh your browser. + +### Testing Bitcoin Stuff + +Bitcoin Core runs in regtest mode - you can generate blocks instantly: + +```bash +# Get a new address +docker exec archy-bitcoin bitcoin-cli -regtest -rpcuser=bitcoin -rpcpassword=bitcoinpass getnewaddress + +# Generate blocks to that address +docker exec archy-bitcoin bitcoin-cli -regtest -rpcuser=bitcoin -rpcpassword=bitcoinpass generatetoaddress 101 [address] +``` + +### Using Lightning + +LND connects to Bitcoin Core automatically. Access the REST API at http://localhost:8080. + +### Adding New Apps + +1. Add service to `docker-compose.yml` +2. Add entry to `neode-ui/src/utils/dummyApps.ts` +3. Restart: `npm stop && npm start` + +## File Structure ``` archy/ -โโโ ๐ INSTALL.sh # Install all dependencies -โโโ ๐ verify-install.sh # Verify installation -โโโ ๐ SETUP_GUIDE.md # Detailed setup guide -โโโ ๐ QUICK_REFERENCE.md # Command reference -โโโ ๐ README.md # Main documentation -โ -โโโ ๐ฆ core/ # Rust backend -โ โโโ archipelago/ # Main binary -โ โโโ container/ # Container management -โ โโโ parmanode/ # Parmanode integration -โ โโโ security/ # Security modules -โ โโโ performance/ # Performance optimization -โ -โโโ ๐จ neode-ui/ # Vue.js frontend -โ โโโ src/ # Source files -โ โโโ public/ # Static assets -โ โโโ package.json # Node dependencies -โ -โโโ ๐ฆ apps/ # Containerized apps -โ โโโ bitcoin-core/ # Bitcoin node -โ โโโ lnd/ # Lightning Network -โ โโโ router/ # Mesh router -โ โโโ did-wallet/ # Web5 wallet -โ โโโ web5-dwn/ # Web5 DWN server -โ -โโโ ๐ docs/ # Documentation -โ โโโ development-setup.md -โ โโโ architecture.md -โ โโโ app-manifest-spec.md -โ -โโโ ๐ง scripts/ # Development scripts - โโโ dev.sh - โโโ dev-start.sh +โโโ docker-compose.yml # All app definitions +โโโ start-docker-apps.sh # Start all Docker apps +โโโ stop-docker-apps.sh # Stop all Docker apps +โโโ DOCKER_DEV_SETUP.md # Detailed documentation +โโโ docker/ # Placeholder app HTML +โ โโโ endurain-placeholder/ +โ โโโ morphos-placeholder/ +โโโ neode-ui/ # Vue.js UI + โโโ package.json + โโโ start-dev.sh # Main dev startup (npm start) + โโโ stop-dev.sh # Stop everything (npm stop) + โโโ src/ + โโโ views/Apps.vue # My Apps page + โโโ utils/dummyApps.ts # App definitions ``` +## Next Steps + +1. **Explore the apps** - Try opening different services +2. **Check the logs** - See what's happening: `docker compose logs -f` +3. **Read the docs** - See [DOCKER_DEV_SETUP.md](DOCKER_DEV_SETUP.md) for details +4. **Start building** - The UI and backend are ready for development! + +## Common Tasks + +### Update all images + +```bash +cd ~/Projects/archy +docker compose pull +docker compose up -d +``` + +### Backup data + +```bash +docker compose down +cp -r /var/lib/docker/volumes ~/docker-backup +``` + +### Check resource usage + +```bash +docker stats +``` + +### See all containers + +```bash +docker ps -a +``` + +## Getting Help + +- **Documentation**: See [DOCKER_DEV_SETUP.md](DOCKER_DEV_SETUP.md) +- **Logs**: `docker compose logs -f [service-name]` +- **Status**: `docker compose ps` +- **Docker issues**: Check Docker Desktop logs + +## Architecture + +This setup mimics production as closely as possible: + +- **Production**: Alpine Linux + Podman + Real apps +- **Development**: macOS/Linux + Docker + Same app images + +Key differences: +- Bitcoin in regtest (no blockchain sync) +- All on localhost (no Tor) +- Simpler networking (single bridge network) + +See [Architecture.md](.cursor/rules/Architecture.mdc) for the full production architecture. + --- -## ๐ ๏ธ Common Commands - -### Development - -```bash -# Start backend -cd core && cargo run --bin archipelago - -# Start frontend -cd neode-ui && npm run dev - -# Start with mock backend -cd neode-ui && npm run dev:mock - -# Build for production -cd neode-ui && npm run build -cd core && cargo build --release -``` - -### Container Apps - -```bash -# Build all apps -cd apps && ./build.sh - -# Build specific app -cd apps && ./build.sh router -``` - -### Maintenance - -```bash -# Update Rust -rustup update - -# Update Node packages -cd neode-ui && npm update - -# Check PostgreSQL status -brew services list | grep postgresql - -# Check Podman status -podman machine list -``` - ---- - -## ๐ Troubleshooting - -### Port Already in Use - -```bash -# Find what's using the port -lsof -i :5959 # Backend -lsof -i :8100 # Frontend - -# Kill the process -kill -9Application platform for decentralized services
+Flexible server platform for applications and services
+