Enhance development workflow and deployment practices for Archipelago

- Updated the Development-Workflow documentation to clarify deployment strategy, emphasizing direct deployment to the live system for testing. - Added detailed instructions for the deployment command, including syncing code, building frontend and backend, and restarting services. - Improved SSH key management section to assist with authentication issues. - Expanded the testing workflow to include steps for checking logs and syncing changes back to the ISO build. - Updated the ISO build integration section to ensure system-level changes are captured for future builds. - Refactored various sections for clarity and completeness, including deployment paths and system configuration files.
2026-02-01 13:24:03 +00:00
parent 00d1af12f0
commit 34fc06726e
28 changed files with 1248 additions and 285 deletions
@@ -0,0 +1,195 @@
+# Live Server to ISO Build Integration Guide
+
+This document explains how to keep the ISO build synchronized with the live development server.
+
+## Development Workflow
+
+### 1. Develop and Test on Live Server
+
+```bash
+# Make changes locally
+vim core/archipelago/src/...
+
+# Deploy to live server for testing
+./scripts/deploy-to-target.sh --live
+
+# Test at http://192.168.1.228
+# Check logs: ssh archipelago@192.168.1.228 'sudo journalctl -u archipelago -f'
+```
+
+### 2. Capture System Changes
+
+When you make system-level changes on the live server (nginx config, systemd service, etc.):
+
+```bash
+cd image-recipe
+./sync-from-live.sh
+```
+
+This automatically captures:
+- `/etc/systemd/system/archipelago.service` → `configs/archipelago.service`
+- `/etc/nginx/sites-available/archipelago` → `configs/nginx-archipelago.conf`
+- `/etc/logrotate.d/archipelago` → `configs/logrotate.conf`
+
+### 3. Build New ISO
+
+```bash
+# Build backend and frontend
+./scripts/build-backend.sh
+./scripts/build-frontend.sh
+
+# Build ISO with latest changes
+./build-debian-iso.sh
+
+# Test in QEMU
+./test-iso-qemu.sh
+```
+
+### 4. Verify Integration
+
+The ISO build script should:
+1. Copy `configs/archipelago.service` to `/etc/systemd/system/`
+2. Copy `configs/nginx-archipelago.conf` to `/etc/nginx/sites-available/archipelago`
+3. Create symlink: `/etc/nginx/sites-enabled/archipelago`
+4. Enable the service: `systemctl enable archipelago`
+5. Install backend to `/usr/local/bin/archipelago`
+6. Install frontend to `/opt/archipelago/web-ui/`
+
+## Critical Configuration Settings
+
+### Backend Service (archipelago.service)
+
+**Must-have settings**:
+```ini
+[Service]
+User=root                                    # Required for root Podman access
+Environment="ARCHIPELAGO_BIND=0.0.0.0:5678"  # Backend API port
+Environment="ARCHIPELAGO_DEV_MODE=true"       # Enable container auto-detection
+```
+
+**Why root?**: The backend must run as root to access containers started with `sudo podman`. Containers in root Podman context are invisible to rootless Podman.
+
+### Nginx Configuration (nginx-archipelago.conf)
+
+**Must-have proxies**:
+```nginx
+location /rpc/ {
+    proxy_pass http://127.0.0.1:5678;  # Backend RPC endpoint
+}
+
+location /ws {
+    proxy_pass http://127.0.0.1:5678;  # WebSocket for real-time updates
+    proxy_http_version 1.1;
+    proxy_set_header Upgrade $http_upgrade;
+    proxy_set_header Connection "upgrade";
+}
+```
+
+## File Paths Reference
+
+### Build Artifacts
+- `build/backend/archipelago` - Compiled Rust backend
+- `build/frontend/` - Built Vue.js frontend
+- `configs/` - System configuration files
+- `results/` - Built ISO images
+
+### Live Server Paths
+- `/usr/local/bin/archipelago` - Backend binary
+- `/opt/archipelago/web-ui/` - Frontend files
+- `/etc/systemd/system/archipelago.service` - Service definition
+- `/etc/nginx/sites-available/archipelago` - Nginx config
+- `/var/lib/archipelago/` - Application data
+
+### ISO Installation Paths
+Same as live server (above) - the ISO must replicate the exact file structure.
+
+## Container Management
+
+### Root vs Rootless Podman
+
+**Current approach**: Root Podman
+- Containers started with: `sudo podman run ...`
+- Backend runs as: `root` user (in systemd)
+- Container detection: Works automatically in dev mode
+
+**Why not rootless?**
+- Would require `User=archipelago` in systemd service
+- All containers must be started as `archipelago` user
+- More complex permission management
+
+### Container Detection
+
+The backend automatically detects running containers when:
+1. `ARCHIPELAGO_DEV_MODE=true` is set
+2. Backend runs with same privileges as container runtime
+3. Containers exist in accessible Podman context
+
+## Troubleshooting
+
+### Issue: Containers not detected in ISO
+
+**Cause**: Backend not running as root, or dev mode disabled
+
+**Fix**: 
+1. Check `configs/archipelago.service` has `User=root`
+2. Check `Environment="ARCHIPELAGO_DEV_MODE=true"` is set
+3. Rebuild ISO and test
+
+### Issue: UI not loading
+
+**Cause**: Nginx config not copied or frontend files missing
+
+**Fix**:
+1. Verify `configs/nginx-archipelago.conf` exists
+2. Check frontend built to `build/frontend/`
+3. Verify ISO build script copies these files
+
+### Issue: Backend won't start
+
+**Cause**: Binary permissions or missing dependencies
+
+**Fix**:
+1. Check backend binary is executable: `chmod +x /usr/local/bin/archipelago`
+2. Check dependencies installed (Podman, nginx)
+3. Review systemd logs: `journalctl -u archipelago`
+
+## Testing Checklist
+
+Before releasing an ISO, verify:
+
+- [ ] Boot ISO in QEMU
+- [ ] Systemd service starts: `systemctl status archipelago`
+- [ ] Backend responds: `curl http://localhost:5678/health`
+- [ ] UI accessible: Open browser to `http://localhost`
+- [ ] Container detection: `sudo podman run -d --name test nginx` → Shows in UI
+- [ ] RPC works: Test login and API calls
+- [ ] WebSocket connects: Check browser console
+
+## Automated Build Pipeline (Future)
+
+To automate this workflow:
+
+1. **CI/CD Integration**
+   - Trigger on main branch commits
+   - Run `sync-from-live.sh` with credentials
+   - Build backend and frontend
+   - Build ISO
+   - Upload to releases
+
+2. **Version Management**
+   - Tag releases with semantic versions
+   - Include git commit hash in ISO metadata
+   - Track which configs were included
+
+3. **Testing Automation**
+   - Boot ISO in headless QEMU
+   - Run API tests
+   - Verify container detection
+   - Generate test report
+
+## Resources
+
+- Development Workflow Rules: `.cursor/rules/Development-Workflow.mdc`
+- Build Checklist: `ISO-BUILD-CHECKLIST.md`
+- Architecture Docs: `.cursor/rules/Architecture.mdc`
+- Deployment Scripts: `scripts/deploy-to-target.sh`
@@ -0,0 +1,145 @@
+# ISO Build Checklist
+
+This checklist ensures that all changes from the live development server are properly integrated into the ISO build.
+
+## Pre-Build Steps
+
+### 1. Sync System Configurations from Live Server
+
+```bash
+cd image-recipe
+./sync-from-live.sh
+```
+
+This captures:
+- [ ] Systemd service configuration (`archipelago.service`)
+- [ ] Nginx configuration (`nginx-archipelago.conf`)
+- [ ] Logrotate configuration (if exists)
+- [ ] Any custom scripts in `/opt/archipelago/scripts/`
+
+### 2. Verify Code Changes
+
+Ensure all code changes are committed:
+- [ ] Backend changes in `core/`
+- [ ] Frontend changes in `neode-ui/`
+- [ ] Script changes in `scripts/`
+
+### 3. Build Components
+
+```bash
+cd image-recipe
+
+# Build backend
+./scripts/build-backend.sh
+
+# Build frontend  
+./scripts/build-frontend.sh
+```
+
+Verify builds:
+- [ ] Backend binary exists: `build/backend/archipelago`
+- [ ] Frontend files exist: `build/frontend/index.html`
+
+## Integration Check
+
+### 4. Update Build Scripts
+
+Review and update if needed:
+- [ ] `integrate-archipelago.sh` - Includes all config files
+- [ ] `build-debian-iso.sh` - Installs to correct paths
+
+### 5. Critical Configuration Values
+
+Verify in `configs/archipelago.service`:
+- [ ] `User=root` (required for Podman root context)
+- [ ] `Environment="ARCHIPELAGO_DEV_MODE=true"` (enables container detection)
+- [ ] `Environment="ARCHIPELAGO_BIND=0.0.0.0:5678"`
+
+Verify in `configs/nginx-archipelago.conf`:
+- [ ] Root path: `/opt/archipelago/web-ui`
+- [ ] RPC proxy: `/rpc/` → `http://127.0.0.1:5678`
+- [ ] WebSocket proxy: `/ws` → `http://127.0.0.1:5678`
+
+## Build Process
+
+### 6. Build the ISO
+
+```bash
+./build-debian-iso.sh
+```
+
+Expected output:
+- [ ] ISO created in `results/` directory
+- [ ] No build errors
+- [ ] File size reasonable (~500MB - 2GB)
+
+### 7. Test in QEMU
+
+```bash
+./test-iso-qemu.sh
+```
+
+Test checklist:
+- [ ] ISO boots successfully
+- [ ] Backend service starts: `systemctl status archipelago`
+- [ ] Nginx serves frontend
+- [ ] Can access UI at `http://localhost:8080` (or mapped port)
+- [ ] Container detection works: Check logs for "Detected container"
+
+## Post-Build
+
+### 8. Write to USB (Optional)
+
+```bash
+./write-usb-dd.sh /dev/diskN
+```
+
+Or use Balena Etcher to flash the ISO.
+
+### 9. Test on Real Hardware
+
+- [ ] Boot from USB
+- [ ] Network configuration works
+- [ ] All services start automatically
+- [ ] Can access web UI
+- [ ] Containers are detected and managed
+
+## Deployment Paths Reference
+
+The ISO build must install to these paths:
+
+| Component | Path | Source |
+|-----------|------|--------|
+| Backend binary | `/usr/local/bin/archipelago` | `build/backend/archipelago` |
+| Frontend files | `/opt/archipelago/web-ui/` | `build/frontend/*` |
+| Systemd service | `/etc/systemd/system/archipelago.service` | `configs/archipelago.service` |
+| Nginx config | `/etc/nginx/sites-available/archipelago` | `configs/nginx-archipelago.conf` |
+| Nginx symlink | `/etc/nginx/sites-enabled/archipelago` | Link to sites-available |
+
+## Common Issues
+
+### Backend Not Detecting Containers
+- Verify service runs as `root` user
+- Check Podman context: `sudo podman ps` should show containers
+- Enable dev mode: `ARCHIPELAGO_DEV_MODE=true`
+
+### UI Not Loading
+- Check nginx configuration paths
+- Verify frontend files deployed to `/opt/archipelago/web-ui/`
+- Check nginx error logs: `/var/log/nginx/error.log`
+
+### Service Fails to Start
+- Check binary permissions: Should be executable
+- Check systemd logs: `journalctl -u archipelago`
+- Test binary manually: `sudo /usr/local/bin/archipelago`
+
+## Version Tracking
+
+When building a new ISO, document:
+- Date: _______________
+- Git commit: _______________
+- Backend version: _______________
+- Frontend version: _______________
+- ISO filename: _______________
+- Tested on hardware: _______________
+- Issues found: _______________
@@ -7,6 +7,14 @@ Build scripts for creating bootable Debian Linux OS images for Archipelago Bitco
 ### Build the ISO

 ```bash
+# 1. Sync latest configs from live dev server
+./sync-from-live.sh
+
+# 2. Build components
+./scripts/build-backend.sh
+./scripts/build-frontend.sh
+
+# 3. Build the ISO
 ./build-debian-iso.sh
 ```

@@ -21,6 +29,8 @@ This creates a bootable Debian Live ISO with Archipelago pre-installed.
 # Or use Balena Etcher to flash the ISO
 ```

+See the **ISO-BUILD-CHECKLIST.md** for a comprehensive build workflow.
+
 See the Architecture documentation for detailed system information.

 ## What's Included
@@ -232,22 +232,53 @@ mkdir -p "$ARCH_DIR/scripts"
 echo "   Including root filesystem..."
 cp "$ROOTFS_TAR" "$ARCH_DIR/rootfs.tar"

-# Copy backend binary
-if [ -f "$SCRIPT_DIR/../core/target/release/archipelago" ]; then
-    echo "   Including backend binary..."
-    cp "$SCRIPT_DIR/../core/target/release/archipelago" "$ARCH_DIR/bin/"
+# Build and copy backend binary
+echo "   Building backend binary for Linux x86_64..."
+BACKEND_DOCKERFILE="$WORK_DIR/Dockerfile.backend"
+cat > "$BACKEND_DOCKERFILE" <<'BACKENDFILE'
+FROM rust:1.93-bookworm as builder
+WORKDIR /build
+COPY core ./core
+RUN cd core && cargo build --release --bin archipelago
+BACKENDFILE
+
+if docker build --platform linux/amd64 -t archipelago-backend -f "$BACKEND_DOCKERFILE" "$SCRIPT_DIR/.." 2>&1 | tail -20; then
+    echo "   Extracting backend binary..."
+    BACKEND_CONTAINER=$(docker create --platform linux/amd64 archipelago-backend)
+    docker cp "$BACKEND_CONTAINER:/build/core/target/release/archipelago" "$ARCH_DIR/bin/" && \
+        echo "   ✅ Backend binary included ($(du -h "$ARCH_DIR/bin/archipelago" | cut -f1))"
+    docker rm "$BACKEND_CONTAINER"
+else
+    echo "   ⚠️  Backend build failed - using existing binary if available"
+    if [ -f "$SCRIPT_DIR/../core/target/release/archipelago" ]; then
+        cp "$SCRIPT_DIR/../core/target/release/archipelago" "$ARCH_DIR/bin/"
+        echo "   Using local backend binary (may not be compatible)"
+    fi
 fi

-# Copy web UI (check both possible locations)
-if [ -d "$SCRIPT_DIR/../web/dist/neode-ui" ]; then
-    echo "   Including web UI from web/dist/neode-ui..."
-    cp -r "$SCRIPT_DIR/../web/dist/neode-ui" "$ARCH_DIR/web-ui"
-elif [ -d "$SCRIPT_DIR/../neode-ui/dist" ]; then
-    echo "   Including web UI from neode-ui/dist..."
-    cp -r "$SCRIPT_DIR/../neode-ui/dist" "$ARCH_DIR/web-ui"
+# Build and copy web UI
+echo "   Building web UI..."
+cd "$SCRIPT_DIR/../neode-ui"
+if npm run build 2>&1 | tail -5; then
+    if [ -d "$SCRIPT_DIR/../web/dist/neode-ui" ]; then
+        echo "   Including web UI from web/dist/neode-ui..."
+        cp -r "$SCRIPT_DIR/../web/dist/neode-ui" "$ARCH_DIR/web-ui"
+        echo "   ✅ Web UI included ($(du -sh "$ARCH_DIR/web-ui" | cut -f1))"
+    fi
 else
-    echo "   ⚠️  Web UI not found - build it first with: cd neode-ui && npm run build"
+    echo "   ⚠️  Web UI build failed"
+    # Try to use existing build
+    if [ -d "$SCRIPT_DIR/../web/dist/neode-ui" ]; then
+        echo "   Using existing web UI build..."
+        cp -r "$SCRIPT_DIR/../web/dist/neode-ui" "$ARCH_DIR/web-ui"
+    elif [ -d "$SCRIPT_DIR/../neode-ui/dist" ]; then
+        echo "   Using neode-ui/dist..."
+        cp -r "$SCRIPT_DIR/../neode-ui/dist" "$ARCH_DIR/web-ui"
+    else
+        echo "   ❌ No web UI available"
+    fi
 fi
+cd "$SCRIPT_DIR"

 # Copy app manifests
 if [ -d "$SCRIPT_DIR/../apps" ]; then
@@ -0,0 +1,16 @@
+[Unit]
+Description=Archipelago Backend
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=root
+Environment="ARCHIPELAGO_BIND=0.0.0.0:5678"
+Environment="ARCHIPELAGO_DEV_MODE=true"
+ExecStart=/usr/local/bin/archipelago
+Restart=on-failure
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,29 @@
+server {
+    listen 80;
+    server_name _;
+    
+    root /opt/archipelago/web-ui;
+    index index.html;
+    
+    # Serve static files (Vue.js SPA)
+    location / {
+        try_files $uri $uri/ /index.html;
+    }
+    
+    # Proxy API requests to backend
+    location /rpc/ {
+        proxy_pass http://127.0.0.1:5678;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+    }
+    
+    # Proxy WebSocket
+    location /ws {
+        proxy_pass http://127.0.0.1:5678;
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection "upgrade";
+        proxy_set_header Host $host;
+    }
+}
@@ -0,0 +1,76 @@
+#!/bin/bash
+# Sync configuration files from live server to ISO build
+#
+# Usage: ./sync-from-live.sh [target-host]
+#
+# This script captures system configuration from the live development
+# server and saves it to the image-recipe/configs/ directory for
+# inclusion in future ISO builds.
+
+set -e
+
+# Configuration
+TARGET_HOST="${1:-archipelago@192.168.1.228}"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+CONFIG_DIR="$SCRIPT_DIR/configs"
+
+echo "╔════════════════════════════════════════════════════════════════╗"
+echo "║  Syncing Configurations from Live Server                      ║"
+echo "╚════════════════════════════════════════════════════════════════╝"
+echo ""
+echo "Target: $TARGET_HOST"
+echo "Output: $CONFIG_DIR"
+echo ""
+
+# Ensure configs directory exists
+mkdir -p "$CONFIG_DIR"
+
+# Sync systemd service
+echo "📋 Capturing systemd service..."
+ssh "$TARGET_HOST" 'sudo cat /etc/systemd/system/archipelago.service' > "$CONFIG_DIR/archipelago.service"
+echo "   ✅ Saved to configs/archipelago.service"
+
+# Sync nginx configuration
+echo "📋 Capturing nginx configuration..."
+ssh "$TARGET_HOST" 'sudo cat /etc/nginx/sites-available/archipelago' > "$CONFIG_DIR/nginx-archipelago.conf"
+echo "   ✅ Saved to configs/nginx-archipelago.conf"
+
+# Sync logrotate if it exists
+if ssh "$TARGET_HOST" 'sudo test -f /etc/logrotate.d/archipelago'; then
+    echo "📋 Capturing logrotate configuration..."
+    ssh "$TARGET_HOST" 'sudo cat /etc/logrotate.d/archipelago' > "$CONFIG_DIR/logrotate.conf"
+    echo "   ✅ Saved to configs/logrotate.conf"
+fi
+
+# Check for custom scripts
+echo ""
+echo "📋 Checking for custom scripts..."
+if ssh "$TARGET_HOST" 'sudo test -d /opt/archipelago/scripts'; then
+    SCRIPT_COUNT=$(ssh "$TARGET_HOST" 'sudo ls /opt/archipelago/scripts/ 2>/dev/null | wc -l' | tr -d ' ')
+    if [ "$SCRIPT_COUNT" -gt 0 ]; then
+        echo "   ⚠️  Found $SCRIPT_COUNT script(s) in /opt/archipelago/scripts/"
+        echo "   Review and manually sync if needed"
+        ssh "$TARGET_HOST" 'sudo ls -lh /opt/archipelago/scripts/'
+    else
+        echo "   ✅ No custom scripts found"
+    fi
+else
+    echo "   ✅ No custom scripts directory"
+fi
+
+# Summary
+echo ""
+echo "╔════════════════════════════════════════════════════════════════╗"
+echo "║  Sync Complete!                                                ║"
+echo "╚════════════════════════════════════════════════════════════════╝"
+echo ""
+echo "Configuration files captured:"
+ls -lh "$CONFIG_DIR"
+echo ""
+echo "Next steps:"
+echo "  1. Review the captured configurations"
+echo "  2. Build backend: ./scripts/build-backend.sh"
+echo "  3. Build frontend: ./scripts/build-frontend.sh"
+echo "  4. Update integration script to use these configs"
+echo "  5. Build ISO: ./build-debian-iso.sh"
+echo ""