security: remove all infrastructure and internal process material from the repo
Demo images / Build & push demo images (push) Failing after 2m13s
Demo images / Build & push demo images (push) Failing after 2m13s
The repo is source code and guidelines only. Nothing about how Archipelago's own fleet is run, or how the team works, stays in it. Untracked (kept on disk, gitignored) — 250 files: - .planning/ (199) and loop/ — internal development process - fleet operations tooling that targets specific nodes: deploy-to-target, deploy-tailscale, deploy-config-defaults, setup-target-dev, setup-aiui-server, setup-https-dev, debug-frontend, node-profile, fleet-fips-pair/unpair, image-recipe/sync-from-live.sh - image-recipe/INTEGRATION-GUIDE.md and docs/multinode-testing-plan.md, both of which are live-server workflow and fleet node inventories - the Phase 10 on-node verification and evidence records, which cite .planning/ as their evidence base KEY-05-ENTROPY-ENFORCEMENT.md was initially moved out with the other Phase 10 docs and then put back: it is cited as normative rationale from ten places in the codebase, including core/clippy.toml, which bans rand::thread_rng and points at it for the reason. That makes it a guideline, not an internal record. Node names removed from source (48 occurrences across comments, manifests and test fixtures): archi-dev-box, archy-x250*, shorty-s, framework-pt, zaza-optiplex, archi-thinkpad. Comments keep the engineering context and the date, which is what carried the meaning; the machine name did not. Three of those were live test values rather than comments and were replaced with valid stand-ins, not prose: two mDNS hostnames and a mesh peer name. An earlier pass substituted "a test node" into a hostname assertion, producing an invalid hostname; caught and fixed as test-node.local. Wipe mechanism: .local-only/manifest.txt inventories every local-only path and .local-only/wipe.sh deletes them on one confirmation, refusing to touch anything git still tracks. Both are themselves untracked, so the public repo does not carry a map of internal filenames. Verified: cargo check -p archipelago --all-features clean; archipelago-container 75/75 tests pass; appOrigin vitest 7/7; audit-secrets 5/5; every relative link in tracked markdown resolves (0 broken). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
co-authored by
Claude Opus 5
Claude Opus 5
parent
cc00884b98
commit
6ba0599639
@@ -1,195 +0,0 @@
|
||||
# Live Server to ISO Build Integration Guide
|
||||
|
||||
This document explains how to keep the ISO build synchronized with the live development server.
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### 1. Develop and Test on Live Server
|
||||
|
||||
```bash
|
||||
# Make changes locally
|
||||
vim core/archipelago/src/...
|
||||
|
||||
# Deploy to live server for testing
|
||||
./scripts/deploy-to-target.sh --live
|
||||
|
||||
# Test at http://192.168.1.228
|
||||
# Check logs: ssh archipelago@192.168.1.228 'sudo journalctl -u archipelago -f'
|
||||
```
|
||||
|
||||
### 2. Capture System Changes
|
||||
|
||||
When you make system-level changes on the live server (nginx config, systemd service, etc.):
|
||||
|
||||
```bash
|
||||
cd image-recipe
|
||||
./sync-from-live.sh
|
||||
```
|
||||
|
||||
This automatically captures:
|
||||
- `/etc/systemd/system/archipelago.service` → `configs/archipelago.service`
|
||||
- `/etc/nginx/sites-available/archipelago` → `configs/nginx-archipelago.conf`
|
||||
- `/etc/logrotate.d/archipelago` → `configs/logrotate.conf`
|
||||
|
||||
### 3. Build New ISO
|
||||
|
||||
```bash
|
||||
# Build backend and frontend
|
||||
./scripts/build-backend.sh
|
||||
./scripts/build-frontend.sh
|
||||
|
||||
# Build ISO with latest changes
|
||||
./build-debian-iso.sh
|
||||
|
||||
# Test in QEMU
|
||||
./test-iso-qemu.sh
|
||||
```
|
||||
|
||||
### 4. Verify Integration
|
||||
|
||||
The ISO build script should:
|
||||
1. Copy `configs/archipelago.service` to `/etc/systemd/system/`
|
||||
2. Copy `configs/nginx-archipelago.conf` to `/etc/nginx/sites-available/archipelago`
|
||||
3. Create symlink: `/etc/nginx/sites-enabled/archipelago`
|
||||
4. Enable the service: `systemctl enable archipelago`
|
||||
5. Install backend to `/usr/local/bin/archipelago`
|
||||
6. Install frontend to `/opt/archipelago/web-ui/`
|
||||
|
||||
## Critical Configuration Settings
|
||||
|
||||
### Backend Service (archipelago.service)
|
||||
|
||||
**Must-have settings**:
|
||||
```ini
|
||||
[Service]
|
||||
User=root # Required for root Podman access
|
||||
Environment="ARCHIPELAGO_BIND=127.0.0.1:5678" # Backend API port
|
||||
Environment="ARCHIPELAGO_DEV_MODE=true" # Enable container auto-detection
|
||||
```
|
||||
|
||||
**Why root?**: The backend must run as root to access containers started with `sudo podman`. Containers in root Podman context are invisible to rootless Podman.
|
||||
|
||||
### Nginx Configuration (nginx-archipelago.conf)
|
||||
|
||||
**Must-have proxies**:
|
||||
```nginx
|
||||
location /rpc/ {
|
||||
proxy_pass http://127.0.0.1:5678; # Backend RPC endpoint
|
||||
}
|
||||
|
||||
location /ws {
|
||||
proxy_pass http://127.0.0.1:5678; # WebSocket for real-time updates
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
}
|
||||
```
|
||||
|
||||
## File Paths Reference
|
||||
|
||||
### Build Artifacts
|
||||
- `build/backend/archipelago` - Compiled Rust backend
|
||||
- `build/frontend/` - Built Vue.js frontend
|
||||
- `configs/` - System configuration files
|
||||
- `results/` - Built ISO images
|
||||
|
||||
### Live Server Paths
|
||||
- `/usr/local/bin/archipelago` - Backend binary
|
||||
- `/opt/archipelago/web-ui/` - Frontend files
|
||||
- `/etc/systemd/system/archipelago.service` - Service definition
|
||||
- `/etc/nginx/sites-available/archipelago` - Nginx config
|
||||
- `/var/lib/archipelago/` - Application data
|
||||
|
||||
### ISO Installation Paths
|
||||
Same as live server (above) - the ISO must replicate the exact file structure.
|
||||
|
||||
## Container Management
|
||||
|
||||
### Root vs Rootless Podman
|
||||
|
||||
**Current approach**: Root Podman
|
||||
- Containers started with: `sudo podman run ...`
|
||||
- Backend runs as: `root` user (in systemd)
|
||||
- Container detection: Works automatically in dev mode
|
||||
|
||||
**Why not rootless?**
|
||||
- Would require `User=archipelago` in systemd service
|
||||
- All containers must be started as `archipelago` user
|
||||
- More complex permission management
|
||||
|
||||
### Container Detection
|
||||
|
||||
The backend automatically detects running containers when:
|
||||
1. `ARCHIPELAGO_DEV_MODE=true` is set
|
||||
2. Backend runs with same privileges as container runtime
|
||||
3. Containers exist in accessible Podman context
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Containers not detected in ISO
|
||||
|
||||
**Cause**: Backend not running as root, or dev mode disabled
|
||||
|
||||
**Fix**:
|
||||
1. Check `configs/archipelago.service` has `User=root`
|
||||
2. Check `Environment="ARCHIPELAGO_DEV_MODE=true"` is set
|
||||
3. Rebuild ISO and test
|
||||
|
||||
### Issue: UI not loading
|
||||
|
||||
**Cause**: Nginx config not copied or frontend files missing
|
||||
|
||||
**Fix**:
|
||||
1. Verify `configs/nginx-archipelago.conf` exists
|
||||
2. Check frontend built to `build/frontend/`
|
||||
3. Verify ISO build script copies these files
|
||||
|
||||
### Issue: Backend won't start
|
||||
|
||||
**Cause**: Binary permissions or missing dependencies
|
||||
|
||||
**Fix**:
|
||||
1. Check backend binary is executable: `chmod +x /usr/local/bin/archipelago`
|
||||
2. Check dependencies installed (Podman, nginx)
|
||||
3. Review systemd logs: `journalctl -u archipelago`
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
Before releasing an ISO, verify:
|
||||
|
||||
- [ ] Boot ISO in QEMU
|
||||
- [ ] Systemd service starts: `systemctl status archipelago`
|
||||
- [ ] Backend responds: `curl http://localhost:5678/health`
|
||||
- [ ] UI accessible: Open browser to `http://localhost`
|
||||
- [ ] Container detection: `sudo podman run -d --name test nginx` → Shows in UI
|
||||
- [ ] RPC works: Test login and API calls
|
||||
- [ ] WebSocket connects: Check browser console
|
||||
|
||||
## Automated Build Pipeline (Future)
|
||||
|
||||
To automate this workflow:
|
||||
|
||||
1. **CI/CD Integration**
|
||||
- Trigger on main branch commits
|
||||
- Run `sync-from-live.sh` with credentials
|
||||
- Build backend and frontend
|
||||
- Build ISO
|
||||
- Upload to releases
|
||||
|
||||
2. **Version Management**
|
||||
- Tag releases with semantic versions
|
||||
- Include git commit hash in ISO metadata
|
||||
- Track which configs were included
|
||||
|
||||
3. **Testing Automation**
|
||||
- Boot ISO in headless QEMU
|
||||
- Run API tests
|
||||
- Verify container detection
|
||||
- Generate test report
|
||||
|
||||
## Resources
|
||||
|
||||
- Development Workflow Rules: `.cursor/rules/Development-Workflow.mdc`
|
||||
- Build Checklist: `ISO-BUILD-CHECKLIST.md`
|
||||
- Architecture Docs: `.cursor/rules/Architecture.mdc`
|
||||
- Deployment Scripts: `scripts/deploy-to-target.sh`
|
||||
@@ -8,7 +8,6 @@ Build scripts for creating bootable Debian Linux OS images for Archipelago Bitco
|
||||
|
||||
```bash
|
||||
# 1. Sync latest configs from live dev server
|
||||
./sync-from-live.sh
|
||||
|
||||
# 2. Build components
|
||||
./scripts/build-backend.sh
|
||||
|
||||
@@ -8,7 +8,6 @@ This checklist ensures that all changes from the live development server are pro
|
||||
|
||||
```bash
|
||||
cd image-recipe
|
||||
./sync-from-live.sh
|
||||
```
|
||||
|
||||
This captures:
|
||||
|
||||
@@ -1257,7 +1257,7 @@ fi
|
||||
# copies everything in archipelago/bin/ to /usr/local/bin, and the mesh
|
||||
# listener spawns /usr/local/bin/archy-reticulum-daemon for RNode radios —
|
||||
# a node imaged without it can never connect a Reticulum stick
|
||||
# (framework-pt, 2026-07-22: silent connect failures until hand-copied).
|
||||
# (a test node, 2026-07-22: silent connect failures until hand-copied).
|
||||
RETICULUM_DAEMON="${ARCHY_RETICULUM_DAEMON:-/usr/local/bin/archy-reticulum-daemon}"
|
||||
if [ -f "$RETICULUM_DAEMON" ]; then
|
||||
cp "$RETICULUM_DAEMON" "$ARCH_DIR/bin/archy-reticulum-daemon"
|
||||
@@ -1270,7 +1270,7 @@ fi
|
||||
# archy-rnodeconf drives the in-app "Flash LoRa" flow for RNode firmware
|
||||
# (mesh/flash.rs spawns /usr/local/bin/archy-rnodeconf --autoinstall). A node
|
||||
# imaged without it fails every RNode flash with "No such file or directory"
|
||||
# (framework-pt, 2026-07-29, v1.7.117).
|
||||
# (a test node, 2026-07-29, v1.7.117).
|
||||
RNODECONF="${ARCHY_RNODECONF:-/usr/local/bin/archy-rnodeconf}"
|
||||
if [ -f "$RNODECONF" ]; then
|
||||
cp "$RNODECONF" "$ARCH_DIR/bin/archy-rnodeconf"
|
||||
|
||||
@@ -13,7 +13,7 @@ Type=oneshot
|
||||
# /var/lib/archipelago/host-secrets-audit.json, all of which are root-owned.
|
||||
User=root
|
||||
# DETECT ONLY. D-06 chose detect-report-then-apply
|
||||
# (docs/security/KEY-02-FLEET-ROTATION.md): rotation is one-way and must never
|
||||
# rotation is one-way and must never
|
||||
# fire unattended across the fleet during an OTA. There is deliberately NO
|
||||
# --apply here. Adding one is a decision, not a configuration change.
|
||||
ExecStart=-/opt/archipelago/scripts/security/host-secrets-audit.sh --detect
|
||||
|
||||
@@ -34,7 +34,7 @@ RestartSec=5
|
||||
# also binds the chromium/Xorg children in this unit's cgroup.
|
||||
# CPUQuota=75% (0.75 cores) was too tight even for normal playback — the kiosk
|
||||
# was throttled ~40% of the time, which is what caused choppy HDMI audio on
|
||||
# archy-x250-exp (2026-06-28 incident). 200% (2 cores) gives enough headroom.
|
||||
# a test node (2026-06-28 incident). 200% (2 cores) gives enough headroom.
|
||||
Delegate=yes
|
||||
CPUQuota=200%
|
||||
# Raised from 1500M/1200M: a Framework (Tiger Lake) kiosk sat at 806M used /
|
||||
|
||||
@@ -32,7 +32,7 @@ ExecStartPre=+/bin/bash -c 'mkdir -p /var/lib/archipelago && chown archipelago:a
|
||||
ExecStartPre=+-/opt/archipelago/scripts/ota-crash-guard.sh
|
||||
ExecStart=/usr/local/bin/archipelago
|
||||
# always (not on-failure): the OTA restart path once stopped the daemon
|
||||
# cleanly and the queued start never fired (framework-pt, v1.7.114->115,
|
||||
# cleanly and the queued start never fired (a test node, v1.7.114->115,
|
||||
# 2026-07-26) — the node sat dead all night behind "server starting up".
|
||||
# Restart=always self-heals any lost start job; an explicit
|
||||
# `systemctl stop` is still honored (systemd never auto-restarts after
|
||||
|
||||
@@ -11,7 +11,7 @@ server {
|
||||
listen 80 default_server;
|
||||
# IPv6 listener is REQUIRED: companion phones reach this node over the
|
||||
# FIPS mesh at its fips0 ULA (http://[fdxx:…]) — without [::]:80 that
|
||||
# address can never connect (found live 2026-07-23, framework-pt).
|
||||
# address can never connect (found live 2026-07-23, a test node).
|
||||
listen [::]:80 default_server;
|
||||
server_name _;
|
||||
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Sync configuration files from live server to ISO build
|
||||
#
|
||||
# Usage: ./sync-from-live.sh [target-host]
|
||||
#
|
||||
# This script captures system configuration from the live development
|
||||
# server and saves it to the image-recipe/configs/ directory for
|
||||
# inclusion in future ISO builds.
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
TARGET_HOST="${1:-archipelago@192.168.1.228}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CONFIG_DIR="$SCRIPT_DIR/configs"
|
||||
|
||||
echo "╔════════════════════════════════════════════════════════════════╗"
|
||||
echo "║ Syncing Configurations from Live Server ║"
|
||||
echo "╚════════════════════════════════════════════════════════════════╝"
|
||||
echo ""
|
||||
echo "Target: $TARGET_HOST"
|
||||
echo "Output: $CONFIG_DIR"
|
||||
echo ""
|
||||
|
||||
# Ensure configs directory exists
|
||||
mkdir -p "$CONFIG_DIR"
|
||||
|
||||
# Sync systemd service
|
||||
echo "📋 Capturing systemd service..."
|
||||
ssh "$TARGET_HOST" 'sudo cat /etc/systemd/system/archipelago.service' > "$CONFIG_DIR/archipelago.service"
|
||||
echo " ✅ Saved to configs/archipelago.service"
|
||||
|
||||
# Sync nginx configuration
|
||||
echo "📋 Capturing nginx configuration..."
|
||||
ssh "$TARGET_HOST" 'sudo cat /etc/nginx/sites-available/archipelago' > "$CONFIG_DIR/nginx-archipelago.conf"
|
||||
echo " ✅ Saved to configs/nginx-archipelago.conf"
|
||||
|
||||
# Sync logrotate if it exists
|
||||
if ssh "$TARGET_HOST" 'sudo test -f /etc/logrotate.d/archipelago'; then
|
||||
echo "📋 Capturing logrotate configuration..."
|
||||
ssh "$TARGET_HOST" 'sudo cat /etc/logrotate.d/archipelago' > "$CONFIG_DIR/logrotate.conf"
|
||||
echo " ✅ Saved to configs/logrotate.conf"
|
||||
fi
|
||||
|
||||
# Check for custom scripts
|
||||
echo ""
|
||||
echo "📋 Checking for custom scripts..."
|
||||
if ssh "$TARGET_HOST" 'sudo test -d /opt/archipelago/scripts'; then
|
||||
SCRIPT_COUNT=$(ssh "$TARGET_HOST" 'sudo ls /opt/archipelago/scripts/ 2>/dev/null | wc -l' | tr -d ' ')
|
||||
if [ "$SCRIPT_COUNT" -gt 0 ]; then
|
||||
echo " ⚠️ Found $SCRIPT_COUNT script(s) in /opt/archipelago/scripts/"
|
||||
echo " Review and manually sync if needed"
|
||||
ssh "$TARGET_HOST" 'sudo ls -lh /opt/archipelago/scripts/'
|
||||
else
|
||||
echo " ✅ No custom scripts found"
|
||||
fi
|
||||
else
|
||||
echo " ✅ No custom scripts directory"
|
||||
fi
|
||||
|
||||
# Summary
|
||||
echo ""
|
||||
echo "╔════════════════════════════════════════════════════════════════╗"
|
||||
echo "║ Sync Complete! ║"
|
||||
echo "╚════════════════════════════════════════════════════════════════╝"
|
||||
echo ""
|
||||
echo "Configuration files captured:"
|
||||
ls -lh "$CONFIG_DIR"
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo " 1. Review the captured configurations"
|
||||
echo " 2. Build backend: ./scripts/build-backend.sh"
|
||||
echo " 3. Build frontend: ./scripts/build-frontend.sh"
|
||||
echo " 4. Update integration script to use these configs"
|
||||
echo " 5. Build ISO: ./build-debian-iso.sh"
|
||||
echo ""
|
||||
Reference in New Issue
Block a user