archy/docs/development-containers.md

Development Container Environment Guide

This guide explains how to develop and test containers in the Archipelago development environment.

Overview

The development server environment enables:

• Testing prepackaged containers (k484 mortar, atob nostrdevs)
• Installing and running containers with port offsetting for dev
• Simulating Bitcoin Core installation and availability
• Supporting both Podman (preferred) and Docker (fallback)

Architecture

┌─────────────────────────────────────────────────────────┐
│              Dev Server Environment                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │  Backend     │  │  Container   │  │  Port        │  │
│  │  (Rust)      │  │  Runtime     │  │  Manager     │  │
│  │              │  │  (Podman/    │  │  (Offset)    │  │
│  │              │  │   Docker)    │  │              │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
│         │                 │                 │          │
│         └─────────────────┼─────────────────┘          │
│                           │                             │
│                  ┌────────▼────────┐                    │
│                  │  Dev Container  │                    │
│                  │  Orchestrator   │                    │
│                  │  - Port offset  │                    │
│                  │  - Bitcoin mock │                    │
│                  │  - Volume dev   │                    │
│                  └─────────────────┘                    │
└─────────────────────────────────────────────────────────┘

Prerequisites

1. Container Runtime: Podman (preferred) or Docker

   • Podman: https://podman.io/getting-started/installation
   • Docker: https://docs.docker.com/get-docker/

2. Rust: Latest stable version

   • Install from: https://rustup.rs/

3. Node.js: v18+ and npm

   • Install from: https://nodejs.org/

Quick Start

1. Check Container Runtime

./scripts/dev-container.sh

This script will:

• Check for Podman and Docker availability
• Show which runtime will be used
• Provide helper commands

2. Start Development Server

./scripts/dev-start.sh
# Choose option 5: Full stack with container support

Or manually:

# Terminal 1: Backend
cd core
ARCHIPELAGO_DEV_MODE=true \
ARCHIPELAGO_CONTAINER_RUNTIME=auto \
ARCHIPELAGO_PORT_OFFSET=10000 \
ARCHIPELAGO_BITCOIN_SIMULATION=mock \
cargo run --bin archipelago

# Terminal 2: Frontend
cd neode-ui
npm run dev

3. Install a Container

Via UI:

1. Open http://localhost:8100
2. Navigate to Marketplace or Apps
3. Install a container app

Via RPC:

curl -X POST http://localhost:5959/rpc/v1 \
  -H "Content-Type: application/json" \
  -d '{
    "method": "container-install",
    "params": {
      "manifest_path": "apps/bitcoin-core/manifest.yml"
    }
  }'

Port Offset Strategy

In development mode, ports are offset by 10000 to prevent conflicts with production services:

Production Port	Dev Port	Example
8332	18332	Bitcoin Core RPC
8333	18333	Bitcoin Core P2P
9735	19735	Lightning Network
8080	18080	Web Services

This is configurable via ARCHIPELAGO_PORT_OFFSET environment variable.

Container Runtime

The system supports three runtime modes:

1. Podman (preferred): Matches production environment
2. Docker: Easier local development
3. Auto: Tries Podman first, falls back to Docker

Set via ARCHIPELAGO_CONTAINER_RUNTIME environment variable.

Bitcoin Simulation

Bitcoin Core dependency can be simulated in three ways:

1. Mock (default): Fast, no actual node required

   • Mocks Bitcoin RPC responses
   • Satisfies dependencies without installation
   • Use for UI and integration testing

2. Testnet: Runs real Bitcoin Core on testnet

   • Slower but more realistic
   • Requires Bitcoin Core container
   • Use for testing Bitcoin integration

3. Mainnet: Runs real Bitcoin Core on mainnet

   • Slowest, most realistic
   • Requires Bitcoin Core container and full sync
   • Use for final testing

4. None: No Bitcoin simulation

   • Apps requiring Bitcoin will fail dependency check
   • Use when testing non-Bitcoin apps

Set via ARCHIPELAGO_BITCOIN_SIMULATION environment variable.

Testing Prepackaged Containers

Test a Single Container

./scripts/test-container.sh <app-id> <package-dir>

Example:

./scripts/test-container.sh k484 ~/k484-package

This script will:

1. Build the container image
2. Create a test manifest
3. Install via RPC
4. Start the container
5. Show status and logs
6. Provide cleanup commands

Test Multiple Containers

./scripts/prepackage-test.sh

This script tests k484 and atob containers if their package directories are found.

Development Data Directories

Container data is stored in isolated directories:

• Location: /tmp/archipelago-dev/{app-id}/
• Purpose: Separate from production data, easy cleanup
• Persistence: Data is preserved between container restarts (optional)

Configure via ARCHIPELAGO_DEV_DATA_DIR environment variable.

RPC Endpoints

All container operations are available via RPC:

Install Container

{
  "method": "container-install",
  "params": {
    "manifest_path": "apps/bitcoin-core/manifest.yml"
  }
}

Start Container

{
  "method": "container-start",
  "params": {
    "app_id": "bitcoin-core"
  }
}

Stop Container

{
  "method": "container-stop",
  "params": {
    "app_id": "bitcoin-core"
  }
}

List Containers

{
  "method": "container-list",
  "params": {}
}

Get Container Status

{
  "method": "container-status",
  "params": {
    "app_id": "bitcoin-core"
  }
}

Get Container Logs

{
  "method": "container-logs",
  "params": {
    "app_id": "bitcoin-core",
    "lines": 100
  }
}

Remove Container

{
  "method": "container-remove",
  "params": {
    "app_id": "bitcoin-core",
    "preserve_data": false
  }
}

Get Health Status

{
  "method": "container-health",
  "params": {}
}

Environment Variables

# Enable dev mode
ARCHIPELAGO_DEV_MODE=true

# Container runtime (podman|docker|auto)
ARCHIPELAGO_CONTAINER_RUNTIME=auto

# Port offset (default: 10000)
ARCHIPELAGO_PORT_OFFSET=10000

# Bitcoin simulation (mock|testnet|mainnet|none)
ARCHIPELAGO_BITCOIN_SIMULATION=mock

# Dev data directory (default: /tmp/archipelago-dev)
ARCHIPELAGO_DEV_DATA_DIR=/tmp/archipelago-dev

# Backend bind address (default: 127.0.0.1:5959)
ARCHIPELAGO_BIND=127.0.0.1:5959

# Log level (default: info)
ARCHIPELAGO_LOG_LEVEL=debug

Helper Commands

List All Containers

podman ps -a
# or
docker ps -a

View Container Logs

podman logs <container-name>
# or
docker logs <container-name>

Stop All Archipelago Containers

podman ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r podman stop
# or
docker ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r docker stop

Remove All Archipelago Containers

podman ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r podman rm -f
# or
docker ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r docker rm -f

Clean Up Dev Data

rm -rf /tmp/archipelago-dev

Troubleshooting

Container Runtime Not Available

Problem: No container runtime available

Solution:

1. Install Podman or Docker
2. Start the daemon:
   • Podman (macOS): podman machine start
   • Docker: Start Docker Desktop or sudo systemctl start docker

Port Already in Use

Problem: Port conflict when starting container

Solution:

1. Change port offset: ARCHIPELAGO_PORT_OFFSET=20000
2. Or stop conflicting service

Bitcoin Dependency Not Satisfied

Problem: App requires Bitcoin Core but simulation is disabled

Solution:

1. Enable Bitcoin simulation: ARCHIPELAGO_BITCOIN_SIMULATION=mock
2. Or install Bitcoin Core container first

Container Fails to Start

Problem: Container exits immediately

Solution:

1. Check logs: container-logs RPC call
2. Verify image exists: podman images or docker images
3. Check manifest configuration
4. Verify port mappings don't conflict

Best Practices

1. Use Mock Bitcoin by Default: Fast iteration, no sync required
2. Test with Real Bitcoin When Needed: Use testnet for integration testing
3. Clean Up Regularly: Remove unused containers and data
4. Check Logs First: Container logs provide detailed error information
5. Use Port Offset: Prevents conflicts with production services
6. Isolate Dev Data: Keep dev and production data separate

Next Steps

• Read App Manifest Specification
• Review Architecture Documentation
• Check Development Setup Guide