archy/docs/development-containers.md

# Development Container Environment Guide

This guide explains how to develop and test containers in the Archipelago development environment.

## Overview

The development server environment enables:
- Testing prepackaged containers (k484 mortar, atob nostrdevs)
- Installing and running containers with port offsetting for dev
- Simulating Bitcoin Core installation and availability
- Supporting both Podman (preferred) and Docker (fallback)

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│              Dev Server Environment                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │  Backend     │  │  Container   │  │  Port        │  │
│  │  (Rust)      │  │  Runtime     │  │  Manager     │  │
│  │              │  │  (Podman/    │  │  (Offset)    │  │
│  │              │  │   Docker)    │  │              │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
│         │                 │                 │          │
│         └─────────────────┼─────────────────┘          │
│                           │                             │
│                  ┌────────▼────────┐                    │
│                  │  Dev Container  │                    │
│                  │  Orchestrator   │                    │
│                  │  - Port offset  │                    │
│                  │  - Bitcoin mock │                    │
│                  │  - Volume dev   │                    │
│                  └─────────────────┘                    │
└─────────────────────────────────────────────────────────┘
```

## Prerequisites

1. **Container Runtime**: Podman (preferred) or Docker
   - Podman: https://podman.io/getting-started/installation
   - Docker: https://docs.docker.com/get-docker/

2. **Rust**: Latest stable version
   - Install from: https://rustup.rs/

3. **Node.js**: v18+ and npm
   - Install from: https://nodejs.org/

## Quick Start

### 1. Check Container Runtime

```bash
./scripts/dev-container.sh
```

This script will:
- Check for Podman and Docker availability
- Show which runtime will be used
- Provide helper commands

### 2. Start Development Server

```bash
./scripts/dev-start.sh
# Choose option 5: Full stack with container support
```

Or manually:

```bash
# Terminal 1: Backend
cd core
ARCHIPELAGO_DEV_MODE=true \
ARCHIPELAGO_CONTAINER_RUNTIME=auto \
ARCHIPELAGO_PORT_OFFSET=10000 \
ARCHIPELAGO_BITCOIN_SIMULATION=mock \
cargo run --bin archipelago

# Terminal 2: Frontend
cd neode-ui
npm run dev
```

### 3. Install a Container

Via UI:
1. Open http://localhost:8100
2. Navigate to Marketplace or Apps
3. Install a container app

Via RPC:
```bash
curl -X POST http://localhost:5959/rpc/v1 \
  -H "Content-Type: application/json" \
  -d '{
    "method": "container-install",
    "params": {
      "manifest_path": "apps/bitcoin-core/manifest.yml"
    }
  }'
```

## Port Offset Strategy

In development mode, ports are offset by 10000 to prevent conflicts with production services:

| Production Port | Dev Port | Example |
|----------------|----------|---------|
| 8332 | 18332 | Bitcoin Core RPC |
| 8333 | 18333 | Bitcoin Core P2P |
| 9735 | 19735 | Lightning Network |
| 8080 | 18080 | Web Services |

This is configurable via `ARCHIPELAGO_PORT_OFFSET` environment variable.

## Container Runtime

The system supports three runtime modes:

1. **Podman** (preferred): Matches production environment
2. **Docker**: Easier local development
3. **Auto**: Tries Podman first, falls back to Docker

Set via `ARCHIPELAGO_CONTAINER_RUNTIME` environment variable.

## Bitcoin Simulation

Bitcoin Core dependency can be simulated in three ways:

1. **Mock** (default): Fast, no actual node required
   - Mocks Bitcoin RPC responses
   - Satisfies dependencies without installation
   - Use for UI and integration testing

2. **Testnet**: Runs real Bitcoin Core on testnet
   - Slower but more realistic
   - Requires Bitcoin Core container
   - Use for testing Bitcoin integration

3. **Mainnet**: Runs real Bitcoin Core on mainnet
   - Slowest, most realistic
   - Requires Bitcoin Core container and full sync
   - Use for final testing

4. **None**: No Bitcoin simulation
   - Apps requiring Bitcoin will fail dependency check
   - Use when testing non-Bitcoin apps

Set via `ARCHIPELAGO_BITCOIN_SIMULATION` environment variable.

## Testing Prepackaged Containers

### Test a Single Container

```bash
./scripts/test-container.sh <app-id> <package-dir>
```

Example:
```bash
./scripts/test-container.sh k484 ~/k484-package
```

This script will:
1. Build the container image
2. Create a test manifest
3. Install via RPC
4. Start the container
5. Show status and logs
6. Provide cleanup commands

### Test Multiple Containers

```bash
./scripts/prepackage-test.sh
```

This script tests k484 and atob containers if their package directories are found.

## Development Data Directories

Container data is stored in isolated directories:

- **Location**: `/tmp/archipelago-dev/{app-id}/`
- **Purpose**: Separate from production data, easy cleanup
- **Persistence**: Data is preserved between container restarts (optional)

Configure via `ARCHIPELAGO_DEV_DATA_DIR` environment variable.

## RPC Endpoints

All container operations are available via RPC:

### Install Container
```json
{
  "method": "container-install",
  "params": {
    "manifest_path": "apps/bitcoin-core/manifest.yml"
  }
}
```

### Start Container
```json
{
  "method": "container-start",
  "params": {
    "app_id": "bitcoin-core"
  }
}
```

### Stop Container
```json
{
  "method": "container-stop",
  "params": {
    "app_id": "bitcoin-core"
  }
}
```

### List Containers
```json
{
  "method": "container-list",
  "params": {}
}
```

### Get Container Status
```json
{
  "method": "container-status",
  "params": {
    "app_id": "bitcoin-core"
  }
}
```

### Get Container Logs
```json
{
  "method": "container-logs",
  "params": {
    "app_id": "bitcoin-core",
    "lines": 100
  }
}
```

### Remove Container
```json
{
  "method": "container-remove",
  "params": {
    "app_id": "bitcoin-core",
    "preserve_data": false
  }
}
```

### Get Health Status
```json
{
  "method": "container-health",
  "params": {}
}
```

## Environment Variables

```bash
# Enable dev mode
ARCHIPELAGO_DEV_MODE=true

# Container runtime (podman|docker|auto)
ARCHIPELAGO_CONTAINER_RUNTIME=auto

# Port offset (default: 10000)
ARCHIPELAGO_PORT_OFFSET=10000

# Bitcoin simulation (mock|testnet|mainnet|none)
ARCHIPELAGO_BITCOIN_SIMULATION=mock

# Dev data directory (default: /tmp/archipelago-dev)
ARCHIPELAGO_DEV_DATA_DIR=/tmp/archipelago-dev

# Backend bind address (default: 127.0.0.1:5959)
ARCHIPELAGO_BIND=127.0.0.1:5959

# Log level (default: info)
ARCHIPELAGO_LOG_LEVEL=debug
```

## Helper Commands

### List All Containers
```bash
podman ps -a
# or
docker ps -a
```

### View Container Logs
```bash
podman logs <container-name>
# or
docker logs <container-name>
```

### Stop All Archipelago Containers
```bash
podman ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r podman stop
# or
docker ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r docker stop
```

### Remove All Archipelago Containers
```bash
podman ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r podman rm -f
# or
docker ps -a --filter 'name=archipelago-' --format '{{.Names}}' | xargs -r docker rm -f
```

### Clean Up Dev Data
```bash
rm -rf /tmp/archipelago-dev
```

## Troubleshooting

### Container Runtime Not Available

**Problem**: `No container runtime available`

**Solution**:
1. Install Podman or Docker
2. Start the daemon:
   - Podman (macOS): `podman machine start`
   - Docker: Start Docker Desktop or `sudo systemctl start docker`

### Port Already in Use

**Problem**: Port conflict when starting container

**Solution**:
1. Change port offset: `ARCHIPELAGO_PORT_OFFSET=20000`
2. Or stop conflicting service

### Bitcoin Dependency Not Satisfied

**Problem**: App requires Bitcoin Core but simulation is disabled

**Solution**:
1. Enable Bitcoin simulation: `ARCHIPELAGO_BITCOIN_SIMULATION=mock`
2. Or install Bitcoin Core container first

### Container Fails to Start

**Problem**: Container exits immediately

**Solution**:
1. Check logs: `container-logs` RPC call
2. Verify image exists: `podman images` or `docker images`
3. Check manifest configuration
4. Verify port mappings don't conflict

## Best Practices

1. **Use Mock Bitcoin by Default**: Fast iteration, no sync required
2. **Test with Real Bitcoin When Needed**: Use testnet for integration testing
3. **Clean Up Regularly**: Remove unused containers and data
4. **Check Logs First**: Container logs provide detailed error information
5. **Use Port Offset**: Prevents conflicts with production services
6. **Isolate Dev Data**: Keep dev and production data separate

## Next Steps

- Read [App Manifest Specification](./app-manifest-spec.md)
- Review [Architecture Documentation](./architecture.md)
- Check [Development Setup Guide](./development-setup.md)