archy/docs/hotfix-process.md

# Hotfix Process

For critical bugs discovered after a tagged release.

## Severity Classification

| Level | Response Time | Examples |
|-------|--------------|---------|
| P0 — Critical | < 4 hours | Data loss, security vulnerability, node bricked |
| P1 — High | < 24 hours | App won't start, auth broken, major UI failure |
| P2 — Medium | < 72 hours | Non-critical feature broken, performance regression |
| P3 — Low | Next release | Cosmetic, minor UX, edge cases |

## Hotfix Workflow

### 1. Triage
- Reproduce the issue on dev server (192.168.1.228)
- Classify severity (P0-P3)
- P0/P1: proceed immediately. P2/P3: add to the next release (`docs/UNIFIED-TASK-TRACKER.md`).

### 2. Fix
- Create branch: `hotfix/vX.Y.Z-description`
- Fix the issue with minimal code changes
- Run full test suite: `cd neode-ui && npm test && npm run type-check`
- Deploy to dev server: `./scripts/deploy-to-target.sh --live`
- Verify fix on live server

### 3. Release
- Merge hotfix branch to `main`
- Tag: `vX.Y.Z` (increment patch version)
- Cut the release with `./scripts/create-release.sh X.Y.Z` (updates
  `releases/manifest.json` and signs it)
- Push `main` + tags to the primary Gitea release server so nodes pick it up OTA

### 4. Communicate
- Update RELEASE-NOTES with hotfix details
- Note in CHANGELOG.md

## Monitoring Dashboards

- **Uptime monitor**: `/var/lib/archipelago/uptime-monitor/summary.json`
- **Soak test**: `/tmp/stability-test-*.log` on dev server
- **Health endpoint**: `http://192.168.1.228/health`

## Rollback

If a hotfix causes regressions:
1. The updater self-verifies after applying (health check on restart) and rolls the
   binary back automatically if the new one fails to come up
2. Point `releases/manifest.json` back at the last-known-good version and push
3. Backend binary backups: `/opt/archipelago/rollback/archipelago.bak` (deploy script)
   and `/var/lib/archipelago/update-backup/archipelago.bak` (`self-update.sh`)