archy/docs/hotfix-process.md

Hotfix Process

For critical bugs discovered after a tagged release.

Severity Classification

Level	Response Time	Examples
P0 — Critical	< 4 hours	Data loss, security vulnerability, node bricked
P1 — High	< 24 hours	App won't start, auth broken, major UI failure
P2 — Medium	< 72 hours	Non-critical feature broken, performance regression
P3 — Low	Next release	Cosmetic, minor UX, edge cases

Hotfix Workflow

1. Triage

• Reproduce the issue on dev server (192.168.1.228)
• Classify severity (P0-P3)
• P0/P1: proceed immediately. P2/P3: add to the next release (docs/UNIFIED-TASK-TRACKER.md).

2. Fix

• Create branch: hotfix/vX.Y.Z-description
• Fix the issue with minimal code changes
• Run full test suite: cd neode-ui && npm test && npm run type-check
• Deploy to dev server: ./scripts/deploy-to-target.sh --live
• Verify fix on live server

3. Release

• Merge hotfix branch to main
• Tag: vX.Y.Z (increment patch version)
• Cut the release with ./scripts/create-release.sh X.Y.Z (updates releases/manifest.json and signs it)
• Push main + tags to the primary Gitea release server so nodes pick it up OTA

4. Communicate

• Update RELEASE-NOTES with hotfix details
• Note in CHANGELOG.md

Monitoring Dashboards

• Uptime monitor: /var/lib/archipelago/uptime-monitor/summary.json
• Soak test: /tmp/stability-test-*.log on dev server
• Health endpoint: http://192.168.1.228/health

Rollback

If a hotfix causes regressions:

1. The updater self-verifies after applying (health check on restart) and rolls the binary back automatically if the new one fails to come up
2. Point releases/manifest.json back at the last-known-good version and push
3. Backend binary backups: /opt/archipelago/rollback/archipelago.bak (deploy script) and /var/lib/archipelago/update-backup/archipelago.bak (self-update.sh)