botfights/loops/plan.md

Overnight Plan — Production Hardening (2-Month Roadmap)

Harden botfights for production: database indexes, memory leak fixes, payment security, Nostr auth bugs, form validation, observability, deployment. Server: /Users/dorian/projects/botfights/server/ — Write/Edit tools blocked by hooks; use Bash heredoc for server file writes. Frontend: /Users/dorian/Projects/botfights/frontend/ — standard Write/Edit tools work. Follow CLAUDE.md conventions: <script setup lang="ts">, conventional commits, no external code copying. Run pnpm typecheck from project root after each phase to verify.

---

Phase 1: Database Indexes (Critical Performance)

• Add SQLite indexes via migration: In server/src/db/startup.ts, add a new migration block after existing migrations. Use the existing pattern of try { sqlite.exec(...) } catch {} for each CREATE INDEX IF NOT EXISTS statement. Add these indexes:

  CREATE INDEX IF NOT EXISTS idx_fights_status ON fights(status);
  CREATE INDEX IF NOT EXISTS idx_fights_bot_a ON fights(bot_a_id);
  CREATE INDEX IF NOT EXISTS idx_fights_bot_b ON fights(bot_b_id);
  CREATE INDEX IF NOT EXISTS idx_fights_created ON fights(created_at);
  CREATE INDEX IF NOT EXISTS idx_fights_winner ON fights(winner_id);
  CREATE INDEX IF NOT EXISTS idx_fights_mode ON fights(mode);
  CREATE INDEX IF NOT EXISTS idx_rounds_fight ON rounds(fight_id);
  CREATE INDEX IF NOT EXISTS idx_payments_fight ON payments(fight_id);
  CREATE INDEX IF NOT EXISTS idx_payments_bot ON payments(bot_id);
  CREATE INDEX IF NOT EXISTS idx_payments_status ON payments(status);
  CREATE INDEX IF NOT EXISTS idx_bets_fight ON bets(fight_id);
  CREATE INDEX IF NOT EXISTS idx_bets_bettor ON bets(bettor_pubkey);
  CREATE INDEX IF NOT EXISTS idx_bets_status ON bets(status);
  CREATE INDEX IF NOT EXISTS idx_bots_pubkey ON bots(public_key);
  CREATE INDEX IF NOT EXISTS idx_bots_elo ON bots(elo_rating);
  CREATE INDEX IF NOT EXISTS idx_bots_active ON bots(is_active);
  CREATE INDEX IF NOT EXISTS idx_bots_type ON bots(bot_type);
  CREATE INDEX IF NOT EXISTS idx_bots_name_lower ON bots(LOWER(name));
  CREATE INDEX IF NOT EXISTS idx_wallet_bot ON wallet_connections(bot_id);
  

  Add a log line: console.log('[botfights] database indexes ensured'). Verify: cd server && npx tsx src/index.ts starts without errors, then Ctrl+C.

• Enable WAL checkpoint strategy: In server/src/db/index.ts, after the existing PRAGMA journal_mode=WAL and PRAGMA foreign_keys=ON, add these pragmas to the sqlite instance:

  PRAGMA wal_autocheckpoint = 1000;
  PRAGMA busy_timeout = 5000;
  PRAGMA cache_size = -20000;
  PRAGMA mmap_size = 268435456;
  

  These improve concurrent read performance and prevent WAL file growth. busy_timeout = 5000 prevents SQLITE_BUSY errors under load. cache_size = -20000 means 20MB page cache. mmap_size enables memory-mapped I/O for reads. Verify: cd server && npx tsx src/index.ts starts without errors.

---

Phase 2: Memory Leak Fixes

• Rate limiter TTL cleanup: In server/src/middleware/rate-limit.ts, the rateLimit() function (line 14) uses an in-memory Map hitCounts with a 5-minute cleanup interval. The cleanup loop (setInterval) runs every 5 minutes but only cleans entries older than the window. Problem: the botRateLimit() function (line 39) uses a Map cooldowns with NO cleanup at all — entries persist forever. Fix: Add a cleanup interval to botRateLimit that runs every 60 seconds and deletes entries older than cooldownMs * 2. Also, cap the hitCounts Map size: if it exceeds 10,000 entries, clear the oldest half. Verify: cd server && npx tsc --noEmit passes.

• Active fighters Set leak prevention: In server/src/engine/orchestrator.ts, the activeFighters Set (around line 37) tracks bots currently in fights but relies on finally blocks for cleanup. Add a safety mechanism: create a cleanupStaleFighters() function that runs every 60 seconds. It should check each botId in activeFighters against the fights table — if no fight with status='live' exists for that bot, remove it from the Set. Call this on a setInterval inside the existing cleanupOrphanedFights() function or alongside it in app.ts. Log when stale entries are cleaned: console.log('[orchestrator] cleaned stale fighter: ${botId}'). Verify: cd server && npx tsc --noEmit passes.

• Betting escrow memory persistence: In server/src/engine/betting.ts, the in-memory escrow Map stores bet data that would be lost on server restart. Add a recoverEscrow() function that runs on startup: query bets table for rows with status = 'locked' and settled_at IS NULL, rebuild the escrow Map from these rows. Export and call this function from server/src/index.ts after seedClassicBots(). Also add a cleanupSettledEscrow() function that runs every 5 minutes and removes entries from the escrow Map where all bets for that fightId have been settled (status not 'locked' or 'pending'). Verify: cd server && npx tsc --noEmit passes.

---

Phase 3: Nostr Auth — Fix Stale Profile Bug

• Clear wallet state on key change: In frontend/src/composables/useNostr.ts, the clearAllState() function (around line 69-76) clears pubkey, bot, and profilePicUrl but does NOT clear wallet-related localStorage keys. When a user generates a new identity, their old wallet connection persists in bf_wallet_method and bf_nwc_url. Fix: In clearAllState(), add these lines after the existing clears:

  localStorage.removeItem('bf_wallet_method')
  localStorage.removeItem('bf_nwc_url')
  

  This ensures wallet state from a previous identity doesn't bleed into a new one.

• Validate stored bot belongs to current pubkey on restore: In frontend/src/composables/useNostr.ts, the auto-restore logic (around line 83-105) loads bf_bot from localStorage and trusts it without checking if it matches the current pubkey. Fix: In the auto-restore function, after loading the stored bot and calling POST /api/auth/login, validate that the returned bot matches what's in localStorage. If the server returns exists: false but localStorage has a bot, clear the stale bot:

  if (!data.exists && bot.value) {
    bot.value = null
    store('bf_bot', null)
    store('bf_pic', null)
  }
  

  Also, if the server returns a different bot ID than what's stored, update localStorage to match the server response. This prevents showing old fighter data after key regeneration. Verify: cd frontend && npx vue-tsc --noEmit passes.

• Clear Nostr profile pic cache on identity change: In frontend/src/composables/useNostr.ts, the generateLogin() function (around line 175-193) calls clearAllState() which clears bf_pic, but there's a subtle race condition: if fetchNostrProfilePic() is still running from a previous login when a new key is generated, it could write the OLD key's profile pic into the new session. Fix: Add a generation counter (simple integer ref that increments in generateLogin() and loginWithNsec()). In fetchNostrProfilePic(), capture the counter value at the start and check it hasn't changed before writing the result:

  const generation = ref(0)
  // In generateLogin():
  generation.value++
  // In fetchNostrProfilePic():
  const gen = generation.value
  // ... after relay fetch ...
  if (generation.value !== gen) return // identity changed during fetch
  

  Verify: cd frontend && npx vue-tsc --noEmit passes.

• Force profile refetch on pubkey change in BotProfilePage: In frontend/src/pages/BotProfilePage.vue, the profile data is loaded once in onMounted() (around line 265-276) with no reactive watcher on the current pubkey. If the user switches identity without a page reload, stale data shows. Fix: Import watch from vue and add a watcher on pubkey from useNostr():

  watch(() => pubkey.value, async (newPk) => {
    // Refetch stats with new pubkey
    const qs = newPk ? `?pubkey=${encodeURIComponent(newPk)}` : ''
    const res = await fetch(`/api/bots/${encodeURIComponent(botName)}/stats${qs}`)
    if (res.ok) stats.value = await res.json()
  })
  

  Verify: cd frontend && npx vue-tsc --noEmit passes.

---

Phase 4: Form Validation Hardening

• Unified name validation with real-time feedback: In frontend/src/pages/JoinBoutPage.vue, the bot name input (around line 608-617) and human name input (around line 866-874) both validate on submit only. Add real-time validation: create a computed nameError that checks: (1) length 2-12, (2) alphanumeric + -_ only via /^[a-zA-Z0-9_-]+$/, (3) no leading/trailing hyphens or underscores. Show the error message below the input in red text (text-red-400 text-xs mt-1). Disable the confirm button when nameError is truthy. Apply to both botName and humanName fields. Also trim whitespace on input (use v-model.trim). Verify: cd frontend && npx vue-tsc --noEmit passes.

• Webhook URL validation with protocol enforcement: In frontend/src/pages/JoinBoutPage.vue, the webhook input (around line 781-788) only validates with new URL() on submit. Add real-time validation: create a computed webhookError that checks: (1) must start with https:// or http:// (show "URL must start with https://"), (2) must be valid URL, (3) must not point to localhost/127.0.0.1/private IPs (mirror isAllowedWebhookUrl check from server/src/engine/orchestrator.ts). Show error below input. Add a "TEST" button next to the URL input that calls the existing test endpoint and shows latency or error inline. Verify: cd frontend && npx vue-tsc --noEmit passes.

• Server-side validation tightening: In server/src/routes/auth.ts, harden all three registration endpoints (/register, /register-human, /login):

  1. Add input sanitization: trim all string inputs, reject strings with null bytes (\0), reject names that are all underscores/hyphens.
  2. In /register (line 86): add webhookUrl length limit (max 2048 chars). Add archetype validation against a known list (import archetype names or validate it's a non-empty alphanumeric string under 32 chars). Add profilePicUrl length limit (max 2048 chars) and validate it starts with https://.
  3. In /register-human (line 185): add avatarSeed validation (alphanumeric, max 32 chars).
  4. In /update (line 247): add the same profilePicUrl validation (https only, max 2048). Verify: cd server && npx tsc --noEmit passes.

• NWC connection string validation: In frontend/src/composables/useWallet.ts, the connectNWC() function (around line 49-75) sends the NWC connection string directly to the server without client-side validation. Add validation before the API call: (1) must start with nostr+walletconnect://, (2) must contain relay= parameter, (3) must contain secret= parameter, (4) secret must be 64-char hex. Return a descriptive error if validation fails. Also in frontend/src/components/WalletConnect.vue, show these validation errors inline below the NWC paste field. Verify: cd frontend && npx vue-tsc --noEmit passes.

---

Phase 5: Payment Security Hardening

• Add pubkey auth to all payment endpoints: In server/src/routes/payments.ts, several endpoints lack proper authentication. Fix:

  1. POST /create-invoice (line 86): require pubkey in body, verify bot belongs to that pubkey before creating invoice.
  2. GET /winnings/:botId (line 162): require pubkey query param, verify bot belongs to that pubkey. Currently returns all Cashu tokens for any botId without auth — this is a critical vulnerability.
  3. POST /submit-cashu (line 144): require pubkey in body, verify bot belongs to that pubkey.
  4. DELETE /disconnect-wallet (line 212): already has pubkey, but add explicit length/format check (64-char hex). Add a helper function verifyBotOwnership(pubkey: string, botId: string): Promise<boolean> to reduce code duplication. Verify: cd server && npx tsc --noEmit passes.

• Rate limit payment endpoints: In server/src/routes/payments.ts, add rate limiting to prevent abuse:

  1. POST /connect-wallet: rateLimit(3600_000, 10) — 10 connections per hour per IP
  2. POST /create-invoice: rateLimit(60_000, 5) — 5 invoices per minute per IP
  3. POST /submit-cashu: rateLimit(60_000, 5) — 5 redemptions per minute per IP
  4. POST /claim/:paymentId: rateLimit(60_000, 10) — 10 claims per minute per IP
  5. POST /confirm/:paymentId: rateLimit(60_000, 10) — 10 confirms per minute per IP Import rateLimit from ../middleware/rate-limit.js. Verify: cd server && npx tsc --noEmit passes.

• NWC relay connection pooling and timeout hardening: In server/src/engine/payments.ts, the nwcRequest() function (around line 300+) opens a new WebSocket to the NWC relay for every request with a 30-second timeout. This blocks the fight completion and can leak connections. Fix:

  1. Create a RelayPool class that maintains a single WebSocket per relay URL, reconnects on disconnect, and queues requests.
  2. Reduce the NWC response timeout from 30s to 10s.
  3. Add a nwcRequestWithRetry(method, params, retries=2) wrapper that retries on timeout with exponential backoff (2s, 4s).
  4. In payWinner(), run the payout in a setTimeout(0) / queueMicrotask so it doesn't block fight completion — the fight result should be returned immediately while payout happens asynchronously. Update fight.payoutStatus to 'pending' before returning, then 'paid' or 'failed' after the async payout completes. Verify: cd server && npx tsc --noEmit passes.

• Cashu token verification hardening: In server/src/engine/betting.ts, the placeBet() function verifies Cashu tokens but may not handle all failure modes. In server/src/engine/payments.ts, the redeemCashuToken() function should also be hardened. Fix:

  1. Add token amount verification: decode the Cashu token and verify the sum of proofs matches amountSats before accepting.
  2. Add mint URL validation: only accept tokens from the configured BOTFIGHTS_CASHU_MINT_URL. Reject tokens from unknown mints.
  3. Add double-spend protection: before accepting a token, check if any proof secret in the token already exists in the payments or bets table (store proof secrets on successful redemption).
  4. Wrap all Cashu operations in try/catch with specific error messages (network error, invalid token, insufficient amount, wrong mint, double spend). Verify: cd server && npx tsc --noEmit passes.

• Payment idempotency keys: In server/src/routes/payments.ts, add idempotency protection:

  1. POST /create-invoice: Accept an optional idempotencyKey in the body. Before creating a new invoice, check if a payment with the same idempotency key exists. If so, return the existing invoice instead of creating a duplicate. Store the idempotency key in a new column on the payments table (add migration in startup.ts: ALTER TABLE payments ADD COLUMN idempotency_key TEXT).
  2. POST /confirm/:paymentId: Already idempotent (returns existing status if already confirmed). No change needed.
  3. POST /submit-cashu: Add idempotency key check similar to create-invoice. Verify: cd server && npx tsc --noEmit passes.

---

Phase 6: Concurrent Fight Limits & Reliability

• Global concurrent fight cap: In server/src/engine/orchestrator.ts, there is no limit on simultaneous fights. Add a constant MAX_CONCURRENT_FIGHTS = 20 at the top of the file. In runFightAsync(), before starting a fight, check activeFighters.size / 2 (each fight has 2 fighters). If >= MAX_CONCURRENT_FIGHTS, reject with error 'Server at capacity — try again in a moment.'. Also add a getActiveFightCount() export that returns Math.floor(activeFighters.size / 2). In server/src/routes/fights.ts, use this in the matchmake and practice endpoints to return 503 with retry-after header when at capacity. Verify: cd server && npx tsc --noEmit passes.

• Webhook circuit breaker: In server/src/engine/orchestrator.ts, bots are deactivated after 5 consecutive webhook errors (consecutiveErrors >= 5). But there's no recovery path — once deactivated, only manual /update re-enables. Add an automatic recovery check: in trackWebhookResult(), if a bot has consecutiveErrors >= 5 and isActive = false, and lastErrorAt is more than 1 hour ago, attempt a single test call to the webhook. If it succeeds, reset consecutiveErrors = 0 and isActive = true. Log: console.log('[orchestrator] bot ${botId} auto-recovered after webhook fix'). Cap the check to once per hour per bot using a simple Map with timestamps. Verify: cd server && npx tsc --noEmit passes.

• Graceful shutdown: In server/src/index.ts, add SIGTERM and SIGINT handlers that:

  1. Call stopBackgroundFights() from server/src/engine/background.ts
  2. Wait up to 30 seconds for activeFighters Set to empty (poll every 1s)
  3. Close the database connection: sqlite.close() from server/src/db/index.ts (export the sqlite instance if not already exported)
  4. Log [botfights] graceful shutdown complete
  5. process.exit(0) This prevents data corruption on deploy/restart. Verify: cd server && npx tsc --noEmit passes.

• Fight timeout enforcement: In server/src/engine/orchestrator.ts, the executeFightRounds() function has per-round timeouts but no overall fight timeout. If a fight hangs (e.g., webhook returns slowly for every round), it could block resources indefinitely. Add an overall fight timeout of 5 minutes: wrap the round loop in a Promise.race() with a 5-minute timer. On timeout, cancel the fight (status = 'cancelled'), clean up activeFighters, and log [orchestrator] fight ${fightId} timed out after 5 minutes. Verify: cd server && npx tsc --noEmit passes.

---

Phase 7: Security Hardening

• CORS origin restriction: In server/src/app.ts, CORS is currently set to '*' (all origins) via the CORS_ORIGIN env var. For production, this should restrict to the actual frontend domain. Fix: In app.ts, where the CORS middleware is configured, change the default from '*' to check NODE_ENV: if production, require CORS_ORIGIN to be explicitly set and warn if it's '*'. Log: console.warn('[botfights] WARNING: CORS_ORIGIN is *, set to your domain in production'). Also add the CORS headers Access-Control-Allow-Credentials: true and restrict Access-Control-Allow-Methods to GET, POST, DELETE, OPTIONS. Verify: cd server && npx tsc --noEmit passes.

• Request body size limits: In server/src/app.ts, there is no request body size limit. A malicious client could send a multi-GB body and crash the server. Fix: Add a body size limit middleware before the routes. In Hono, use the built-in body limit middleware: app.use('/api/*', bodyLimit({ maxSize: 64 * 1024 })) (64KB max). Import bodyLimit from hono/body-limit. For the webhook test endpoint specifically, the 10KB limit is already in the orchestrator, but the route itself should also enforce it. Verify: cd server && npx tsc --noEmit passes.

• Helmet-style security headers: In server/src/app.ts, add a middleware that sets security headers on all responses:

  app.use('*', async (c, next) => {
    await next()
    c.header('X-Content-Type-Options', 'nosniff')
    c.header('X-Frame-Options', 'DENY')
    c.header('X-XSS-Protection', '0')
    c.header('Referrer-Policy', 'strict-origin-when-cross-origin')
    c.header('Permissions-Policy', 'camera=(), microphone=(), geolocation=()')
  })
  

  Place this BEFORE the CORS middleware so it applies to all responses. Verify: cd server && npx tsc --noEmit passes.

• Webhook SSRF hardening: In server/src/engine/orchestrator.ts, the isAllowedWebhookUrl() function (around line 50) blocks private IPs but uses simple string matching. Harden it: (1) resolve the hostname to IP via DNS lookup (dns.promises.lookup) and check the resolved IP against private ranges — this catches DNS rebinding attacks where a domain resolves to 127.0.0.1. (2) Block file://, ftp://, data: schemes. (3) Block URLs with userinfo (user:pass@host). (4) Block URLs with ports below 80 or above 65535 (except 443, 8080, 8443, 3000-9999). Make the function async and update all callers (auth.ts register and update endpoints). Verify: cd server && npx tsc --noEmit passes.

• Wallet encryption key enforcement in production: In server/src/engine/crypto.ts (38 lines), if BOTFIGHTS_WALLET_ENCRYPTION_KEY is not set, a random key is generated. This means wallet data encrypted in one session can't be decrypted after a restart. Fix: In production (NODE_ENV === 'production'), throw an error on startup if BOTFIGHTS_WALLET_ENCRYPTION_KEY is not set, rather than silently using a random key. In dev mode, keep the random key behavior but log a warning. Also validate that the key is exactly 64 hex characters (32 bytes). Verify: cd server && npx tsc --noEmit passes.

---

Phase 8: Observability & Health

• Health check endpoint: In server/src/app.ts, add a GET /health endpoint (outside /api/* so it's not rate-limited) that returns:

  {
    "status": "ok",
    "uptime": process.uptime(),
    "activeFights": activeFighters.size / 2,
    "dbOk": true/false,
    "timestamp": new Date().toISOString()
  }
  

  Check DB health by running SELECT 1 via sqlite.prepare('SELECT 1').get(). If it throws, set dbOk: false and status: 'degraded'. Import activeFighters from orchestrator or use the new getActiveFightCount(). Return 200 for ok, 503 for degraded. This endpoint will be used by the VPS monitoring and deployment scripts. Verify: cd server && npx tsc --noEmit passes.

• Structured logging utility: Create server/src/engine/logger.ts with a simple structured logger. No dependencies — just wrap console.log/console.error with JSON output in production and human-readable in dev:

  export function log(level: 'info' | 'warn' | 'error', component: string, message: string, data?: Record<string, unknown>) {
    const entry = { ts: Date.now(), level, component, msg: message, ...data }
    if (process.env.NODE_ENV === 'production') {
      console[level === 'error' ? 'error' : 'log'](JSON.stringify(entry))
    } else {
      console[level === 'error' ? 'error' : 'log'](`[${component}] ${message}`, data || '')
    }
  }
  

  Replace 5-10 key console.log calls in orchestrator.ts, payments.ts, and background.ts with this logger (focus on fight start/end, payment events, and errors). Do NOT replace every console.log — just the critical operational ones. Verify: cd server && npx tsc --noEmit passes.

• Request timing middleware: In server/src/app.ts, add a middleware that logs slow requests. Before the routes, add:

  app.use('/api/*', async (c, next) => {
    const start = Date.now()
    await next()
    const ms = Date.now() - start
    if (ms > 1000) {
      log('warn', 'http', `slow request: ${c.req.method} ${c.req.path}`, { ms, status: c.res.status })
    }
  })
  

  Import log from ./engine/logger.js. This catches endpoints that are unexpectedly slow (DB queries without indexes, webhook timeouts). Verify: cd server && npx tsc --noEmit passes.

• Fight metrics counter: In server/src/engine/orchestrator.ts, add a simple metrics object that tracks counts since startup:

  export const metrics = {
    fightsStarted: 0,
    fightsCompleted: 0,
    fightsCancelled: 0,
    webhookErrors: 0,
    webhookTimeouts: 0,
    avgRoundMs: 0,
    totalRoundMs: 0,
    totalRounds: 0,
  }
  

  Increment fightsStarted in runFightAsync(), fightsCompleted/fightsCancelled in executeFightRounds(), webhookErrors/webhookTimeouts in callWebhook(). Track round duration and compute running average. Expose these via GET /health endpoint (add metrics field to the health response). Verify: cd server && npx tsc --noEmit passes.

---

Phase 9: Frontend Polish & Error States

• Global API error handler: In frontend/src/composables/useNostr.ts and frontend/src/composables/useWallet.ts, API calls use raw fetch() without consistent error handling. Create frontend/src/utils/api.ts with a thin wrapper:

  export async function apiFetch<T>(path: string, options?: RequestInit): Promise<T> {
    const res = await fetch(path, {
      ...options,
      headers: { 'Content-Type': 'application/json', ...options?.headers },
    })
    if (!res.ok) {
      const body = await res.json().catch(() => ({ error: res.statusText }))
      throw new ApiError(res.status, body.error || 'Unknown error', body.details)
    }
    return res.json()
  }
  export class ApiError extends Error {
    constructor(public status: number, message: string, public details?: string) {
      super(message)
    }
  }
  

  Replace 5-8 of the most important fetch() calls in useNostr.ts (login, register, registerHuman, update) and useWallet.ts (connectNWC, payEntryFee, createInvoice) with apiFetch(). Handle ApiError in the callers to show user-friendly error messages. Do NOT replace every fetch call — focus on the ones where errors are user-facing. Verify: cd frontend && npx vue-tsc --noEmit passes.

• Loading states for all async operations in JoinBoutPage: In frontend/src/pages/JoinBoutPage.vue, there are multiple async operations (login, name check, webhook test, registration) that should show loading states. Check each submit button: if it doesn't already have a loading/disabled state, add one. Specifically:

  1. Login button: add isLoggingIn ref, set true during handleLogin(), disable button and show "LOGGING IN..." text while true.
  2. Name confirm buttons (both bot and human): already have isCheckingName — verify it works and the button text changes.
  3. Webhook confirm: add isTestingWebhook ref, show "TESTING..." during the webhook verification.
  4. Register: add isRegistering ref, show "REGISTERING..." during the API call. All buttons should use :disabled="isLoading" and show the loading text via ternary in the button content. Verify: cd frontend && npx vue-tsc --noEmit passes.

• Error toast/notification system: Create frontend/src/composables/useToast.ts — a minimal toast notification composable:

  const toasts = ref<Array<{ id: number; message: string; type: 'error' | 'success' | 'info'; timeout: number }>>([])
  let nextId = 0
  export function useToast() {
    function show(message: string, type: 'error' | 'success' | 'info' = 'info', duration = 4000) {
      const id = nextId++
      toasts.value.push({ id, message, type, timeout: window.setTimeout(() => dismiss(id), duration) })
    }
    function dismiss(id: number) {
      toasts.value = toasts.value.filter(t => t.id !== id)
    }
    return { toasts: readonly(toasts), show, dismiss }
  }
  

  Create frontend/src/components/ToastContainer.vue — renders toasts fixed at bottom-right with appropriate colors (red for error, green for success, cyan for info). Style matches existing neon aesthetic: font-display, tracking-wider, dark background with colored border. Add <ToastContainer /> to frontend/src/App.vue (or the root layout). Verify: cd frontend && npx vue-tsc --noEmit passes.

---

Phase 10: Deployment & Operations

• Database backup script: Create server/scripts/backup-db.sh:

  #!/bin/bash
  DB_PATH="${DB_PATH:-./data/botfights.db}"
  BACKUP_DIR="${BACKUP_DIR:-./backups}"
  mkdir -p "$BACKUP_DIR"
  TIMESTAMP=$(date +%Y%m%d_%H%M%S)
  sqlite3 "$DB_PATH" ".backup '$BACKUP_DIR/botfights_$TIMESTAMP.db'"
  # Keep only last 30 backups
  ls -t "$BACKUP_DIR"/botfights_*.db | tail -n +31 | xargs -r rm
  echo "[backup] created botfights_$TIMESTAMP.db ($(du -h "$BACKUP_DIR/botfights_$TIMESTAMP.db" | cut -f1))"
  

  Make it executable: chmod +x server/scripts/backup-db.sh. This uses SQLite's online backup API (safe during writes). The 30-backup retention keeps ~1 day of hourly backups. Verify: script is syntactically valid with bash -n server/scripts/backup-db.sh.

• Systemd service file: Create server/deploy/botfights.service:

  [Unit]
  Description=Botfights Server
  After=network.target
  
  [Service]
  Type=simple
  User=botfights
  WorkingDirectory=/opt/botfights/server
  ExecStart=/usr/bin/node --experimental-specifier-resolution=node dist/index.js
  Restart=on-failure
  RestartSec=5
  Environment=NODE_ENV=production
  Environment=PORT=9100
  EnvironmentFile=-/opt/botfights/.env
  StandardOutput=journal
  StandardError=journal
  SyslogIdentifier=botfights
  
  # Security hardening
  NoNewPrivileges=true
  ProtectSystem=strict
  ProtectHome=true
  ReadWritePaths=/opt/botfights/server/data
  PrivateTmp=true
  
  [Install]
  WantedBy=multi-user.target
  

  Also create server/deploy/botfights-backup.timer and server/deploy/botfights-backup.service for hourly database backups via systemd timer. The backup service runs /opt/botfights/server/scripts/backup-db.sh. Verify: files are valid INI format.

• Zero-downtime deploy script: Create server/scripts/deploy.sh:

  #!/bin/bash
  set -euo pipefail
  DEPLOY_DIR="/opt/botfights"
  echo "[deploy] pulling latest..."
  cd "$DEPLOY_DIR" && git pull origin main
  echo "[deploy] installing deps..."
  cd server && pnpm install --frozen-lockfile
  echo "[deploy] building..."
  pnpm build
  echo "[deploy] backing up database..."
  bash scripts/backup-db.sh
  echo "[deploy] restarting service..."
  sudo systemctl restart botfights
  echo "[deploy] waiting for health check..."
  for i in $(seq 1 30); do
    if curl -sf http://localhost:9100/health > /dev/null 2>&1; then
      echo "[deploy] healthy after ${i}s"
      exit 0
    fi
    sleep 1
  done
  echo "[deploy] FAILED: health check timeout"
  exit 1
  

  Make executable: chmod +x server/scripts/deploy.sh. Verify: bash -n server/scripts/deploy.sh.

---

Phase 11: Typecheck & Final Verification

• Full typecheck: Run pnpm typecheck from project root (/Users/dorian/Projects/botfights). Fix ALL TypeScript errors in both server and frontend. This is the final gate — every previous phase should have verified individually, but this catches any cross-package issues.

• Server startup smoke test: Run cd /Users/dorian/projects/botfights/server && npx tsx src/index.ts and verify:

  1. [botfights] database migrated appears
  2. [botfights] database indexes ensured appears
  3. No crash on startup
  4. curl http://localhost:9100/health returns {"status":"ok",...}
  5. curl http://localhost:9100/api/bots returns a list Stop the server after verification.

• Frontend build: Run cd /Users/dorian/Projects/botfights/frontend && pnpm build. Verify zero errors. Check the output bundle size is reasonable (should be under 5MB total).

• Commit all changes: Stage all modified and new files. Commit with message: feat: production hardening — indexes, auth fixes, payment security, observability. Do NOT push — the user will review and push manually.