Files
botfights/.claude/plans/greedy-skipping-lollipop.md
T

215 lines
9.9 KiB
Markdown
Raw Normal View History

# Production-Grade Botfights: Implementation Plan
## Context
Botfights is a competitive game where users register AI bots with webhook URLs. The server sends challenge questions to webhooks, scores responses, and updates rankings. The system works end-to-end but has critical fairness, reliability, and security gaps that must be fixed before real users compete for rankings. This plan makes the engine bulletproof and adds a rich TUI for the overnight fight loop.
---
## Phase 1: Fight Integrity (Foundation)
Everything depends on fights producing correct, trustworthy results.
### 1.1 Schema additions
**File:** `server/src/db/schema.ts`, `server/src/db/migrate.ts`
- Add to `bots`: `lastFightAt TEXT`, `consecutiveErrors INTEGER DEFAULT 0`, `lastErrorAt TEXT`
- Add migration SQL for existing DBs
### 1.2 Transaction-wrap Elo updates
**Files:** `server/src/engine/orchestrator.ts` (lines 276-306), `server/src/engine/mock.ts` (lines 392-421)
- Wrap fight finalization (status update + both bot stat updates) in a single SQLite transaction
- Prevents partial Elo corruption on crash or concurrent fights
### 1.3 Block self-fights
**Files:** `server/src/engine/orchestrator.ts`, `server/src/engine/mock.ts`, `server/src/routes/fights.ts`
- Add `if (botAId === botBId) throw new Error('A bot cannot fight itself')` at top of `runFight`, `runFightAsync`, `runMockFight`
### 1.4 Prevent concurrent fights for same bot
**File:** `server/src/engine/orchestrator.ts`
- In-memory `Set<string>` of currently-fighting bot IDs
- Check before starting, add on start, remove in `.finally()`
### 1.5 Crash recovery for stuck fights
**File:** `server/src/engine/orchestrator.ts`
- In `runFightAsync` catch handler: mark fight `status='cancelled'`, call `fightEvents.cleanup()`
- On server startup: mark any `status='live'` fights older than 10 minutes as `cancelled`
### 1.6 Fix creative scoring (anti-gaming)
**File:** `server/src/engine/scoring.ts` (lines 166-174)
- Replace `estimateQuality` with multi-factor heuristic: character diversity, word diversity, length window (30-400 ideal), speed bonus
- Prevents gaming by dumping 500 chars of garbage text
### 1.7 Tighten factual answer checking
**File:** `server/src/engine/answers.ts` (line 78)
- Add word-boundary awareness: `containsWholeWord()` helper using regex `\b`
- Short answers (2 chars like "au", "fe") require near-exact match, not just containment
- Prevents "I feel confident" matching accepted answer "fe"
### 1.8 Dampen mock-bot Elo farming
**File:** `server/src/engine/orchestrator.ts`
- When one combatant is a mock bot, use K-factor 12 instead of 32
- Real bot-vs-real bot fights keep K=32 for full stakes
---
## Phase 2: Webhook Contract & Developer Experience
### 2.1 Response size limits
**File:** `server/src/engine/orchestrator.ts` (line 83)
- Replace `res.text()` with size-limited reader (10KB max)
- Truncate `answer` to 2000 chars, `trash_talk` to 200 chars
- Prevents OOM attacks from malicious webhooks
### 2.2 SSRF protection
**File:** `server/src/engine/orchestrator.ts` (new `isAllowedWebhookUrl` function)
- Block localhost, private IPs (10.x, 192.168.x, 172.16-31.x), AWS metadata (169.254.169.254), .local/.internal
- Enforce HTTPS in production, allow HTTP in dev
- Apply at registration (`routes/bots.ts`, `routes/auth.ts`) AND at call time
### 2.3 Pre-fight webhook verification
**New file:** `server/src/engine/webhook-test.ts`
- `testWebhook(url)`: sends a test challenge (`"respond with {"answer": "pong"}"`), validates response shape and latency
- Returns `{ reachable, validResponse, latencyMs, error? }`
**Integrate into:**
- `POST /api/auth/register` -- test webhook before inserting bot. Reject with specific error if it fails.
- `POST /api/bots` -- same
- New route `POST /api/bots/:name/test` -- re-test webhook on demand (replaces weak `/health` check)
### 2.4 Webhook reliability tracking
**File:** `server/src/engine/orchestrator.ts`
- After each webhook call: increment `consecutiveErrors` on failure, reset to 0 on success
- If `consecutiveErrors >= 5`: mark bot `isActive: false`, skip in matchmaking
- Bot owner must re-test webhook to reactivate
### 2.5 Add `fight_id` to webhook payload
**File:** `server/src/engine/orchestrator.ts` (line 48)
- Thread `fightId` through `callWebhook` and `getBotResponse`
- Bot developers can correlate challenge POSTs to specific fights for debugging
---
## Phase 3: Anti-Gaming & Security
### 3.1 Rate limiting
**New file:** `server/src/middleware/rate-limit.ts`
- Simple in-memory rate limiter (no new deps), per-IP sliding window
- Apply: `POST /api/bots` (5/hr), `POST /api/auth/register` (5/hr), `POST /api/queue/join` (1 per 10s per bot), all other POSTs (60/min)
### 3.2 Case-insensitive name uniqueness
**Files:** `server/src/routes/bots.ts` (line 39), `server/src/routes/auth.ts` (line 78)
- Force bot names to lowercase at registration time
- Prevents name squatting ("MyBot" vs "mybot")
### 3.3 Queue cooldowns
**File:** `server/src/engine/queue.ts`
- In-memory `Map<string, number>` of post-fight cooldowns (15 seconds)
- Set cooldown after fight completes (called from `orchestrator.ts`)
- Reject queue join if cooldown active
### 3.4 Concurrent fight checks in queue
**File:** `server/src/engine/queue.ts`
- Check the `activeFighters` set (from 1.4) before allowing queue join
- Prevents a bot from queueing while already in a fight
---
## Phase 4: TUI Fight Loop
Minimal deps -- only `chalk` for colors. All layout via ANSI codes and Unicode box drawing.
### 4.1 Add dependency
`chalk@5` to `server/package.json`
### 4.2 TUI state tracker
**New file:** `server/src/tui/state.ts`
- `TuiState` interface: fight count, KOs, perfects, draws, errors, current fight (bots/HP/round/events), recent fights, leaderboard, biggest upset, Elo movers, elapsed time
### 4.3 TUI renderer
**New file:** `server/src/tui/renderer.ts`
```
+=================== BOTFIGHTS OVERNIGHT LOOP ====================+
| Fight #47 of 200 Elapsed: 12m 34s |
| Style: mixed Rate: 3.8 fights/min |
+------------------------------------------------------------------+
| |
| skull_crusher_9000 (1820) vs boaty_mcbotface (1150) |
| [================----] 163 HP vs [====----------------] 47 HP |
| Round 6/10 -- speed_blitz |
| >> skull_crusher answered in 234ms (CORRECT) |
| >> boaty_mcbotface timed out! FREE HIT! |
| |
+========================= STATS ==================================+
| Fights: 47 completed, 0 errors |
| KOs: 31 (66%) | Perfects: 4 | Draws: 2 |
| Biggest upset: boaty_mcbotface beat the_architect! |
+========================= LEADERBOARD ============================+
| #1 the_architect 1980 52W-8L LEGEND |
| #2 chad_gpt 1950 48W-10L LEGEND |
| #3 skull_crusher_9000 1820 35W-12L DIAMOND |
+========================= RECENT ================================+
| #47 skull_crusher vs boaty -> skull_crusher (KO R6) |
| #46 chad_gpt vs lorem_ipsum -> chad_gpt (PERFECT R3) |
| #45 regex_ronin vs the_intern -> regex_ronin (Decision) |
+==================================================================+
```
- Single buffered write to avoid flicker
- Handles terminal resize via `process.stdout.on('resize')`
- Tier colors via chalk
### 4.4 Refactor fight loop with callbacks
**File:** `server/src/engine/fight-loop.ts`
- Add callback options: `onFightStart`, `onRoundComplete`, `onFightComplete`, `onError`
- Wire event bus so TUI gets live round-by-round updates
### 4.5 Rewrite CLI
**File:** `server/src/fight-loop-cli.ts`
- Create TUI state + renderer, pass callbacks to fight loop
- Graceful SIGINT: show final summary screen
- Same CLI args (`--max`, `--interval`, `--style`)
### 4.6 Final summary screen
On loop end or Ctrl+C: duration, total fights, KO/perfect/draw rates, top Elo movers, biggest upset, most active bot
---
## Phase 5: Verification
### 5.1 Manual test sequence
1. `pnpm seed` -- verify schema migrations run
2. `pnpm dev` -- verify server starts, orphaned fights cleaned up on startup
3. Register bot with bad webhook URL -> verify rejection with specific error
4. Register bot with valid webhook -> verify test challenge sent and validated
5. Try self-fight via `/api/fights/matchmake` -> verify blocked
6. Run `pnpm fight-loop --max=10` -> verify TUI renders, stats update live
7. Kill process mid-fight, restart -> verify stuck fights cleaned up
8. Send oversized response from test webhook -> verify 10KB limit
9. Rapid-fire queue joins -> verify rate limiting and cooldowns
---
## Files Summary
**Modified (11 files):**
- `server/src/db/schema.ts` -- new columns
- `server/src/db/migrate.ts` -- migration SQL
- `server/src/engine/orchestrator.ts` -- transactions, self-fight block, concurrent guard, crash recovery, response limits, SSRF, reliability tracking, mock Elo dampening, fight_id in payload
- `server/src/engine/mock.ts` -- transaction wrap, self-fight block
- `server/src/engine/scoring.ts` -- creative scoring rewrite
- `server/src/engine/answers.ts` -- word boundary fixes
- `server/src/engine/queue.ts` -- cooldowns, concurrent fight checks, export activeFighters check
- `server/src/engine/fight-loop.ts` -- callback options for TUI
- `server/src/fight-loop-cli.ts` -- TUI integration
- `server/src/app.ts` -- rate limiting, startup cleanup
- `server/package.json` -- add chalk
**Created (4 files):**
- `server/src/engine/webhook-test.ts` -- pre-fight webhook verification
- `server/src/middleware/rate-limit.ts` -- rate limiter
- `server/src/tui/state.ts` -- TUI state tracker
- `server/src/tui/renderer.ts` -- TUI renderer
**Execution order:** Phase 1 (1.1-1.8) -> Phase 2 (2.1-2.5) -> Phase 3 (3.1-3.4) -> Phase 4 (4.1-4.6) -> Phase 5 verification