DorianandClaude Opus 4.6
610e799605
Enable real-time fight spectating for all live fights (not just human fights). Multiple spectators can watch simultaneously via SSE. Spectator count is tracked per-fight and broadcast with every SSE event. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
215 lines
9.9 KiB
Markdown
215 lines
9.9 KiB
Markdown
# Production-Grade Botfights: Implementation Plan
|
|
|
|
## Context
|
|
|
|
Botfights is a competitive game where users register AI bots with webhook URLs. The server sends challenge questions to webhooks, scores responses, and updates rankings. The system works end-to-end but has critical fairness, reliability, and security gaps that must be fixed before real users compete for rankings. This plan makes the engine bulletproof and adds a rich TUI for the overnight fight loop.
|
|
|
|
---
|
|
|
|
## Phase 1: Fight Integrity (Foundation)
|
|
|
|
Everything depends on fights producing correct, trustworthy results.
|
|
|
|
### 1.1 Schema additions
|
|
**File:** `server/src/db/schema.ts`, `server/src/db/migrate.ts`
|
|
- Add to `bots`: `lastFightAt TEXT`, `consecutiveErrors INTEGER DEFAULT 0`, `lastErrorAt TEXT`
|
|
- Add migration SQL for existing DBs
|
|
|
|
### 1.2 Transaction-wrap Elo updates
|
|
**Files:** `server/src/engine/orchestrator.ts` (lines 276-306), `server/src/engine/mock.ts` (lines 392-421)
|
|
- Wrap fight finalization (status update + both bot stat updates) in a single SQLite transaction
|
|
- Prevents partial Elo corruption on crash or concurrent fights
|
|
|
|
### 1.3 Block self-fights
|
|
**Files:** `server/src/engine/orchestrator.ts`, `server/src/engine/mock.ts`, `server/src/routes/fights.ts`
|
|
- Add `if (botAId === botBId) throw new Error('A bot cannot fight itself')` at top of `runFight`, `runFightAsync`, `runMockFight`
|
|
|
|
### 1.4 Prevent concurrent fights for same bot
|
|
**File:** `server/src/engine/orchestrator.ts`
|
|
- In-memory `Set<string>` of currently-fighting bot IDs
|
|
- Check before starting, add on start, remove in `.finally()`
|
|
|
|
### 1.5 Crash recovery for stuck fights
|
|
**File:** `server/src/engine/orchestrator.ts`
|
|
- In `runFightAsync` catch handler: mark fight `status='cancelled'`, call `fightEvents.cleanup()`
|
|
- On server startup: mark any `status='live'` fights older than 10 minutes as `cancelled`
|
|
|
|
### 1.6 Fix creative scoring (anti-gaming)
|
|
**File:** `server/src/engine/scoring.ts` (lines 166-174)
|
|
- Replace `estimateQuality` with multi-factor heuristic: character diversity, word diversity, length window (30-400 ideal), speed bonus
|
|
- Prevents gaming by dumping 500 chars of garbage text
|
|
|
|
### 1.7 Tighten factual answer checking
|
|
**File:** `server/src/engine/answers.ts` (line 78)
|
|
- Add word-boundary awareness: `containsWholeWord()` helper using regex `\b`
|
|
- Short answers (2 chars like "au", "fe") require near-exact match, not just containment
|
|
- Prevents "I feel confident" matching accepted answer "fe"
|
|
|
|
### 1.8 Dampen mock-bot Elo farming
|
|
**File:** `server/src/engine/orchestrator.ts`
|
|
- When one combatant is a mock bot, use K-factor 12 instead of 32
|
|
- Real bot-vs-real bot fights keep K=32 for full stakes
|
|
|
|
---
|
|
|
|
## Phase 2: Webhook Contract & Developer Experience
|
|
|
|
### 2.1 Response size limits
|
|
**File:** `server/src/engine/orchestrator.ts` (line 83)
|
|
- Replace `res.text()` with size-limited reader (10KB max)
|
|
- Truncate `answer` to 2000 chars, `trash_talk` to 200 chars
|
|
- Prevents OOM attacks from malicious webhooks
|
|
|
|
### 2.2 SSRF protection
|
|
**File:** `server/src/engine/orchestrator.ts` (new `isAllowedWebhookUrl` function)
|
|
- Block localhost, private IPs (10.x, 192.168.x, 172.16-31.x), AWS metadata (169.254.169.254), .local/.internal
|
|
- Enforce HTTPS in production, allow HTTP in dev
|
|
- Apply at registration (`routes/bots.ts`, `routes/auth.ts`) AND at call time
|
|
|
|
### 2.3 Pre-fight webhook verification
|
|
**New file:** `server/src/engine/webhook-test.ts`
|
|
- `testWebhook(url)`: sends a test challenge (`"respond with {"answer": "pong"}"`), validates response shape and latency
|
|
- Returns `{ reachable, validResponse, latencyMs, error? }`
|
|
|
|
**Integrate into:**
|
|
- `POST /api/auth/register` -- test webhook before inserting bot. Reject with specific error if it fails.
|
|
- `POST /api/bots` -- same
|
|
- New route `POST /api/bots/:name/test` -- re-test webhook on demand (replaces weak `/health` check)
|
|
|
|
### 2.4 Webhook reliability tracking
|
|
**File:** `server/src/engine/orchestrator.ts`
|
|
- After each webhook call: increment `consecutiveErrors` on failure, reset to 0 on success
|
|
- If `consecutiveErrors >= 5`: mark bot `isActive: false`, skip in matchmaking
|
|
- Bot owner must re-test webhook to reactivate
|
|
|
|
### 2.5 Add `fight_id` to webhook payload
|
|
**File:** `server/src/engine/orchestrator.ts` (line 48)
|
|
- Thread `fightId` through `callWebhook` and `getBotResponse`
|
|
- Bot developers can correlate challenge POSTs to specific fights for debugging
|
|
|
|
---
|
|
|
|
## Phase 3: Anti-Gaming & Security
|
|
|
|
### 3.1 Rate limiting
|
|
**New file:** `server/src/middleware/rate-limit.ts`
|
|
- Simple in-memory rate limiter (no new deps), per-IP sliding window
|
|
- Apply: `POST /api/bots` (5/hr), `POST /api/auth/register` (5/hr), `POST /api/queue/join` (1 per 10s per bot), all other POSTs (60/min)
|
|
|
|
### 3.2 Case-insensitive name uniqueness
|
|
**Files:** `server/src/routes/bots.ts` (line 39), `server/src/routes/auth.ts` (line 78)
|
|
- Force bot names to lowercase at registration time
|
|
- Prevents name squatting ("MyBot" vs "mybot")
|
|
|
|
### 3.3 Queue cooldowns
|
|
**File:** `server/src/engine/queue.ts`
|
|
- In-memory `Map<string, number>` of post-fight cooldowns (15 seconds)
|
|
- Set cooldown after fight completes (called from `orchestrator.ts`)
|
|
- Reject queue join if cooldown active
|
|
|
|
### 3.4 Concurrent fight checks in queue
|
|
**File:** `server/src/engine/queue.ts`
|
|
- Check the `activeFighters` set (from 1.4) before allowing queue join
|
|
- Prevents a bot from queueing while already in a fight
|
|
|
|
---
|
|
|
|
## Phase 4: TUI Fight Loop
|
|
|
|
Minimal deps -- only `chalk` for colors. All layout via ANSI codes and Unicode box drawing.
|
|
|
|
### 4.1 Add dependency
|
|
`chalk@5` to `server/package.json`
|
|
|
|
### 4.2 TUI state tracker
|
|
**New file:** `server/src/tui/state.ts`
|
|
- `TuiState` interface: fight count, KOs, perfects, draws, errors, current fight (bots/HP/round/events), recent fights, leaderboard, biggest upset, Elo movers, elapsed time
|
|
|
|
### 4.3 TUI renderer
|
|
**New file:** `server/src/tui/renderer.ts`
|
|
|
|
```
|
|
+=================== BOTFIGHTS OVERNIGHT LOOP ====================+
|
|
| Fight #47 of 200 Elapsed: 12m 34s |
|
|
| Style: mixed Rate: 3.8 fights/min |
|
|
+------------------------------------------------------------------+
|
|
| |
|
|
| skull_crusher_9000 (1820) vs boaty_mcbotface (1150) |
|
|
| [================----] 163 HP vs [====----------------] 47 HP |
|
|
| Round 6/10 -- speed_blitz |
|
|
| >> skull_crusher answered in 234ms (CORRECT) |
|
|
| >> boaty_mcbotface timed out! FREE HIT! |
|
|
| |
|
|
+========================= STATS ==================================+
|
|
| Fights: 47 completed, 0 errors |
|
|
| KOs: 31 (66%) | Perfects: 4 | Draws: 2 |
|
|
| Biggest upset: boaty_mcbotface beat the_architect! |
|
|
+========================= LEADERBOARD ============================+
|
|
| #1 the_architect 1980 52W-8L LEGEND |
|
|
| #2 chad_gpt 1950 48W-10L LEGEND |
|
|
| #3 skull_crusher_9000 1820 35W-12L DIAMOND |
|
|
+========================= RECENT ================================+
|
|
| #47 skull_crusher vs boaty -> skull_crusher (KO R6) |
|
|
| #46 chad_gpt vs lorem_ipsum -> chad_gpt (PERFECT R3) |
|
|
| #45 regex_ronin vs the_intern -> regex_ronin (Decision) |
|
|
+==================================================================+
|
|
```
|
|
|
|
- Single buffered write to avoid flicker
|
|
- Handles terminal resize via `process.stdout.on('resize')`
|
|
- Tier colors via chalk
|
|
|
|
### 4.4 Refactor fight loop with callbacks
|
|
**File:** `server/src/engine/fight-loop.ts`
|
|
- Add callback options: `onFightStart`, `onRoundComplete`, `onFightComplete`, `onError`
|
|
- Wire event bus so TUI gets live round-by-round updates
|
|
|
|
### 4.5 Rewrite CLI
|
|
**File:** `server/src/fight-loop-cli.ts`
|
|
- Create TUI state + renderer, pass callbacks to fight loop
|
|
- Graceful SIGINT: show final summary screen
|
|
- Same CLI args (`--max`, `--interval`, `--style`)
|
|
|
|
### 4.6 Final summary screen
|
|
On loop end or Ctrl+C: duration, total fights, KO/perfect/draw rates, top Elo movers, biggest upset, most active bot
|
|
|
|
---
|
|
|
|
## Phase 5: Verification
|
|
|
|
### 5.1 Manual test sequence
|
|
1. `pnpm seed` -- verify schema migrations run
|
|
2. `pnpm dev` -- verify server starts, orphaned fights cleaned up on startup
|
|
3. Register bot with bad webhook URL -> verify rejection with specific error
|
|
4. Register bot with valid webhook -> verify test challenge sent and validated
|
|
5. Try self-fight via `/api/fights/matchmake` -> verify blocked
|
|
6. Run `pnpm fight-loop --max=10` -> verify TUI renders, stats update live
|
|
7. Kill process mid-fight, restart -> verify stuck fights cleaned up
|
|
8. Send oversized response from test webhook -> verify 10KB limit
|
|
9. Rapid-fire queue joins -> verify rate limiting and cooldowns
|
|
|
|
---
|
|
|
|
## Files Summary
|
|
|
|
**Modified (11 files):**
|
|
- `server/src/db/schema.ts` -- new columns
|
|
- `server/src/db/migrate.ts` -- migration SQL
|
|
- `server/src/engine/orchestrator.ts` -- transactions, self-fight block, concurrent guard, crash recovery, response limits, SSRF, reliability tracking, mock Elo dampening, fight_id in payload
|
|
- `server/src/engine/mock.ts` -- transaction wrap, self-fight block
|
|
- `server/src/engine/scoring.ts` -- creative scoring rewrite
|
|
- `server/src/engine/answers.ts` -- word boundary fixes
|
|
- `server/src/engine/queue.ts` -- cooldowns, concurrent fight checks, export activeFighters check
|
|
- `server/src/engine/fight-loop.ts` -- callback options for TUI
|
|
- `server/src/fight-loop-cli.ts` -- TUI integration
|
|
- `server/src/app.ts` -- rate limiting, startup cleanup
|
|
- `server/package.json` -- add chalk
|
|
|
|
**Created (4 files):**
|
|
- `server/src/engine/webhook-test.ts` -- pre-fight webhook verification
|
|
- `server/src/middleware/rate-limit.ts` -- rate limiter
|
|
- `server/src/tui/state.ts` -- TUI state tracker
|
|
- `server/src/tui/renderer.ts` -- TUI renderer
|
|
|
|
**Execution order:** Phase 1 (1.1-1.8) -> Phase 2 (2.1-2.5) -> Phase 3 (3.1-3.4) -> Phase 4 (4.1-4.6) -> Phase 5 verification
|