Files
ssmithx bb03bfc365 Go global: worldwide BTC Map sync, marker clustering, response compression
- Sync now pulls every BTC Map place worldwide (~29k live), not just
  Madeira — dropped BTCMAP_CENTER_LAT/LON/RADIUS_KM entirely.
- Incremental via a stored watermark (settings.btcmap_synced_since):
  only fetches what changed since the last sync, not the whole dataset
  every time. Handles deletions too (BTC Map delistings correctly evict
  the local place + cascade its claim/links, not just go unclaimed).
- Real bug caught and fixed before it shipped: tried BTC Map's own
  recommended updated_since+limit pagination first, but their timestamps
  mix millisecond and whole-second precision, and naive string comparison
  across that isn't chronologically safe — silently truncated a real
  sync to ~4,000 of ~42,000 records with no error. Measured the
  alternative (single unpaginated request) instead: 2-3s for the full
  dataset, simpler and actually correct. Full writeup in
  services/btcmap.ts, regression test for the specific bug in
  services/btcmap.test.ts.
- @fastify/compress added — the places list is now tens of thousands of
  rows, gzip/br/zstd auto-negotiated.
- Also fixed while touching dependencies: @fastify/static had a real
  high-severity path-traversal/auth-bypass advisory (GHSA-pr96-94w5-mx2h
  et al) affecting the version we were pinned to — bumped to the patched
  10.1.2.
- Frontend: leaflet.markercluster (raw per-marker rendering doesn't scale
  to tens of thousands of points), cyberpunk-themed cluster icons to
  match the existing HUD styling, batch marker insertion (addLayers, not
  a per-marker addLayer loop) since that's dramatically faster at this
  scale. Default map view is now world-scale, recentering on the
  player's location if geolocation is available.
- 60 backend tests passing throughout (7 new for the sync rewrite).
2026-08-05 17:32:54 +00:00

197 lines
9.6 KiB
Markdown

# Regress
A location-based capture game in the spirit of *Ingress*, built as a nostr-native
web app. Real Bitcoin-accepting businesses from [BTC Map](https://btcmap.org)
stand in for Ingress's portals — players physically visit them to claim them
for their team (orange or green), then link claimed places into triangular
control fields. **Global** — every BTC Map place worldwide (~29k live, plus
historical deletions tracked so delisted businesses correctly drop out).
Started as a Madeira, Portugal-only pilot; see git history/`DESIGN.md` for
that phase.
See [`DESIGN.md`](./DESIGN.md) for the full research/design writeup.
## Stack
- `server/` — Fastify + better-sqlite3 + nostr-tools (TypeScript), same pattern
as [podsteadr](http://146.59.87.168:3000/ssmithx/podsteadr).
- `frontend/` — Vue 3 + Vite + Pinia + Tailwind + Leaflet (+
`leaflet.markercluster` — required at global scale; tens of thousands of
raw markers would choke the DOM otherwise).
- Auth is NIP-98 (signed-event login → session cookie), same as podsteadr.
`frontend/public/nostr-provider.js` is vendored so the app can also be
launched identity-aware from inside an Archipelago dashboard, the same way
as podsteadr — see that repo's README for how the registration side works.
## Running locally
```bash
# server
cd server
npm install
npm run dev # http://localhost:8096
# frontend (separate terminal)
cd frontend
npm install
npm run dev # http://localhost:5173, proxies /api to :8096
```
On first run, the `places` table is empty — call `POST /api/sync` (or click
"Sync from BTC Map" in the UI) to pull the current worldwide dataset from BTC
Map's public API (~29k places, a few seconds, single request — see "Global
sync" below for why it's one request rather than paginated).
## Global sync
`POST /api/sync` pulls **every** BTC Map place worldwide via the
chronological `/v4/places` endpoint, incrementally: a watermark
(`btcmap_synced_since` in the `settings` table) tracks the last sync's
newest `updated_at`, and each subsequent call only fetches what changed
since then — call it as often as you like (e.g. a cron hitting it hourly)
without re-pulling the full dataset every time.
Deleted places (BTC Map returns these too, via `include_deleted=true`) are
evicted locally — `ON DELETE CASCADE` on `claims`/`links` means a delisted
business correctly loses its claim and any links through it, not just the
place row.
**Why one unpaginated request, not BTC Map's own recommended
`updated_since`+`limit` pagination loop:** tried that first. BTC Map's
`updated_at` timestamps mix millisecond-precision and whole-second values,
and naive string comparison across mixed precision isn't chronologically
safe (`"...27Z"` sorts *after* `"...27.137Z"` lexicographically, even though
`.137` happened later within that same second) — this silently truncated a
real sync to the first ~4,000 of ~42,000 records with no error, since the
pagination cursor got stuck comparing strings instead of real timestamps.
Measured the alternative instead: omitting `limit` entirely just returns the
whole dataset — currently ~42k records including historical deletions, ~9MB
uncompressed, in 2-3 seconds. Simpler and correct beats "matches the docs'
suggested pattern" here. `services/btcmap.ts` has the full writeup;
`services/btcmap.test.ts` has a regression test for the specific
mixed-precision comparison bug. Revisit with real pagination (numeric/parsed
timestamp cursor, not string comparison) if the dataset ever grows enough to
make a single request unwieldy.
Response payloads (`/api/places` is the big one, tens of thousands of rows)
are compressed (`@fastify/compress`, gzip/brotli/zstd auto-negotiated).
## Game rules (v0)
- Claiming or linking from a place requires being within `CLAIM_RADIUS_METERS`
(default 40m) of it, via browser geolocation — no claiming from the couch.
- Claiming an enemy-held place captures it for your team and tears down any
links running through it.
- A link can be created between any two places your team currently holds, as
long as you're standing at the origin place, up to `MAX_LINK_DISTANCE_KM`
(default 500km) apart. Ingress scales its link range off portal/resonator
level (up to ~655km at max level); Regress has no leveling system to scale
off of, so this is a single flat cap instead.
- A new link can't cross any existing link (either team) — same as Ingress.
Links sharing an endpoint don't count as crossing. See
`services/geo.ts#segmentsIntersect` and `routes/links.ts`.
- Three mutually-linked, same-team places form a field (computed live, not
stored) — see `server/src/services/fields.ts`.
- Team choice is one-way once made.
- **Claims expire after `PORTAL_TIMEOUT_BLOCKS` Bitcoin blocks (default
2100)** and revert to neutral, tearing down any links through them — the
game's clock is block height, not wall-clock time. Current height comes
from a public block explorer API (`BLOCK_HEIGHT_API_URL`, defaults to
mempool.space), same trust model as pulling place data from BTC Map. A
background sweep (`EXPIRY_SWEEP_INTERVAL_MS`, default 5 min) actively
reverts timed-out claims; claim/link requests also check inline so
correctness doesn't depend on sweep timing (`services/expiry.ts`).
- Score (`GET /api/score`) reports claimed places, **link count**, field
count, and total field area per team.
## Tests
```bash
cd server
npm test
```
## Deployment
Live at **`https://regress.atobitcoin.io`** — own subdomain, own Let's
Encrypt cert (`certbot certonly --webroot`, same box/method as podsteadr's
cert), same host (`23.182.128.130`) as
[podsteadr](http://146.59.87.168:3000/ssmithx/podsteadr) but fully
independent nginx server blocks. Deployed at root (no `ROUTE_PREFIX`/
`VITE_BASE`) now that it has its own domain — simpler than the path-prefix
setup below, which was a temporary measure before DNS for the subdomain
existed. Migrated here from an earlier `archy-x250-dev3` trial deploy
(Tailscale-only Archipelago node), then briefly lived at
`podsteadr.atobitcoin.io/regress/` before the subdomain was added — see git
history for both phases.
- Single container (`Dockerfile`, server + built frontend in one image),
`podman run --restart unless-stopped`, data bind-mounted at
`/var/lib/archipelago/regress-data` on the host, internal port **8199**
(bound to `127.0.0.1` only — nginx is the only way in).
- `deploy/nginx-regress-domain.conf` — dedicated nginx site
(`/etc/nginx/sites-available/regress-domain` on the host), same structure
as podsteadr's own site file (HTTP→HTTPS redirect + ACME webroot
challenge on 80, `proxy_pass` to `127.0.0.1:8199` on 443).
- Cert renewal is certbot's standard cron/systemd timer, already running on
this host for podsteadr's cert — no extra setup needed, it picks up the
new cert automatically.
### Path-prefix deployment (`ROUTE_PREFIX` / `VITE_BASE`) — not currently used, but still supported
The app still supports being served from a path prefix under someone else's
domain (that's how it briefly ran at `podsteadr.atobitcoin.io/regress/`) —
kept working and tested in case it's useful again, e.g. for a future
deployment that doesn't get its own subdomain. Two coordinated changes are
required together if you use this:
1. **Frontend build**: `VITE_BASE=/regress/` (Vite's `base` config) so built
asset URLs and the vendored `nostr-provider.js` script tag/data
attributes (via `%BASE_URL%` in `index.html`) resolve under the prefix.
2. **Backend runtime**: `ROUTE_PREFIX=/regress` env var — every route
(including `/api/health`) is registered under this prefix via Fastify's
plugin-encapsulation `{ prefix }` option (`app.ts`).
**The nginx location must forward the full prefixed path unchanged — do
NOT strip it**, unlike podsteadr's own `/player/`, `/hls/`, etc. blocks
which all strip their prefix (`proxy_pass http://127.0.0.1:PORT/;` with a
trailing slash). Regress's NIP-98 login signs the *exact* URL it's about to
call, prefix included (`frontend/src/lib/api.ts#apiUrl`); if nginx stripped
the prefix before forwarding, the backend would reconstruct a different
(unprefixed) URL to check the signature against, and every login would fail
with "u tag does not match the request URL". So the location block uses
`proxy_pass http://127.0.0.1:8199;`**no trailing path at all** — which
tells nginx to forward the original URI verbatim, prefix included. This is
covered by `server/src/routePrefix.test.ts`, including a negative test that
a signed-for-the-unprefixed-URL login is correctly rejected (proving the
prefix check is real, not accidentally bypassed).
Deploying at root (no prefix) needs neither variable — both default to `''`/`'/'`.
### Migrating the SQLite database between hosts
The whole game state (places/claims/links/users) is one file,
`regress.sqlite3`. To move it: stop the container, copy the file (plus its
`-wal`/`-shm` siblings if present, or checkpoint first), start the new
container pointed at the copy. No export/import tooling needed — see git
history for the exact commands used for the archy-x250-dev3 → podsteadr
machine move.
To redeploy after a code change (current setup, root deployment on
`regress.atobitcoin.io`): `git pull` in `/var/lib/archipelago/regress-src`
on the host, then:
```bash
podman build -t localhost/regress:latest .
podman run -d --name regress-app --replace --restart unless-stopped \
-p 127.0.0.1:8199:8199 \
-v /var/lib/archipelago/regress-data:/data \
-e PUBLIC_URL=https://regress.atobitcoin.io \
localhost/regress:latest
```
(If ever redeploying under a path prefix instead, add back
`--build-arg VITE_BASE=/prefix/ --build-arg ROUTE_PREFIX=/prefix` to the
build and `-e ROUTE_PREFIX=/prefix` to the run command — see the
path-prefix section above.)