fix(mempool): self-healing nginx backend proxy (v3.0.1) + gate timeout

The frontend nginx used a literal proxy_pass host with no resolver, so it
pinned mempool-api's IP at worker startup. When the backend restarts (gate,
OTA, crash, reboot re-IPAM) podman reassigns its IP and nginx keeps proxying
to the dead one -> /api hangs, websocket 502s, UI shows 'offline' until a
manual nginx reload. Same stale-upstream-IP class as the netbird 502.

Fix: mempool-frontend:v3.0.1 rewrites the generated nginx-mempool.conf to
re-resolve the backend per-request via 'resolver' + a variable proxy_pass.
Resolver address is read from /etc/resolv.conf (podman aardvark-dns answers
on the network gateway, not Docker's 127.0.0.11). Per-location path mapping
preserved (ws -> '/', /api/v1 identity via no-URI, /api/ -> /api/v1/ rewrite).
Proven on .228: backend IP change now auto-recovers with no reload; the
literal-host control still 502s. Migrated the manifest off the retired
tx1138 registry to vps2.

Also: mempool.bats #74 waited only 180s post-restart (the slow path) and
called an undefined 'fail' helper (status 127). Bumped to 300s to match the
passing parity probes and emit a real failure instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago
2026-06-22 18:07:07 -04:00
co-authored by Claude Opus 4.8
parent c8acc84506
commit 0f05f73a23
8 changed files with 167 additions and 8 deletions
+10 -2
View File
@@ -129,14 +129,22 @@ mempool_skip_if_absent() {
mempool_skip_if_absent
# mempool-api on :8999 — same probe required-stack.bats uses for parity.
local deadline=$(( $(date +%s) + 180 ))
# This case runs immediately after package.restart, so mempool-api has just
# dropped + must re-establish its electrs/bitcoin connection (it reports
# "offline" in the frontend during this window). Give it the same recovery
# budget the passing parity probes use (required-stack-destructive: 240s,
# package-update-smoke: 300s) — 180s was too tight for the post-restart path.
local deadline=$(( $(date +%s) + 300 ))
while (( $(date +%s) < deadline )); do
if curl -fsS -m 5 "http://127.0.0.1:8999/api/v1/backend-info" >/dev/null 2>&1; then
return 0
fi
sleep 3
done
fail "mempool-api never responded on :8999"
# NB: bats-assert's `fail` is not loaded in this file (only ../lib/rpc.bash),
# so emit + return non-zero directly rather than calling an undefined helper.
echo "mempool-api never responded on :8999 within 300s" >&2
return 1
}
# ────────────────────────────────────────────────────────────────────