Commit Graph
5 Commits
Author SHA1 Message Date
archipelago 0d0e2e243a feat(lnd): channel-peer watchdog — a dropped peer link heals itself
Demo images / Build & push demo images (push) Successful in 3m49s
LND normally reconnects channel peers after a restart, but not reliably:
after long or repeated downtime (an app update, a node reboot,
reconciler churn) the peer link can stay down for hours while BOTH
endpoints keep the channel flagged disabled in the routing graph. The
node looks perfectly healthy, the wallet shows balance, and every
payment in either direction fails "no route to the recipient" —
observed live on framework-pt (2026-09-01): its only channel sat
disabled on both policy sides for ~17 hours after the LND 0.21.2
update, while shorty had 583k spendable and the user was told, by a
mis-mapped modal, that they had 'no payment channel'.

The channel graph is desired state — every open channel should have a
live peer connection. A daemon-side watchdog now enforces it:

- every 2 minutes, list channels + peers over LND REST
- for each channel whose remote peer is not connected, look the peer's
  advertised addresses up in the public graph and dial one
- per-peer retries throttled to 10 minutes so an unreachable peer is
  not hammered; 'already connected' counts as done; a peer with no
  advertised address is logged once per pass (cannot be dialed)
- no-ops quietly on nodes without LND (missing macaroon) and while a
  wallet is locked (503 body has no channels)

Unit tests pin the selection against the live REST shapes
(remote_pubkey in /v1/channels vs pub_key in /v1/peers).

v1.8.10 CHANGELOG + What's New entries staged so the next release run
is clean first time.
2026-09-01 17:51:15 -04:00
archipelago 9c49b502e3 docs: post-1.8.9 verification — pipeline confirmed, routing failure root-caused to framework-pt's disabled channel 2026-09-01 16:36:27 -04:00
archipelago d68a013e35 docs: tracker — v1.8.9 published, NPM live-healed on shorty via the signed catalog; funding-gate fix staged for v1.8.10 2026-09-01 11:43:33 -04:00
archipelago ed49cc974f docs: tracker updated — fixes landed, tests green, remaining steps are the two mnemonic ceremonies + node updates 2026-09-01 10:31:36 -04:00
archipelago 4849186ab9 docs: incident tracker for the 2026-09-01 https/launcher/LND breakage + v1.8.9 notes
Demo images / Build & push demo images (push) Successful in 3m57s
Root-cause table, fix inventory, regression-test inventory and the
deploy/live-verification checklist for today's fleet incident — written
as the working record while the fixes land, so the deploy + verify steps
can be checked off against real nodes rather than memory. CHANGELOG
carries the user-facing notes for the release these fixes ship in.
2026-09-01 10:29:10 -04:00