feat(fips): auto-peer LAN-discovered federation nodes directly over FIPS

Mesh/federation messages between co-located nodes were always falling back to
Tor because the FIPS overlay had no direct peering — every node depended on the
global anchor's spanning tree, and when that anchor link flaps a node is
isolated and all FIPS dials time out. (Diagnosed live on .116/.198: pure-FIPS
direct peering over UDP 8668 fixes it — 2.5ms vs timeout.)

Generalize the manual fix: in the existing 5-min FIPS seed-anchor apply loop,
also auto-connect every federation peer the PeerRegistry knows both a LAN
address AND a FIPS npub for, dialing its FIPS UDP transport (port 8668) at its
LAN IP via the same idempotent `fipsctl connect` path (new
anchors::lan_fips_anchors). This is FIPS's own transport over the LAN — NOT
Tailscale, NOT the HTTP/LAN messaging port. Transient (recomputed each tick from
live mDNS discovery, never persisted) so changing IPs self-correct. Remote peers
with no LAN address are untouched (still routed via the anchor).

Registry Arc hoisted out of the transport-init block so the loop can read
all_peers(). cargo check green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
archipelago 2026-06-29 06:42:18 -04:00
parent 11155055aa
commit 20f762cb2c
2 changed files with 60 additions and 0 deletions

View File

@ -216,6 +216,44 @@ pub struct ApplyResult {
pub message: String,
}
/// FIPS UDP transport port (matches `transports.udp.bind_addr` in the generated
/// `fips.yaml`). Direct peer links dial this, NOT the HTTP/LAN messaging port.
const FIPS_UDP_PORT: u16 = 8668;
/// Build transient seed-anchor entries that dial LAN-discovered federation peers
/// directly over their FIPS UDP transport. For each peer the registry knows both
/// a LAN socket address AND a FIPS npub for, point a `udp` anchor at
/// `<lan-ip>:8668`. This lets co-located federation nodes form a DIRECT FIPS link
/// instead of depending on the global anchor's spanning tree to route between
/// them (the cause of every dial falling back to Tor when the anchor link flaps).
///
/// This is FIPS's own UDP transport over the LAN — not Tailscale, not the LAN
/// HTTP messaging port. NOT persisted to `seed-anchors.json`: recomputed each
/// apply tick from live LAN discovery, so a peer's changing IP self-corrects and
/// stale entries never accumulate. `fipsctl connect` is idempotent, so
/// re-applying just keeps the link warm.
pub fn lan_fips_anchors(peers: &[crate::transport::PeerRecord]) -> Vec<SeedAnchor> {
let mut out = Vec::new();
for p in peers {
let (Some(lan), Some(npub)) = (p.lan_address.as_deref(), p.fips_npub.as_deref()) else {
continue;
};
// lan_address is the peer's HTTP/LAN socket ("ip:port"); reuse only its IP
// and target the FIPS UDP port. SocketAddr::new(...).to_string() formats
// IPv6 with brackets correctly.
let Ok(sa) = lan.parse::<std::net::SocketAddr>() else {
continue;
};
out.push(SeedAnchor {
npub: npub.to_string(),
address: std::net::SocketAddr::new(sa.ip(), FIPS_UDP_PORT).to_string(),
transport: "udp".to_string(),
label: "LAN federation peer (direct FIPS)".to_string(),
});
}
out
}
#[cfg(test)]
mod tests {
use super::*;

View File

@ -355,6 +355,9 @@ impl Server {
}
// Initialize transport router (unified routing: mesh > lan > tor)
// Hoisted so the FIPS seed-anchor loop below can auto-peer LAN-discovered
// federation peers directly over FIPS (see that loop).
let mut fips_peer_registry: Option<std::sync::Arc<crate::transport::PeerRegistry>> = None;
{
let data_dir = config.data_dir.clone();
let did =
@ -368,6 +371,7 @@ impl Server {
match crate::transport::PeerRegistry::load(&data_dir).await {
Ok(registry) => {
let registry = std::sync::Arc::new(registry);
fips_peer_registry = Some(registry.clone());
let mut transports: Vec<Box<dyn crate::transport::NodeTransport>> = Vec::new();
// Tor transport (always register — availability checked dynamically)
@ -641,6 +645,7 @@ impl Server {
// onboarding before we start dialing.
{
let data_dir = config.data_dir.clone();
let fips_peer_registry = fips_peer_registry.clone();
tokio::spawn(async move {
tokio::time::sleep(Duration::from_secs(30)).await;
let mut interval = tokio::time::interval(Duration::from_secs(300));
@ -655,6 +660,23 @@ impl Server {
tracing::debug!("Seed-anchor apply: load failed (non-fatal): {}", e)
}
}
// Auto-peer federation nodes we've discovered on the LAN
// directly over FIPS, so co-located peers don't depend on the
// (often flaky) global anchor's spanning tree to route to each
// other. For every peer the registry knows both a LAN address
// AND a FIPS npub for, dial it on its FIPS UDP transport port
// (8668) at its LAN IP. This is FIPS's own transport over the
// LAN — NOT Tailscale, NOT the HTTP/LAN messaging port. Pure
// FIPS. `fipsctl connect` is idempotent, so re-applying every
// tick just keeps the direct link warm; unknown/remote peers
// (no LAN address) are left to the anchor as before.
if let Some(reg) = fips_peer_registry.as_ref() {
let direct = crate::fips::anchors::lan_fips_anchors(&reg.all_peers().await);
if !direct.is_empty() {
let _ = crate::fips::anchors::apply(&direct).await;
}
}
}
});
}