archy/docs/RETICULUM-TRANSPORT-PROGRESS.md

Reticulum mesh transport — progress tracker

Living status doc for the Reticulum (RNS+LXMF) third-transport work. Update this after every meaningful step. If a session is cut off mid-work, read this file first, then the plan, then resume at "Next up."

Full plan: .claude/plans/enchanted-strolling-rocket.md. Memory pointer: project_reticulum_transport_plan.md (auto-memory index).

Coordination note (2026-06-30): a separate agent owns concurrent Meshtastic work, scoped to mesh/meshtastic.rs + mesh/protocol.rs (see docs/SESSION-1.8.0-OTA-PROGRESS.md) and explicitly avoiding mesh/listener/session.rs transport plumbing + mesh/mod.rs routing, which this work owns. Stay out of meshtastic.rs/protocol.rs to avoid collisions.

Status at a glance

Phase	What	Status
0	Gate #1 — deterministic identity from Archy keys	✅ DONE, verified in venv AND in the PyInstaller binary (same dest hash)
0	Gate #2 — two-node LXMF-over-LoRa on real hardware	✅ PASSED 2026-06-30 — real RF announce + encrypted DM exchanged between .116's Heltec V3 RNode and a phone-flashed second RNode running Sideband
0	Gate #3 — external Sideband/MeshChat interop	✅ PASSED 2026-06-30 — same session as gate #2; Sideband is the stock external client this gate calls for
1	reticulum-daemon/ (Python rns+lxmf, Unix-socket RPC)	✅ scaffolded + tested (no radio); signed-identity announce also done (see below)
1	Packaging — PyInstaller single binary	✅ DONE + verified — reticulum-daemon/build.sh, 16M standalone binary, selftest passes run from /tmp with no venv on PATH
2	Rust wiring (DeviceType, MeshRadioDevice, ReticulumLink, stamp sites)	✅ cargo check/cargo test -p archipelago GREEN (99 mesh tests pass) — still untested on real hardware
2c	MeshConfig.device_kind reflashable-board pin	✅ DONE this session (was the one open Phase-2 item)
3	Frontend (~8 label/CSS spots)	✅ DONE (scoped down — see note below)
4	Multi-device (run all 3 radios at once) + per-network channels	⏳ not started (follow-on, after 0–3)

Checkpoint 2026-06-30 (late session — read this first if cut off)

This session picked up after Phase 2/3 were already green, and closed out everything that didn't need real RNode hardware:

1. Corrected two stale tracker entries (both were already done, just not reflected here):
   • The _announce_app_data "TODO" was actually already implemented: reticulum_daemon.py's _announce_app_data() embeds ARCHY:2:{ed}:{x25519} when --archy-ed-pubkey-hex/--archy-x25519-pubkey-hex are passed, and reticulum.rs's daemon_command()/open() already forward our_ed_pubkey_hex/our_x25519_pubkey_hex from session.rs (run_mesh_session → auto_detect_and_open/open_preferred_path → ReticulumLink::open). Confirmed end-to-end by reading the call chain, not just grepping.
   • Phase 3 frontend was already done (see prior entry below) — tracker table above said "not started", now corrected.
2. Added MeshConfig.device_kind: Option<DeviceType> (plan §2c, the one explicitly-listed open Phase-2 item) — mesh/mod.rs (field + Default + threaded into start()'s spawn_mesh_listener call), listener/mod.rs (spawn_mesh_listener param → run_mesh_session arg), listener/session.rs (run_mesh_session param; auto_detect_and_open skips non-matching probes per-path via device_kind.is_none_or(|k| k == ...); open_preferred_path restructured to a match kind { ... } that tries only the pinned driver and surfaces its real error, instead of silently falling through to another firmware's handshake on the same port). None (default) preserves today's strict Meshcore→Meshtastic→Reticulum auto-detect — fully backward compatible, no config migration needed. cargo check + cargo test -p archipelago both green after (99 mesh tests, 0 failed).
3. Built and verified the PyInstaller packaging (plan's Phase 1 "Packaging" + the file list's "Ops: release packaging to include the daemon binary" item — previously undone):
   • reticulum-daemon/build.sh (new) — reproducible build, installs requirements-build.txt (new, pyinstaller==6.21.0, build-only/not shipped) into the existing .venv, runs PyInstaller with flags discovered by trial: --collect-submodules RNS --collect-submodules LXMF --collect-data RNS -d noarchive.
   • Non-obvious gotcha, written up in build.sh's comments so it isn't re-discovered: RNS.Interfaces/__init__.py builds its __all__ via glob.glob(os.path.dirname(__file__) + "/*.py") at import time (Reticulum.py does from RNS.Interfaces import *). PyInstaller's default --onefile zips pure-Python modules into an in-binary PYZ archive, so __file__ doesn't point at a real directory and the glob comes back empty → NameError: name 'Interface' is not defined the moment RNS.Reticulum(...) is constructed. -d noarchive (keep modules as loose .pyc files on disk inside the onefile bundle's runtime-extraction dir) fixes it — confirmed by reproducing the failure first, then fixing it.
   • Verified, not just built: ran the resulting dist/archy-reticulum-daemon binary's --check (dest hash matches the venv-derived 06bb31e16f4f8d46a8ae8eac23a4fd21 for the test seed) and --selftest (full RNS+LXMF bring-up, no radio) both from /tmp with the binary copied away from the repo and the .venv not on PATH — confirms it's genuinely self-contained, not accidentally still depending on the dev venv.
   • dist//build//*.spec are already gitignored (reticulum-daemon/.gitignore); only build.sh + requirements-build.txt are new tracked files.

NOT done this session (still genuinely open):

• Everything hardware-dependent (Phase 0 gates #2/#3, real RNode probe/spawn). The .116 Heltec V3 reflash mentioned in the prior session's memory was not done in this session — no physical hardware access was exercised, only software.
• /dev/reticulum-radio udev symlink (plan §2c) — deliberately not added: the existing 99-mesh-radio.rules keys on USB vendor/product ID (e.g. CP2102 0x10c4/0xea60), but the whole point of device_kind is that the same chip can run any of the three firmwares — a vendor/product udev rule can't disambiguate them, and a fabricated rule would just be misleading. Real fix needs either a per-device ATTRS{serial}==... rule the operator fills in once they know their specific board's serial (no such board exists in-repo to template from yet), or rely on device_kind alone (already done, works regardless of /dev path naming). Revisit once a real RNode-flashed board's serial is known.
• PyInstaller binary not yet wired into the release tarball / scripts/deploy-to-target.sh (the daemon binary path is currently resolved via ARCHY_RETICULUM_DAEMON_BIN env or the dev venv fallback in reticulum.rs's daemon_command() — production default /usr/local/bin/archy-reticulum-daemon is a real path convention now that build.sh produces exactly that filename, but nothing copies it there yet). Left undone deliberately — wiring release-tarball plumbing for a binary that's never been run against real RNS network traffic felt premature; do this once Phase 0 gates #2/#3 pass.

Phase 2 — Rust wiring detail (what's done vs left)

Done — cargo check -p archipelago is GREEN:

• core/archipelago/src/mesh/types.rs — DeviceType::Reticulum (+ Display arm) + a radio_transport_label(DeviceType) -> &'static str helper ("reticulum" vs "lora").
• core/archipelago/src/mesh/mod.rs — all 4 outbound stamp sites use radio_transport_label(...); use_typed_envelope (~1571) extended to matches!(device_type, Meshcore | Reticulum); data_dir threaded into spawn_mesh_listener(...) call (was: MeshService::start() → spawn_mesh_listener).
• core/archipelago/src/mesh/listener/mod.rs — spawn_mesh_listener takes data_dir: PathBuf, passes &data_dir into run_mesh_session.
• core/archipelago/src/mesh/listener/decode.rs:406,639 and dispatch.rs:79 — all 3 inbound stamp sites now use radio_transport_label(state.status.read().await.device_type).
• core/archipelago/src/mesh/listener/session.rs:
  • MeshRadioDevice enum has Reticulum(ReticulumLink); all 18 method arms wired (no-ops: ensure_lora_region, ensure_channel, send_keepalive, send_nodeinfo_advert, reboot, reset_contact_path; everything else forwards to ReticulumLink).
  • auto_detect_and_open(data_dir: &Path) and open_preferred_path(path, data_dir: &Path) both now try ReticulumLink::open(path, data_dir) last, after Meshcore/Meshtastic — cheap raw-serial KISS-detect probe runs first; the daemon only spawns on a confirmed match.
  • reticulum_contact_id() helper added (delegates to the canonical reticulum::reticulum_contact_id_from_hash, masked & 0x7FFF_FFFF, avoids 0).
  • refresh_contacts() has an is_reticulum branch parallel to is_meshtastic; reachable flows through contact.path_len != 0 unchanged (ReticulumLink::get_contacts() already encodes daemon-reported reachability into path_len).
  • data_dir: &Path threaded through run_mesh_session → both probe functions.
• core/archipelago/src/mesh/reticulum.rs — created. ReticulumLink: spawns/supervises the daemon as a child process, Unix-socket RPC client (matches the tested daemon contract), prefix_to_hash: HashMap<[u8;6],[u8;16]> (mandatory per the plan), synthetic InboundFrame builder byte-matching meshtastic.rs's layout, Drop impl that kills the daemon + cleans up the socket. Has unit tests (KISS-detect byte matching, contact-id masking, synthetic-frame layout) — passing, see below.

Concurrent-edit note: a separate in-flight change (not mine) added MeshPeer.pkc_capable and ParsedContact.pkc_capable (Meshtastic PKI-capability tracking) while this work was in progress. Accounted for: reticulum.rs's ParsedContact literal sets pkc_capable: false (Reticulum/LXMF is unconditionally E2E via take_rx_encrypted(), this field has no analogue); two incomplete MeshPeer literals in decode.rs (lines ~330, ~548) were completed with pkc_capable: false to unblock the build for everyone — not reverted, not worked around.

Self-review fix applied: the RPC Unix socket originally lived in the shared system temp dir; moved to {data_dir}/reticulum/ (0700) instead — archipelago-owned, not shared /tmp, matching the security posture. Re-confirmed cargo check -p archipelago GREEN after the move.

NOT yet done:

• MeshConfig.device_kind: Option<DeviceType> hint (optional reflashable-board disambiguator, plan §2c) — not added. Auto-detect ordering (Meshcore→Meshtastic→Reticulum, strict probes) is the only disambiguator right now.
• Phase 3 frontend — DONE, but smaller scope than originally inventoried: only Mesh.vue's transportLabel() (per-message field) + mesh-styles.css .transport-reticulum
  • the mesh.ts doc comment needed the addition. transport.ts TransportKind, federation/types.ts last_transport, NodeList.vue transportBadge, and PeerFiles.vue transportPill are a COARSER routing-layer category (mesh/lan/fips/tor) where 'mesh' already covers any radio (meshcore/meshtastic/reticulum) — adding a separate 'reticulum' there would be inconsistent with how meshcore/meshtastic are handled. Confirmed via vue-tsc --noEmit (exit 0, zero errors).
• Everything hardware-dependent: real daemon spawn/probe against an actual RNode (the .116 Heltec V3, once reflashed), two-node LXMF-over-LoRa, the _announce_app_data signed-identity TODO in the daemon (currently carries only the plaintext display name, not a verified Archy DID/pubkey — needed for bind_federation_twins-style auto-binding across protocols).

Verified facts to reuse (don't re-derive)

RNode KISS-detect handshake (confirmed against the canonical Reticulum source, not guessed):

constants: FEND=0xC0 FESC=0xDB TFEND=0xDC TFESC=0xDD CMD_DETECT=0x08 DETECT_REQ=0x73 DETECT_RESP=0x46
probe tx:  C0 08 73 C0 50 00 C0 48 00 C0 49 00 C0   (detect + fw_version + platform + mcu queries)
success:   response contains byte sequence ... C0 08 46 ...  (FEND, CMD_DETECT, DETECT_RESP)

Source: RNS/Interfaces/RNodeInterface.py (Liberated Systems mirror), detect()/readLoop().

Synthetic InboundFrame layout for a 1:1 DM, copied exactly from meshtastic.rs:1031-1047 (ReticulumLink must build the same shape so frames::handle_frame needs zero changes):

data = [snr(1)=0][reserved(2)=00,00][sender_prefix(6)][path(1)=0xff][type(1)=0][rx_time(4 LE)][payload…]
code = RESP_CONTACT_MSG_V3_E2E if encrypted else RESP_CONTACT_MSG_V3   (RNS/LXMF is always E2E, so always _E2E)

Channel/broadcast equivalent (RESP_MESHTASTIC_CHANNEL_TEXT, meshtastic.rs:1019-1028) — N/A for Reticulum in single-device Phase 2 (LXMF has no shared-channel concept); revisit in Phase 4.

resolve_peer (decode.rs:316) matches inbound sender_prefix against peer.pubkey_hex.starts_with(prefix) — so as long as refresh_contacts/announce-handling populates pubkey_hex = full 16-byte RNS hash hex BEFORE a message arrives (same precondition meshtastic relies on via its peer_pubkeys map), no Reticulum-specific fallback is needed there.

ParsedContact.public_key_hex for Reticulum = hex of the 16-byte RNS dest hash (32 hex chars, NOT 32 bytes) — the hex::decode(...).len()==32 checks elsewhere (e.g. the auto-heal reset_contact_path loop in refresh_contacts) will naturally skip Reticulum contacts since their key decodes to 16 bytes, not 32. That's fine — no special-casing needed, just don't "fix" it to be 32 bytes.

data_dir.join("identity").join("node_key") is the 32-byte raw Ed25519 seed file — this is exactly what reticulum_daemon.py --identity-key <path> expects (confirmed against identity.rs NODE_KEY_FILE/load_or_create). The daemon reads the file itself — Rust should pass the path, not pipe the raw key bytes through more hops than already exist.

Hardware update (2026-06-30)

.116 has a Heltec V3 available to reflash with RNode firmware. This unblocks Phase 0 gates #2/#3 (previously marked blocked — .198's radio is dead, but .116's Heltec V3 is a real path forward without needing new hardware). Next concrete step once reflashed: run reticulum-daemon/reticulum_daemon.py pointed at the RNode's serial path, confirm --check hash matches --selftest, then bring up two instances (.116 + .228, after .228 also gets an RNode-capable board) for the real two-node LXMF-over-LoRa gate.

Daemon contract (already built + tested — Phase 2 codes against this, no changes needed)

reticulum-daemon/reticulum_daemon.py, RPC over Unix socket (0600), one JSON object per line:

• in: {"cmd":"send","dest_hash":hex16,"content":...} / {"cmd":"announce"} / {"cmd":"status"} / {"cmd":"shutdown"}
• out: {"event":"ready",...} / {"event":"recv",...} / {"event":"announce",...} / {"event":"delivered",...} / {"event":"status",...} Verified: --check (hash only), --selftest (boots real RNS+LXMF, no radio), and a live socket round-trip (ready→status→shutdown, clean exit) — see reticulum-daemon/README.md.

Checkpoint 2026-06-30 (hardware session — gates #2/#3 PASSED)

Picked up after a session pipe-break; the live system (archipelago.service + the spawned archy-reticulum-daemon) had kept running uninterrupted the whole time, so nothing was lost.

What happened, in order:

1. .116's Heltec V3 (CP2102, USB vendor/product 10c4:ea60, serial 0001) was reflashed with RNode firmware and plugged into /dev/mesh-radio (generic udev symlink → ttyUSB0, not a per-serial rule). mesh-config.json has device_path: null — pure auto-detect, no device_kind pin needed.
2. Auto-detect correctly tried Meshcore → Meshtastic → Reticulum and found it: journal shows Found Reticulum (RNode) device via auto-detect path=/dev/mesh-radio — but only after ~4 min of Failed to spawn reticulum-daemon — is it installed/packaged? retries, because /usr/local/bin/archy-reticulum-daemon hadn't been copied into place yet from reticulum-daemon/dist/ (built via ./build.sh). Once copied (sha256-verified match to the dist/ build), auto-detect succeeded on the very next retry.
3. mesh.status RPC confirmed live: device_type: "reticulum", device_connected: true, dest_hash: 5d146f6e1c9707f89468b5016ed6dfad. Periodic self-advert (send_self_advert → {"cmd":"announce"} → real RNS Identity.announce()) firing every ~30s — confirmed this is not the send_nodeinfo_advert no-op arm (that one's still legitimately a no-op for Reticulum; the real announce path is send_self_advert, wired correctly).
4. Second RNode flashed onto a phone running Sideband. First attempt showed RF energy (interference_last_dbm climbing) but rxb: 0 — a parameter mismatch, not a frequency problem (energy was detected, just not demodulated). Root cause: Spreading Factor mismatch in Sideband's manual RNode interface config (frequency display rounds to one decimal so "869.5" silently passed at first glance — bandwidth/SF/CR are separate fields and SF was wrong). Once SF was corrected to match (freq 869525000, BW 125000, SF 8, CR 5), rxb went non-zero immediately and a real {"event":"announce","dest_hash":"1870744d...", "app_data":"7a617a61"} (hex for "zaza") arrived over the air.
5. Gate #2 + gate #3 both passed in the same exchange: zaza shows up as a real, reachable mesh.peers contact; an inbound encrypted LXMF message ("Yoooo") arrived and was correctly stamped encrypted: true, transport: "reticulum"; a reply was sent back and round-tripped. Sideband is exactly the stock external client gate #3 calls for, so one real RNode-to-RNode LoRa link covered both gates — no need for a second dedicated archy node.
6. Two real bugs found from this, both fixed:
   • record_sent_typed's encrypted flag was hardcoded false/archy || pkc_capable on the Reticulum send path (both the native-text path in send_message and the typed-envelope path in send_typed_wire) — correct for Meshcore/Meshtastic (where E2E really is conditional on PKI/session state not yet threaded through), wrong for Reticulum: LXMF encrypts every send to the destination identity key unconditionally, archy peer or not. Fixed: both call sites now OR in device_type == DeviceType::Reticulum.
   • radio_transport_label() collapsed Meshcore and Meshtastic into one generic "lora" string, so the per-message pill couldn't distinguish them. User asked for 3 distinct pill colors (Meshtastic mint, Meshcore orange, Reticulum blue) — extended the label fn to return "meshtastic"/"meshcore"/"reticulum" distinctly, updated Mesh.vue's transportLabel() switch and mesh-styles.css (.transport-meshtastic #3eb489, .transport-meshcore #fb923c, .transport-reticulum #60a5fa; kept .transport-lora #f59e0b as a fallback for any already-stored legacy-labelled messages). cargo check + vue-tsc --noEmit both green after.

NOT yet done:

• The Rust-side fix above (encrypted flag, transport-label split) is built but not yet deployed to .116's running binary — the live daemon/auto-detect verification above was all against the binary already running before this session's edits. Rebuild + redeploy to see the fix live.
• tests/lifecycle/run-gate.sh not re-run after these mesh changes yet (project convention: run after backend changes land).
• Multi-device (3 radios at once, Phase 4) and the release-tarball/udev-rule wiring (originally "Next up" #6 below) are both still untouched.

Next up (resume here)

Phase 0 gates #1–#3 are now all passed. What's left:

1. Rebuild the backend + frontend and redeploy to .116 so the encrypted-flag fix and the 3-way transport-pill color split actually take effect on the live node (currently only checked in with cargo check/vue-tsc, not deployed).
2. Re-verify on-device after redeploy: send another Sideband↔archy DM, confirm the Sent bubble now shows E2E + a blue "Reticulum" pill, and confirm Meshtastic/Meshcore pills (if any messages exist) render mint/orange instead of the old generic amber "LoRa".
3. Exercise the rest of the plan's "Verification (definition of done)" items: hot-swap detection (unplug the RNode mid-session, confirm fallback to FIPS/Tor on the same contact; replug, confirm it picks Reticulum back up), and device_kind: Some(Reticulum) pin path (currently only auto-detect has been exercised on real hardware).
4. Run tests/lifecycle/run-gate.sh to confirm no regression from the mesh changes landing.
5. Only after the above: wire dist/archy-reticulum-daemon into the release tarball / scripts/deploy-to-target.sh (target path /usr/local/bin/archy-reticulum-daemon, matching reticulum.rs's default) and add a per-serial-number /dev/reticulum-radio udev rule now that a real board's serial number (0001 on the CP2102, .116's board) is known — though a second board will likely report the same 0001 stock serial since CP2102 modules commonly ship with an unprogrammed default, so this may still need a different disambiguator.
6. Phase 4 (run all 3 radios at once) — still not started, follow-on after the above.