archy/docs/dht-RESUME.md
archipelago f14829542b docs(dht): RESUME checkpoint — state, next steps, build/worktree rules
Single source of truth for picking the DHT work back up after a restart:
worktree/branch rules, all phase commits, the exact next task (#12 Phase 3
glue), build-time facts, and the Phase 0 go-live ceremony.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:18:00 -04:00

5.8 KiB

DHT work — RESUME HERE

Last updated: 2026-06-16 · Branch: agent-trust-wip · Worktree: ~/Projects/archy-dht

This file is the single source of truth for resuming the DHT / peer-distribution work after a restart. Read it top to bottom, run the Verify state block, then continue at Next step.


⚠️ CRITICAL — where to work (do not skip)

  • Work ONLY in the worktree ~/Projects/archy-dht on branch agent-trust-wip.
  • NEVER run git checkout / branch-switch / commit in the shared tree ~/Projects/archy. Another agent cuts releases on main there. Git branch state is global to one working tree, so a checkout in the shared tree drags every session onto that branch and can clobber uncommitted work. That already happened once — the worktree exists specifically to prevent it. See memory feedback_concurrent_agent_tree.
  • The shared tree stays on main for the release agent. Leave it alone.

Build facts (so you don't get surprised)

  • It's a binary crate: test with cargo test --bin archipelago -- <filter> (there is no lib target).
  • The test profile is opt-level=3 → every incremental test rebuild of the archipelago crate is ~5 min; a cold build of the iroh feature tree is ~19 min. Budget for it. Run builds in the background and poll.
  • Default build = no iroh. The iroh swarm engine is behind the iroh-swarm Cargo feature (off by default): cargo build --features iroh-swarm.
  • Plain cargo build (no feature) is the fleet build and is unaffected by any DHT work.

Verify state (run these first on resume)

cd ~/Projects/archy-dht
git branch --show-current            # → agent-trust-wip
git log --oneline -7                 # see the commit list below
git status --short                   # should be clean (or your in-progress edits)
git worktree list                    # archy-dht → agent-trust-wip; archy → main
# sanity compile (default, fast-ish):
cargo build --bin archipelago 2>&1 | tail -3

What is DONE (committed on agent-trust-wip)

Design doc: docs/dht-distribution-design.md (the full plan).

Commit Phase Summary
0fef8086 base parked trust module + seed::derive_release_root_ed25519 (pre-existing)
27f11bf8 0 signed-catalog authenticity wired: trust/ module verifies the release-root detached signature in app_catalog::fetch_one; release-root KAT pinned
f0cb91ed 1 BLAKE3 alongside SHA-256: content_hash.rs, ComponentUpdate.blake3, BlobMeta.blake3
2523c9e3 2 seam swarm/mod.rsBlobProvider + fetch_content_addressed (verify peer bytes, origin-always-wins); iroh-swarm flag; wired into update.rs
082946aa 2 engine real swarm/iroh_provider.rs over iroh 1.0 + iroh-blobs 0.103 (optional deps). Dep tree proven to resolve+compile against the pinned stack
9fa56a82 3 core swarm/seed_advert.rs — signed Nostr seed-advertisement protocol (NIP-33 kind 30081, d-tag=blake3)

All tests green at each step. Total new modules: trust/, content_hash.rs, swarm/.

NEXT STEP — task #12 (Phase 3 glue + wiring)

Implement, in the worktree:

  1. NostrSeedDiscovery (feature-gated, in swarm/iroh_provider.rs or a new swarm/discovery.rs): implement the ProviderDiscovery trait by querying relays with seed_advert::advertisement_filter(hash), then seed_advert::endpoint_ids_from_events(...) → parse each string into iroh::EndpointId (EndpointId::from_str / parse). Skip ids that don't parse.
    • NOTE: ProviderDiscovery::providers_for is currently sync. The relay query is async → either change the trait to #[async_trait] async fn, or back it with an in-memory cache refreshed by a background subscription. Async trait is cleaner (the caller try_fetch is already async).
  2. Publish path: when a node finishes downloading / already holds a public release/app-image blob, publish seed_advert::advertisement_builder(blake3, my_endpoint_id) signed with the node's Nostr key (nostr_discovery.rs has the load_or_create_nostr_keys + Client + send_event_builder patterns to reuse). Scope: releases/catalog blobs only — never private user blobs.
  3. Wire swarm::providers() to construct an IrohProvider (with the NostrSeedDiscovery) from runtime config — needs an enable flag + relay list + data_dir. Likely make providers() async / build it once at startup and pass a handle into the update path. Until this is wired, providers() returns empty and everything uses origin (safe).

Then verify: cargo build --features iroh-swarm + cargo test --bin archipelago -- swarm::.

After Phase 3

  • Phase 4 — IndeeHub films on the same blob layer (Blossom catalog + iroh swarm; MinIO origin). Each HLS .ts segment = a content-addressed blob.
  • Phase 0 GO-LIVE (needs the user) — the catalog/manifest signature anchor trust::anchor::RELEASE_ROOT_PUBKEY_HEX is still None; the pinned KAT is the TEST mnemonic, not the real key. Going live = signing ceremony with the real release master seed (only the user has it) → derive release-root → bake its pubkey into anchor.rs → sign the real releases/app-catalog.json. Until then verification is advisory (verify-if-present, anchor not enforced).

Mergeability

As of last check we were only ~4 commits diverged from main; the only shared-file overlap is seed.rs + update.rs. Do NOT merge to main while the release is in flight — that's the user's call. Sync (merge main → agent-trust-wip) once the release lands and main is clean.

Background build logs from the last session (may be stale)

/tmp/dht-*.log — phase test/build outputs. Safe to ignore/delete on resume.