docs(master-plan): §8b session-b state — connection-lost+netbird+UX-merge shipped to .228, uninstall ghost fix, workstream F in progress

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

core/archipelago/src/api/rpc/package/runtime.rs

@@ -312,7 +312,16 @@ impl RpcHandler {
 
         let mut stopped = 0u32;
         let mut removed = 0u32;
-        let mut errors = Vec::new();
+        // Two distinct failure classes, kept separate so they don't get
+        // conflated (the old single `errors` vec did, which caused the "ghost in
+        // My Apps" bug): `container_errors` means a container could NOT be
+        // removed (force-rm failed too) — the app is genuinely still present, so
+        // we keep its state entry and surface a hard error. `cleanup_errors`
+        // means volume/network/data-dir teardown left residue — the containers
+        // are already gone, so the app IS uninstalled and MUST disappear from My
+        // Apps; the residue is logged but never ghosts the app.
+        let mut container_errors: Vec<String> = Vec::new();
+        let mut cleanup_errors: Vec<String> = Vec::new();
 
         self.set_uninstall_stage(
             package_id,
@@ -370,7 +379,7 @@ impl RpcHandler {
                             let msg =
                                 format!("Failed to remove {}: {}; {}", name, stderr.trim(), e);
                             tracing::error!("Uninstall {}: {}", package_id, msg);
-                            errors.push(msg);
+                            container_errors.push(msg);
                         }
                     }
                 }
@@ -379,12 +388,35 @@ impl RpcHandler {
                     Err(force_err) => {
                         let msg = format!("Failed to remove {}: {}; {}", name, e, force_err);
                         tracing::error!("Uninstall {}: {}", package_id, msg);
-                        errors.push(msg);
+                        container_errors.push(msg);
                     }
                 },
             }
         }
 
+        // A container that survived even force-remove means the app is NOT
+        // actually uninstalled — keep its state entry and fail so the spawned
+        // task reverts it to its prior state (and the user can retry), rather
+        // than orphaning a live container that's missing from My Apps.
+        if !container_errors.is_empty() {
+            tracing::error!(
+                "Uninstall {}: containers could not be removed: {:?}",
+                package_id,
+                container_errors
+            );
+            return Err(anyhow::anyhow!(
+                "Uninstall {} failed: {}",
+                package_id,
+                container_errors.join("; ")
+            ));
+        }
+
+        // Containers are gone → the app is uninstalled. Remove its state entry
+        // NOW, before the (possibly slow, possibly fallible) volume/data
+        // teardown below, so My Apps updates immediately and a residue failure
+        // can never leave a ghost. Reinstall/scan no longer see a stale entry.
+        self.remove_package_state_entry(package_id).await;
+
         self.set_uninstall_stage(package_id, "Cleaning up volumes")
             .await;
         // Avoid global Podman volume prune on production nodes: store-wide
@@ -432,45 +464,55 @@ impl RpcHandler {
                         let stderr = String::from_utf8_lossy(&o.stderr);
                         let msg = format!("Failed to remove data {}: {}", dir, stderr.trim());
                         tracing::error!("Uninstall {}: {}", package_id, msg);
-                        errors.push(msg);
+                        cleanup_errors.push(msg);
                     }
                     Err(e) => {
                         let msg = format!("Failed to remove data {}: {}", dir, e);
                         tracing::error!("Uninstall {}: {}", package_id, msg);
-                        errors.push(msg);
+                        cleanup_errors.push(msg);
                     }
                     _ => {}
                 }
             }
         }
 
-        if !errors.is_empty() {
+        // The app is already gone from My Apps (entry removed above). Residual
+        // volume/data cleanup failures are logged but NEVER ghost the app — a
+        // reinstall and the next uninstall both tolerate leftover dirs.
+        if !cleanup_errors.is_empty() {
             tracing::error!(
-                "Uninstall {} completed with errors: {:?}",
+                "Uninstall {} removed but left cleanup residue: {:?}",
                 package_id,
-                errors
+                cleanup_errors
             );
-            return Err(anyhow::anyhow!(
-                "Uninstall {} partially failed: {}",
-                package_id,
-                errors.join("; ")
-            ));
         }
 
         tracing::info!(
-            "Uninstall {} complete: stopped={}, removed={}",
+            "Uninstall {} complete: stopped={}, removed={}, cleanup_errors={}",
             package_id,
             stopped,
-            removed
+            removed,
+            cleanup_errors.len()
         );
 
-        // Immediately remove from in-memory state so the UI updates without
-        // waiting for the scanner's absence threshold (3 scans × 60s each).
-        {
+        Ok(serde_json::json!({
+            "status": "uninstalled",
+            "stopped": stopped,
+            "removed": removed,
+            "cleanup_warnings": cleanup_errors,
+        }))
+    }
+
+    /// Remove a package's entry (and any alias keys) from persisted state so it
+    /// disappears from My Apps immediately, without waiting for the scanner's
+    /// absence threshold (3 scans × 60s). Called as soon as an uninstall has
+    /// removed the app's containers — before the slower volume/data teardown —
+    /// so a residue failure can never leave a ghost entry behind.
+    async fn remove_package_state_entry(&self, package_id: &str) {
         let (mut data, _rev) = self.state_manager.get_snapshot().await;
         let before = data.package_data.len();
         data.package_data.remove(package_id);
-            // Also remove any alias keys (e.g. "bitcoin-knots" vs "bitcoin")
+        // Also remove any alias keys (e.g. "bitcoin-knots" vs "bitcoin").
         let aliases: Vec<String> = data
             .package_data
             .keys()
@@ -489,13 +531,6 @@ impl RpcHandler {
         }
     }
 
-        Ok(serde_json::json!({
-            "status": "uninstalled",
-            "stopped": stopped,
-            "removed": removed,
-        }))
-    }
-
     /// Start a bundled app (create container from pre-loaded image if needed).
     pub(in crate::api::rpc) async fn handle_bundled_app_start(
         &self,

docs/PRODUCTION-MASTER-PLAN.md

@@ -245,7 +245,26 @@ phases 2–6 (`dual-ecash-design.md`).
 
 ## 8b. SESSION STATE + RESUME (updated 2026-06-23) — READ §8b "CURRENT STATE + RESUME" FIRST
 
-### ▶ CURRENT STATE + RESUME (2026-06-23) — RESUME FROM HERE (works from any device)
+### ▶ SESSION b (2026-06-23 PM) — LATEST, RESUME FROM HERE
+
+**Canonical resume detail: memory `project_session_resume_2026_06_23b` (▶️ top of MEMORY.md).**
+`gitea-vps2/main = 4346007d` pushed; local HEAD `e57514b6` (uninstall fix, committed, **not pushed/deployed**).
+
+Shipped + verified live on .228 (all in 4346007d):
+- **Connection-lost FULLY fixed** — companion `image_exists` journal-flood (Stdio::null) + netbird UDP-port reconcile churn (`wait_for_manifest_host_ports` tcp-only). .228: flood→0, ws/db→0 disconnects, load 3.95→2.26.
+- **netbird → manifest-driven** (#20 ph4) — 3 manifests + 4 orchestrator primitives (base64 secret, GeneratedCert+`ensure_manifest_certs`, templated-file render `{{HOST_IP}}/{{NETWORK_GATEWAY}}/{{secret:}}`, udp port protocol). Live: https 8087→200, OIDC→200, resolver=gateway. Legacy-Rust delete deferred to post-full-verify.
+- **registry-manifest flip (code)** — `EMBED_MANIFESTS` default-on, `main.rs` bounded pre-load `refresh_catalog`. Catalog regenerated w/ 52 embedded manifests but **NOT published** (gitignored + never committed; publish = force-add to gitea-vps2 main). Do after fleet binary roll.
+- **UX regression root-caused + fixed** — the mobile/desktop UX (loader/AppLoadingScreen, store-driven launch, app icons, android webview footer) was on `companion-mobile-ux` and **never merged to main**, so any main build silently dropped it. **Merged → main**, frontend redeployed to .228. Android 0.4.9/code13 pushed for user to build APK elsewhere.
+
+In progress — **Workstream F lifecycle bugs** (this §, user-picked next):
+- **uninstall ghost — FIXED (e57514b6), not deployed.** `handle_package_uninstall` returned Err on any cleanup-residue failure *before* removing the package state entry → ghost in My Apps + revert-to-Installed. Now: split container vs cleanup errors; remove state entry as soon as containers gone (before slow data rm). NEXT: deploy + verify a real uninstall clears My Apps (CAUTION: destroys app data — get user OK).
+- grafana reinstall-stops (#14, likely same root cause, re-test after) + fedimint guardian wait-vs-stuck (#15, not started).
+
+Next: deploy+verify uninstall fix on .228 → push e57514b6 → roll binary to fleet → publish catalog → finish F → Phase 3 → multinode.
+
+---
+
+### ▶ CURRENT STATE + RESUME (2026-06-23) — earlier session-a baseline (historical)
 
 **✅ HEADLINE (2026-06-23): single-node gate GREEN (`run-gate.sh` 5/5 on .228, 0 not-ok) +
 multinode test deploy DONE to 6 nodes.** The exit criterion (§5) is met. Green took fixing **two real