archy/.planning/phases/13-aiui-functional-conversational-node-control-and-content-surf/13-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created

phase	slug	status	nyquist_compliant	wave_0_complete	created
13	aiui-functional-conversational-node-control-and-content-surf	draft	true	true	2026-08-03

Phase 13 — Validation Strategy

Per-phase validation contract for feedback sampling during execution. Seeded from 13-RESEARCH.md § Validation Architecture. Task IDs are filled in by the planner.

---

Test Infrastructure

This phase spans three test surfaces in two repositories (now one in-repo tree — D-19).

Property	Value
Framework (Rust)	cargo test — in-tree unit/integration tests (precedent: swarm/payment.rs #[tokio::test], pine_ha.rs #[test])
Framework (neode-ui)	Vitest 3.1 — neode-ui/package.json "test": "vitest run"
Framework (AIUI repo)	Vitest — aiui/packages/app: pnpm test (vitest run). Confirmed 2026-08-07 (353 passed / 3 pre-existing fixture failures, run repeatedly).
Config file	core/Cargo.toml (Rust) · neode-ui/vitest.config.ts (frontend) · aiui/packages/app/vite.config.ts
Quick run command	cd core && cargo test --package archipelago assistant:: · cd neode-ui && npx vitest run src/services/__tests__/contextBroker.test.ts
Full suite command	cd core && cargo test · cd neode-ui && npm run test · cd aiui/packages/app && pnpm test
Estimated runtime	Rust assistant suite ~2s (post-compile); Vitest targeted <10s; AIUI full ~19s

Build gotcha (CLAUDE.md): if cargo test hits rust-lld: undefined hidden symbol, that is incremental-cache corruption — rebuild with CARGO_INCREMENTAL=0. Not a real failure. (Hit twice on 2026-08-07 under parallel-build load; both times the clean rebuild confirmed green.)

---

Sampling Rate

• After every task commit: the quick-run command for the touched module (cargo test assistant::, or the specific Vitest file)
• After every plan wave: full cargo test + npm run test (neode-ui) + AIUI's own test command (once confirmed in Wave 0)
• Before /gsd-verify-work: full suite green and the AIUI-06 on-device pass on archi-dev-box (desktop + mobile)
• Max feedback latency: targeted Vitest < 30s; Rust module tests < 120s

---

Per-Task Verification Map

Requirement-level map seeded from research. The planner fills Task ID / Plan / Wave / Threat Ref columns as it decomposes; every row below must end up owned by at least one task.

Task ID	Plan	Wave	Requirement	Threat Ref	Secure Behavior	Test Type	Automated Command	File Exists	Status
13-01 T1	13-01	1	AIUI-01	—	Typed chat request executes a real read-only tool ("how much space is left" → system.disk-status) and returns the real result	integration (Rust)	cargo test assistant::tests::disk_status_tool_executes	✅	✅ green (assistant suite 130/130 on 2026-08-07)
13-08 T2	13-08	3	AIUI-01	D-07/D-11	A write request ("restart bitcoin") produces a pending confirmation, never an executed action, until the human confirms	integration (Rust) + component (Vue)	cargo test assistant::tests::destructive_tool_requires_confirm; npx vitest run src/services/__tests__/toolConfirm.test.ts	✅	✅ green; live-verified 2026-08-07 (app_install pending → decline → nothing executed)
13-02 T1	13-02	1	AIUI-01 / AIUI-04	Phase-10 D-01..D-04	An unauthenticated caller cannot reach any new assistant.* RPC method	integration (Rust)	cargo test rpc::middleware::tests::assistant_methods_require_session	✅	✅ green
13-02/13-09	13-02+13-09	1/4	AIUI-04	live exposure	/aiui/api/claude/ and /aiui/api/openrouter/ are no longer reachable without a session	integration/shell	live curl (see note)	✅	✅ green 2026-08-07: claude → 401; openrouter → SPA-fallback (indistinguishable from a nonexistent path, 13-02's recorded finding); web-search → 401 after same-day heal (482c4e30 + live hand-patch on the dev box, whose /home/archipelago/archy symlink skips the self-heal by design)
13-05 T2	13-05	2	AIUI-02	D-16	A conversational settings change is scoped to a granted permission category and refused when not granted	unit (Rust)	cargo test assistant::tools::tests::settings_tool_respects_category_grant	✅	✅ green
13-12 T1	13-12	5	AIUI-04	D-10	Peer-supplied text inside untrusted-content delimiters cannot escalate tool authority; an injected "restart bitcoin" still requires a human confirm naming the real action	unit (Rust)	cargo test assistant::tests::injected_instruction_does_not_grant_authority	✅	✅ green
13-06 T1	13-06	2	AIUI-03	—	content.* RPC data renders in FilmGrid/SongGrid through the new adapter (pins the shape mismatch found in research)	unit (Vue/TS)	npx vitest run src/composables/__tests__/archyContentAdapter.test.ts	✅	✅ green (37 tests, incl. same-day owner-lock and per-onion regressions)
13-06 T3	13-06	2	AIUI-03	—	Audio routes to the global bottom-bar player, never the lightbox (regression-pins the rule enforced in 5 call sites)	unit (Vue/TS)	npx vitest run src/composables/__tests__/useAudioPlayer.test.ts	✅	✅ green (11 tests, 2026-08-07)
13-09 T1	13-09	4	AIUI-05	D-15	Build enforces VITE_BASE_PATH=/aiui/; script exits non-zero if unset	shell/CI	scripts/build-aiui.sh require_base_path	✅	✅ green (in daily use; the known post-success hang is timeboxed by callers)
13-09 T3	13-09	4	AIUI-05	D-15	Post-deploy check fetches a live asset over HTTP rather than trusting a directory listing	shell	resolve via index.html + fetch	✅	✅ green 2026-08-07: live bundle hash diffing node↔tree and inside the RC ISOs (caught a stale-AIUI ISO bake, fixed in e669a3e4)
13-15 T2	13-15	8	AIUI-06	—	Embedded iframe on archi-dev-box, desktop + mobile	manual	N/A	—	⚠️ desktop + narrow-viewport PASS (see 13-UAT.md); real-handheld pass outstanding

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

Rows added at close-out (phase reality, post-research)

Task ID	Plan	Wave	Requirement	Threat Ref	Secure Behavior	Test Type	Automated Command	File Exists	Status
S-01…S-15 structural invariants	13-01…13-14	1–7	AIUI-01/02/04	S-01…S-15	Each invariant has a named test in assistant:: (grants default-closed, confirm-gate nonce binding, untrusted wrap per-call, excluded-authority absence, egress fail-closed, prompt/authority split)	unit (Rust)	cargo test --bin archipelago assistant::	✅	✅ green (130/130)
E-01…E-18 eval cases	13-13	6	AIUI-01	—	assistant::evals golden behaviors	unit (Rust)	same suite	✅	✅ green
—	13-12	5	AIUI-04	G-B1/G-B2	Cloud egress screen: BIP39-checksum, secrets corpus, ecash/nostr/macaroon shapes, 64KB cap, fail-closed; replayed history allowlisted, fabricated turns truncated	unit (Rust)	cargo test assistant::egress	✅	✅ green (incl. S6 fix tests, 7686a486)
—	13-13	6	AIUI-01	D-05	Routstr budget hard-stops (BudgetExhausted), zero-allowance never selects Routstr	unit (Rust)	same suite	✅	✅ green
—	2026-08-07 close	8	AIUI-03	—	Prod AIUI bundle carries zero mock hosts (mock quarantine, W1.4)	shell grep	grep -c 'plex://|cloud.example.com|spotify.com/track/example|image.tmdb.org' dist/assets/index-*.js → 0	✅	✅ green 2026-08-07

---

Wave 0 Requirements

• core/archipelago/src/assistant/ + its #[cfg(test)] module — built out; 130 tests in the assistant suite alone by close-out
• neode-ui/src/services/__tests__/toolConfirm.test.ts — exists and green (incl. the hydration-narrowing fix)
• neode-ui/src/composables/__tests__/archyContentAdapter.test.ts — pins ContentItem → Film/Song/Podcast/ImageItem; 37 green
• scripts/build-aiui.sh — exists, enforces VITE_BASE_PATH=/aiui/ + dist verification (D-15)
• AIUI's own test command confirmed — pnpm test (vitest run), exercised repeatedly on 2026-08-07
• Keep green: contextBroker.test.ts (25), chatAiuiEmbed.test.ts — green

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Embedded AIUI works in the real iframe	AIUI-06	Real-device rendering in the actual embed context; dev:mock does not reproduce it	DONE on archi-dev-box 2026-08-07 (desktop + narrow viewports; see 13-UAT.md). A physical-handset pass remains owed.
Frontend bundle actually shipped	AIUI-05	Node assets/ is a never-pruned graveyard — a disk grep reports "deployed" before the deploy	DONE 2026-08-07: live chunks resolved via index.html and diffed node↔tree and inside the RC ISO images (that pass caught the stale-AIUI bake, e669a3e4)
Confirm dialog is un-spoofable by the iframe	AIUI-04 / D-11	Anti-spoofing is a visual/trust property of the host chrome	DONE: modal Teleports to body, full-screen backdrop, description node-authored via assistant.pending; the iframe never sees the nonce (asserted by iframe_message_cannot_open_or_resolve_confirmation)
Routstr pays a live request	D-04 / D-05	Research confidence on the Routstr protocol is MEDIUM — cited from docs, never run against a live provider	NOT DONE — recorded open; 13-ROUTSTR-FINDINGS.md stands (0/9 live claims). The backend ships behind the budget ceiling but stays unverified against a live provider.

---

Edge-Probe Reconciliation

The audit trail for the deterministic edge probe, counted against the plan files rather than asserted. An earlier summary claimed "5 truths + 4 unclassified = 9, nothing dropped"; that total was right by coincidence and wrong by composition, because it omitted the backstop scalar and silently absorbed three planner-authored edges into the probe's own count. The real numbers:

Line	Count	Where
Requirements probed	6	AIUI-01 … AIUI-06
Probes resolved covered	2	AIUI-01, AIUI-03
Probes returned unclassified — flagged, never auto-resolved and never auto-backstopped	4	AIUI-02 → 13-05 · AIUI-04 → 13-09 · AIUI-05 → 13-09 · AIUI-06 → 13-15
Probe-surfaced findings authored as covered truths	5	1 in 13-01 (AIUI-01 concurrency) · 4 in 13-06 (AIUI-03 adjacency, empty, ordering, concurrency)
Probe-surfaced findings authored as verification: backstop scalars	1	13-01 — the two-tab confirmation-nonce case
Probe findings total	6 covered + 4 unclassified = 10	5 truths + 1 backstop + 4 flagged
Planner-authored edge truths — not probe output	3	13-07 — concurrency, ordering and empty re-applied to the persisted music index, tagged — authored, not probe-surfaced
Edge-tagged truths across all plans	8	1 (13-01) + 4 (13-06) + 3 (13-07)
Edge entries across all plans, incl. the backstop scalar	9	the 8 above + 13-01's backstop

Two numbers are easy to conflate and are deliberately kept apart here: 10 probe findings (what the probe produced) and 9 edge entries in the plan files (what was written, including three authored edges the probe never surfaced and excluding the four unclassified probes, which are prose in <flagged_assumptions> rather than truths). Nothing was dropped in either direction — every one of the 6 probes is accounted for, and every edge-tagged truth states whether it came from the probe or from the planner.

Also verified and unchanged: the 4 unclassified entries sit under <flagged_assumptions> and are never promoted to must_haves.truths; the 3 prohibitions in 13-08, 13-12 and 13-14 are flat scalars under prohibitions, never under truths, and carry no check_* keys.

---

Open Questions Blocking Full Validation

Carried from 13-RESEARCH.md § Open Questions — each needs a planner decision, and two change what "validated" even means:

1. The port-3142 proxy — RESOLVED (13-02): deleted the anonymous relays; a session-gated Rust forwarder replaced them. Live-verified 2026-08-07 (/aiui/api/claude/ → 401 unauthenticated; web-search → 401 after the same-day self-heal, 482c4e30).
2. Iframe sandbox mechanism — RESOLVED (13-09): a /aiui/-scoped CSP plus G-B3 (postMessage origin check + broker-side sanitization). sandbox attribute was rejected (breaks required same-origin bridge semantics); the residual is documented in 13-09-SUMMARY.
3. Routstr protocol accuracy — OPEN: 13-13 entered the backend behind D-05's budget ceiling; 13-ROUTSTR-FINDINGS.md's 0/9 live-protocol claims stand. The funding-UX phase must spike a live provider before Routstr is load-bearing.
4. RBAC integration — RESOLVED (13-01): a single assistant. prefix arm, so the existing role.can_access gate applies unchanged before dispatch (assistant_methods_require_session green).

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies
• Sampling continuity: no 3 consecutive tasks without automated verify
• Wave 0 covers all MISSING references
• No watch-mode flags
• Feedback latency < 120s
• AIUI repo test command confirmed
• nyquist_compliant: true set in frontmatter — set below: every map row has an automated verify or a discharged manual entry; the single honest exception is the physical-handset pass, recorded in 13-UAT.md as owed, not silently skipped.

Approval: pending — awaits the operator's read of 13-UAT.md and the handset decision (the flagged AIUI-06 assumption asks for BOTH a devtools mobile viewport and a real phone; only the former is on record).