feat(kiosk): companion remote drives app iframes via trusted CDP input
Demo images / Build & push demo images (push) Successful in 3m11s
Demo images / Build & push demo images (push) Successful in 3m11s
Companion tap/scroll/type now works INSIDE cross-origin app iframes and kiosk tabs. The web relay synthesizes untrusted DOM events in the top document, which can never cross an origin boundary — so apps served through the appgate were dead to the remote. The kiosk Chromium now exposes a loopback-only CDP port (default origin check intact, no --remote-allow-origins) and a backend bridge (api/handler/cdp.rs) dispatches validated companion input as Input.dispatchKeyEvent / dispatchMouseEvent / mouseWheel — trusted events that hit-test through any frame, move real focus, and insert text like a physical device. - Session keeper self-heals across kiosk Chromium restarts; inert on nodes without a kiosk unit (falls back to the existing relay path). - The kiosk relay subscriber self-tags (?kiosk=1) and the backend mutes its key/click/scroll messages while the bridge is live, so input never applies twice; cursor moves still flow for the on-screen cursor. - While companion input is active the native OS pointer is hidden (cursor:none, auto-restores 30s after the last event) so the dead physical-mouse cursor doesn't sit next to the virtual one. - docs/tv-input-iframe-apps.md scope note updated: gamepad keys stay on uinput; CDP is for companion pointer/typing only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
876ecc4bdf
commit
3a3077529b
@@ -1,4 +1,5 @@
|
||||
mod blob;
|
||||
mod cdp;
|
||||
mod content;
|
||||
mod dwn;
|
||||
mod model_proxy;
|
||||
@@ -51,6 +52,10 @@ pub struct ApiHandler {
|
||||
/// to the phone's default browser. Lets "open in external browser" apps —
|
||||
/// which the kiosk can't usefully open itself — launch on the controller.
|
||||
external_open_tx: broadcast::Sender<String>,
|
||||
/// Bridge that dispatches companion input into the local kiosk Chromium
|
||||
/// as trusted CDP events (reaches inside cross-origin app iframes).
|
||||
/// Inert (never connects) on nodes without a kiosk.
|
||||
cdp_bridge: cdp::CdpBridge,
|
||||
/// Content-addressed blob store for attachments shared over mesh/federation.
|
||||
blob_store: Arc<BlobStore>,
|
||||
/// Our own node pubkey (hex) — used to self-sign debug/test capabilities.
|
||||
@@ -79,6 +84,7 @@ impl ApiHandler {
|
||||
);
|
||||
let (input_relay_tx, _) = broadcast::channel(64);
|
||||
let (external_open_tx, _) = broadcast::channel(16);
|
||||
let cdp_bridge = cdp::CdpBridge::spawn();
|
||||
|
||||
// Derive a blob-store capability key from the node's Ed25519 signing
|
||||
// key. SHA-256 domain-separated so rotating the identity rotates
|
||||
@@ -109,6 +115,7 @@ impl ApiHandler {
|
||||
session_store,
|
||||
input_relay_tx,
|
||||
external_open_tx,
|
||||
cdp_bridge,
|
||||
blob_store,
|
||||
self_pubkey_hex,
|
||||
})
|
||||
@@ -402,6 +409,7 @@ impl ApiHandler {
|
||||
req,
|
||||
self.input_relay_tx.clone(),
|
||||
self.external_open_tx.subscribe(),
|
||||
self.cdp_bridge.clone(),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
@@ -416,6 +424,7 @@ impl ApiHandler {
|
||||
req,
|
||||
self.input_relay_tx.subscribe(),
|
||||
self.external_open_tx.clone(),
|
||||
self.cdp_bridge.clone(),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user