feat(security): enforce declared cosign image signatures at the pull sites

New container::image_verify gates PodmanClient::pull_image and the
dev-only DockerRuntime::pull_image. Signature claims classify three
ways: absent/empty (pull unverified, logged), the literal
'cosign://...' placeholder every fleet manifest carries today (same —
enforcement stays dormant until the signing ceremony ships real
values), or a declared signature, which must verify via
'cosign verify --key /etc/archipelago/cosign.pub
--insecure-ignore-tlog=true' (plus --allow-insecure-registry
--allow-http-registry for the HTTP mirror; flags checked against
cosign's own docs) before anything is fetched. Missing key, missing
cosign binary, timeout, or verification failure all hard-fail the
pull — a declared signature cannot be skipped on either runtime. Key
path overridable via ARCHIPELAGO_COSIGN_PUBKEY for tests/staging.

Deletes security::ImageVerifier: zero callers, blocking
std::process::Command on would-be async paths, and a fantasy
'cosign verify --signature' invocation (that flag belongs to
verify-blob).

Activation ships with the Workstream B ceremony, in order: pin
cosign.pub on nodes + install cosign, then publish real
image_signature values in the catalog.

Tests: archipelago-container 58/58 (5 new), archipelago container::
159/159, security check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

core/container/src/image_verify.rs

@@ -0,0 +1,207 @@
+//! Container image signature verification (cosign).
+//!
+//! The manifest/catalog `image_signature` field is a *claim* that the image
+//! is signed with the fleet's cosign key. Verification runs at the pull
+//! choke points (`PodmanClient::pull_image`, `DockerRuntime::pull_image`);
+//! a declared signature that cannot be verified hard-fails the pull.
+//!
+//! Every manifest has carried the literal placeholder `cosign://...` since
+//! the field was introduced — that means "not signed yet" and is treated as
+//! no claim, so enforcement stays dormant until the signing ceremony
+//! publishes real signatures AND nodes carry the pinned cosign public key.
+//! Ship order matters: key + cosign binary reach the fleet first, real
+//! signature values in the catalog come after.
+
+use anyhow::{bail, Context, Result};
+use std::path::PathBuf;
+
+/// The literal placeholder every pre-ceremony manifest carries.
+pub const SIGNATURE_PLACEHOLDER: &str = "cosign://...";
+
+/// Env override for the pinned cosign public key path (tests, staging).
+pub const COSIGN_PUBKEY_ENV: &str = "ARCHIPELAGO_COSIGN_PUBKEY";
+
+const DEFAULT_PUBKEY_PATH: &str = "/etc/archipelago/cosign.pub";
+
+const COSIGN_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(60);
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum SignatureClaim {
+    /// No signature declared (field absent or empty).
+    None,
+    /// The literal `cosign://...` placeholder — manifest predates real signing.
+    Placeholder,
+    /// A real declared signature reference; MUST verify or the pull fails.
+    Declared(String),
+}
+
+pub fn classify_signature(signature: Option<&str>) -> SignatureClaim {
+    match signature.map(str::trim) {
+        None | Some("") => SignatureClaim::None,
+        Some(SIGNATURE_PLACEHOLDER) => SignatureClaim::Placeholder,
+        Some(s) => SignatureClaim::Declared(s.to_string()),
+    }
+}
+
+fn pinned_pubkey_path() -> PathBuf {
+    std::env::var(COSIGN_PUBKEY_ENV)
+        .map(PathBuf::from)
+        .unwrap_or_else(|_| PathBuf::from(DEFAULT_PUBKEY_PATH))
+}
+
+/// Verify a declared image signature with the fleet's pinned cosign key.
+/// Any failure — missing key, missing cosign binary, verification error —
+/// is a hard error: an image that CLAIMS to be signed must never be pulled
+/// on a node that can't prove the claim.
+pub async fn verify_declared_signature(
+    image: &str,
+    sig_ref: &str,
+    allow_insecure_registry: bool,
+) -> Result<()> {
+    verify_with_key_path(image, sig_ref, &pinned_pubkey_path(), allow_insecure_registry).await
+}
+
+async fn verify_with_key_path(
+    image: &str,
+    sig_ref: &str,
+    key_path: &std::path::Path,
+    allow_insecure_registry: bool,
+) -> Result<()> {
+    if !key_path.exists() {
+        bail!(
+            "Image '{image}' declares signature '{sig_ref}' but the pinned cosign \
+             public key is missing at {} (override with {COSIGN_PUBKEY_ENV}). \
+             Refusing to pull an image whose signature claim cannot be verified.",
+            key_path.display()
+        );
+    }
+
+    // Self-managed key => signatures aren't in the public Rekor transparency
+    // log, so tlog verification must be disabled explicitly (cosign v2
+    // defaults it on and would fail every private-key signature otherwise).
+    let mut cmd = tokio::process::Command::new("cosign");
+    cmd.arg("verify")
+        .arg("--key")
+        .arg(key_path)
+        .arg("--insecure-ignore-tlog=true");
+    if allow_insecure_registry {
+        // podman's --tls-verify=false covers both plain HTTP and bad TLS;
+        // cosign splits those into two flags — pass both to match.
+        cmd.arg("--allow-insecure-registry");
+        cmd.arg("--allow-http-registry");
+    }
+    cmd.arg(image);
+
+    let output = tokio::time::timeout(COSIGN_TIMEOUT, cmd.output())
+        .await
+        .map_err(|_| {
+            anyhow::anyhow!(
+                "cosign verify timed out after {}s for image '{image}'",
+                COSIGN_TIMEOUT.as_secs()
+            )
+        })?
+        .with_context(|| {
+            format!(
+                "Failed to run cosign for image '{image}' which declares signature \
+                 '{sig_ref}' — is cosign installed? A declared signature cannot be \
+                 skipped."
+            )
+        })?;
+
+    if !output.status.success() {
+        let stderr = String::from_utf8_lossy(&output.stderr);
+        bail!(
+            "Signature verification FAILED for image '{image}' (declared: '{sig_ref}'): \
+             {stderr}"
+        );
+    }
+
+    tracing::info!("cosign signature verified for image {image}");
+    Ok(())
+}
+
+/// Shared pull-site gate: decide whether the pull may proceed.
+/// Returns Ok(()) for unsigned/placeholder claims (with a log line) and only
+/// after successful cosign verification for declared ones.
+pub async fn enforce_signature_claim(
+    image: &str,
+    signature: Option<&str>,
+    allow_insecure_registry: bool,
+) -> Result<()> {
+    match classify_signature(signature) {
+        SignatureClaim::None => {
+            tracing::debug!("image {image}: no signature declared, pulling unverified");
+            Ok(())
+        }
+        SignatureClaim::Placeholder => {
+            tracing::debug!(
+                "image {image}: signature is the pre-ceremony placeholder, pulling unverified"
+            );
+            Ok(())
+        }
+        SignatureClaim::Declared(sig_ref) => {
+            verify_declared_signature(image, &sig_ref, allow_insecure_registry).await
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn absent_and_empty_signatures_are_no_claim() {
+        assert_eq!(classify_signature(None), SignatureClaim::None);
+        assert_eq!(classify_signature(Some("")), SignatureClaim::None);
+        assert_eq!(classify_signature(Some("   ")), SignatureClaim::None);
+    }
+
+    #[test]
+    fn literal_placeholder_is_not_a_claim() {
+        assert_eq!(
+            classify_signature(Some("cosign://...")),
+            SignatureClaim::Placeholder
+        );
+        assert_eq!(
+            classify_signature(Some("  cosign://...  ")),
+            SignatureClaim::Placeholder
+        );
+    }
+
+    #[test]
+    fn real_values_are_declared_claims() {
+        assert_eq!(
+            classify_signature(Some("cosign://sha256-abc.sig")),
+            SignatureClaim::Declared("cosign://sha256-abc.sig".to_string())
+        );
+        // Unknown schemes still count as a claim — better to fail closed on
+        // a value we don't understand than to pull unverified.
+        assert_eq!(
+            classify_signature(Some("sigstore://whatever")),
+            SignatureClaim::Declared("sigstore://whatever".to_string())
+        );
+    }
+
+    #[tokio::test]
+    async fn declared_signature_without_pinned_key_hard_fails() {
+        let err = verify_with_key_path(
+            "registry.example/app:1.0",
+            "cosign://sha256-abc.sig",
+            std::path::Path::new("/nonexistent/cosign.pub"),
+            false,
+        )
+        .await
+        .unwrap_err();
+        assert!(err.to_string().contains("pinned cosign public key is missing"));
+    }
+
+    #[tokio::test]
+    async fn unsigned_and_placeholder_claims_pass_the_gate() {
+        enforce_signature_claim("registry.example/app:1.0", None, false)
+            .await
+            .unwrap();
+        enforce_signature_claim("registry.example/app:1.0", Some("cosign://..."), false)
+            .await
+            .unwrap();
+    }
+}

core/container/src/lib.rs

@@ -1,5 +1,6 @@
 pub mod bitcoin_simulator;
 pub mod health_monitor;
+pub mod image_verify;
 pub mod manifest;
 pub mod podman_client;
 pub mod port_manager;

core/container/src/podman_client.rs

@@ -252,7 +252,17 @@ impl PodmanClient {
 
     // ─── Container Operations ────────────────────────────────────
 
-    pub async fn pull_image(&self, image: &str, _signature: Option<&str>) -> Result<()> {
+    pub async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()> {
+        // A declared (non-placeholder) signature must verify before we fetch
+        // anything; placeholder/absent claims pull unverified until the
+        // signing ceremony ships real signatures (see image_verify).
+        crate::image_verify::enforce_signature_claim(
+            image,
+            signature,
+            image_uses_insecure_registry(image),
+        )
+        .await?;
+
         // Image pull uses CLI — it's a streaming operation that the API handles differently
         let mut cmd = tokio::process::Command::new("podman");
         cmd.arg("pull");

core/container/src/runtime.rs

@@ -546,7 +546,12 @@ impl DockerRuntime {
 
 #[async_trait]
 impl ContainerRuntime for DockerRuntime {
-    async fn pull_image(&self, image: &str, _signature: Option<&str>) -> Result<()> {
+    async fn pull_image(&self, image: &str, signature: Option<&str>) -> Result<()> {
+        // Same signature gate as the podman path — the docker fallback is
+        // dev-only, but a declared signature must never be skippable by
+        // switching runtimes.
+        crate::image_verify::enforce_signature_claim(image, signature, false).await?;
+
         let mut cmd = self.docker_async();
         cmd.arg("pull").arg(image);
 

core/security/src/image_verifier.rs

@@ -1,111 +0,0 @@
-// Container image signature verification using Cosign
-// Verifies that container images are signed and trusted
-
-use anyhow::{Context, Result};
-use std::process::Command;
-use tracing::{info, warn};
-
-pub struct ImageVerifier {
-    cosign_public_key: Option<String>,
-    require_signatures: bool,
-}
-
-impl ImageVerifier {
-    pub fn new(cosign_public_key: Option<String>) -> Self {
-        Self {
-            cosign_public_key,
-            require_signatures: false,
-        }
-    }
-
-    /// Create a verifier that requires all images to be signed.
-    pub fn new_strict(cosign_public_key: Option<String>) -> Self {
-        Self {
-            cosign_public_key,
-            require_signatures: true,
-        }
-    }
-
-    /// Verify a container image signature
-    pub async fn verify_image(&self, image: &str, signature: Option<&str>) -> Result<bool> {
-        if signature.is_none() && self.cosign_public_key.is_none() {
-            if self.require_signatures {
-                return Err(anyhow::anyhow!(
-                    "Image '{}' has no signature and no cosign key is configured. \
-                     All container images must be signed for production use.",
-                    image
-                ));
-            }
-            warn!("No signature provided for image: {}", image);
-            return Ok(false);
-        }
-
-        // Check if cosign is available
-        let cosign_available = Command::new("cosign").arg("version").output().is_ok();
-
-        if !cosign_available {
-            if self.require_signatures {
-                return Err(anyhow::anyhow!(
-                    "Cosign binary not found. Install cosign to verify container image signatures."
-                ));
-            }
-            warn!("Cosign not available, skipping signature verification");
-            return Ok(false);
-        }
-
-        // If public key is provided, use it for verification
-        if let Some(ref public_key) = self.cosign_public_key {
-            let output = Command::new("cosign")
-                .arg("verify")
-                .arg("--key")
-                .arg(public_key)
-                .arg(image)
-                .output()
-                .context("Failed to run cosign verify")?;
-
-            if output.status.success() {
-                info!("Image signature verified: {}", image);
-                return Ok(true);
-            } else {
-                let stderr = String::from_utf8_lossy(&output.stderr);
-                return Err(anyhow::anyhow!("Signature verification failed: {}", stderr));
-            }
-        }
-
-        // If signature URL is provided, verify using that
-        if let Some(sig_url) = signature {
-            if sig_url.starts_with("cosign://") {
-                // Extract signature reference
-                let sig_ref = sig_url.strip_prefix("cosign://").unwrap();
-                let output = Command::new("cosign")
-                    .arg("verify")
-                    .arg("--signature")
-                    .arg(sig_ref)
-                    .arg(image)
-                    .output()
-                    .context("Failed to run cosign verify")?;
-
-                if output.status.success() {
-                    info!("Image signature verified: {}", image);
-                    return Ok(true);
-                } else {
-                    let stderr = String::from_utf8_lossy(&output.stderr);
-                    return Err(anyhow::anyhow!("Signature verification failed: {}", stderr));
-                }
-            }
-        }
-
-        Ok(false)
-    }
-
-    /// Check if an image has a signature
-    pub async fn has_signature(&self, image: &str) -> bool {
-        // Try to find signature in registry
-        let output = Command::new("cosign")
-            .arg("triangulate")
-            .arg(image)
-            .output();
-
-        output.is_ok() && output.unwrap().status.success()
-    }
-}

core/security/src/lib.rs

@@ -1,7 +1,5 @@
 pub mod container_policies;
-pub mod image_verifier;
 pub mod secrets_manager;
 
 pub use container_policies::ContainerPolicyGenerator;
-pub use image_verifier::ImageVerifier;
 pub use secrets_manager::SecretsManager;

docs/1.8.0-RELEASE-HARDENING-PLAN.md

@@ -58,11 +58,19 @@ arbitrary app catalog to the entire fleet — fully unattended under
       `scripts/sign-manifest.sh` exists for re-signs. **Still open:** move the mirror
       to HTTPS + pinned cert (tracked with the next item); flip unsigned-manual-apply →
       hard-reject once the fleet is on a pinned-anchor binary.
-- [ ] 🔴 **Implement container image signature verification (cosign).**
-      `container/src/podman_client.rs:255` — `pull_image(.., _signature)` silently discards
-      the signature that the manifest threads all the way down
-      (`prod_orchestrator.rs:1978/2435`). Wire `sigstore-rs`/`cosign verify` (or
-      `podman pull --signature-policy`); hard-fail when a declared signature doesn't verify.
+- [x] 🔴 **Implement container image signature verification (cosign).** DONE 2026-07-04
+      (code path; enforcement dormant until the ceremony): new `container::image_verify`
+      gates BOTH pull sites (`PodmanClient::pull_image` + the dev-only `DockerRuntime`).
+      Claims classify as None / the literal `cosign://...` placeholder (every fleet
+      manifest today → pull proceeds, logged) / Declared → `cosign verify --key
+      /etc/archipelago/cosign.pub --insecure-ignore-tlog=true` (+ both insecure-registry
+      flags for the HTTP mirror; flags verified against cosign docs), hard-fail on missing
+      key, missing cosign binary, timeout, or bad signature — a declared signature can
+      never be skipped, on either runtime. Key path overridable via
+      `ARCHIPELAGO_COSIGN_PUBKEY`. Deleted the caller-less, blocking, wrong-CLI
+      `security::ImageVerifier`. **Activation = ceremony work**: pin cosign.pub on nodes +
+      install cosign + publish real `image_signature` values (in that order); tracked with
+      the Workstream B signing ceremony item.
 - [ ] 🟠 **Move the image mirror to HTTPS; drop `--tls-verify=false`.**
       `podman_client.rs:641` `INSECURE_REGISTRY_HOSTS = ["146.59.87.168:3000"]` +
       `config.rs:104,124` allowlist pull images over unauthenticated HTTP. Remove the raw-IP