archipelago 891cbba469 fix(lifecycle): serialize per-app start/stop/restart workers (FIFO app-op lock)
package.start/stop/restart reply immediately and run their multi-container
sequences in spawned tasks with no coordination. Back-to-back RPCs on the
same app interleave those sequences: gate run D (.228, 2026-07-09 04:28)
fired stop→start→restart on mempool; the start's member bring-up raced the
still-running stop's member shutdown — archy-mempool-db was stopped 2s
after it started and the stack finished with ZERO containers (tests 95/123/
124). A user double-clicking restart in the UI can do the same.

Workers now take a per-app tokio Mutex (fair/FIFO, keyed by the normalized
orchestrator app id) as their first await, so queued ops run in RPC arrival
order and the final state matches the last request. The RPC reply stays
immediate.

Known limit: container lists are still computed at RPC time (pre-lock), so
a stop landing mid-start can still return 'No containers found' — it can no
longer interleave/corrupt, only error. Reconciler/health-monitor restarts
don't take the lock yet (separate paths, same candidate follow-up).

Also from run D, filed separately: jellyfin/vaultwarden (legacy containers
on a quadlet node) stop fine but the package state sticks at 'stopping' —
'Failed to stop jellyfin.service: Unit not loaded' error path suspected.

Tests: runtime 9/9; archipelago builds clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 01:10:47 -04:00
..
2026-01-24 22:59:20 +00:00
2026-01-24 22:59:20 +00:00