13 Commits

Author SHA1 Message Date
Dorian
9d1baf75d5 perf: skip missed ticks on all intervals, reduce scan frequency
Prevents burst of health checks, scans, and snapshots after slow
podman responses by using MissedTickBehavior::Skip. Bumps container
scan interval from 30s to 60s to reduce DB lock contention.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 20:25:09 +01:00
Dorian
64b57dca7d fix: overhaul container lifecycle — recovery, health, uninstall, UI state
Container recovery:
- Health monitor: MAX_RESTART_ATTEMPTS 3→10, interval 60s→120s
- Dependency-aware restarts: won't restart services before their deps
- Reset dependent counters when a dependency recovers
- Handle "created" state containers (were invisible to health monitor)
- Added IndeedHub, mempool-api, mysql to tier system
- Crash recovery: podman start timeout 30s→120s with retry
- Podman client: socket timeout 5s→30s, added restart policy

UI state representation:
- Exit code 0 shows "stopped" (gray), not "crashed" (red)
- Exit code 137 shows "killed (OOM)"
- Non-zero exit shows "crashed" (red)
- Added exit_code field to PackageDataEntry

Install/uninstall fixes:
- Install returns error when container doesn't start (was silent success)
- Post-install hooks awaited instead of fire-and-forget tokio::spawn
- Uninstall: graceful rm before force, volume prune, network cleanup
- Uninstall returns error on partial failure (was 200 OK)

Config consistency:
- DB passwords read from /var/lib/archipelago/secrets/ (was hardcoded)
- Bitcoin: added ZMQ ports 28332/28333 for LND block notifications
- IndeedHub port 7777→8190 (was conflicting with strfry)
- Marketplace versions: LND 0.17.4→0.18.4, Mempool 2.5.0→3.0.0

Performance:
- Metrics collector interval 60s→300s (was duplicating health monitor)
- Podman client: proper error propagation instead of unwrap_or_default

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:03:57 +01:00
Dorian
13e4a738be bug fixing and deploy and build diagnostics 2026-03-22 03:30:21 +00:00
Dorian
f20f0650cf feat: Discover view, Fleet dashboard, MeshMap, type fixes
- New Discover.vue (app store redesign)
- Fleet.vue dashboard for .228
- MeshMap.vue component
- Fixed Discover.vue type errors (unused var, type predicate)
- Various UI updates (Apps, Dashboard, Marketplace, Mesh, Web5)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:12:01 +00:00
Dorian
aad98dec08 fix: telemetry reporter field name cpu_percent, add type annotation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 23:53:17 +00:00
Dorian
41ab499698 feat(TASK-12): periodic telemetry reporter — 15min interval, collector POST
Background task spawned on server startup: every 15 minutes, checks opt-in
status, builds anonymous health report (node ID hash, version, uptime,
CPU/RAM/disk %, container states, recent alerts), saves to disk, and POSTs
to TELEMETRY_COLLECTOR_URL env var if configured. Non-fatal on failure.

Fixed FiredAlert field references (kind not rule_type, timestamp not
fired_at) in both monitoring and analytics modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 23:36:57 +00:00
Dorian
25ad68ac4c fix: BUG-33 CPU threshold, TASK-27 tab icons, TASK-36 iframe errors
- BUG-33: CPU load alert threshold increased from 2x to 4x core count
  (8→16 on 4-core machine) to reduce false alerts during container ops
- TASK-27: Launch buttons for new-tab apps now show external link icon
  (BTCPay, Grafana, PhotoPrism, Portainer, OnlyOffice, etc.)
- TASK-36: Iframe error screen now distinguishes between X-Frame-Options
  blocked vs container not reachable, with appropriate messaging

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 19:24:52 +00:00
Dorian
176336b555 fix: add missing tier field to all AppMetadata, fix build errors
- Add tier: "" to all AppMetadata match arms (was missing from 30+ arms)
- Use std:🧵:available_parallelism() instead of num_cpus crate
- Remove unused num_cpus dependency
- Fix unused variable warning in health_monitor.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 03:36:44 +00:00
Dorian
ebad38cdaf feat: add CPU load alert, lower disk/RAM thresholds (SCALE-04)
- Add CpuLoad alert rule: fires when 5min load > 2x core count
- Lower disk usage alert from 90% to 80%
- Lower RAM usage alert from 90% to 80%
- Add num_cpus dependency for runtime core detection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 03:29:29 +00:00
Dorian
701b202b41 fix: add webhook delivery for monitoring alerts
DiskUsage and ContainerCrash alerts now fire webhooks via
send_webhook() after pushing WebSocket notifications. Added
data_dir parameter to spawn_metrics_collector for webhook config
access.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 22:32:19 +00:00
Dorian
5e6aaa74aa patches on sxsw ai working api key working container hardened plus many more 2026-03-12 22:19:04 +00:00
Dorian
1b8a8cfd32 feat: add alerting system with configurable rules and UI (MON-02, MON-03)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:28:44 +00:00
Dorian
592548066e feat: add real-time metrics collection with ring buffer storage (MON-01)
Implements monitoring/collector.rs that collects per-container CPU/RAM/network/disk,
system-wide metrics, RPC latency, and WebSocket connection count every 60 seconds.
Data stored in dual ring buffers: 1-min resolution (24h) and 15-min resolution (7d).
Three new RPC endpoints: monitoring.current, monitoring.history, monitoring.containers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:11:02 +00:00