archy/docs/MIGRATION_STATUS_REPORT.md

253 lines
25 KiB
Markdown
Raw Normal View History

# Migration Status Report
Last updated: 2026-06-14
## RESUME CHECKPOINT (2026-06-14, after SSH drop)
State right now, so any disconnect resumes cleanly:
- **`main` = `a483fe4b`** = the other agent's 4 fixes (`0ed892a4`: wallet receive / bitcoin
install self-heal / ElectrumX tile / extended test gate) + **my F1 fix committed on top**
(`launch_url_port` in `docker_packages.rs` + 3 regression tests). Tree is clean (only two
untracked `docs/*.md` tracking files remain). Not pushed.
- The old isolated `archy-f1` worktree was **removed** — built the combined tree in-place.
-**DONE — combined backend release build** (`cd core && TMPDIR=/home/archipelago/.buildtmp
cargo build --release -p archipelago`, 7m46s, exit 0). `/tmp` is a full tmpfs so `TMPDIR`
MUST point at `/home/archipelago/.buildtmp`.
-**DONE — sideloaded + restarted on `.116`.** Backed up old binary to
`/usr/local/bin/archipelago.pre-f1.bak`, `install`ed new binary (root:root 755),
`sudo systemctl restart archipelago` (new MainPID 2885863).
-**F1 VALIDATED LIVE on `.116` (2026-06-14).** See "FINDING F1" below — before/after proves
the fix. Harness focused audit `jellyfin,filebrowser`**all checks passed, exit 0**.
- **IMPORTANT — restart is SAFE on this node:** containers run rootless under
`user-1000.slice/user@1000.service/app.slice`, a DIFFERENT cgroup from
`/system.slice/archipelago.service`. They survived both the 01:47 and this restart
(bitcoin/lnd/btcpay/immich/indeedhub all intact, count stayed 36). The
`feedback_no_systemctl_deploy_until_quadlet` cgroup-cascade warning does NOT apply to `.116`'s
current config. (The reconciler does recreate a few app containers like jellyfin/fedimint on
adoption — normal level-triggered behavior, not casualties.)
- **RELEASE IN PROGRESS — v1.7.91-alpha (user approved 2026-06-14).** Bundles the other agent's
4 fixes (`0ed892a4`) + F1 (`a483fe4b`) + changelog (`ab858271`). Steps:
1. ✅ Freed `/tmp` (removed stale published frontend tarballs 1.7.83→1.7.89; ~1.1G free) —
`create-release.sh` writes the 184MB frontend tarball to `/tmp` (hardcoded, NOT TMPDIR).
2.`cargo fmt -p archipelago --check` clean; curated layman changelog added + committed.
3. 🔄 `TMPDIR=/home/archipelago/.buildtmp scripts/create-release.sh 1.7.91-alpha`
(runs `tests/release/run.sh` gate → bumps Cargo.toml/package.json → builds backend+frontend
→ manifest → commit "chore: release v1.7.91-alpha" → tag `v1.7.91-alpha`). MUST set TMPDIR
or cargo's ring C-build fails on the full `/tmp` tmpfs.
- **AFTER create-release.sh:** `scripts/publish-release-assets.sh 1.7.91-alpha gitea-vps2`
`git push origin main && git push gitea-local main``git push --tags` (origin+gitea-local).
Ship target per memory: vps2 (146.59.87.168) is PRIMARY OTA manifest; tx1138 RETIRED.
- Verify packaged tarball actually contains the new version string before trusting the build
(npm run build can silently produce stale dist — see `feedback_frontend_build_verify`).
## Validation node (ACTIVE)
As of 2026-06-14 the app-migration lifecycle validation moves from `.198` (remote, OVH) to
**`.116` — the local dev node (`archi-thinkpad`, `192.168.1.116`)** because it is the machine
this session runs on, so the harness drives it over loopback instead of SSH (much faster, no
network latency). A separate agent owns OS-level fixes + its own test harness; this track owns
the **app-packaging migration** lifecycle validation only.
How to drive the harness against `.116` (local):
```bash
ARCHY_HOST=127.0.0.1 ARCHY_SCHEME=http ARCHY_PASSWORD='ThisIsWeb54321@' \
ARCHY_APPS='meshtastic,jellyfin,filebrowser,uptime-kuma' \
tests/lifecycle/remote-lifecycle.sh # focused, audit-only (non-destructive)
```
- `.116` serves nginx on **:80 only** (443 is tailscale's) → use `ARCHY_SCHEME=http`, `ARCHY_HOST=127.0.0.1`.
- Local node is healthy: `update_state.json.current_version == 1.7.90-alpha`, `update_in_progress=false`
(the OTA self-heal that was a follow-up gap in PROGRESS_MEMORY is now confirmed resolved on .116).
- Login password for `.116`: `ThisIsWeb54321@` (verified against `auth.login`). Note: auth.login
has a login rate-limiter — avoid rapid repeated attempts.
- `.198` results below remain the prior baseline; new results are tagged `[.116]`.
### [.116] audit log (newest first)
- **2026-06-14 — focused audit `meshtastic,jellyfin,filebrowser,uptime-kuma` (audit-only, non-destructive):**
harness exit 1, FAILED checks: 1.
- `filebrowser` — running, pass (also passed a standalone single-app smoke run).
- `uptime-kuma` — running, pass.
- `meshtastic``state=absent`. Not installed on `.116` (was installed/validated on `.198`).
Not a regression; just node state. To exercise meshtastic here, install it first (it needs
`/dev/ttyUSB0`, which `.116` may not have) or drop it from the focused set on this node.
- `jellyfin`**running but FAILED: "launch metadata missing: jellyfin has no lan_address".**
**ROOT-CAUSED 2026-06-14 — real, current bug in the working tree (a regression).** See
"FINDING F1" below.
### [.116] FINDING F1 — manifest launch URLs with a path are silently dropped (OPEN, fix pending)
**Symptom:** `jellyfin` is `running` and genuinely serving (`curl 127.0.0.1:8096/` → 302), but
`container-list` reports `lan_address: null`, so the UI/harness sees no launch URL.
**Root cause:** `core/archipelago/src/container/docker_packages.rs::reachable_lan_address()` parses
the port out of the candidate URL with `url.rsplit(':').next()`. When the candidate comes from the
manifest `interfaces.main` (via `PodmanClient::lan_address_for`
`core/container/src/podman_client.rs::manifest_primary_interface_url`), the URL **includes the
manifest `path`** — e.g. jellyfin → `http://localhost:8096/`. Then `rsplit(':').next()` yields
`"8096/"`, which **fails to `parse::<u16>()`**, so the function hits its `else { return None }`
branch and drops a perfectly reachable launch URL. (Diagnostic tell: the dropped-at-parse path
emits **no** log, whereas a genuine unreachable port logs "suppressing unreachable launch URL".
jellyfin has no such log; uptime-kuma — whose candidate `…:3002` has no path — does.)
**Why it's a regression:** the old `extract_lan_address(ports)` produced `http://localhost:PORT`
(no path), which parsed fine. The newer manifest-interface feature appends the declared `path`,
so any app routed through `lan_address_for` now yields `…:PORT/` and trips the parser.
**Blast radius (apps in `requires_reachable_launch` whose `interfaces.main.path` = `/`):**
`botfights`, `btcpay-server`, `fedimint`, `jellyfin`, `gitea`, `nextcloud`, `portainer`.
(`filebrowser`/`nextcloud`/`nginx-proxy-manager`/`vaultwarden` are in `uses_allocated_launch_port`
so they hit `extract_lan_address` first and dodge it; `grafana`/`mempool`/`uptime-kuma`/`searxng`
have no manifest `interfaces.main` path.) On `.198` this likely went unnoticed because those apps
weren't all running during the launch-metadata assertion, or predated the interfaces.main addition.
**Fix (IMPLEMENTED in working tree, uncommitted):**
`docker_packages.rs::reachable_lan_address` now parses the port via a new `launch_url_port()`
helper that reads digits after the final colon (`take_while(is_ascii_digit)`), mirroring the
RPC-layer `port_from_url`, so `http://localhost:8096/``Some(8096)`. Added unit tests
(`launch_url_port_tests`) covering the trailing-path regression, the bare-authority case, and a
no-port reject. The existing `lan_address_prefers_manifest_main_interface` test only exercised
`lan_address_for` (which always returned `…:8175/`) and never the `reachable_lan_address` wrapper,
which is why the bug slipped through.
**Unit validation: GREEN (2026-06-14).** `cargo test -p archipelago --bin archipelago launch_url_port`
→ 3 passed / 0 failed (trailing-path, bare-authority, no-port-reject); crate compiles clean.
**Coordination note (shared tree):** the repo is on branch `fix/wallet-receive-portdrift-secrets`
at commit `bb808df8` (= the deployed 1.7.90-alpha). A parallel agent has uncommitted changes here
(lnd `wallet.rs`, `bitcoin_relay.rs`, `prod_orchestrator.rs`, electrumx manifest, neode-ui, new
bats). To validate F1 in isolation (and NOT deploy their in-flight work onto the live node, nor
disturb their tree), the live-validation build is done in a detached git worktree at
`/home/archipelago/archy-f1` = clean `bb808df8` + only the F1 `docker_packages.rs` change. Build:
`cd /home/archipelago/archy-f1/core && TMPDIR=/home/archipelago/.buildtmp cargo build --release -p archipelago`
(`.116`'s `/tmp` is a 7.7G tmpfs that runs 100% full → the ring crate's C compile fails with
"No space left on device"; redirect `TMPDIR` to `/` which has ~399G). After validation the
worktree is removed (`git worktree remove`). NOTE: sideloading replaces the OTA-managed
`/usr/local/bin/archipelago` with a local 1.7.90-alpha+F1 build until the next OTA — back up the
current binary first (`/usr/local/bin/archipelago.pre-f1.bak`).
**Live validation status — ✅ GREEN on `.116` (2026-06-14).** Built combined tree (`a483fe4b`),
sideloaded, restarted `archipelago.service`. Before/after on the live node (old buggy binary → new):
| app | OLD lan_address | NEW lan_address |
|---|---|---|
| jellyfin | `None` ❌ | `http://localhost:8096/` ✅ |
| btcpay-server | `None` ❌ | `http://localhost:23000/` ✅ |
| fedimint | `None` ❌ | `http://localhost:8175/` ✅ |
| gitea | `None` ❌ | `http://localhost:3001/` ✅ |
| portainer | `None` ❌ | `http://localhost:9000/` ✅ |
| botfights | `None` ❌ | `http://localhost:9100/` ✅ |
| nextcloud | `:8085` ✓ | `:8085` (unchanged — allocated-port path) |
| filebrowser | `:8083` ✓ | `:8083` (unchanged) |
Harness focused audit `jellyfin,filebrowser`**all checks passed, exit 0**. Unit tests green.
No container casualties (all 36 survived; see RESUME CHECKPOINT for the cgroup detail).
NOTE: Do NOT run the prod binary directly to "check a version" —
`/usr/local/bin/archipelago <anyflag>` boots a whole second node instance (learned the hard way
2026-06-14; it exited without leaving a stray, but don't repeat).
## Goal
Make Archipelago's app/container system developer-ready and release-ready: app installs, lifecycle, recovery, and integrations should be portable, manifest-driven, and not rely on one-off OS-level changes or hardcoded Rust branches for each new app. The OS/backend should provide generic primitives for manifests, Quadlet rendering, lifecycle, health/readiness, dependency ordering, data ownership, image availability, bind mounts, secrets, app files, networking, bridge/signer integrations, and recovery.
The developer contract should be clear enough that a third-party developer can build and ship an Archipelago app from documentation plus manifest/schema examples. If an app needs a capability the platform does not yet expose, the release direction is to add a reusable manifest/orchestrator primitive rather than a special case tied to that app. This is the standard for the `1.8-alpha` app migration: professional app delivery, predictable behavior after restart/reboot, and a path for user-installed/community apps that does not require rebuilding the OS image for every app.
Release quality bar: every supported app must install, stop, start, restart, uninstall, survive host reboot, report accurate status, and expose clear install/uninstall progress. Stale health notifications must not persist across login or refresh after the underlying condition has cleared. Final release validation should run on the intended release validation server, not drift between appliances without an explicit checkpoint.
Target release: `1.8-alpha`, including a cut and smoke-tested ISO once validation is green.
Current release readiness estimate: about `82%`. The remaining percentage is mostly post-reboot recovery confidence, repeated reboot validation, and ISO creation/smoke testing rather than the core manifest/catalog migration itself.
## Current Result
- The migration is not final-release complete yet, but the core direction is being met.
- Portainer, Filebrowser, BTCPay, Grafana, Nostr Relay, SearXNG, Gitea, and key dependency units have moved further into the manifest/orchestrator path.
- `.198` has passed focused and broad lifecycle audits for the already migrated set.
- Meshtastic is now routed through the orchestrator path, no longer falls back to legacy `localhost/meshtastic:latest`, and has passed full lifecycle validation on `.198`.
- On 2026-06-02, focused and broad `.198` non-destructive lifecycle audits passed after clearing a wedged `nextcloud` Podman record. The live registry config already has OVH primary plus tx1138 mirror, and Meshtastic/Portainer were added to the catalog surfaces.
- Later on 2026-06-02, the current release backend hash `579b823cf4a4b8c50bb3d0c3d49449c58101b016eb6ebc8049975dce98e34265` was found active and stable on `.198`. Meshtastic `app.files` rendering was proven live by removing `/var/lib/archipelago/meshtastic/config.yaml`, restarting through `package.restart`, and verifying the manifest recreated the file. Focused Meshtastic, focused `meshtastic,jellyfin,filebrowser`, and broad non-destructive audits all passed afterward; raw Podman sweep was clean.
- The remaining release gate was continued on 2026-06-02: bounded disk cleanup, journal retention, backend-backup retention, and release-focused catalog drift classification were added. `.198` is active on backend hash `e285d421cef497beb6b4b929f36fb4296d6db1f4a4c786157b6751eec51619ca`; focused and broad post-cleanup lifecycle audits passed, and final raw Podman sweep was clean.
- Follow-up found Podman store commands can hang on `.198` beyond image prune (`podman system df`, image list/exists, and sometimes broad ps/inspect). The release cleanup path now skips Podman image/volume prune rather than touching that unstable path. `.198` is active on backend hash `c9695dc3db10ff6e593cdbcfbbdc94b2e98b6008aa62655bba51b9879b549e8c`; Uptime Kuma was repaired with a normal `package.restart`; focused and broad post-repair lifecycle audits passed, and final raw bad-state sweep was clean.
- On 2026-06-03, startup/adoption scanner hardening and pasta restart repair were deployed. `.198` is active on backend hash `2b72e83ff368e4a696ad701f8985b0a8e1e889d9f4844056dc063455df973b28`; `package.restart` for Uptime Kuma now returns successfully and restores the `3002` pasta listener; focused `meshtastic,jellyfin,filebrowser,uptime-kuma` and broad lifecycle audits passed.
- Later on 2026-06-03, expanded rollback cleanup and store-safe uninstall hardening were deployed. `.198` is active on backend hash `7f90345b75148b7ed748e1a417f31d1273e1646a9b742891858df11c5397051b`; `system.disk-cleanup` reclaimed `10.3 GB` from old backend and web UI rollback artifacts while still skipping Podman prune, and focused `meshtastic,jellyfin,filebrowser,uptime-kuma` lifecycle passed afterward.
- Latest 2026-06-03 follow-up deployed backend hash `d21202cd79794e3bfc882d37134afd7a41dac766bae386a675714e5fa030e94e`. It mitigates stale cached `container-list` state during Podman scan backoff, adds a bounded TCP reachability fallback for `container-health`, and adds Jellyfin `8096` to legacy pasta host-listener repair. Focused `meshtastic,jellyfin,filebrowser,uptime-kuma` lifecycle passed on this hash. Broad lifecycle still needs rerun on this latest hash.
- Current validation backend hash is `14d360a206d1e58f287c5722d709dace0284b0dea56b66aa4bce0f57c631631b`. It keeps the generic host-listener health direction, preserves the `container-health` fallback fix from `be95ea...`, hardens fresh local-build installs so `podman image exists <local-build-tag>` failures/timeouts rebuild instead of failing the lifecycle operation, and reduces duplicated legacy runtime port repair by deriving host ports from manifests. Targeted PhotoPrism and broad non-destructive `.198` lifecycle audits passed on this hash.
- Catalog metadata generation from manifests is now implemented via `scripts/generate-app-catalog.py`. The canonical catalog and UI public catalog are synced from manifest-owned fields, strict release drift is zero, and frontend build validation passed.
- Current live `.198` validation backend hash is `95dfd8530ae9621b2f16da05d2229fe40bed7e5f6e2097cf4c87000fe97b92de`. Broad non-destructive lifecycle is green on that deployed line after app health/port recovery, IndeedHub recovery, scoped legacy install hardening, and bounded Podman pull hardening.
- Local release validation now passes the full backend binary test target and every Rust workspace member after release cleanup fixes for scanner backoff wakeups, crash-recovery tests, manifest-port lookup, journal parsing, and boot-reconciler test determinism.
- Frontend release validation now passes `npm run type-check`, `npm test` (`548` tests), and `npm run build` after fixing mobile app-launch routing for new-tab apps and updating stale launch tests. Local `npm ci` is blocked by root-owned `neode-ui/node_modules` entries, so dependency reinstall remains a local environment cleanup item requiring explicit approval.
- Reboot validation is not yet green. User reported that a reboot test left IndeeHub stopped afterward, with multiple containers killed by SIGKILL during shutdown/reboot and at least one crash. Treat post-reboot recovery as the active release blocker.
- Local follow-up now hardens IndeeHub stack boot recovery and updates lifecycle validation so IndeeHub must still serve the Nostr signer bridge (`/nostr-provider.js`) before a launch probe passes.
## Completed In This Pass
- Pause checkpoint for resume: generated app-session metadata now covers manifest-owned launch ports, titles, and new-tab behavior. The next migration step should continue from proxy path/companion UI alias generation or return to the release blocker around post-reboot IndeeHub recovery.
- Updated `docs/APP-PACKAGING-MIGRATION-PLAN.md` to reflect the current `apps/<app-id>/manifest.yml` contract, replacing stale `archy-app.yml` next-step language with the actual parser/generator/orchestrator progress and the remaining migration blockers.
- Updated `docs/app-developer-guide.md` so developers see the current manifest fields, generated catalog flow, validation commands, and release lifecycle expectations instead of the older Nostr marketplace publish/trust-score draft.
- Verified the developer-guide manifest example parses as YAML, `scripts/generate-app-catalog.py` is idempotent, strict release catalog drift remains zero, and `git diff --check` is clean for the migration docs.
- Extended `scripts/generate-app-catalog.py` to also emit `neode-ui/src/views/appSession/generatedAppSessionConfig.ts` from manifests, and wired `appSessionConfig.ts` to merge generated launch ports/titles/new-tab launch behavior with the existing manual overrides for companion UIs and aliases.
- Added a Fedimint `interfaces.main` launch declaration for the Guardian wait/proxy UI on port `8175`, so that public launch surface is now represented in the manifest.
- Focused validation passed for the generated app-session path: Python helper compile, generator idempotence, strict catalog drift, `appSessionConfig.test.ts`, and frontend type-check.
- Aligned `docs/APP-PACKAGING-MIGRATION-PLAN.md` and `docs/app-developer-guide.md` with the current manifest/runtime contract so the release docs no longer describe the stale marketplace-style schema.
- Removed the hardcoded Portainer host-prep path and replaced it with a manifest plus generic Podman socket bind-mount preparation.
- Added generic Quadlet health drift detection for command, interval, timeout, and retry changes.
- Made rendered HTTP health helpers honor manifest timeouts.
- Added image availability guards before Quadlet starts/restarts so pruned images are pulled or built before systemd tries to start them.
- Fixed stale dependency handling so active manifest dependencies are not suppressed by old `user-stopped.json` entries.
- Added parent-app reconcile syncing for dependency Quadlet units.
- Validated Portainer, Filebrowser, BTCPay, and broad non-destructive audits on `.198`.
- Updated Meshtastic manifest to use a real available image, the real `/dev/ttyUSB0` device, the actual daemon data path, and a non-HTTP health check.
- Updated the lifecycle harness so non-HTTP apps do not require launch metadata.
- Added a generic manifest-owned file rendering primitive under `app.files` so apps can declare required bind-mounted config files without adding app-specific Rust/OS branches.
## Current `.198` State
- `archipelago.service`: active.
- `archipelago-doctor.timer`: inactive.
- `archipelago-reconcile.timer`: inactive.
- Current validation backend hash: `95dfd8530ae9621b2f16da05d2229fe40bed7e5f6e2097cf4c87000fe97b92de`.
- `.198` root filesystem pressure is currently resolved for release validation: latest sweep showed `/` at 65% used with about 9.6G free after expanded rollback cleanup.
- Latest focused Fedimint, Immich, IndeedHub, and PhotoPrism audits passed on the current hash.
- Broad non-destructive lifecycle passed on the current hash before and after backend restart validation.
## Meshtastic Status
- Orchestrator routing is fixed and verified by the generated Quadlet unit.
- Current generated unit uses:
- `Image=docker.io/meshtastic/meshtasticd:daily-alpine`
- `Volume=/var/lib/archipelago/meshtastic:/var/lib/meshtasticd:Z`
- `AddDevice=/dev/ttyUSB0`
- `HealthCmd=test -f /var/lib/meshtasticd/config.yaml`
- The daemon starts and accepts TCP API connections on port `4403`.
- Full lifecycle passed on `.198`: install, stop, start, restart, uninstall with preserved data, and reinstall.
- A persisted `config.yaml` is required. The release path is now the generic `app.files` manifest primitive rather than a Meshtastic-specific backend hook, and this has been verified live on `.198` by deleting the file and proving `package.restart` recreates it from the manifest.
## Release Blockers
- Continue monitoring the current optimized release backend on `.198`; the previously observed release-binary segfault is not reproducing with hash `95dfd8530ae9621b2f16da05d2229fe40bed7e5f6e2097cf4c87000fe97b92de`.
- `system.disk-cleanup` now handles journal, backend-backup, legacy backend rollback, and web UI rollback retention while intentionally skipping Podman image/volume prune because Podman store commands can hang on `.198` under current load. Diagnose Podman store health separately from the release cleanup path.
- Release image probes have been further quarantined from the fragile Podman store commands and deployed to `.198` on backend hash `7e82532137292e91111f63819d1be7fa69f994ce20d6b5e0194915f194f20412`: runtime, legacy install, and companion image checks now use bounded targeted `podman image inspect` instead of `podman image exists` or `podman images -q`. Focused and broad non-destructive lifecycle validation passed on the deployed hash.
- Podman socket/runtime health remains a release blocker: `package.restart jellyfin` stopped the container but failed to complete because Podman reported `Cannot connect to Podman socket at /run/user/1000/podman/podman.sock: Permission denied`; `package.start jellyfin` recovered the app and the focused lifecycle passed afterward.
- Release-focused catalog drift now has zero missing catalog/manifest entries and zero metadata drift after generating catalog metadata from manifests.
- Backend-restart validation passed. Host-reboot validation is currently failed/pending due to post-reboot IndeeHub recovery. Reboot retests should run only after an explicit release checkpoint/approval.
- Local code-review/refactor cleanup gate has full local validation coverage now:
- `cargo test --manifest-path core/Cargo.toml -p archipelago --bin archipelago` passed (`688` tests);
- all other workspace packages check/test clean;
- frontend type-check/tests/build passed;
- release build, catalog drift, catalog idempotence, Python helper compile, and whitespace checks passed.
- Before `1.8-alpha` release:
- deploy the post-reboot recovery fixes;
- prove focused IndeeHub lifecycle with Nostr signer injection intact;
- update the app packaging/developer docs so `docs/APP-PACKAGING-MIGRATION-PLAN.md` and `docs/app-developer-guide.md` match the current manifest/runtime contract and release-quality lifecycle expectations;
- complete the required refactor/remove-dead-code gate after correctness validation: remove obsolete transitional code, stale per-app hacks, duplicate lifecycle paths, and misleading compatibility fallbacks, then rerun release validation;
- require at least 3 consecutive clean post-fix reboots with broad non-destructive lifecycle green after each;
- prefer 5 consecutive clean reboots for production-release confidence;
- cut and smoke-test the `1.8-alpha` ISO.
## Bottom Line
We are working toward the intended goal: better than Umbrel/StartOS by making app behavior declarative and registry/manifest-owned. The migration is substantially advanced, Meshtastic manifest-owned config generation is verified live, catalog metadata is generated from manifests, disk cleanup/backup retention is in place without Podman prune risk, and full local backend/frontend workspace validation has been green. Remaining follow-up for `1.8-alpha` is post-reboot recovery validation, especially IndeeHub plus Nostr signer behavior, repeated reboot passes, ISO cut/smoke test, separate Podman socket/store-health diagnosis, and optional local cleanup of root-owned frontend dependencies before rerunning `npm ci`.