Files
archy/.planning/phases/02-ui-performance/02-10-SUMMARY.md
T

186 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: 02-ui-performance
plan: 10
subsystem: ui
tags: [playwright, performance-profiling, keepalive, dispersion-analysis, gap-closure]
# Dependency graph
requires:
- phase: 02-ui-performance/02-09
provides: "Confirmation that Server's/Web5's KeepAlive 'remount' was a probe-measurement artifact — 02-10's re-measure runs against the build that includes this finding (no source change, so trivially satisfied)"
- phase: 02-ui-performance/02-01
provides: "The frozen perf harness and 02-PERF-BASELINE.json"
- phase: 02-ui-performance/02-08
provides: "02-PERF-AFTER.json, the second data point in this plan's three-way comparison"
provides:
- "02-PERF-REMEASURE.json — third on-device measurement (5 runs/surface) against archi-dev-box"
- "02-FINDINGS.md § Re-measurement (gap closure) — three-way dispersion table and a verdict per named surface"
- "REQUIREMENTS.md PERF-02/PERF-03 traceability rows updated to point at the resolved verdicts"
affects: []
# Tech tracking
tech-stack:
added: []
patterns:
- "Three-way dispersion comparison (min/median/max across all samples, not bare medians) to distinguish a real, monotonically-worsening regression from run-to-run noise — a regression that shrinks toward baseline as conditions improve is noise; one that grows despite improving conditions is real"
- "Git-log bisection against the pre-phase-2 baseline commit (a75b6709) to confirm a regression traces to a phase-2/gap-closure commit before naming it 'phase-2-caused' rather than pre-existing"
key-files:
created:
- .planning/phases/02-ui-performance/02-PERF-REMEASURE.json
modified:
- .planning/phases/02-ui-performance/02-FINDINGS.md
- .planning/REQUIREMENTS.md
key-decisions:
- "Deployed archi-dev-box to 3e3159fa (frontend-only) before measuring — the previously-deployed 8fe6217b predated the 02-review commits touching 4 of the 6 named surfaces (Web5/Discover/Server/OpenWrtGateway); a stale-bundle re-measure would have been worthless for this plan's purpose"
- "No further deploy performed after the coordinator flagged a shared-tree hazard mid-plan (concurrent security-follow-up and BotFights sessions leaving uncommitted edits in neode-ui/src) — every confirmed regression is recorded as an accepted deviation rather than a landed fix, since the plan's own 'fixed' branch requires a deploy-and-re-measure step that was unavailable this round"
- "First ARCHY_PERF_RUNS=5 attempt hit the harness's own hardcoded 20-minute test.setTimeout under concurrent node load (an unrelated podman build) and aborted after 9/15 surfaces; re-ran once per the plan's own contingency rather than hand-editing the partial artifact — the second attempt completed cleanly in 7.9 minutes"
- "Wallet/send-flow cleared as noise (re-measure median and 3/5 samples at or below the baseline's own minimum); Discover/Server/Web5/AppDetails/OpenWrtGateway confirmed as real, phase-2-caused regressions because their medians climbed monotonically across all three independent runs despite disk pressure genuinely easing between runs — the opposite of what the noise theory predicts"
- "Named cause for all five confirmed regressions is the same client-side render/reactivation 'split-signal' class 02-08 already identified for Web5/Fleet: RPC count flat-or-improved and (for Server/Web5) instance survival independently confirmed by 02-09, yet wall-clock revisit time keeps climbing — pointing at Vue-reactivity/re-render cost from the useCachedResource/KeepAlive conversion itself, not a network wait or a remount"
requirements-completed: []
coverage:
- id: D1
description: "Third measurement (02-PERF-REMEASURE.json) taken against archi-dev-box with the frozen harness (git diff --stat 3ee20430 -- neode-ui/e2e/perf/ shows only 02-09's separate probe spec), runs=5, 15/15 rows, run conditions (clock, df, load, concurrent activity) recorded before and after"
requirement: "PERF-02"
verification:
- kind: e2e
ref: "neode-ui/e2e/perf/surface-perf.spec.ts run against http://archi-dev-box with ARCHY_PERF_RUNS=5 — first attempt timed out under concurrent load (harness's own 20-min test.setTimeout, not a harness modification), re-run once, second attempt exit code 0, 15/15 rows"
status: pass
human_judgment: false
- id: D2
description: "Every one of the six surfaces 02-VERIFICATION.md named for gap 2 carries an explicit verdict backed by three-way dispersion data and git-log bisection: 5 confirmed regressions (accepted deviations, named cause + destination), 1 cleared as noise"
requirement: "PERF-02"
verification:
- kind: other
ref: "02-FINDINGS.md § Re-measurement (gap closure) — three-way min/median/max table + per-surface verdict + Accepted Deviations table"
status: pass
human_judgment: false
- id: D3
description: "The environmental-noise theory is settled with data, not left as a caveat: current run conditions (79% disk, load, concurrent podman/vitest/vite/typeorm activity) recorded quantitatively; prior runs' known conditions restated for comparison"
requirement: "PERF-03"
verification:
- kind: other
ref: "02-FINDINGS.md § Re-measurement (gap closure), 'Run header and recorded conditions' table"
status: pass
human_judgment: false
- id: D4
description: "Full vitest suite and type-check confirmed green (required whether or not a fix landed); no fix landed this session (deploy blocked), so this is a sanity check against the tree as it stood"
requirement: "PERF-02"
verification:
- kind: automated
ref: "npm test -- --run: 95 test files / 785 tests passed; npm run type-check: clean"
status: pass
human_judgment: false
duration: ~55min (includes one ~20min harness timeout-and-retry cycle and an 8min successful harness run)
completed: 2026-07-31
status: complete
---
# Phase 02 Plan 10: Timing-Regression Verdict — Three-Way Re-Measure Summary
**Re-measured all 15 surfaces a third time (5 runs each) against a freshly-deployed archi-dev-box and, using full sample dispersion rather than bare medians, found the environmental-noise theory only half holds: Wallet/send-flow's apparent regression clears as noise, but Discover/Server/Web5/AppDetails/OpenWrtGateway all show a real, monotonically-worsening revisit-ms regression across three independent runs even as disk pressure genuinely eased — confirmed phase-2-caused via git-log bisection, named as the same client-side render/reactivation cost 02-08 already flagged for Web5/Fleet, and recorded as accepted deviations rather than fixed because a mid-plan shared-tree hazard blocked any further deploy this session.**
## Performance
- **Duration:** ~55 min (Task 1: deploy ~3min, first harness attempt timed out at 20min under concurrent podman-build load, second attempt completed in 7.9min; Task 2: dispersion analysis + findings write-up + requirements update + test-suite sanity check)
- **Completed:** 2026-07-31
- **Tasks:** 2/2 completed, no checkpoints (autonomous plan)
- **Files modified:** 1 created (`02-PERF-REMEASURE.json`), 2 modified (`02-FINDINGS.md`, `.planning/REQUIREMENTS.md` — the latter is a scope deviation from `files_modified`, see below)
## Accomplishments
- Deployed archi-dev-box (`--frontend-only`) to commit `3e3159fa` before measuring, since the previously-deployed `8fe6217b` predated the `02-review` commits touching Web5/Discover/Server/OpenWrtGateway and 02-09's own investigation — confirmed via `scripts/deploy-history.log` and a clean-tree (`dirty=false`) deploy.
- Confirmed the harness is byte-for-byte frozen since the after-run (`git diff --stat 3ee20430 -- neode-ui/e2e/perf/` shows only 02-09's separately-added `keepalive-remount-probe.spec.ts`, zero changes to the three measured files).
- Ran the harness at `ARCHY_PERF_RUNS=5`. First attempt hit the harness's own hardcoded 20-minute `test.setTimeout` while an unrelated `podman build` consumed CPU on the shared box, aborting after 9/15 surfaces with the remainder failing "browser has been closed" — recognized as a harness-timeout artifact, not measurement data, and not written into the artifact as such. Re-ran once per the plan's own contingency; the second attempt completed cleanly in 7.9 minutes once the concurrent build finished. `02-PERF-REMEASURE.json` committed: 15/15 rows, `runs: 5`, `baseUrl: http://archi-dev-box`.
- Recorded run conditions quantitatively for the first time in this three-run series: 79% disk (1.4T/1.8T, down from the qualitative "85%" both prior runs cited — confirming the ~118G disk reclaim actually shows up in the numbers), load averages 8.7011.88 across pre/post checks, and named concurrent activity (a BotFights `podman build` pre-run; `npm run build`/`vitest`/`typeorm migration:run` from other sessions post-run).
- Built the three-way min/median/max dispersion table (not bare medians) for all six of 02-VERIFICATION.md's named surfaces plus two bonus data points (Fleet, Chat), then wrote a verdict per surface into a new `## Re-measurement (gap closure)` section in `02-FINDINGS.md`:
- **Wallet/send-flow: cleared as noise.** Re-measure spread (17292551ms) sits at or below the baseline's own spread (25622619ms); 3 of 5 samples are below the baseline minimum entirely. The separate, pre-existing revisit-slower-than-first-visit anomaly (unrelated to phase 2) is unchanged and stays in Outstanding.
- **Discover, Server, Web5, AppDetails, OpenWrtGateway: confirmed real regressions**, each with a monotonically-climbing median across all three independent runs (e.g., Web5: 566→709→1329ms with zero sample overlap between any two runs) — the opposite of what shrinking disk pressure should produce if this were noise. Each traced via `git log a75b6709..HEAD` to specific phase-2/`02-review` commits confirming phase-2 origin, not pre-existing behavior.
- Named cause for all five: the same client-side render/reactivation "split-signal" class 02-08 already identified for Web5/Fleet — RPC count flat or improved, and for Server/Web5 instance survival independently proven by 02-09, yet wall-clock revisit time keeps growing. Most plausibly Vue-reactivity/watcher/re-render cost from the `useCachedResource`/KeepAlive conversion itself.
- Recorded as **accepted deviations**, not fixes: the plan's own "fixed" branch requires a deploy-and-re-measure step to prove a fix moved the number, and that step became unavailable mid-plan (see Deviations below).
- Fleet (out-of-scope bonus, same mechanism, most severe magnitude: 330→1054→2631ms) and Chat (measured for the first time this phase, no baseline counterpart) recorded as data points, not verdicts.
- Updated `.planning/REQUIREMENTS.md`'s PERF-02/PERF-03 traceability rows to point at this section, per the plan's own Task 2 instruction to reconcile the record.
- Full `npm test` (95 files / 785 tests) and `npm run type-check` confirmed green.
## Task Commits
Each task was committed atomically, staging only the exact paths this plan owns (never `git add -A`, given a confirmed shared-tree hazard mid-plan — see Deviations):
1. **Task 1: Re-run the frozen harness against archi-dev-box under recorded conditions** - `db629f6f` (feat)
2. **Task 2: Three-way comparison, a verdict per named surface, and the resulting action** - `1d6b6c22` (docs)
## Files Created/Modified
- `.planning/phases/02-ui-performance/02-PERF-REMEASURE.json` - Third on-device measurement, 15 rows, `runs: 5`, `baseUrl: http://archi-dev-box`
- `.planning/phases/02-ui-performance/02-FINDINGS.md` - New `## Re-measurement (gap closure)` section: run conditions for all three runs, three-way dispersion table, per-surface verdict, Accepted Deviations table
- `.planning/REQUIREMENTS.md` - PERF-02/PERF-03 traceability rows updated to cite the resolved verdicts (scope deviation — not in this plan's `files_modified`; called out here per the plan checker's hygiene note, and directly instructed by this plan's own Task 2 action text)
## Decisions Made
See `key-decisions` in frontmatter for the full list. Highlights:
- **Deploy-then-lock:** deployed once at the start of Task 1 (clean tree, needed for measurement validity against 4 of 6 named surfaces), then performed **no further deploy** after a mid-plan coordinator directive identified concurrent uncommitted work from other sessions sharing the tree — every confirmed regression is an accepted deviation rather than a fix for exactly this reason.
- **Dispersion over medians:** the whole verdict rests on full min/median/max ranges across all samples in all three artifacts, not the single bare medians 02-08 compared. This is what let Wallet/send-flow clear (its re-measure spread undercuts the baseline's own minimum) while catching that the other five aren't noise (their spreads never shrink back toward baseline — they climb monotonically across three independent, differently-loaded runs).
- **Git-bisection-confirmed phase-2 origin:** every confirmed regression's implicated view file was checked against `a75b6709` (the pre-phase-2 baseline commit) to name the specific phase-2/`02-review` commit responsible, rather than asserting "must be phase 2" from timing alone.
- **Harness-timeout handling:** treated the first attempt's 20-minute Playwright `test.setTimeout` cutoff (hardcoded in the frozen `surface-perf.spec.ts`, sized for 3 runs) as a legitimate execution constraint under `ARCHY_PERF_RUNS=5` plus concurrent node load — not a reason to edit the frozen harness, and not a reason to hand-edit the partial artifact. Re-ran once per the plan's own contingency instead.
## Deviations from Plan
### Auto-fixed Issues
None — no bugs found in this plan's own scope requiring a Rule 1/2/3 fix. (The five confirmed timing regressions are pre-existing phase-2 defects this plan's own scope explicitly limits to "fix only if small and deployable this session" — see below for why that branch wasn't available.)
### Scope deviations (documented, not auto-fixed)
**1. Confirmed regressions recorded as accepted deviations instead of fixed, due to a mid-plan deploy block**
- **Found during:** Task 2 analysis
- **Issue:** Discover/Server/Web5/AppDetails/OpenWrtGateway all confirmed as real, phase-2-caused regressions with a plausible, well-evidenced cause (client-side render/reactivation cost). The plan's own "fix it" branch requires: a small, single-file, test-covered fix, followed by `npm test`/type-check/build, an `ARCHIPELAGO_TARGET=archi-dev-box` deploy, and a targeted harness re-run proving the number moved.
- **Why not fixed:** Partway through this plan's execution, the coordinator identified that a concurrent security-follow-up session and a BotFights session had left uncommitted edits in shared `neode-ui/src` files (`resources.ts`, `useCachedResource.ts`, `Cloud.vue`, `AppDetails.vue`, and others) — files this plan never touches. A second deploy this session would have shipped that in-progress, unreviewed work to the node alongside anything this plan fixed, which the coordinator explicitly instructed against. Without the deploy-and-re-measure step, "fixed" cannot be proven, so all five are recorded as accepted deviations with a named cause, why they're out of bounds, and a destination (carried forward in `STATE.md`'s Blockers/Concerns for a future dedicated Vue-render-profiling / phase-2 gap-closure plan).
- **Files modified:** None (no source change attempted).
- **Committed in:** `1d6b6c22` (the verdict itself, not a fix).
**2. `.planning/REQUIREMENTS.md` modified outside this plan's `files_modified`**
- **Found during:** Task 2, per the plan's own action text ("update the status note for those rows... so the next verification pass reads the resolved state")
- **Issue:** The plan's frontmatter `files_modified` lists only `02-PERF-REMEASURE.json` and `02-FINDINGS.md`, but Task 2's own instructions explicitly direct a `REQUIREMENTS.md` coverage-table update.
- **Resolution:** Updated only the PERF-02/PERF-03 traceability rows (not the requirement checkboxes, which are unrelated to this plan's evidence-note scope). Called out here per the plan checker's own hygiene note.
- **Files modified:** `.planning/REQUIREMENTS.md`.
- **Committed in:** `1d6b6c22`.
### Harness-execution deviation (not a scope deviation — anticipated by the plan's own contingency)
**3. First `ARCHY_PERF_RUNS=5` attempt aborted at the harness's hardcoded 20-minute test timeout**
- **Found during:** Task 1
- **Issue:** `surface-perf.spec.ts`'s `test.setTimeout(20 * 60 * 1000)` is sized for "15 surfaces x 3 runs" per its own inline comment; at `runs=5` under concurrent podman-build load, the first attempt reached only 9/15 surfaces before timing out, force-closing the browser and failing every remaining surface with "Target page, context or browser has been closed."
- **Fix:** Did not edit the frozen harness file (would violate this plan's own freeze gate) and did not hand-edit the partial artifact. Re-ran the identical command once per the plan's explicit contingency; the concurrent build had finished by then and the second attempt completed cleanly in 7.9 minutes.
- **Files modified:** None.
- **Verification:** `02-PERF-REMEASURE.json` from the second run has 15/15 rows, `runs: 5`, no `null`/error rows except Mesh (unmeasured for the same device-not-connected reason as both prior runs).
---
**Total deviations:** 2 scope deviations (both documented, neither a code bug), 1 harness-execution retry anticipated by the plan itself.
**Impact on plan:** The plan's own success criteria are still met — every one of the six named surfaces has an explicit, data-backed verdict (5 confirmed + named cause + destination, 1 cleared with data) rather than being left in the "regressed or noisy, unresolved" state that blocked the phase's clean pass. No regression was silently dropped; the deploy block only changed which branch ("fixed" vs. "accepted deviation") each confirmed regression landed in.
## Issues Encountered
- **Shared working tree, mid-plan coordinator directive:** a concurrent security-follow-up session and a BotFights session were both actively committing to this same tree during this plan's execution. Handled per instruction: staged only exact paths this plan owns for every commit (never `git add -A`), unstaged one file (`02-REVIEW.md`) that another session had staged into the shared index before my own commit (using `git restore --staged`, which does not alter their working-tree content), and performed no deploy after the coordinator's directive. Both other sessions' commits (`5bfe6088`, `b8391115`, `16102558`) landed cleanly on top of/alongside this plan's commits with no conflicts, confirmed via `git log`.
- **Harness timeout under real concurrent load** — see Deviations #3 above; resolved by a single re-run, no artifact corruption.
- **`02-PERF-REMEASURE.json`'s `commit` header field reads `7c063a20`** (the local working-tree HEAD at the moment the spec ran) rather than `3e3159fa` (the commit actually deployed and served) — the two docs-only/unrelated-app commits between them (`d7fbba98`, `058d7609`, `002de661`) touch no `neode-ui/src` file, confirmed via `git diff --stat 3e3159fa..7c063a20 -- neode-ui/src` (empty), so this is a cosmetic artifact-header discrepancy, not a measurement-validity problem, and is explicitly called out in `02-FINDINGS.md`.
## User Setup Required
None for this plan's own execution — `ARCHY_PASSWORD` was supplied out-of-band by the coordinator per the plan's `user_setup` block (as in 02-01/02-08/02-09), passed only as a runtime environment variable, never written to any file, commit, or this summary.
## Next Phase Readiness
- Verification gap 2 is closed: every one of the six surfaces 02-VERIFICATION.md named now carries an explicit, data-backed verdict — no surface remains in the "regressed or noisy, unresolved" state.
- **Carried forward, not resolved:** Discover/Server/Web5/AppDetails/OpenWrtGateway/Fleet's confirmed client-side render/reactivation regressions need (a) DevTools Performance-panel profiling to pinpoint the exact reactivation cost per surface, and (b) a clean deploy window once the shared tree's concurrent security-follow-up and BotFights work land — tracked in `STATE.md`'s Blockers/Concerns, not silently dropped.
- Combined with 02-09 (gap 1, Server/Web5 remount proven to be a probe artifact, no fix needed), both of `02-VERIFICATION.md`'s gaps now have committed, data-backed resolutions ready for a `gsd-verifier` re-run on phase 02, per the phase's own `.continue-here.md` handoff note.
- No blockers for the milestone's next phase from this plan specifically; the five accepted-deviation regressions are a known, quantified, non-blocking follow-up item, not a new blocker.
---
*Phase: 02-ui-performance*
*Completed: 2026-07-31*