refactor: Add cache validation methods for use in ui5 serve#1450
Open
RandomByte wants to merge 7 commits into
Open
refactor: Add cache validation methods for use in ui5 serve#1450RandomByte wants to merge 7 commits into
RandomByte wants to merge 7 commits into
Conversation
RandomByte
changed the title
…h build
A cold cache lookup for a build-as-side-effect dependency currently waits
for the first HTTP request against it. Move that lookup between build
cycles so the server reports IDLE or STALE while it happens.
New surface:
- `ProjectBuildCache.validateCache(dependencyReader)` — split out of
`prepareProjectBuildAndValidateCache`, which now only performs build-prep
side effects and delegates.
- `ProjectBuildContext.validateCache()` — logs perf timing, propagates
changed paths to dependents. Cache.Force still throws on stale caches.
- `ProjectBuilder.validateCaches({projects, signal}, callback)` — mirrors
`#build`: guards `#buildIsRunning`, walks contexts dependency-first via
`getRequiredProjectContexts` so cold contexts (and their transitive
build-time dependencies' source indices) initialize inside the pass. A
`willValidate` hook fires before each project so the caller can claim it.
State machine:
- `ProjectBuildStatus` gains a VALIDATING state; `setReader` accepts both
BUILDING and VALIDATING as legitimate prior states.
- `SERVER_STATES.VALIDATING` is distinct from BUILDING and STALE. After a
cycle drains, BuildServer transitions BUILDING → VALIDATING when any
project is still INITIAL, then to IDLE once the pass confirms every
cache is fresh, or to STALE when the pass surfaces a stale one.
Previously the post-build transition pinned to STALE, so a fully-cached
graph never returned to IDLE on its own.
- VALIDATING → STALE mirrors IDLE → STALE, so a source change during a
pass surfaces immediately. `getReaderForProject` skips enqueueing a
build while VALIDATING (the reader request joins the pass); on settle,
any non-fresh project with pending readers gets a real build queued so
callers don't hang.
Error handling matches the build path in two directions:
- A non-abort failure inside the pass emits `error` and transitions to
SERVER_STATES.ERROR; the post-pass finally skips the IDLE/STALE
transition when the run ended in ERROR.
- A failed build cycle short-circuits the cycle-end transition. Before,
`#build` resolved normally after setting ERROR and the cycle-end
`.then()` scheduled a validation pass over every INITIAL project — in
Cache.Force mode with no cache, that threw `Cache is in "Force" mode
but no cache found` on the first dependency it visited, on top of the
build error that just fired.
BuildServer preempts the in-flight pass before each build cycle and on
`destroy()`, composes every targeted project's per-project AbortSignal
into the pass signal, and releases VALIDATING on every targeted project
in a finally so none are stranded after an abort.
The `Serve` logger gains `validating(projects)`, emitting
`serve-validating`. The InteractiveConsole writer renders
STATES.VALIDATING with a cyan spinner and keeps the tick loop running.
Two dedup passes on the logger code that grew with the VALIDATING
state.
`Serve.#emitStatus(status, payload, fallbackMessage, {level})` replaces
the 4-line pattern each status method (`ready`, `stale`, `building`,
`validating`, `buildDone`, `serveError`) repeated: build the
`ui5.serve-status` payload, emit it, and if no listener responded, log
at the same level. Public method names, argument shapes, event payload
keys and fallback messages are unchanged. New statuses land as
two-line wrappers.
`SPINNING_STATES` moves onto `state/build.js`. Both
`InteractiveConsole.#scheduleTick` and `renderStatusLine` gated on
`state === BUILDING || state === VALIDATING`; a third animated state
meant matching changes at each site. The tick scheduler now consults
the set. The renderer keeps its per-state cases because each spinning
state renders distinct colour, label and payload; only the tick gate
benefits from a shared predicate today.
Every serve-status test open-coded the same listener wiring (`statusEvents.push` on `ui5.serve-status`, teardown), and the two VALIDATING end-state tests re-declared the same root/lib graph literal. Add two helpers: - `makeStatusRecorder(t)` — accumulating event array plus teardown. - `makeGraphWithLib()` — graph with one library dependency, the shape needed to leave one INITIAL project after a build cycle (the trigger for VALIDATING). No runtime impact; the 16 tests continue to pass.
Both loops open-coded the same three-step setup: call
`getRequiredProjectContexts(requestedProjects)`, traverse with
`traverseDependenciesDepthFirst(true)`, and materialize a DFS-ordered
queue with a parallel `processedProjectNames` array. They then iterated
with `while (queue.length) { queue.shift() }`, O(n^2) in the queue
length for no reason.
Extract `#buildProjectQueue(requestedProjects)` returning
`{projectBuildContexts, queue, processedProjectNames}`; the
`getRequiredProjectContexts` perf-log line moves inside. Both callers
start with a one-line destructure and keep their differing pre/post
work inline. Both loops switch to `for (const projectBuildContext of
queue)` — the queue is built once from the pre-computed traversal and
is not mutated during iteration.
The per-project loop bodies stay in each caller: their contents differ
(build vs. validate step sequences), and the shared shape
(`possiblyRequiresBuild` gate, abort classification) is short enough
that closure-passing wouldn't earn its keep.
`ProjectBuildContext.prepareProjectBuildAndValidateCache` and
`ProjectBuildContext.validateCache` routed through the same
`#runCacheValidation` helper, differing only in the underlying cache
call and the perf-log label. `ProjectBuildCache` mirrored the split
with a delegator that captured readers, discarded pending stage-cache
entries, and called `validateCache`.
Collapse to a single `validateCache(depReader?, {prepareForBuild})`,
the flag gating pre-build side effects (`stageCache.discardPending()`
+ reader capture). `ProjectBuildContext.#runCacheValidation` goes
away; both `ProjectBuilder` call sites log under `validateCache`.
Tests and skill docs mirror the API change.
Two ad-hoc protocols around the VALIDATING state each move onto the type
that owns them.
Reader-request protocol → `ProjectBuildStatus`. It was spread across three
call sites: `#getReaderForProject` skipped `#enqueueBuild` while
`isValidating()`, `#runBackgroundValidation`'s finally re-enqueued a build
when the project had pending reader requests, and `#triggerRequestQueue`
called `#stopActiveValidation` to make room for the build. Each site
carried a long block comment explaining why the others must bail.
`ProjectBuildStatus` now takes an `onBuildRequired` callback at
construction; BuildServer wires it to `#enqueueBuild(projectName)`.
`releaseValidating()` fires the callback whenever the state isn't FRESH and
the reader queue is non-empty, covering both the cache-stale case
(VALIDATING → INITIAL with queued readers) and the invalidated-mid-pass
case (INVALIDATED with queued readers). Call sites collapse:
`#getReaderForProject` calls `addReaderRequest` unconditionally,
`#runBackgroundValidation`'s finally shrinks to
`status?.releaseValidating()`. `hasPendingReaderRequests()` had no other
callers and goes away.
Cycle-end reconciliation → `#reconcileServerState`. Both producer cycles
(build cycle end, background validation pass end) open-coded the same
terminal-state decision: bail if another actor already owns the next
transition, then pick IDLE / VALIDATING / STALE based on the stale set.
The post-validation finally carried a five-part guard whose intent
("only transition if nobody else has") was spread across three field
checks and two state checks.
`#reconcileServerState({hrtime, mayValidate} = {})` bails on `destroyed`
or `ERROR`, then on `activeBuild || pendingBuildRequest.size > 0`.
Otherwise: zero stale → IDLE; stale set non-empty → try
`#scheduleBackgroundValidation` when `mayValidate` is true, fall back to
STALE. Post-build passes `mayValidate: true`; post-validation passes
`mayValidate: false` to avoid re-scheduling the pass that just settled.
The old `state === BUILDING` branch is dropped. BUILDING is only set
inside `#triggerRequestQueue`'s setTimeout callback, itself gated on
`await #stopActiveValidation` — the same promise whose finally was
performing the check. A future refactor introducing a synchronous
BUILDING setter would need to re-add the guard.
…d claim While the request-queue timer awaits `#stopActiveValidation`, validation's finally can call `releaseValidating` → `onBuildRequired` → schedule a second timer. When that timer fires with `#activeBuild` already set, it calls `projectBuilder.build` concurrently; the builder's `buildIsRunning` guard rejects with 'A build is already running' and the server transitions to ERROR, emitting a spurious 'error' event. Re-check `#activeBuild` after the await. The active build's `#reconcileServerState` hook drains `#pendingBuildRequest` on its own.
RandomByte
force-pushed
the
refactor/serve-cache-validation
branch
from
fa7a612 to
5e57496
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main change:
refactor(project): Validate dependency caches in background after each buildAfter server startup (and initial project build), and while no HTTP requests have been received yet, this will initialize and validate the ProjectBuildCache instances of all projects in the dependency tree. The user will see this activity as a new
VALIDATINGserver state in the interactive console. The resulting state will be either "stale" or "ready", depending on whether the restored cache is "valid" (i.e. can be used without a build) or not.The other commits are cleanups on top of this: consolidation of BuildServer state transitions, collapse of the two cache-validation entry points on
ProjectBuildContext, deduplication ofProjectBuilder.#build/#validate, a guard against a concurrent build claim in#triggerRequestQueue, plus a matching pair of logger extractions and test-scaffolding helpers.