Skip to content

feat: add storage-backed build cache support#233

Merged
TorstenDittmann merged 11 commits into
mainfrom
feat/node-modules-cache
Jun 9, 2026
Merged

feat: add storage-backed build cache support#233
TorstenDittmann merged 11 commits into
mainfrom
feat/node-modules-cache

Conversation

@TorstenDittmann

@TorstenDittmann TorstenDittmann commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add optional cacheKey for opt-in build cache reuse on POST /v1/runtimes
  • validate cacheKey with a safe path-friendly format and reject invalid keys
  • restore and store build cache artifacts through Utopia Storage instead of Docker volumes
  • use a single SquashFS package-store artifact at /build-cache/<cacheKey>/lz4-b1M/stores.sqfs
  • mount a temporary executor-owned /cache directory into build containers for artifact restore/save
  • delegate cache restore/save inside runtimes to /usr/local/server/helpers/build-cache.sh
  • cache package-manager stores for npm, Yarn, pnpm, and Bun instead of restoring node_modules directly
  • document cacheKey and OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE
  • add E2E coverage for cache save/hit behavior and executable build output

Runtime Dependency

  • cached builds require runtime images that include /usr/local/server/helpers/build-cache.sh, mksquashfs, and unsquashfs
  • this is provided by open-runtimes/open-runtimes#497
  • verified with released openruntimes/node:v5-18.0 and openruntimes/node:v5-22 images

Testing

  • php -l app/controllers.php
  • php -l app/http.php
  • php -l src/Executor/Runner/Adapter.php
  • php -l src/Executor/Runner/Docker.php
  • php -l tests/e2e/ExecutorTest.php
  • composer format:check
  • composer refactor:check
  • composer analyze
  • composer validate --strict
  • git diff --check
  • focused build cache E2E against released runtime image: OK (1 test, 8 assertions)

Notes

  • cache invalidation is caller-managed through cacheKey
  • OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE falls back to OPR_EXECUTOR_CONNECTION_STORAGE when empty
  • old Docker volume cache env vars are removed: OPR_EXECUTOR_BUILD_CACHE_VOLUME and OPR_EXECUTOR_BUILD_CACHE_HELPER_IMAGE

@greptile-apps

greptile-apps Bot commented Jun 1, 2026

Copy link
Copy Markdown

Greptile Summary

This PR introduces opt-in build cache support for POST /v1/runtimes via a new cacheKey parameter. Cache artifacts are stored as SquashFS files through Utopia Storage (with a separate OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE DSN that falls back to the main storage), and the actual cache save/restore is delegated to a build-cache.sh helper that lives inside the runtime container.

  • cacheKey validation (regex + Text(128,0)) is sound; the allowed character set prevents path traversal in the storage path construction.
  • restoreBuildCacheArtifact and storeBuildCacheArtifact are clean; directory creation, existence checks, and transfer calls follow the existing pattern for source/build artifacts.
  • The build-cache.sh helper path is v5-specific; enabling cache for v2 builds (not currently guarded) will cause the build to fail because the script does not exist in v2 images.

Confidence Score: 4/5

Safe to merge for v5-only deployments; v2 builds with a non-empty cacheKey will fail at runtime.

The cache logic is well-structured and the cacheKey validation is correct. The one current defect is that cacheEnabled does not exclude v2 builds, so callers who pass a cacheKey with a v2 image will see their build fail — the helper script is v5-only and the guard is a one-line fix.

src/Executor/Runner/Docker.php — specifically the cacheEnabled condition on line 322

Important Files Changed

Filename Overview
src/Executor/Runner/Docker.php Core cache implementation: adds restoreBuildCacheArtifact, storeBuildCacheArtifact, prepareBuildCacheScripts, and getBuildCacheDevice helpers; cacheEnabled guard does not exclude v2 builds, which will fail at runtime because build-cache.sh does not exist in v2 images
app/controllers.php Adds cacheKey param with Text(128,0) validation and a regex guard; param order and action signature are consistent
src/Executor/Runner/Adapter.php Adds optional cacheKey parameter to the abstract createRuntime signature with a safe default
tests/e2e/ExecutorTest.php Adds invalid-cacheKey rejection test and a full save/hit E2E test; coverage is reasonable for the happy path
README.md Documents cacheKey param and OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE env var
.env Adds empty OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE placeholder
docker-compose.yml Passes OPR_EXECUTOR_CONNECTION_BUILD_CACHE_STORAGE through to the executor service
.gitignore Adds .DS_Store to gitignore

Reviews (11): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile

Comment thread src/Executor/Runner/Docker.php Outdated
Comment thread src/Executor/Runner/Docker.php Outdated
Comment thread src/Executor/Runner/Docker.php Outdated
Comment thread src/Executor/Runner/Docker.php Outdated
@TorstenDittmann TorstenDittmann changed the title feat: add node_modules build cache feat: add build cache support Jun 1, 2026
@TorstenDittmann TorstenDittmann changed the title feat: add build cache support feat: add storage-backed build cache support Jun 9, 2026
Comment thread src/Executor/Runner/Docker.php
@TorstenDittmann TorstenDittmann merged commit c3ce886 into main Jun 9, 2026
12 of 13 checks passed
@TorstenDittmann TorstenDittmann deleted the feat/node-modules-cache branch June 9, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant