Add --deps-only flag to separate dependency fetching from source builds#1336
Add --deps-only flag to separate dependency fetching from source builds#1336sbryngelson wants to merge 11 commits intoMFlowCode:masterfrom
Conversation
This allows CI to fetch and build dependencies (FFTW, HDF5, etc.) on login nodes with internet access, then build MFC source code on compute nodes that may have no network connectivity. Key changes: - New --deps-only CLI flag for ./mfc.sh build - Already-configured dependencies are skipped entirely during regular builds, guaranteeing no network access in the source build step - Frontier and Frontier AMD now follow the pattern: deps on login node, source build + test on compute node
📝 WalkthroughWalkthroughThis pull request refactors the CI/CD build workflows to separate dependency fetching from source building. Changes include removing the 🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1336 +/- ##
==========================================
+ Coverage 45.01% 47.42% +2.41%
==========================================
Files 70 70
Lines 20562 19202 -1360
Branches 1962 1634 -328
==========================================
- Hits 9255 9106 -149
+ Misses 10179 9229 -950
+ Partials 1128 867 -261 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
When __build_target recursively called build() to resolve sub-deps (e.g. SILO depends on HDF5), the --deps-only guard intercepted the recursive call and only built deps-of-deps (none for HDF5), never building HDF5 itself. This caused SILO's configure to fail with 'HDF5 was not found'. Fix: only enter the --deps-only path on the top-level call (history is empty), letting recursive calls proceed normally.
|
Claude Code Review Incremental review from: 407974b New findings since last Claude review:
|
Frontier/Frontier AMD now follow the same deps-on-login, source-on-compute pattern for case optimization tests. Previously, prebuild-case-optimization.sh built deps + source on the login node. Now: - Login node: build.sh fetches deps via --deps-only - Compute node: run_case_optimization.sh builds case-optimized binaries then runs them Phoenix is unchanged (prebuild + run both in SLURM).
…se-opt builds Remove -h ipa0, --debug, and -j 1 workarounds for Frontier Cray builds. These did not address the root cause: corrupted intermediate files from prior failed builds persisting on self-hosted CI runners and poisoning subsequent compilations. Instead, clean stale hash-named MFC target staging/install dirs before case-opt builds while preserving dependency dirs (hipfort, fftw, etc.) that cannot be re-fetched without internet.
…d files Add clean: false to benchmark checkout steps, matching the test workflow. This prevents actions/checkout from failing on NFS silly-rename files (.nfs*) left by still-running SLURM processes, and preserves .slurm_job_id files so submit-slurm-job.sh can detect and cancel stale jobs on retry.
Summary
--deps-onlyCLI flag to./mfc.sh buildthat only fetches and builds dependencies (FFTW, HDF5, etc.) without building MFC source targetsTest plan
./mfc.sh build --deps-only -j 8builds only dependency targets./mfc.sh build -j 8skips already-configured deps