ci: remove unnecessary conda env and Azure auth from Style job#2544
ci: remove unnecessary conda env and Azure auth from Style job#2544BrendanWalsh wants to merge 2 commits intomasterfrom
Conversation
The Style job was using AzureCLI@2 for sbt scalastyle (which doesn't need Azure auth — Secrets.scala values are all lazy) and installing the full conda environment (R, torch, horovod, langchain, etc.) just to run black. Replace with direct sbt invocation and pip install of black[jupyter]. Validated in a clean container: both scalastyle and black produce identical results without conda or Azure auth. History: Python style was originally added via pip install (PR #1520, Jun 2022). The conda template was added later during a refactor, not for any functional reason.
|
Hey @BrendanWalsh 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
There was a problem hiding this comment.
Pull request overview
This PR simplifies the Azure DevOps Style job in pipeline.yaml by removing Azure authentication and the full conda environment setup, replacing them with direct bash execution for Scala style checks and a minimal pip-based install for Black.
Changes:
- Replace
AzureCLI@2wrapper with a plainbashstep forsbt scalastyle test:scalastyle. - Remove
templates/conda.ymlfrom the Style job and switch Python formatting topip install black[jupyter]==22.3.0before runningblack.
Show a summary per file
| File | Description |
|---|---|
pipeline.yaml |
Simplifies the Style job by removing AzureCLI + conda env usage and installing Black via pip for Python formatting checks. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 1
| - bash: | | ||
| set -e | ||
| source activate synapseml | ||
| pip install -q "black[jupyter]==22.3.0" | ||
| black --diff --color . && black --check -q . |
There was a problem hiding this comment.
The Style job now installs Black via pip, but unlike other jobs it no longer provisions a known Python runtime (e.g., via UsePythonVersion in templates/update_cli.yml) and also removed the conda environment that previously guaranteed a writable Python env. This can make pip install and/or the black executable resolution fail depending on the hosted agent image.
Consider adding an explicit Python setup step (or creating a venv) before installing/running Black, and optionally add the same retry/timeout pattern used elsewhere in the pipeline for pip installs to reduce transient CI failures.
See below for a potential fix:
- task: UsePythonVersion@0
displayName: 'Use Python 3'
inputs:
versionSpec: '3.10'
- bash: |
set -e
python -m pip install --upgrade pip
for i in 1 2 3; do
python -m pip install -q "black[jupyter]==22.3.0" && break
if [ "$i" -eq 3 ]; then
exit 1
fi
sleep 10
done
python -m black --diff --color . && python -m black --check -q .
Address review feedback — add UsePythonVersion to ensure a consistent Python runtime on the hosted agent.
What changes are proposed in this pull request?
Removes unnecessary dependencies from the CI Style job:
bashstep forsbt scalastyle. The Azure auth is not needed —Secrets.scalavalues are alllazy valand never accessed during scalastyle.pip install black[jupyter]==22.3.0. The only tool needed is black.Validation
Tested in a clean container (
eclipse-temurin:8-jdk-jammy):sbt scalastyle test:scalastyle— ✅ passes without Azure CLI or condablack --check— ✅ identical results (233 .py + 57 .ipynb files) with pip-only installHistory
Python style checking was originally added via
pip install -r requirements.txt(PR #1520, Jun 2022). The conda template was added later during a refactor — not for any functional reason.How is this patch tested?
The PR CI itself validates this change — if the Style job passes, the change works.