Add actionlint checks for GitHub Actions - #16316
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Example of what a failing run looks like: https://github.com/python/typeshed/actions/runs/33195282726/job/98930729363?pr=16316 |
This comment has been minimized.
This comment has been minimized.
This reverts commit 0a36c86.
This comment has been minimized.
This comment has been minimized.
6611c6c to
97378d8
Compare
This comment has been minimized.
This comment has been minimized.
This reverts commit 0a36c86.
97378d8 to
fc05e21
Compare
|
(I updated the PR description with details of exactly what's being fixed here) |
This comment has been minimized.
This comment has been minimized.
| PACKAGE_ARGS=() | ||
| while IFS= read -r package; do | ||
| PACKAGE_ARGS+=("$package") | ||
| done <<< "$PACKAGES" |
There was a problem hiding this comment.
Why do we need this? I assume this was flagged due to white space handling. But since package names can not contain white space, I think this only complicates the action unnecessarily and makes it harder to understand and verify its correctness.
(Same pattern a few times below.)
There was a problem hiding this comment.
Yeah, there were a huge number of SC2086 errors reported on our workflows:
Details
.github/workflows/tests.yml:64:9: shellcheck reported issue in this script: SC2086:info:3:46: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:64:9: shellcheck reported issue in this script: SC2086:info:4:54: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:117:9: shellcheck reported issue in this script: SC2086:info:3:64: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:152:9: shellcheck reported issue in this script: SC2086:info:3:64: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:187:9: shellcheck reported issue in this script: SC2086:info:3:46: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:187:9: shellcheck reported issue in this script: SC2086:info:4:54: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:196:9: shellcheck reported issue in this script: SC2086:info:3:51: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:196:9: shellcheck reported issue in this script: SC2086:info:4:66: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/tests.yml:203:9: shellcheck reported issue in this script: SC2086:info:1:26: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:54:9: shellcheck reported issue in this script: SC2086:info:10:24: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:10:55: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:11:23: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:15:57: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:16:24: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:2:63: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:5:50: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:67:9: shellcheck reported issue in this script: SC2086:info:6:58: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/stubtest_third_party.yml:89:9: shellcheck reported issue in this script: SC2086:info:10:72: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/mypy_primer.yml:40:9: shellcheck reported issue in this script: SC2086:info:13:10: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/mypy_primer.yml:40:9: shellcheck reported issue in this script: SC2086:info:13:33: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/mypy_primer.yml:40:9: shellcheck reported issue in this script: SC2086:info:15:18: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/mypy_primer.yml:40:9: shellcheck reported issue in this script: SC2086:info:4:40: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/daily.yml:86:9: shellcheck reported issue in this script: SC2086:info:10:53: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/daily.yml:86:9: shellcheck reported issue in this script: SC2086:info:11:21: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/daily.yml:86:9: shellcheck reported issue in this script: SC2086:info:15:55: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/daily.yml:86:9: shellcheck reported issue in this script: SC2086:info:16:22: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/daily.yml:86:9: shellcheck reported issue in this script: SC2086:info:5:48: Double quote to prevent globbing and word splitting [shellcheck]
.github/workflows/daily.yml:86:9: shellcheck reported issue in this script: SC2086:info:6:56: Double quote to prevent globbing and word splitting [shellcheck]
I'm reverting most of the changes you're talking about; I agree they made things less readable for a purely theoretical problem. Sorry about that.
Co-authored-by: Sebastian Rittau <sebastian.rittau@zfutura.de>
This comment has been minimized.
This comment has been minimized.
| run: | | ||
| PACKAGES=$(python tests/get_external_stub_requirements.py) | ||
| if [ -n "$PACKAGES" ]; then | ||
| uv pip install --python-version ${{ matrix.python-version }} $PACKAGES | ||
| PACKAGE_ARGS=() | ||
| while IFS= read -r package; do | ||
| PACKAGE_ARGS+=("$package") | ||
| done <<< "$PACKAGES" | ||
| uv pip install --python-version ${{ matrix.python-version }} "${PACKAGE_ARGS[@]}" | ||
| fi | ||
| # Published stub packages can shadow the checked-in stubs when ty | ||
| # resolves their relative imports. |
There was a problem hiding this comment.
@srittau -- codex reckons that the shellcheck diagnostic here (and for other places where we use get_external_stub_requirements.py in our workflows) was not a false positive, and I think codex/shellcheck have a point here. Here's codex's reasoning:
get_external_stub_requirements.pyreturns complete requirement specifiers, not just package names. A valid requirement can contain whitespace, such asexample>=1; python_version < "3.14", or characters with special meaning to shell globbing, such as the brackets inexample[extra]. Expanding$PACKAGESwithout quotes would split the first example into multiple arguments and could interpret the second as a filename pattern. Quoting"$PACKAGES"as a whole would not solve this: it would pass the entire list of requirements as one argument.Those steps therefore still read the helper's output into an array, one requirement per line, and pass
"${PACKAGE_ARGS[@]}"to uv. This preserves each requirement as exactly one argument. It protects the requirement syntax the helper supports; it isn't a claim that one of our current requirements is already causing an installation failure.
|
According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉 |
Add actionlint with ShellCheck to catch workflow syntax and shell-script errors, and update the existing workflows to pass the new checks.
The findings in
tests.yml,daily.yml,stubtest_third_party.yml, andmypy_primer.ymlcame from actionlint's ShellCheck integration:$GITHUB_SHA,$MYPY_VERSION,$GITHUB_ENV, and$GITHUB_PATHwere unquoted too. The PR quotes scalar values and passes lists through quoted array expansions. Package requirements are read one line at a time, preserving spaces and wildcard characters within each requirement; the array-building loops also work with macOS's Bash 3.2.printfformat strings; SC2001: unnecessarysedsubstitutions. Package logging usedecho | sedto build an indented list and interpolated it into the format string, where%or backslashes in package names could be interpreted byprintf. The PR prints each array element with the fixed formatprintf ' %s\n', removing theecho | sedpipeline.readwithout-r. The loop collecting changed stubs could interpret backslashes in its input. It now usesIFS= read -rto read each line literally.egrep. The changed-stub filter now usesgrep -Einstead.The pre-commit hook hit pre-commit.ci's 180-second timeout, so actionlint runs in a dedicated GitHub Actions workflow using its official prebuilt Docker image, which includes ShellCheck.
Stacked on #16314.