Skip to content

ci(cmd-bot): post PR comments via the App token, not GITHUB_TOKEN#246

Merged
ilchu merged 2 commits into
devfrom
ic/cmd-bot-app-token-comments
Jun 29, 2026
Merged

ci(cmd-bot): post PR comments via the App token, not GITHUB_TOKEN#246
ilchu merged 2 commits into
devfrom
ic/cmd-bot-app-token-comments

Conversation

@ilchu

@ilchu ilchu commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Problem

Every non---quiet /cmd run currently dies at its first step. The cmd-run.yml jobs comment/react on the PR using secrets.GITHUB_TOKEN, and that token gets HTTP 403 "Resource not accessible by integration" on POST /repos/.../issues/{n}/comments — even though the repo's Actions workflow-permissions setting reads write. So before-cmd → Comment PR (Start) fails, the cmd (e.g. bench) step is skipped, and the only visible signal is the 👀 acknowledgement reaction (the failure comment 403s too).

Evidence: the same workflow commented successfully on a PR as recently as Jun 13 (run 27460868652, PR #172), then began 403ing (Jun 17, Jun 26). The repo token setting is write, yet runs still 403 — i.e. the cap is at the org level (a policy override the repo setting can't relax), applied in that window.

Fix

Route the comment + reaction steps in before-cmd, after-cmd, and finish through the CMD_BOT GitHub App token (scoped to issues + pull-requests: write) instead of GITHUB_TOKEN. This is the same App token already used for the result push in after-cmd, which works precisely because an App token's scope comes from the App installation and isn't subject to the org GITHUB_TOKEN cap. The read-only "Build workflow link" curl keeps using GITHUB_TOKEN.

Prerequisites / notes

  • The CMD_BOT App installation must grant issues + pull-requests: write (it already grants contents: write for the push). If it doesn't, the new create-github-app-token steps will fail at token generation and need that scope added in the App settings.
  • /cmd dispatches cmd-run.yml from the cmd-bot branch (gh workflow run --ref cmd-bot), so this change only takes effect for live runs once it reaches cmd-bot via the usual devcmd-bot sync.
  • Workflow changes can't be exercised pre-merge (the dispatch input contract requires the default-branch/cmd-bot copy), so this hasn't been run end-to-end.

The /cmd dispatcher's before-cmd / after-cmd / finish jobs comment and react on
the PR using `secrets.GITHUB_TOKEN`. That token is capped read-only by org
policy — the repo's workflow-permissions setting reads "write", yet runs still
get HTTP 403 "Resource not accessible by integration" on
`POST /issues/{n}/comments`. So every non-quiet /cmd run dies at the first step
("Comment PR (Start)") before the command itself runs, and only the 👀 reaction
ever appears (the failure comment 403s too).

Route the comment + reaction steps through the CMD_BOT GitHub App token (scoped
to issues + pull-requests write) — the same App token already used for the push
in after-cmd, which works precisely because it is not subject to the org
GITHUB_TOKEN cap. The read-only "Build workflow link" curl keeps GITHUB_TOKEN.

Prerequisite: the CMD_BOT App installation must grant issues + pull-requests
write (it already grants contents write for the push). To take effect for live
/cmd runs this must also reach the `cmd-bot` branch the dispatch runs from
(via the usual dev -> cmd-bot sync).
@ilchu ilchu self-assigned this Jun 26, 2026

@danielbui12 danielbui12 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For cmd-run.yml changes, you can manual verify by below cli, then post result!

  gh workflow run cmd-run.yml \
            --ref <YOUR_BRANCH_NAME> \
            -f cmd="${CMD}" \
            -f repo="${REPO}" \
            -f pr_branch="${PR_BRANCH}" \
            -f pr_num="${PR_NUMBER}" \
            -f runner="${RUNNER}" \
            -f is_org_member="${IS_ORG_MEMBER}" \
            -f comment_id="${COMMENT_ID}" \
            -f image="${IMAGE}" \
            -f is_quiet="${IS_QUIET}"

Reminder: Also merge cmd-run related changes to cmd-bot branch after this PR merged.

@ilchu

ilchu commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Manually verified by dispatching this branch's cmd-run.yml per the suggested CLI (gh workflow run cmd-run.yml --ref ic/cmd-bot-app-token-comments … -f is_quiet=false): run 28356077303.

All App-token comment/reaction steps pass (these 403'd on GITHUB_TOKEN before):

  • before-cmd: Generate App token ✅ → Comment PR (Start) ✅
  • finish: Generate App token ✅ → Comment PR (Failure) ✅ → 😕 reaction ✅

The bot posted the start/failure comments on this PR (throwaway test comments, since deleted). The cmd job failure is expected — I passed an intentionally-invalid command (verify-app-token-fix) so it fails fast without running a bench or pushing; only the comment/reaction steps (what this PR changes) matter, and they all pass. App-token generation succeeding also confirms the CMD_BOT App already grants issues + pull-requests write, so no App-installation change is needed.

Reminder per review: this also needs mirroring to the cmd-bot branch after merge, since /cmd dispatches cmd-run.yml from there.

@ilchu ilchu added this pull request to the merge queue Jun 29, 2026
Merged via the queue into dev with commit eb3d541 Jun 29, 2026
38 checks passed
@ilchu ilchu deleted the ic/cmd-bot-app-token-comments branch June 29, 2026 09:38
@bkontur

bkontur commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Why 😕 here: #125 (comment)?

image

:D :D could be related to this PR?

@danielbui12

Copy link
Copy Markdown
Member

@bkontur it is :D https://github.com/paritytech/web3-storage/actions/runs/28356077303/job/83999454192
Screenshot 2026-06-29 at 17 14 31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants