Skip to content

Make quality snapshot comments safe for fork PRs #522

Description

Problem

Pull requests from forks receive a read-only GITHUB_TOKEN, even when a pull_request workflow declares pull-requests: write. The performance and coverage workflows currently invoke marocchino/sticky-pull-request-comment before running their substantive work.

PR #521 reproduced the failure across all performance and coverage jobs:

  • the workflow token had PullRequests: read and Secret source: None;
  • the initial sticky-comment step failed with Resource not accessible by integration;
  • normal benchmark/coverage steps were then skipped because the preceding step failed;
  • always() upload/comparison steps ran without metrics.json or lcov.info, producing cascading snapshot failures; and
  • the final sticky-comment step failed with the same permission error.

The regular build, test, lint, and environment jobs passed, confirming that this was a fork-permission failure rather than a problem with the action pins in PR #521.

Security constraint

Fork workflows must remain untrusted and read-only:

  • do not grant fork pull requests write access or repository secrets;
  • do not use pull_request_target to execute or otherwise consume untrusted pull-request code; and
  • do not weaken the fail-closed behavior of the actual performance and coverage gates.

Tasks

  • Detect fork pull requests before invoking PR-comment-writing actions.
  • Skip both in-progress and final sticky-comment steps when the token cannot write, while allowing the benchmark/coverage work and snapshot comparisons to continue.
  • Ensure comment publishing itself is non-gating: a comment API failure must not suppress or replace the underlying quality-gate result.
  • Preserve sticky performance and coverage comments for same-repository pull requests.
  • Keep reports available to fork contributors through job summaries and uploaded artifacts.
  • Audit other pull_request workflows for write operations with the same fork-token assumption.

Acceptance criteria

  • A fork PR runs the complete performance and coverage jobs using only a read-only token.
  • Fork runs do not attempt to create or update PR comments and do not report Resource not accessible by integration.
  • Performance or coverage regressions still fail their jobs for both fork and same-repository PRs.
  • Missing or malformed benchmark/coverage output still fails closed.
  • Same-repository PRs continue receiving sticky snapshot comments.
  • Fork reports remain inspectable in the GitHub step summary and artifacts.
  • The solution does not use pull_request_target, expose secrets, or grant write permissions to untrusted fork code.

Reproduction

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIssue identified by VS Code Team member as probable bugneeds PR

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions