Skip to content

add cypress-flake-fixer skill - #4870

Open
qlqlmi wants to merge 1 commit into
github:mainfrom
qlqlmi:add-cypress-flake-fixer
Open

qlqlmi wants to merge 1 commit into
github:mainfrom
qlqlmi:add-cypress-flake-fixer

Conversation

@qlqlmi

@qlqlmi qlqlmi commented Oct 10, 2026

Copy link
Copy Markdown

Pull Request Checklist

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have read and followed the Guidance for submissions involving paid services.
  • My contribution adds a new instruction, prompt, agent, skill, workflow, or canvas extension file in the correct directory.
  • The file follows the required naming convention.
  • The content is clearly structured and follows the example format.
  • I have tested my instructions, prompt, agent, skill, workflow, or canvas extension with GitHub Copilot.
  • I have run npm start and verified that README.md is up to date.
  • I am targeting the main branch for this pull request.

Description

Adding cypress-flake-fixer, a skill for auditing and repairing an existing Cypress test suite.

What it does

  • Scans specs, support files and config for patterns that commonly cause flakiness in Cypress test suites, then guides the fix in a set order: broken command flow first, then time-based waits, retry-ability, force: true, state leaks and selectors.
  • Requires a measured baseline before changing anything, and a matching number of clean runs afterwards.
  • Rules out the typical workarounds used on Cypress tests: longer timeouts, added retries, force: true, or a larger cy.wait().

What it adds beyond default model behavior

  • Removed APIs. Most Cypress code a model has seen is from versions 9 to 13. Cypress 16 removed Cypress.env(), cy.exec() and .end(), and turned the cookie and storage getters into queries. The skill covers each removal from Cypress 12 to 16 with replacement code.
  • Bundled scanner. scripts/scan-cypress.mjs is a zero-dependency Node script with 16 rules. It reads the installed Cypress version from package.json and grades each removed API as already failing or failing on upgrade. It supports JSON output, a --fail-on threshold and inline suppression comments.
  • Non-obvious flakiness causes. The gotchas cover clicks hitting before hydration in server-rendered apps, .should('not.exist') passing before the element renders, intercepts registered after the request has fired, aliasing GraphQL operations, and the keystrokeDelay default changing to 0 in Cypress 16.

Contents

  • SKILL.md: workflow, fix order, gotchas, report format
  • references/anti-patterns.md: before/after code for every scanner rule
  • references/removed-apis.md: removals by Cypress major, with migration code
  • scripts/scan-cypress.mjs: the scanner (--help, --list-rules)

Testing

  • npm run skill:validate and npm run build pass.
  • Used with GitHub Copilot (standalone app and in VSCode) against real Cypress projects and it correctly detected anti-patterns and suggested relevant fixes.

Type of Contribution

  • New instruction file.
  • New prompt file.
  • New agent file.
  • New plugin.
  • New skill file.
  • New agentic workflow.
  • New canvas extension.
  • Update to existing instruction, prompt, agent, plugin, skill, workflow, or canvas extension.
  • Other (please specify):

By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

🚦 Submission status: 👀 Ready for review

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 1 approval from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier
  • skills/cypress-flake-fixer/scripts/scan-cypress.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
  • Spawns processes or evaluates code in skills/cypress-flake-fixer/references/removed-apis.md

Automated checks

Check Status Details
Line endings ✅ Passed Passed · logs
Spelling ✅ Passed Passed · logs
Generated README consistency ✅ Passed Passed · logs
Skill validation ✅ Passed Passed · logs
Skill lint (vally) ✅ Passed Passed · logs
Risk scan ✅ Passed Passed · logs
Contributor reputation ✅ Passed Passed · logs
Duplicate resource scan ✅ Passed Passed · logs
PR quality signal ⏭️ Skipped Skipped by its workflow · logs

Review

  • Approvals: 0/1
  • Assigned reviewer: not assigned yet — comment /request-review to ask for one
  • Review target date: not set
  • Still needed: 1 more approval(s); an approval from a maintainer with admin or maintain permission
  • The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command Who What it does
/rerun-checks PR author, maintainers Re-runs failed or incomplete checks and re-evaluates this gate
/request-review PR author, maintainers Asks the review rotation to assign a reviewer (adds needs-reviewer)

Updated for 638657f · gate run · This comment is maintained automatically — see submission gate docs.

@github-actions

Copy link
Copy Markdown
Contributor

🔒 PR Risk Scan Results

Scanned 4 changed file(s).

Severity Count
🔴 High 0
🟠 Medium 2
ℹ️ Info 1
Severity Rule File Line Match
🟠 package-exec-command skills/cypress-flake-fixer/SKILL.md 45 for i in $(seq 1 20); do npx cypress run --spec "cypress/e2e/checkout.cy.ts" --quiet || echo "FAIL $i"; done
🟠 package-exec-command skills/cypress-flake-fixer/SKILL.md 74 | Unknown | Run the test in `npx cypress open`, select the wait in the Command Log, and read which requests and DOM changes happened during it |
ℹ️ skill-script-touched skills/cypress-flake-fixer/scripts/scan-cypress.mjs 1 skills/cypress-flake-fixer/scripts/scan-cypress.mjs

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

@github-actions github-actions Bot added new-submission PR adds at least one new contribution skills PR touches skills labels Oct 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Vally Lint Results

✅ All checks passed

Scope Checked
Skills 1
Agents 0
Total 1
Severity Count
❌ Errors 0
⚠️ Warnings 0
ℹ️ Advisories 0

Summary

Level Finding
ℹ️ ✅ cypress-flake-fixer (2/2 checks passed)
ℹ️ ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
ℹ️ ✓ spec-compliance: All spec checks passed.
ℹ️ ✓ [valid-refs] All file references across 1 skill(s) are valid.
ℹ️ ✓ valid-refs: All file references resolve to existing files within the skill directory.
ℹ️ 1 skill(s) linted, 1 passed
Full linter output
### Linting skills/cypress-flake-fixer
✅ cypress-flake-fixer (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

@github-actions github-actions Bot added ready-for-review Submission passed intake validation and is ready for maintainer review and removed awaiting-automation labels Oct 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk:high new-submission PR adds at least one new contribution ready-for-review Submission passed intake validation and is ready for maintainer review skills PR touches skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant