From 638657fd064d3d057f642273a95df1faa35f7c42 Mon Sep 17 00:00:00 2001 From: t3rry-davis Date: Sat, 10 Oct 2026 16:07:37 +0200 Subject: [PATCH] add cypress-flake-fixer skill --- docs/README.skills.md | 1 + skills/cypress-flake-fixer/SKILL.md | 154 ++++++ .../references/anti-patterns.md | 241 +++++++++ .../references/removed-apis.md | 151 ++++++ .../scripts/scan-cypress.mjs | 506 ++++++++++++++++++ 5 files changed, 1053 insertions(+) create mode 100644 skills/cypress-flake-fixer/SKILL.md create mode 100644 skills/cypress-flake-fixer/references/anti-patterns.md create mode 100644 skills/cypress-flake-fixer/references/removed-apis.md create mode 100755 skills/cypress-flake-fixer/scripts/scan-cypress.mjs diff --git a/docs/README.skills.md b/docs/README.skills.md index 2e3530a45e..c09597d684 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -148,6 +148,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [csharp-nunit](../skills/csharp-nunit/SKILL.md)
`gh skills install github/awesome-copilot csharp-nunit` | Get best practices for NUnit unit testing, including data-driven tests | None | | [csharp-tunit](../skills/csharp-tunit/SKILL.md)
`gh skills install github/awesome-copilot csharp-tunit` | Get best practices for TUnit unit testing, including data-driven tests | None | | [csharp-xunit](../skills/csharp-xunit/SKILL.md)
`gh skills install github/awesome-copilot csharp-xunit` | Get best practices for XUnit unit testing, including data-driven tests | None | +| [cypress-flake-fixer](../skills/cypress-flake-fixer/SKILL.md)
`gh skills install github/awesome-copilot cypress-flake-fixer` | Find and fix flake-prone anti-patterns in an existing Cypress suite, and APIs removed in Cypress 12 to 16. Bundles a zero-dependency scanner for fixed cy.wait() delays, assigned command return values, async/await, force: true, chaining after actions, one-shot expect() inside .then(), conditional testing, and removed APIs such as Cypress.env(), cy.exec(), cy.server()/cy.route(). Use when asked to fix flaky Cypress tests, remove cy.wait, stabilize or harden a Cypress suite, audit Cypress specs, debug "detached from the DOM" errors, or prepare an upgrade to Cypress 15 or 16. | `references/anti-patterns.md`
`references/removed-apis.md`
`scripts/scan-cypress.mjs` | | [d365-solution-blueprint](../skills/d365-solution-blueprint/SKILL.md)
`gh skills install github/awesome-copilot d365-solution-blueprint` | Authors a Dynamics 365 Finance and Supply Chain Management Solution Blueprint from scratch through a structured, section-by-section architect interview, establishing scope, target operating model, application and data architecture, integration landscape, migration strategy, security model, ALM, testing, deployment, and support approach, with a decision log capturing rationale and rejected alternatives. Use when the user wants to create D365 implementation architecture documentation, start a D365 implementation, design the architecture, prepare a Solution Blueprint, or identify the architectural decisions the programme must make. Do not use for critique of an existing design; that is a review task rather than blueprint authoring. | `assets/blueprint-template.md`
`references/section-guide.md` | | [daily-focus-board](../skills/daily-focus-board/SKILL.md)
`gh skills install github/awesome-copilot daily-focus-board` | Spin up a personal, motivating daily focus board that renders in a browser canvas and that the user drives by talking to their AI partner. Tasks track status (to-do → in progress → done) with timestamped progress notes and roll up into a "today's momentum" feed; numeric-goal tasks (pages, pomodoros, reps) render as progress-bar counters. Executive-function / neurodivergent-friendly by design: Focus mode, kind "not today" carryover (no overdue-shaming), a brain-dump box, reduced-motion, and gentle deadline countdowns. Add, reorder, and relabel tasks live, assign Eisenhower priority (Do first / Schedule / Delegate / Later), open with an above/below-the-line check-in and a daily mantra, and save an end-of-day recap. Use when someone wants to plan their day, stay focused, kick off a work session, or track progress. Progress persists in the browser (localStorage). | `assets/board.template.html`
`examples/sample-board.html`
`references/customize.md`
`references/neurodivergent-design.md`
`references/tutorial.md`
`scripts/serve-board.ps1` | | [daily-prep](../skills/daily-prep/SKILL.md)
`gh skills install github/awesome-copilot daily-prep` | Prepare for tomorrow's meetings and tasks. Pulls calendar from Outlook via WorkIQ, cross-references open tasks and workspace context, classifies meetings, detects conflicts and day-fit issues, finds learning and deep-work slots, and generates a structured HTML prep file with productivity recommendations. | None | diff --git a/skills/cypress-flake-fixer/SKILL.md b/skills/cypress-flake-fixer/SKILL.md new file mode 100644 index 0000000000..bfd12cae6f --- /dev/null +++ b/skills/cypress-flake-fixer/SKILL.md @@ -0,0 +1,154 @@ +--- +name: cypress-flake-fixer +description: 'Find and fix flake-prone anti-patterns in an existing Cypress suite, and APIs removed in Cypress 12 to 16. Bundles a zero-dependency scanner for fixed cy.wait() delays, assigned command return values, async/await, force: true, chaining after actions, one-shot expect() inside .then(), conditional testing, and removed APIs such as Cypress.env(), cy.exec(), cy.server()/cy.route(). Use when asked to fix flaky Cypress tests, remove cy.wait, stabilize or harden a Cypress suite, audit Cypress specs, debug "detached from the DOM" errors, or prepare an upgrade to Cypress 15 or 16.' +--- + +# Cypress Flake Fixer + +Audit an existing Cypress suite for the code patterns that cause intermittent failures, then fix them without hiding the problem behind retries, longer timeouts, or `force: true`. + +## When to Use This Skill + +- A Cypress spec passes locally and fails in CI, or fails once in every few runs +- The suite contains `cy.wait()`, `{ force: true }`, or tests that only pass in order +- Errors mention "detached from the DOM", "timed out retrying", or a subject that "is no longer attached" +- The project is upgrading Cypress across a major version (12 through 16) +- A review of Cypress test code is requested + +For writing new Cypress tests from scratch, coding standards are enough. This skill is for diagnosing and repairing an existing suite. + +## Prerequisites + +- Node.js 18 or later to run the scanner (it has no dependencies and does not need Cypress installed) +- The Cypress version, read from `package.json`. Fixes differ by major version + +## Workflow + +### 1. Scan + +```bash +node ./scripts/scan-cypress.mjs +node ./scripts/scan-cypress.mjs --json # machine-readable +node ./scripts/scan-cypress.mjs --strict # add low-confidence rules +node ./scripts/scan-cypress.mjs cypress/e2e/checkout.cy.ts # one file +``` + +The scanner reads the Cypress version from the nearest `package.json` and grades removed APIs as `error` when the installed major already rejects them and `warn` when they break on upgrade. Run `--list-rules` for the catalog. + +Findings are heuristic. Open each location and confirm before editing. + +### 2. Measure before changing anything + +When the goal is to fix a specific flaky spec, record how often it fails first. Without a baseline there is no way to tell a fix from luck. + +```bash +for i in $(seq 1 20); do npx cypress run --spec "cypress/e2e/checkout.cy.ts" --quiet || echo "FAIL $i"; done +``` + +A spec that fails 1 run in 10 needs about 30 clean runs after the fix before the fix is believable. + +### 3. Fix in this order + +| Order | Rules | Why first | +|---|---|---| +| 1 | CY002, CY003, CY007 graded `error` | The test is wrong or already broken; other fixes build on correct command flow | +| 2 | CY001, intercept ordering | Time-based waits are the largest single source of flake | +| 3 | CY005, CY006 | Broken retry-ability causes detached-DOM and one-shot assertion failures | +| 4 | CY004 | `force: true` hides a real overlap, animation, or disabled state | +| 5 | CY008, CY010, CY012 | State leaks and order dependence | +| 6 | CY013, CY101, CY102 | Selector and assertion hygiene | + +Before-and-after code for every rule is in [references/anti-patterns.md](./references/anti-patterns.md). Replacements for removed APIs are in [references/removed-apis.md](./references/removed-apis.md). Read the section for the rule being fixed; do not load both files upfront. + +### 4. Replace each fixed wait with the condition it was standing in for + +Do not delete a `cy.wait(n)` or swap it for a guess. Work out what the delay covered: + +| The wait covered | Replace with | +|---|---| +| A network request | `cy.intercept(...).as('x')` registered before the trigger, then `cy.wait('@x')` | +| A render that follows a request or state change | A `.should()` on the DOM that results | +| A debounce, throttle, polling interval, or toast timeout | `cy.clock()` before `cy.visit()`, then `cy.tick(ms)` | +| A CSS animation or transition | An assertion on the final state; Cypress already waits for the element to stop moving before acting | +| Client-side hydration of server-rendered HTML | A signal the app sets when hydration finishes (see Gotchas) | +| Unknown | Run the test in `npx cypress open`, select the wait in the Command Log, and read which requests and DOM changes happened during it | + +### 5. Verify + +1. Run the changed spec alone, then repeat it at least as many times as the baseline run. +2. Rerun the scanner and compare counts per rule. +3. Run the full suite once to catch order dependence exposed by the change. + +## Rules for the fix itself + +- Change test code only. If the test exposes an application bug (double submit, missing loading state, race in the UI), report it instead of working around it. +- Do not raise `defaultCommandTimeout`, add `retries`, or add `{ force: true }` to make a test pass. Each one converts a visible failure into a slower or hidden one. +- Do not replace `cy.wait(2000)` with `cy.wait(5000)`, or with `.should('exist')` on an element that was already present. +- Keep one rule category per commit so a regression can be bisected. +- Fix shared custom commands in `cypress/support` first. A fixed wait inside a command runs in every test that calls it. + +## Gotchas + +- **Register `cy.intercept()` before the request can fire.** For requests made on page load that means before `cy.visit()`. An intercept registered after the request has left never matches, and `cy.wait('@alias')` times out intermittently depending on network speed. +- **Each `cy.wait('@alias')` consumes one request.** If the app calls the endpoint twice, the first wait resolves on the first call and the assertion runs against stale data. Wait once per expected call, or assert on the DOM. +- **GraphQL uses one URL for every operation.** Alias inside the handler: `cy.intercept('POST', '/graphql', (req) => { if (req.body.operationName === 'GetOrders') req.alias = 'getOrders' })`. +- **`.should('not.exist')` passes before the element has rendered.** Using it to wait for a spinner to disappear succeeds instantly on a slow page. Assert the positive end state. +- **Server-rendered apps accept clicks before handlers are attached.** In Next.js, Nuxt, SvelteKit, and Remix the button is visible and enabled in the server HTML, so Cypress clicks it during hydration and nothing happens. Have the app set a marker when hydration completes (for example a `data-hydrated` attribute on `` from a mount effect) and assert on it after `cy.visit()`. +- **A `.should()` in the middle of a chain pins the subject.** Queries after it cannot re-query past the assertion, so a re-render produces a detached-DOM error. Split the chain at the assertion. +- **`.then()` breaks the retry chain even with no assertion inside.** Everything before it runs once. +- **Cypress 16 types with no delay between keys** (`keystrokeDelay` default changed from 10 to 0). Inputs that re-render per keystroke or debounce on input may now drop characters or fire once. Assert the field value after typing instead of restoring the delay. +- **Cypress 16 turned cookie and storage getters into queries.** `cy.getCookie()`, `cy.getCookies()`, `cy.getAllCookies()`, `cy.getAllLocalStorage()`, and `cy.getAllSessionStorage()` now retry chained `.should()` assertions, which removes the need for waits around auth cookies. +- **The page is blank after `cy.session()`.** A missing `cy.visit()` after it looks like a flaky selector failure. +- **`testIsolation: false` shares state across tests.** If the config sets it, every order-dependent failure is expected behavior, not flake. Check the config before diagnosing. +- **Retries hide flake unless configured to report it.** To find flaky tests instead of masking them, use the experimental strategy that retries and still fails: + + ```js + retries: { + experimentalStrategy: 'detect-flake-but-always-fail', + experimentalOptions: { maxRetries: 2, stopIfAnyPassed: true }, + openMode: true, + runMode: true, + } + ``` + +- **CY005 is a judgement call.** `cy.get(input).clear().type('x')` on an input that never re-renders is fine. Split the chain when the element belongs to a list, a form library that remounts fields, or anything driven by async state. + +## Report format + +```markdown +## Cypress flake audit: + +- Cypress version: , major +- Files scanned: +- Baseline: failed of runs + +| Rule | Before | After | Notes | +|---|---|---|---| +| CY001 fixed-wait | 14 | 0 | 9 replaced by intercept aliases, 5 by assertions | + +### Not fixed +- : + +### Application issues found +- + +### Verification +- : of runs passed after the change +``` + +## Troubleshooting + +| Issue | Solution | +|---|---| +| Scanner reports "No Cypress files found" | Pass the folder that contains `cypress/` or `*.cy.*` files. Specs outside those conventions can be passed as explicit file paths | +| A finding is a false positive | Add `// cy-scan-ignore CY0xx` on the same or previous line, with the reason | +| Version shows "not detected" | Pass `--cypress-major `; in a monorepo, run the scanner from the package that depends on Cypress | +| The test fails only in CI after fixes | Compare viewport, browser, and CPU. Run locally with the CI browser and `--browser chrome --headless`, and throttle CPU to surface timing gaps | +| `cy.wait('@alias')` times out after the change | The intercept is registered after the request, the URL pattern misses a query string (add `*`), or the request is served from cache and never reaches the network | + +## References + +- [references/anti-patterns.md](./references/anti-patterns.md): before-and-after code for every scanner rule +- [references/removed-apis.md](./references/removed-apis.md): removed APIs by Cypress major, with migration code for `Cypress.env()` and `cy.exec()` +- [Cypress retry-ability](https://docs.cypress.io/app/core-concepts/retry-ability) +- [Cypress migration guide](https://docs.cypress.io/app/references/migration-guide) diff --git a/skills/cypress-flake-fixer/references/anti-patterns.md b/skills/cypress-flake-fixer/references/anti-patterns.md new file mode 100644 index 0000000000..a0195f6e02 --- /dev/null +++ b/skills/cypress-flake-fixer/references/anti-patterns.md @@ -0,0 +1,241 @@ +# Cypress anti-patterns: before and after + +One section per scanner rule. Each shows the failing shape, why it flakes, and the replacement. + +## CY001 fixed-wait + +```js +// Before +cy.get('[data-cy=save]').click() +cy.wait(3000) +cy.get('[data-cy=toast]').should('contain', 'Saved') + +// After: the wait was covering a request +cy.intercept('PUT', '/api/profile').as('saveProfile') +cy.get('[data-cy=save]').click() +cy.wait('@saveProfile').its('response.statusCode').should('eq', 200) +cy.get('[data-cy=toast]').should('contain', 'Saved') +``` + +The delay is either too short on a slow machine (flake) or too long everywhere (slow suite). When the wait covers a timer in the app, fake the clock: + +```js +cy.clock() +cy.visit('/search') +cy.get('[data-cy=query]').type('shoes') +cy.tick(300) // the debounce interval +cy.get('[data-cy=result]').should('have.length.greaterThan', 0) +``` + +`cy.clock()` must run before `cy.visit()` to capture timers the app creates while loading. + +## CY002 assigned-return-value + +```js +// Before +const row = cy.get('[data-cy=row]').first() +row.click() +row.should('have.class', 'selected') + +// After +cy.get('[data-cy=row]').first().as('row') +cy.get('@row').click() +cy.get('@row').should('have.class', 'selected') +``` + +A command returns a chainable, not an element. Reusing the variable chains new commands onto a subject captured once. An alias re-runs the query each time it is read. + +To use a value later in the test, nest or alias the value: + +```js +cy.get('[data-cy=order-id]').invoke('text').as('orderId') +cy.get('@orderId').then((orderId) => { + cy.visit(`/orders/${orderId}`) +}) +``` + +`cy.stub()` and `cy.spy()` are synchronous and may be assigned. + +## CY003 async-await + +```js +// Before +it('loads the user', async () => { + const user = await cy.request('/api/me') + expect(user.body.name).to.eq('Ada') +}) + +// After +it('loads the user', () => { + cy.request('/api/me').its('body.name').should('eq', 'Ada') +}) +``` + +Commands are not promises. `await` resolves at the wrong time or never, and an `async` test function returns a promise that Cypress races against its own queue. Awaiting a real promise inside `cy.then()` or `cy.wrap(promise)` is fine. + +## CY004 force-true + +```js +// Before +cy.get('[data-cy=submit]').click({ force: true }) + +// After: the button was covered by a cookie banner +cy.get('[data-cy=cookie-accept]').click() +cy.get('[data-cy=cookie-banner]').should('not.exist') +cy.get('[data-cy=submit]').click() +``` + +Read the actionability error Cypress printed before `force` was added. It names the cause: + +| Error says | Usual cause | Fix | +|---|---|---| +| is being covered by another element | Overlay, sticky header, banner, toast | Dismiss it, or scroll with `scrollIntoView()` and assert position | +| is not visible | Collapsed menu, `display: none`, zero size | Open the parent first | +| is disabled | Form not yet valid, request in flight | Assert `.should('be.enabled')` before acting | +| is animating | Transition still running | Assert the final state; do not disable the check globally | + +`force: true` is legitimate for hidden native inputs behind a styled control, such as a custom file input or checkbox. Leave those and add a comment. + +## CY005 chain-after-action + +```js +// Before +cy.get('[data-cy=todo]').first().click().find('[data-cy=delete]').click() + +// After +cy.get('[data-cy=todo]').first().click() +cy.get('[data-cy=todo]').first().find('[data-cy=delete]').click() +``` + +An action runs once. If it triggers a re-render, the subject passed down the chain is the old element, and the next command fails with a detached-DOM error or acts on a node that is no longer on screen. A new `cy.get()` re-queries. + +Chaining is fine when the element cannot be replaced: `cy.get(input).focus().clear().type('x')` on a plain input. + +## CY006 one-shot-assertion + +```js +// Before +cy.get('[data-cy=count]').then(($el) => { + expect(Number($el.text())).to.eq(3) +}) + +// After: simple cases +cy.get('[data-cy=count]').should('have.text', '3') + +// After: when the value needs processing +cy.get('[data-cy=count]').should(($el) => { + expect(Number($el.text())).to.eq(3) +}) +``` + +`.then()` runs its callback once with whatever the element contained at that instant. `.should(callback)` re-runs the query and the callback until it stops throwing. Do not call `cy` commands inside a `.should()` callback. + +`.then()` is correct after one-shot commands such as `cy.request()`, `cy.task()`, `cy.wait('@alias')`, and `cy.env()`. + +## CY008 conditional-on-dom + +```js +// Before +cy.get('body').then(($body) => { + if ($body.find('[data-cy=promo-modal]').length) { + cy.get('[data-cy=promo-close]').click() + } +}) + +// After: control the condition +cy.intercept('GET', '/api/promotions', { body: [] }) +cy.visit('/') +cy.get('[data-cy=promo-modal]').should('not.exist') +``` + +The `if` runs once, at an arbitrary moment. If the modal renders 50 ms later, the test continues with it open. Remove the uncertainty at its source: stub the response, seed the data, set the cookie or feature flag that decides the branch. + +## CY009 exclusive-or-debug + +Remove `.only`, `cy.pause()`, and `cy.debug()` before committing. A committed `.only` silently disables every other test in the spec. + +## CY010 cleanup-in-after-hook + +```js +// Before +afterEach(() => { cy.task('db:reset') }) + +// After +beforeEach(() => { cy.task('db:reset') }) +``` + +An `after` hook does not run when the run is cancelled, the browser crashes, or a developer refreshes the runner. The next run then starts from dirty state. Resetting first also leaves the failing state in place for debugging. + +## CY011 absolute-local-visit + +Set `baseUrl` in `cypress.config` and call `cy.visit('/login')`. Without it Cypress loads on a random localhost port and reloads when the first `cy.visit()` runs, and the same spec cannot run against another environment. + +## CY012 ui-login-without-session + +```js +// cypress/support/commands.js +Cypress.Commands.add('login', (email) => { + cy.session(['login', email], () => { + cy.env(['userPassword']).then(({ userPassword }) => { + cy.request('POST', '/api/login', { email, password: userPassword }) + }) + }, { + validate() { + cy.request('/api/me').its('status').should('eq', 200) + }, + }) +}) + +// in specs +beforeEach(() => { + cy.login('ada@example.com') + cy.visit('/dashboard') +}) +``` + +- Put every input that changes the session in the `id` array, never the password. +- The page is blank after `cy.session()`; call `cy.visit()`. +- On Cypress older than 15.10, read the password with `Cypress.env('userPassword')` instead of `cy.env()`. +- Keep one test that logs in through the UI. Everything else should use the cached session. + +## CY013 positional-selector and CY101 brittle-selector + +```js +// Before +cy.get('ul.results > li:nth-child(2) .btn-primary').click() + +// After +cy.get('[data-cy=result]').eq(1).find('[data-cy=open]').click() +// or, when the row is identified by content +cy.contains('[data-cy=result]', 'Invoice 1042').find('[data-cy=open]').click() +``` + +Class names change with styling work and positions change with sorting and data. Add `data-cy` attributes to the application when they are missing; that is a legitimate application change for a test fix. + +## CY102 negative-assertion-as-wait + +```js +// Before +cy.get('[data-cy=save]').click() +cy.get('[data-cy=spinner]').should('not.exist') +cy.get('[data-cy=status]').should('have.text', 'Saved') + +// After +cy.get('[data-cy=save]').click() +cy.get('[data-cy=status]').should('have.text', 'Saved') +``` + +`should('not.exist')` passes immediately if the spinner has not rendered yet. The assertion on the final state already waits for the right moment. Keep a negative assertion only after a positive one has proved the page reached the expected state. + +## CY103 variable-wait + +`cy.wait(TIMEOUT)` with a numeric constant is CY001 behind a name. `cy.wait(alias)` where the variable holds `'@orders'` is fine. Check the declaration. + +## Patterns the scanner cannot see + +- **Intercept registered too late.** Compare the order of `cy.intercept()` and the action or `cy.visit()` that triggers the request. +- **Shared mutable test data.** Two specs that edit the same seeded record collide when specs run in parallel. Create a record per test. +- **Order dependence.** Run the failing test with `.only`. If it fails alone, an earlier test was setting state for it. +- **Time and locale.** Assertions on formatted dates or "today" fail near midnight and in other time zones. Fix the clock with `cy.clock(new Date(2025, 0, 15))`. +- **Third-party scripts.** Analytics and chat widgets slow page load unpredictably. Block them with the `blockHosts` config option. +- **Viewport-dependent layout.** A menu that collapses below a breakpoint behaves differently in CI when the viewport differs. Set `viewportWidth` and `viewportHeight` in the config. diff --git a/skills/cypress-flake-fixer/references/removed-apis.md b/skills/cypress-flake-fixer/references/removed-apis.md new file mode 100644 index 0000000000..9c8bb63c14 --- /dev/null +++ b/skills/cypress-flake-fixer/references/removed-apis.md @@ -0,0 +1,151 @@ +# Cypress APIs removed or changed, by major version + +The scanner reports these as CY007. An entry is graded `error` when the installed major already rejects it. Always confirm against the [official migration guide](https://docs.cypress.io/app/references/migration-guide) for the exact target version. + +## Cypress 16 + +Requires Node.js 22, 24, or 26+. + +| Removed or changed | Replacement | +|---|---| +| `Cypress.env()` | `cy.env([...])` for secrets, `Cypress.expose()` for public values | +| `env` in a test or suite config object | `expose` | +| `allowCypressEnv` config option | delete | +| `cy.exec()`, `execTimeout` | `cy.task()` in `setupNodeEvents`, `taskTimeout` | +| `.end()` | delete the call | +| `cy.getCookie()`, `cy.getCookies()`, `cy.getAllCookies()`, `cy.getAllLocalStorage()`, `cy.getAllSessionStorage()` | now queries: chain `.should()`; override with `Cypress.Commands.overwriteQuery()` | +| `Cypress.config('viewportWidth' / 'viewportHeight' / 'blockHosts', v)` during a test | `cy.viewport(w, h)`; `{ blockHosts: '...' }` in the test config object | +| `experimentalMemoryManagement: true` | delete (on by default as `manageBrowserMemory`) | +| `experimentalMemoryManagement: false` | `manageBrowserMemory: false` | +| `experimentalFastVisibility` | delete; `visibilityStrategy` defaults to `'modern'` | +| `experimentalSourceRewriting` | delete; `removeSRIAttributes` if it was used for SRI errors | +| `keystrokeDelay` default 10 | default is 0; pass `{ delay: 10 }` to a single `.type()` only where the app needs it | +| CoffeeScript specs, support files, fixtures | convert to JavaScript or TypeScript | +| `cypress/angular-zoneless` | `cypress/angular` | +| Electron as the test browser | deprecated: set `defaultBrowser: 'chrome'` or pass `--browser chrome` | + +Component testing minimums in 16: Angular 21, Vite 8, Next.js 15.0.4. + +### Migrating `Cypress.env()` + +Decide per key whether the value is sensitive. + +**Public values** (feature flags, API version, public URLs) are a mechanical change, because `Cypress.expose()` is synchronous like the API it replaces: + +```js +// cypress.config.js +module.exports = defineConfig({ + expose: { apiVersion: 'v2', featureNewCheckout: true }, // moved out of env + env: { apiKey: process.env.API_KEY }, // secrets stay in env +}) + +// Before +const version = Cypress.env('apiVersion') +// After +const version = Cypress.expose('apiVersion') +``` + +- CLI: `--env apiVersion=v2` becomes `--expose apiVersion=v2` (short form `-x`). +- Per test: `it('...', { env: { flag: true } }, ...)` becomes `{ expose: { flag: true } }`. +- `CYPRESS_*` environment variables populate `env`, not `expose`. A public value that CI injects as `CYPRESS_apiVersion` must be either read with `cy.env()` or copied into `expose` in the config file (`expose: { apiVersion: process.env.API_VERSION }`). + +**Sensitive values** require restructuring, because `cy.env()` is an asynchronous command: + +```js +// Before: read at module scope, used anywhere +const apiKey = Cypress.env('apiKey') +it('lists users', () => { + cy.request({ url: '/api/users', headers: { Authorization: `Bearer ${apiKey}` } }) +}) + +// After: read inside the test, used inside the callback +it('lists users', () => { + cy.env(['apiKey']).then(({ apiKey }) => { + cy.request({ url: '/api/users', headers: { Authorization: `Bearer ${apiKey}` } }) + }) +}) +``` + +- `cy.env()` takes an array of keys and yields an object. An undefined key yields `undefined`. +- It cannot be called at module scope or outside a test or hook. +- It cannot set values. Code that used `Cypress.env('token', value)` to pass state between steps should use an alias within a test, or a `set`/`get` pair of tasks across tests. +- Yielded values are not masked. Do not assert on them or pass them through `.its()`. +- Inside `cy.origin()`, call `cy.env()` within the callback. + +**Plugins** that call `Cypress.env()` throw on Cypress 16. Upgrade them in the same change. Known minimums: `@cypress/grep` 6.0.0, `@cypress/code-coverage` 4.0.0, `cypress-mailpit` 2.0.0. Search `node_modules` of other Cypress plugins for `Cypress.env(` before upgrading. + +On Cypress 15.10 through 15.x, set `allowCypressEnv: false` after migrating to make any remaining `Cypress.env()` call fail early. Remove the option when moving to 16. + +### Migrating `cy.exec()` + +```js +// cypress.config.js +const { execFileSync } = require('node:child_process') + +module.exports = defineConfig({ + e2e: { + setupNodeEvents(on) { + on('task', { + seedDatabase(fixture) { + execFileSync('node', ['scripts/seed.js', fixture], { stdio: 'inherit' }) + return null + }, + }) + }, + }, +}) + +// Before +cy.exec('node scripts/seed.js users') +// After +cy.task('seedDatabase', 'users') +``` + +- A task must return a value or `null`. Returning `undefined` fails the task. +- Use `execFileSync` with an argument array so test input never reaches a shell. +- Tasks time out after 60 seconds by default; set `taskTimeout` for longer work. + +## Cypress 15 + +Requires Node.js 20, 22, or 24+ (18 and 23 dropped). + +| Removed or changed | Replacement | +|---|---| +| `cy.stub(obj, 'method', fn)` | `cy.stub(obj, 'method').callsFake(fn)` | +| `cy.exec(...).its('code')` | `exitCode` | +| `Cypress.SelectorPlayground` | `Cypress.ElementSelector` | +| Webpack 4 | Webpack 5 | +| Firefox automation over CDP | WebDriver BiDi only; Firefox 135+ (140+ from 15.19) | +| CommonJS config with `@cypress/vite-dev-server` | ESM config (`.mjs`, `"type": "module"`, or `cypress.config.ts`) | + +## Cypress 14 + +| Removed or changed | Replacement | +|---|---| +| Commands on a different subdomain worked without `cy.origin()` | wrap them in `cy.origin()`; `injectDocumentDomain` is a deprecated escape hatch | +| `experimentalSkipDomainInjection` | removed | +| `experimentalJustInTimeCompile` | `justInTimeCompile` | + +## Cypress 13 + +| Removed or changed | Replacement | +|---|---| +| `video` default `true` | default `false`; enable explicitly if CI relies on videos | +| `videoUploadOnPasses` | removed | + +## Cypress 12 + +| Removed or changed | Replacement | +|---|---| +| `cy.server()`, `cy.route()` | `cy.intercept()` | +| `Cypress.Cookies.preserveOnce()`, `Cypress.Cookies.defaults()` | `cy.session()` | +| `experimentalSessionAndOrigin` | delete; `cy.session()` and `cy.origin()` are stable | +| Tests shared page and storage state | `testIsolation` is on: each test starts on a blank page with cleared cookies and storage | + +`cy.route()` to `cy.intercept()` is not a rename. Differences that change test behavior: + +- `cy.intercept()` glob-matches the full URL including the query string. `cy.intercept('/api/users')` never matches `/api/users?page=2`; use `'/api/users*'` or `{ pathname: '/api/users' }`. +- `cy.intercept()` matches every HTTP method when none is given. `cy.route()` defaulted to `GET`. +- When several intercepts match, the most recently defined one runs first. Define general stubs in `beforeEach` and overrides inside the test. +- A response served from the browser cache never reaches the network layer, so the intercept does not fire and `cy.wait('@alias')` times out. Disable cache headers on the test server. +- `cy.intercept()` also sees `fetch` requests and static resources, which `cy.route()` did not. diff --git a/skills/cypress-flake-fixer/scripts/scan-cypress.mjs b/skills/cypress-flake-fixer/scripts/scan-cypress.mjs new file mode 100755 index 0000000000..d217ddebf3 --- /dev/null +++ b/skills/cypress-flake-fixer/scripts/scan-cypress.mjs @@ -0,0 +1,506 @@ +#!/usr/bin/env node +// Scan Cypress specs, support files, and config for flake-prone anti-patterns +// and for APIs removed in Cypress 12 through 16. +// +// Zero dependencies. Requires Node.js 18 or later. +// The scan is heuristic (text based, no AST). Treat findings as leads to +// confirm in the code, not as proof. + +import fs from 'node:fs' +import path from 'node:path' +import process from 'node:process' + +const HELP = `Usage: node scan-cypress.mjs [paths...] [options] + +Scans Cypress test code for flake-prone anti-patterns and removed APIs. +With no path, scans the current directory. + +Options: + --json Print machine-readable JSON instead of text + --strict Also report low-confidence rules (CY1xx) + --rule Only report these rule ids, comma separated (CY001,CY005) + --max-per-rule Findings listed per rule in text output (default 20, 0 = all) + --fail-on Exit 1 if a finding at this level or above exists: + error | warn | info | never (default never) + --cypress-major Override the Cypress major version detected from package.json + --list-rules Print the rule catalog and exit + -h, --help Show this help + +Suppress a finding with a comment on the same or the previous line: + // cy-scan-ignore (all rules) + // cy-scan-ignore CY001 (one or more rule ids) + +Exit codes: 0 ok, 1 threshold from --fail-on reached, 2 usage or I/O error.` + +const RULES = { + CY001: ['fixed-wait', 'warn', 'cy.wait() waits on time. Wait on a cy.intercept() alias or a retrying assertion.'], + CY002: ['assigned-return-value', 'error', 'A cy command return value is assigned. Commands are queued and yield later; use .then(), .should(), or an alias.'], + CY003: ['async-await', 'error', 'async/await is used with Cypress commands. Commands are not promises; remove async and await.'], + CY004: ['force-true', 'warn', '{ force: true } skips actionability checks. Find what covers or disables the element.'], + CY005: ['chain-after-action', 'warn', 'A command is chained after an action. Safe only if the element cannot re-render; otherwise start a new cy.get().'], + CY006: ['one-shot-assertion', 'warn', 'expect() inside .then() runs once and does not retry. Use .should() or .should(callback).'], + CY007: ['removed-api', 'error', 'API removed or changed in a newer Cypress major.'], + CY008: ['conditional-on-dom', 'warn', 'Test branches on DOM state. Make the state deterministic, then assert it.'], + CY009: ['exclusive-or-debug', 'warn', '.only, cy.pause(), or cy.debug() left in the code.'], + CY010: ['cleanup-in-after-hook', 'info', 'State is cleaned in after/afterEach. Reset state in before/beforeEach so interrupted runs do not leak.'], + CY011: ['absolute-local-visit', 'info', 'cy.visit() uses an absolute local URL. Set baseUrl and visit a relative path.'], + CY012: ['ui-login-without-session', 'info', 'Password is typed through the UI in several files and cy.session() is never used.'], + CY013: ['positional-selector', 'info', 'Selector depends on element position. Prefer a data-* attribute.'], + CY101: ['brittle-selector', 'info', 'Selector relies on a CSS class or id. Prefer a data-* attribute.'], + CY102: ['negative-assertion-as-wait', 'info', 'A negative assertion passes before the element has rendered. Confirm a positive state was asserted first.'], + CY103: ['variable-wait', 'info', 'cy.wait() is called with an identifier. Confirm it is an alias, not a number of milliseconds.'], +} + +const LEVELS = { info: 1, warn: 2, error: 3 } + +const REMOVED = [ + [/\bCypress\s*\.\s*env\s*\(/g, 16, 'Cypress.env() was removed in 16.0 (deprecated 15.10). Use cy.env([...]) for secrets or Cypress.expose() for public values.'], + [/\bcy\s*\.\s*exec\s*\(/g, 16, 'cy.exec() was removed in 16.0. Register a task in setupNodeEvents and call cy.task().'], + [/\)\s*\.\s*end\s*\(\s*\)/g, 16, '.end() was removed in 16.0. Delete the call.'], + [/\bexecTimeout\b/g, 16, 'execTimeout was removed in 16.0 with cy.exec(). Use taskTimeout.'], + [/\bexperimentalMemoryManagement\b/g, 16, 'experimentalMemoryManagement was removed in 16.0. Delete it, or set manageBrowserMemory: false to opt out.'], + [/\bexperimentalFastVisibility\b/g, 16, 'experimentalFastVisibility was removed in 16.0. visibilityStrategy defaults to "modern".'], + [/\bexperimentalSourceRewriting\b/g, 16, 'experimentalSourceRewriting was removed in 16.0. Delete it; use removeSRIAttributes for SRI errors.'], + [/\ballowCypressEnv\b/g, 16, 'allowCypressEnv was removed in 16.0. Delete it.'], + [/\bCypress\s*\.\s*config\s*\(\s*['"`](?:viewportWidth|viewportHeight|blockHosts)['"`]\s*,/g, 16, 'Setting this through Cypress.config() during a test throws in 16.0. Use cy.viewport() or the test config object.'], + [/\bCypress\s*\.\s*Commands\s*\.\s*overwrite\s*\(\s*['"`](?:getCookie|getCookies|getAllCookies|getAllLocalStorage|getAllSessionStorage)['"`]/g, 16, 'This command is a query in 16.0. Use Cypress.Commands.overwriteQuery().'], + [/\bCypress\s*\.\s*SelectorPlayground\b/g, 15, 'Cypress.SelectorPlayground was renamed Cypress.ElementSelector in 15.0.'], + [/\bexperimentalSkipDomainInjection\b/g, 14, 'experimentalSkipDomainInjection was removed in 14.0. Use cy.origin() for subdomains.'], + [/\bexperimentalJustInTimeCompile\b/g, 14, 'experimentalJustInTimeCompile became justInTimeCompile in 14.0.'], + [/\bvideoUploadOnPasses\b/g, 13, 'videoUploadOnPasses was removed in 13.0.'], + [/\bcy\s*\.\s*(?:server|route)\s*\(/g, 12, 'cy.server() and cy.route() were removed in 12.0. Use cy.intercept().'], + [/\bCypress\s*\.\s*Cookies\s*\.\s*(?:preserveOnce|defaults)\b/g, 12, 'Cypress.Cookies.preserveOnce() and .defaults() were removed in 12.0. Use cy.session().'], + [/\bexperimentalSessionAndOrigin\b/g, 12, 'experimentalSessionAndOrigin was removed in 12.0. cy.session() and cy.origin() are stable.'], + [/\bcy\s*\.\s*route2\s*\(/g, 7, 'cy.route2() was renamed cy.intercept() in 7.0.'], + [/\bcy\s*\.\s*xpath\s*\(/g, null, 'cy.xpath() comes from the deprecated cypress-xpath plugin. Use cy.get() or cy.contains().'], +] + +const ACTIONS = 'click|dblclick|rightclick|type|clear|check|uncheck|select|selectFile|trigger|focus|blur|submit|scrollIntoView|scrollTo' +const SKIP_DIRS = new Set(['node_modules', '.git', 'dist', 'build', 'coverage', '.next', '.nuxt', 'out', 'screenshots', 'videos', 'downloads']) +const CODE_EXT = /\.(?:[cm]?[jt]sx?)$/ + +function rel(file) { + const relative = path.relative(process.cwd(), file) + const shown = relative && !relative.startsWith('..') && !path.isAbsolute(relative) ? relative : file + return shown.split(path.sep).join('/') +} + +function fail(message) { + process.stderr.write(`scan-cypress: ${message}\n`) + process.exit(2) +} + +function parseArgs(argv) { + const opts = { paths: [], json: false, strict: false, rules: null, maxPerRule: 20, failOn: 'never', major: undefined } + for (let i = 0; i < argv.length; i++) { + const arg = argv[i] + const next = () => { + if (i + 1 >= argv.length) fail(`${arg} needs a value`) + return argv[++i] + } + if (arg === '-h' || arg === '--help') { process.stdout.write(`${HELP}\n`); process.exit(0) } + else if (arg === '--list-rules') { + for (const [id, [name, level, text]] of Object.entries(RULES)) process.stdout.write(`${id} ${name.padEnd(28)} ${level.padEnd(5)} ${text}\n`) + process.exit(0) + } + else if (arg === '--json') opts.json = true + else if (arg === '--strict') opts.strict = true + else if (arg === '--rule') opts.rules = new Set(next().split(',').map((s) => s.trim().toUpperCase()).filter(Boolean)) + else if (arg === '--max-per-rule') { + opts.maxPerRule = Number(next()) + if (!Number.isInteger(opts.maxPerRule) || opts.maxPerRule < 0) fail('--max-per-rule needs a non-negative integer') + } + else if (arg === '--fail-on') { + opts.failOn = next() + if (!['error', 'warn', 'info', 'never'].includes(opts.failOn)) fail('--fail-on must be error, warn, info, or never') + } + else if (arg === '--cypress-major') { + opts.major = Number(next()) + if (!Number.isInteger(opts.major) || opts.major < 1) fail('--cypress-major needs a positive integer') + } + else if (arg.startsWith('-')) fail(`unknown option ${arg} (try --help)`) + else opts.paths.push(arg) + } + if (opts.paths.length === 0) opts.paths.push('.') + if (opts.rules) for (const id of opts.rules) if (!RULES[id]) fail(`unknown rule ${id} (try --list-rules)`) + return opts +} + +function isCypressFile(file) { + const normalized = rel(file) + const base = path.basename(normalized) + if (/^cypress\.config\.[cm]?[jt]s$/.test(base)) return true + if (!CODE_EXT.test(base)) return false + if (/\.cy\.[cm]?[jt]sx?$/.test(base)) return true + return /(?:^|\/)cypress\//.test(normalized) +} + +function walk(target, files, coffee) { + let stat + try { stat = fs.statSync(target) } catch { fail(`cannot read ${target}`) } + if (stat.isFile()) { + if (isCypressFile(target) || CODE_EXT.test(target)) files.push(target) + return + } + for (const entry of fs.readdirSync(target, { withFileTypes: true })) { + const full = path.join(target, entry.name) + if (entry.isDirectory()) { + if (!SKIP_DIRS.has(entry.name)) walk(full, files, coffee) + } else if (entry.isFile()) { + if (isCypressFile(full)) files.push(full) + else if (/\.coffee$/.test(entry.name) && /(?:^|\/)cypress\//.test(full.split(path.sep).join('/'))) coffee.push(full) + } + } +} + +function detectCypress(startDir) { + let dir = path.resolve(startDir) + for (let depth = 0; depth < 8; depth++) { + const file = path.join(dir, 'package.json') + if (fs.existsSync(file)) { + try { + const pkg = JSON.parse(fs.readFileSync(file, 'utf8')) + const range = pkg.devDependencies?.cypress ?? pkg.dependencies?.cypress + if (range) { + const match = String(range).match(/\d+/) + return { range: String(range), major: match ? Number(match[0]) : null, source: file } + } + } catch { /* unreadable package.json: keep walking up */ } + } + const parent = path.dirname(dir) + if (parent === dir) break + dir = parent + } + return { range: null, major: null, source: null } +} + +// Replace comments with spaces so offsets and line numbers stay valid. +function blankComments(src) { + let out = '' + let state = 'code' + for (let i = 0; i < src.length;) { + const c = src[i] + const d = src[i + 1] + if (state === 'code') { + if (c === '\\') { out += c + (d ?? ''); i += 2; continue } + if (c === '/' && d === '/') { state = 'line'; out += ' '; i += 2; continue } + if (c === '/' && d === '*') { state = 'block'; out += ' '; i += 2; continue } + if (c === "'") state = 'sq' + else if (c === '"') state = 'dq' + else if (c === '`') state = 'tpl' + out += c; i++; continue + } + if (state === 'line') { + if (c === '\n') { state = 'code'; out += c } else out += ' ' + i++; continue + } + if (state === 'block') { + if (c === '*' && d === '/') { state = 'code'; out += ' '; i += 2; continue } + out += c === '\n' ? '\n' : ' ' + i++; continue + } + if (c === '\\') { out += c + (d ?? ''); i += 2; continue } + if ((state === 'sq' && c === "'") || (state === 'dq' && c === '"') || (state === 'tpl' && c === '`')) state = 'code' + else if ((state === 'sq' || state === 'dq') && c === '\n') state = 'code' + out += c; i++ + } + return out +} + +// Index of the parenthesis that closes the one at `open`, or -1. +function matchParen(src, open) { + let depth = 0 + let quote = null + for (let i = open; i < src.length; i++) { + const c = src[i] + if (quote) { + if (c === '\\') { i++; continue } + if (c === quote) quote = null + continue + } + if (c === "'" || c === '"' || c === '`') { quote = c; continue } + if (c === '(') depth++ + else if (c === ')') { depth--; if (depth === 0) return i } + } + return -1 +} + +// Start offset of the statement whose chain contains offset `idx`. +function chainStart(src, idx) { + let depth = 0 + for (let j = idx - 1; j >= 0; j--) { + const c = src[j] + if (c === ')' || c === ']' || c === '}') depth++ + else if (c === '(' || c === '[' || c === '{') { + if (depth === 0) return j + 1 + depth-- + } else if (depth === 0) { + if (c === ';') return j + 1 + if (c === '\n') { + const rest = src.slice(j + 1, idx).trimStart() + if (rest && rest[0] !== '.') return j + 1 + } + } + } + return 0 +} + +function countTopLevelCommas(args) { + let depth = 0 + let quote = null + let commas = 0 + for (let i = 0; i < args.length; i++) { + const c = args[i] + if (quote) { + if (c === '\\') { i++; continue } + if (c === quote) quote = null + continue + } + if (c === "'" || c === '"' || c === '`') quote = c + else if (c === '(' || c === '[' || c === '{') depth++ + else if (c === ')' || c === ']' || c === '}') depth-- + else if (c === ',' && depth === 0 && args.slice(i + 1).trim()) commas++ + } + return commas +} + +function scanFile(file, opts, major, findings) { + const raw = fs.readFileSync(file, 'utf8') + const rawLines = raw.split('\n') + const src = blankComments(raw) + const lineStarts = [0] + for (let i = 0; i < src.length; i++) if (src[i] === '\n') lineStarts.push(i + 1) + + const lineOf = (offset) => { + let lo = 0 + let hi = lineStarts.length - 1 + while (lo < hi) { + const mid = (lo + hi + 1) >> 1 + if (lineStarts[mid] <= offset) lo = mid + else hi = mid - 1 + } + return lo + 1 + } + + const suppressed = (line, rule) => { + for (const text of [rawLines[line - 1], rawLines[line - 2]]) { + const match = text?.match(/cy-scan-ignore\b([^\n]*)/) + if (!match) continue + const ids = match[1].match(/CY\d{3}/g) + if (!ids || ids.includes(rule)) return true + } + return false + } + + const add = (rule, offset, message, level) => { + if (opts.rules && !opts.rules.has(rule)) return + if (!opts.strict && /^CY1\d\d$/.test(rule) && !(opts.rules && opts.rules.has(rule))) return + const line = lineOf(offset) + if (suppressed(line, rule)) return + const snippet = (rawLines[line - 1] ?? '').trim() + findings.push({ + rule, + name: RULES[rule][0], + level: level ?? RULES[rule][1], + file: rel(file), + line, + message: message ?? RULES[rule][2], + snippet: snippet.length > 140 ? `${snippet.slice(0, 137)}...` : snippet, + }) + } + + const each = (regex, fn) => { + regex.lastIndex = 0 + let match + while ((match = regex.exec(src)) !== null) { + fn(match) + if (match[0].length === 0) regex.lastIndex++ + } + } + + // CY001 fixed waits + each(/(?:\bcy|\))\s*\.\s*wait\s*\(\s*\d[\d_.]*\s*[,)]/g, (m) => add('CY001', m.index + m[0].indexOf('wait'))) + + // CY103 wait on an identifier (strict) + each(/\bcy\s*\.\s*wait\s*\(\s*[A-Za-z_$][\w$.]*\s*\)/g, (m) => add('CY103', m.index)) + + // CY002 assigned return values. cy.stub() and cy.spy() are synchronous and exempt. + each(/\b(?:const|let|var)\s+[^=;\n]+=\s*cy\s*\.\s*(\w+)/g, (m) => { + if (!['stub', 'spy', 'state'].includes(m[1])) add('CY002', m.index) + }) + + // CY003 async tests and awaited commands + each(/\b(?:it|specify|before|beforeEach|after|afterEach)(?:\s*\.\s*(?:only|skip))?\s*\(\s*(?:(['"`])(?:\\.|(?!\1)[^\\])*\1\s*,\s*)?(?:\{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}\s*,\s*)?async\b/g, (m) => add('CY003', m.index)) + each(/\bawait\s+cy\s*\./g, (m) => add('CY003', m.index, 'A cy command is awaited. Commands are not promises; use .then().')) + + // CY004 force + each(/\bforce\s*:\s*true\b/g, (m) => add('CY004', m.index)) + + // CY005 chaining after an action + each(new RegExp(`\\.\\s*(${ACTIONS})\\s*\\(`, 'g'), (m) => { + const open = m.index + m[0].length - 1 + const close = matchParen(src, open) + if (close === -1) return + const tail = src.slice(close + 1).match(/^\s*\.\s*([A-Za-z_$][\w$]*)/) + if (!tail) return + const next = tail[1] + if (['then', 'as', 'catch', 'finally', 'end'].includes(next)) return + const start = chainStart(src, m.index) + if (!/^\s*(?:return\s+)?cy\b/.test(src.slice(start, m.index))) return + const soft = next === 'should' || next === 'and' + add( + 'CY005', + m.index, + soft + ? `.${next}() is chained after .${m[1]}(). If the action re-renders the element the subject is stale; re-query before asserting.` + : `.${next}() is chained after .${m[1]}(). Start a new cy.get() unless the element cannot re-render.`, + soft ? 'info' : 'warn', + ) + }) + + // CY006 expect() inside .then() after a DOM or page query + each(/\.\s*then\s*\(/g, (m) => { + const open = m.index + m[0].length - 1 + const close = matchParen(src, open) + if (close === -1) return + const body = src.slice(open + 1, close) + if (!/\bexpect\s*\(|\bassert\s*[.(]/.test(body) || /\bcy\s*\./.test(body)) return + const chain = src.slice(chainStart(src, m.index), m.index) + if (!/^\s*(?:return\s+)?cy\b/.test(chain)) return + if (/\bcy\s*\.\s*(?:request|task|wait|env|fixture|readFile|wrap|window|exec|origin|session)\s*\(/.test(chain)) return + if (/\bcy\s*\.\s*get\s*\(\s*['"`]@/.test(chain)) return + if (!/\bcy\s*\.\s*(?:get|contains|url|title|location|hash|focused|getCookie|getCookies|getAllCookies|getAllLocalStorage|getAllSessionStorage)\s*\(/.test(chain)) return + add('CY006', m.index) + }) + + // CY007 removed APIs + for (const [regex, removedIn, text] of REMOVED) { + each(regex, (m) => { + const broken = removedIn === null ? false : major === null ? null : major >= removedIn + const level = broken === true ? 'error' : 'warn' + const suffix = removedIn === null || major === null ? '' : broken ? ` Installed major is ${major}: this already fails.` : ` Installed major is ${major}: fix before upgrading.` + add('CY007', m.index, `${text}${suffix}`, level) + }) + } + each(/\bcy\s*\.\s*stub\s*\(/g, (m) => { + const open = m.index + m[0].length - 1 + const close = matchParen(src, open) + if (close === -1) return + if (countTopLevelCommas(src.slice(open + 1, close)) >= 2) { + const broken = major !== null && major >= 15 + add('CY007', m.index, 'cy.stub(obj, name, fn) was removed in 15.0. Use cy.stub(obj, name).callsFake(fn).', broken ? 'error' : 'warn') + } + }) + + // CY008 conditional testing + each(/\bif\s*\(\s*!?\s*\$\w+\s*\.\s*(?:find|is|hasClass|children|text|length)\b/g, (m) => add('CY008', m.index)) + + // CY009 exclusive tests and debug commands + each(/\b(?:it|describe|context|specify)\s*\.\s*only\s*\(/g, (m) => add('CY009', m.index, '.only is committed: the rest of the suite does not run.')) + each(/(?:\bcy|\))\s*\.\s*(?:pause|debug)\s*\(\s*\)/g, (m) => add('CY009', m.index, 'cy.pause() or cy.debug() left in the code.')) + + // CY010 cleanup hooks + each(/(?:^|\n)[ \t]*(?:after|afterEach)\s*\(/g, (m) => add('CY010', m.index + m[0].search(/a/))) + + // CY011 absolute local visit + each(/\bcy\s*\.\s*visit\s*\(\s*['"`]https?:\/\/(?:localhost|127\.0\.0\.1|0\.0\.0\.0)/g, (m) => add('CY011', m.index)) + + // CY013 and CY101 selectors + each(/(?:\bcy\s*\.\s*get|\.\s*find)\s*\(\s*(['"`])((?:\\.|(?!\1)[^\\\n])*)\1/g, (m) => { + const selector = m[2] + if (selector.startsWith('@') || selector.includes('${')) return + if (/:nth-(?:child|of-type|last-child)\(|:eq\(|:first\b|:last\b/.test(selector)) add('CY013', m.index) + else if (!/\[data-|\[aria-|\[role=|\[name=/.test(selector) && /(?:^|[\s>+~])[.#][\w-]+/.test(selector)) add('CY101', m.index) + }) + + // CY102 negative assertions + each(/\.\s*should\s*\(\s*['"`]not\.(?:exist|be\.visible)['"`]\s*\)/g, (m) => add('CY102', m.index)) + + return { + typesPassword: /(?:password|passwd|pwd)[^\n]{0,80}\.\s*type\s*\(|type\s*=\s*["']?password/i.test(src), + usesSession: /\bcy\s*\.\s*session\s*\(/.test(src), + } +} + +function main() { + const opts = parseArgs(process.argv.slice(2)) + const files = [] + const coffee = [] + for (const target of opts.paths) walk(target, files, coffee) + const unique = [...new Set(files)].sort() + + const firstPath = opts.paths[0] + const startDir = fs.statSync(firstPath).isDirectory() ? firstPath : path.dirname(firstPath) + const detected = detectCypress(startDir) + const major = opts.major ?? detected.major + + const findings = [] + const passwordFiles = [] + let sessionUsed = false + for (const file of unique) { + const facts = scanFile(file, opts, major, findings) + if (facts.typesPassword) passwordFiles.push(rel(file)) + if (facts.usesSession) sessionUsed = true + } + + const wants = (rule) => !opts.rules || opts.rules.has(rule) + if (wants('CY012') && !sessionUsed && passwordFiles.length >= 2) { + findings.push({ + rule: 'CY012', name: RULES.CY012[0], level: RULES.CY012[1], file: passwordFiles[0], line: 1, + message: `${RULES.CY012[2]} Files: ${passwordFiles.slice(0, 8).join(', ')}${passwordFiles.length > 8 ? ', ...' : ''}`, + snippet: '', + }) + } + if (wants('CY007')) { + for (const file of coffee) { + findings.push({ + rule: 'CY007', name: RULES.CY007[0], level: major !== null && major >= 16 ? 'error' : 'warn', + file: rel(file), line: 1, + message: 'Built-in CoffeeScript support was removed in 16.0. Convert the file to JavaScript or TypeScript.', snippet: '', + }) + } + } + + findings.sort((a, b) => a.rule.localeCompare(b.rule) || a.file.localeCompare(b.file) || a.line - b.line) + const summary = {} + for (const f of findings) summary[f.rule] = (summary[f.rule] ?? 0) + 1 + + if (opts.json) { + process.stdout.write(`${JSON.stringify({ + cypress: { range: detected.range, major, source: detected.source }, + filesScanned: unique.length, + strict: opts.strict, + summary, + findings, + }, null, 2)}\n`) + } else { + const lines = ['Cypress anti-pattern scan', ''] + lines.push(major === null + ? 'Cypress version: not detected (pass --cypress-major to grade removed APIs)' + : `Cypress version: major ${major}${detected.range && opts.major === undefined ? ` (range ${detected.range} in ${detected.source})` : ' (from --cypress-major)'}`) + lines.push(`Files scanned: ${unique.length}`) + lines.push(`Findings: ${findings.length}`, '') + if (unique.length === 0) lines.push('No Cypress files found. Pass the folder that contains cypress/ or *.cy.* files.') + for (const rule of Object.keys(summary)) { + const group = findings.filter((f) => f.rule === rule) + lines.push(`${rule} ${RULES[rule][0]} (${group.length})`) + lines.push(` ${RULES[rule][2]}`) + const shown = opts.maxPerRule === 0 ? group : group.slice(0, opts.maxPerRule) + for (const f of shown) { + lines.push(` [${f.level}] ${f.file}:${f.line} ${f.snippet}`) + if (f.message !== RULES[rule][2]) lines.push(` ${f.message}`) + } + if (shown.length < group.length) lines.push(` ... ${group.length - shown.length} more (use --max-per-rule 0 or --json)`) + lines.push('') + } + if (!opts.strict && findings.length > 0) lines.push('Run with --strict to include low-confidence rules (CY1xx).') + process.stdout.write(`${lines.join('\n')}\n`) + } + + if (opts.failOn !== 'never') { + const threshold = LEVELS[opts.failOn] + if (findings.some((f) => LEVELS[f.level] >= threshold)) process.exit(1) + } +} + +main()