Treat disabled managed tests as skipped#74
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1b9616c134
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Suite: suite, | ||
| Name: name, | ||
| } | ||
| disabledTests[disabledTest.FQN()] = true |
There was a problem hiding this comment.
When a managed disabled test has parameters in local discovery, this key is built with an empty Parameters field, but recordFullDiscoveryResults checks skippableTests[test.FQN()], and RSpec/minitest discovery commonly populates Parameters (for example scoped metadata). In that case Datadog returns the disabled test by module/suite/name, but the discovered test key is module.suite.name.<params>, so the disabled test remains runnable instead of being skipped.
Useful? React with 👍 / 👎.
1 participant
What
Treat disabled tests from Datadog Test Management as skipped during planning, alongside TIA skippable tests. Module is now part of test FQNs for both TIA skippable data and test-management disabled data so same suite/name tests in different modules do not collide.
Why
Test Management can mark tests as disabled, and ddtest should honor that state when deciding which discovered tests to skip. This keeps planner behavior aligned with Datadog-managed test state while preserving the existing rule that full test discovery only depends on TIA/test-skipping settings.
E2E testing
In a repository with TIA and Test Management enabled, mark one test as disabled in Datadog Test Management and run
ddtest plan; confirm the generated plan skips that disabled test even when it is not returned as TIA-skippable, while an identically named test in another module remains runnable. Also validated withgo test ./internal/planner,make test, andmake lint.