INTEROP-9384: Add OCS_TEST_FILTER to exclude NooBaa timeout test on vSphere by amp-rh · Pull Request #82996 · openshift/release

amp-rh · 2026-08-05T18:55:58Z

What

Adds an OCS_TEST_FILTER env var to the interop-tests-ocs-tests step registry ref, allowing per-job pytest -k filtering. Sets it in the vSphere 4.22 OPP config to exclude test_bucket_creation_deletion.

Why

The nb_ensure_endpoint_count fixture consistently times out on vSphere 4.22 (7 consecutive failures since Jul 15, 2025). This is 1 of 28 acceptance tests; the remaining 27 pass reliably. The ODF/OCS team has an open fix tracked in OCSQE-5148 but has not responded to multiple engagement attempts.

Changes

File	Change
`ci-operator/step-registry/interop-tests/ocs-tests/interop-tests-ocs-tests-ref.yaml`	Add `OCS_TEST_FILTER` env var (default empty)
`ci-operator/step-registry/interop-tests/ocs-tests/interop-tests-ocs-tests-commands.sh`	Use `${OCS_TEST_FILTER}` in pytest `-k` flag
`ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml`	Set filter in vSphere config only

Caveats

AWS unaffected: The filter is only set in interop-opp-vsphere; AWS continues running the full suite.
Re-enablement: Remove the OCS_TEST_FILTER override once OCSQE-5148 is resolved. Tracked in INTEROP-9384.
Safe to exclude: Chai Bot analysis confirmed this test is pure NooBaa functional with no cross-product integration value.

References

Jira: INTEROP-9384
Upstream fix: OCSQE-5148

Summary by CodeRabbit

Adds per-job pytest -k filtering to the OCS interop test step.
Applies the filter only to vSphere 4.22 to skip the timing-out test_bucket_creation_deletion test.
AWS continues to run the full test suite.
The filter can be removed after OCSQE-5148 is resolved, as tracked by INTEROP-9384.

The nb_ensure_endpoint_count fixture consistently times out on vSphere 4.22 (7 consecutive failures since Jul 15). This adds an OCS_TEST_FILTER env var to the interop-tests-ocs-tests step and sets it in the vSphere config to skip the affected test. The test continues to run on AWS. Tracked by OCSQE-5148 for re-enablement when the upstream fix lands. Jira: INTEROP-9384

coderabbitai · 2026-08-05T18:56:33Z

Walkthrough

The OCS interoperability test step now supports a configurable pytest -k filter. The vSphere configuration excludes test_bucket_creation_deletion.

Changes

OCS test filter

Layer / File(s)	Summary
Configure and apply the OCS test filter `ci-operator/step-registry/interop-tests/ocs-tests/interop-tests-ocs-tests-ref.yaml`, `ci-operator/step-registry/interop-tests/ocs-tests/interop-tests-ocs-tests-commands.sh`, `ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml`	The step defines `OCS_TEST_FILTER` with an empty default and passes it to pytest through `-k`. The vSphere configuration sets the filter to exclude `test_bucket_creation_deletion`.

Estimated code review effort: 2 (Simple) | ~5 minutes

Suggested labels: jira/valid-reference

🚥 Pre-merge checks | ✅ 15

✅ Passed checks (15 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The commit changes only YAML and a shell command; it adds no Ginkgo test titles or dynamic title construction. The filter uses a static pytest identifier.
Test Structure And Quality	✅ Passed	The commit changes only YAML configuration and a Bash pytest invocation; it adds no Ginkgo test code, so the listed Ginkgo quality checks are not applicable.
Microshift Test Compatibility	✅ Passed	The exact diff changes only CI YAML and a shell command; it adds no Go files, Ginkgo declarations, or MicroShift-sensitive API usage.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The PR changes only CI YAML and a shell command; it adds no Ginkgo e2e tests, so SNO compatibility checks do not apply.
Topology-Aware Scheduling Compatibility	✅ Passed	The changes only add a CI environment variable, pass it to pytest, and scope it to the vSphere job; the inspected files contain no scheduling constraints or operator/controller changes.
Ote Binary Stdout Contract	✅ Passed	The PR changes only one shell script and two YAML files; it adds no OTE binary, Ginkgo suite setup, or process-level stdout write.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The PR changes only CI YAML and an OCS test runner filter; it adds no Ginkgo e2e tests or new IPv4 or external-connectivity behavior.
No-Weak-Crypto	✅ Passed	The PR only adds a pytest filter variable and passes it to run-ci; the complete parent-to-HEAD diff contains no weak-crypto APIs, algorithms, custom crypto, or secret comparisons.
Container-Privileges	✅ Passed	The commit only adds OCS_TEST_FILTER and passes it to pytest; no privileged, host namespace, SYS_ADMIN, allowPrivilegeEscalation, or root settings are added.
No-Sensitive-Data-In-Logs	✅ Passed	The PR adds only a non-sensitive pytest filter and passes it to run-ci; no new logging or secret, PII, hostname, or customer-data output is added.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes adding a filter to exclude the affected OCS test on vSphere.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

redhat-chai-bot · 2026-08-05T18:57:48Z

Review: `OCS_TEST_FILTER` env var for per-variant test exclusion

I reviewed the three-file diff for correctness against step-registry conventions. Everything looks clean.

✅ Ref YAML declaration (`interop-tests-ocs-tests-ref.yaml`)

OCS_TEST_FILTER is properly declared with default: "" and clear documentation including a usage example.
Empty default preserves backward compatibility — all existing consumers of this step are unaffected.
Placement is consistent with the surrounding env var declarations (DR__RP__CR_COMP_NAME, DISABLE_ENVIRONMENT_CHECKER).

✅ Shell script usage (`interop-tests-ocs-tests-commands.sh`)

The change from -k '' to -k "${OCS_TEST_FILTER}" is correct:
- Double-quoted to prevent word splitting — proper shell practice.
- When the default empty string is used, -k "" behaves identically to the original -k '' in pytest (both mean "no filter, run everything").
- When set, the value is passed as a single argument to pytest's -k flag as intended.

✅ Config placement (`stolostron-policy-collection-main__ocp4.22.yaml`)

The filter not test_bucket_creation_deletion is set only in the vSphere 4.22 OPP config, which is the correct scope for this exclusion.
The value is valid pytest -k syntax.
YAML-safe: not is not a YAML boolean keyword, so the unquoted value will not be misinterpreted.
Other variants (including AWS) remain unaffected and continue running the full suite.

Coverage note

This exclusion is safe from a cross-product perspective: test_bucket_creation_deletion is blocked on vSphere by a NooBaa endpoint timeout tracked in OCSQE-5148. The AWS variant still runs the full acceptance suite (no OCS_TEST_FILTER override), so there is no gap in overall product coverage. The exclusion should be re-enabled once OCSQE-5148 is resolved.

Summary

Follows step-registry conventions ✅
Backward compatible ✅
Minimal and well-scoped ✅
No concerns

LGTM from a structural review standpoint.

Review performed by Chai Bot at the request of a team member (thread).

AI-generated. Review for accuracy.

openshift-merge-bot · 2026-08-05T18:59:27Z

[REHEARSALNOTIFIER]
@amp-rh: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
periodic-ci-RedHatQE-interop-testing-cnv-4.18-cnv-odf-ocp4.18-lp-interop-cnv-odf-tests-aws-ipi-ocp418	N/A	periodic	Registry content changed
periodic-ci-stolostron-policy-collection-main-ocp4.22-interop-opp-vsphere	N/A	periodic	Ci-operator config changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.15-lp-interop-odf-interop-aws	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp-4.21-lp-interop-cr-cnv-component-readiness-aws-ipi-ocp421	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp-4.20-lp-interop-cnv-odf-tests-aws-ipi-ocp420	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp-4.20-lp-interop-cr-cnv-component-readiness-aws-ipi-ocp420	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.17-lp-interop-odf-interop-aws-fips	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-ocp-4.22-lpGA-lp-ocp-compat-aws-fips	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp-4.21-lp-interop-cr-odf-interop-aws	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp4.19-lp-interop-cnv-odf-tests-aws-ipi-ocp419	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp-4.20-lp-interop-cnv-component-readiness-aws-ipi-ocp420	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.18-lp-interop-odf-interop-aws	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.18-lp-interop-odf-interop-aws-fips	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.16-lp-interop-odf-interop-aws	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.16-lp-interop-odf-interop-aws-fips	N/A	periodic	Registry content changed
periodic-ci-stolostron-policy-collection-main-ocp5.0-interop-opp-vsphere	N/A	periodic	Registry content changed
periodic-ci-stolostron-policy-collection-main-ocp5.0-interop-opp-aws	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.21-lp-interop-odf-interop-aws-fips	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.20-lp-interop-odf-interop-aws	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.20-lp-interop-odf-interop-aws-fips	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.19-lp-interop-odf-interop-aws-fips	N/A	periodic	Registry content changed
periodic-ci-stolostron-policy-collection-main-ocp4.22-interop-opp-aws	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp-4.20-lp-interop-cnv-odf-tests-aws-ipi-ocp420-fips	N/A	periodic	Registry content changed
periodic-ci-red-hat-storage-ocs-ci-master-odf-ocp4.17-lp-interop-odf-interop-aws	N/A	periodic	Registry content changed
periodic-ci-RedHatQE-interop-testing-master-cnv-odf-ocp4.19-lp-interop-cnv-odf-tests-aws-ipi-ocp419-fips	N/A	periodic	Registry content changed

A total of 28 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

openshift-ci · 2026-08-05T19:06:39Z

@amp-rh: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

sg-rh

/lgtm

openshift-ci · 2026-08-07T17:24:45Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: amp-rh, sg-rh
Once this PR has been reviewed and has the lgtm label, please assign dhaiducek for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

ci-operator/config/stolostron/policy-collection/OWNERS
~~ci-operator/step-registry/interop-tests/ocs-tests/OWNERS~~ [amp-rh,sg-rh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2026-08-07T18:48:21Z

@amp-rh: This pull request references INTEROP-9384 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

What

Adds an OCS_TEST_FILTER env var to the interop-tests-ocs-tests step registry ref, allowing per-job pytest -k filtering. Sets it in the vSphere 4.22 OPP config to exclude test_bucket_creation_deletion.

Why

The nb_ensure_endpoint_count fixture consistently times out on vSphere 4.22 (7 consecutive failures since Jul 15, 2025). This is 1 of 28 acceptance tests; the remaining 27 pass reliably. The ODF/OCS team has an open fix tracked in OCSQE-5148 but has not responded to multiple engagement attempts.

Changes

File Change

ci-operator/step-registry/interop-tests/ocs-tests/interop-tests-ocs-tests-ref.yaml Add OCS_TEST_FILTER env var (default empty)

ci-operator/step-registry/interop-tests/ocs-tests/interop-tests-ocs-tests-commands.sh Use ${OCS_TEST_FILTER} in pytest -k flag

ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml Set filter in vSphere config only

Caveats

AWS unaffected: The filter is only set in interop-opp-vsphere; AWS continues running the full suite.

Re-enablement: Remove the OCS_TEST_FILTER override once OCSQE-5148 is resolved. Tracked in INTEROP-9384.

Safe to exclude: Chai Bot analysis confirmed this test is pure NooBaa functional with no cross-product integration value.

References

Jira: INTEROP-9384

Upstream fix: OCSQE-5148

Summary by CodeRabbit

Adds per-job pytest -k filtering to the OCS interop test step.

Applies the filter only to vSphere 4.22 to skip the timing-out test_bucket_creation_deletion test.

AWS continues to run the full test suite.

The filter can be removed after OCSQE-5148 is resolved, as tracked by INTEROP-9384.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

amp-rh · 2026-08-07T18:55:44Z

Superseded by #82999, which replaces the entire OCS acceptance step with a lightweight ODF health check rather than filtering individual tests.

openshift-ci Bot requested review from jan-law and sg-rh August 5, 2026 18:57

sg-rh approved these changes Aug 7, 2026

View reviewed changes

openshift-ci Bot assigned sg-rh Aug 7, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 7, 2026

amp-rh changed the title ~~Add OCS_TEST_FILTER to exclude NooBaa timeout test on vSphere~~ INTEROP-9384: Add OCS_TEST_FILTER to exclude NooBaa timeout test on vSphere Aug 7, 2026

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 7, 2026

amp-rh closed this Aug 7, 2026

Navigation Menu

Uh oh!

INTEROP-9384: Add OCS_TEST_FILTER to exclude NooBaa timeout test on vSphere - #82996

INTEROP-9384: Add OCS_TEST_FILTER to exclude NooBaa timeout test on vSphere#82996
amp-rh wants to merge 1 commit into
openshift:mainfrom
amp-rh:ocs-test-filter-vsphere

amp-rh commented Aug 5, 2026 •

edited by openshift-ci Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 5, 2026 •

edited

Loading

Uh oh!

redhat-chai-bot commented Aug 5, 2026 •

edited by atlassian Bot

Loading

Uh oh!

openshift-merge-bot Bot commented Aug 5, 2026

Uh oh!

openshift-ci Bot commented Aug 5, 2026

Uh oh!

sg-rh left a comment

Uh oh!

openshift-ci Bot commented Aug 7, 2026

Uh oh!

openshift-ci-robot commented Aug 7, 2026 •

edited by openshift-ci Bot

Loading

What

Why

Changes

Caveats

References

Summary by CodeRabbit

Uh oh!

amp-rh commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

amp-rh commented Aug 5, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Changes

Caveats

References

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

redhat-chai-bot commented Aug 5, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review: OCS_TEST_FILTER env var for per-variant test exclusion

✅ Ref YAML declaration (interop-tests-ocs-tests-ref.yaml)

✅ Shell script usage (interop-tests-ocs-tests-commands.sh)

✅ Config placement (stolostron-policy-collection-main__ocp4.22.yaml)

Coverage note

Summary

Uh oh!

openshift-merge-bot Bot commented Aug 5, 2026

Uh oh!

openshift-ci Bot commented Aug 5, 2026

Uh oh!

sg-rh left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Aug 7, 2026

Uh oh!

openshift-ci-robot commented Aug 7, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Changes

Caveats

References

Summary by CodeRabbit

Uh oh!

amp-rh commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

amp-rh commented Aug 5, 2026 •

edited by openshift-ci Bot

Loading

coderabbitai Bot commented Aug 5, 2026 •

edited

Loading

redhat-chai-bot commented Aug 5, 2026 •

edited by atlassian Bot

Loading

Review: `OCS_TEST_FILTER` env var for per-variant test exclusion

✅ Ref YAML declaration (`interop-tests-ocs-tests-ref.yaml`)

✅ Shell script usage (`interop-tests-ocs-tests-commands.sh`)

✅ Config placement (`stolostron-policy-collection-main__ocp4.22.yaml`)

openshift-ci-robot commented Aug 7, 2026 •

edited by openshift-ci Bot

Loading