Skip to content

[set_containers] Add cifmw.general.set_containers module#3955

Open
arxcruz wants to merge 1 commit into
openstack-k8s-operators:mainfrom
arxcruz:feature/set-containers-module
Open

[set_containers] Add cifmw.general.set_containers module#3955
arxcruz wants to merge 1 commit into
openstack-k8s-operators:mainfrom
arxcruz:feature/set-containers-module

Conversation

@arxcruz

@arxcruz arxcruz commented May 22, 2026

Copy link
Copy Markdown
Contributor

Replace the update_containers role invocation in deploy_architecture with the new cifmw.general.set_containers module. The module generates and optionally applies an OpenStackVersion CR, accepting a dynamic images list where each entry can override registry, org, name_prefix, and tag individually. This removes the rigid Jinja2 template in favor of a Python module that is easier to test, extend, and call from any playbook without role-level variable coupling.

Key design points:

  • _OPENSTACK_SUFFIXES table enables partial overrides (e.g. tag-only) for any standard OpenStack image field without repeating the suffix.
  • Empty name_prefix is supported so images that ship without the "openstack-" prefix (EDPM, IPA) can be built from parts rather than requiring a pre-assembled full_registry URL.
  • backends list handles cinderVolumeImages / manilaShareImages nested dicts.
  • Unit tests cover _build_url, _resolve_image, _resolve_backend, _build_cr, and the full run_module flow including apply, absent, and validation paths.

@openshift-ci

openshift-ci Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign dasm for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/59cc46027da04af38a632c597c515613

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 18m 45s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 26m 14s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 41m 05s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 1h 59m 58s
✔️ cifmw-pod-zuul-files SUCCESS in 5m 08s
✔️ adoption-standalone-to-crc-ceph-provider SUCCESS in 3h 05m 41s
✔️ noop SUCCESS in 0s
cifmw-pod-ansible-test FAILURE in 6m 00s
cifmw-pod-pre-commit FAILURE in 8m 09s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 38s

@arxcruz arxcruz force-pushed the feature/set-containers-module branch from 4b08e81 to 7ff95ec Compare May 22, 2026 16:19
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/0eaf31714d824b02baf38086cb3ec17a

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 26m 34s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 26m 24s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 33m 28s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 1h 59m 17s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 49s
✔️ adoption-standalone-to-crc-ceph-provider SUCCESS in 3h 11m 41s
✔️ noop SUCCESS in 0s
cifmw-pod-ansible-test FAILURE in 6m 43s
cifmw-pod-pre-commit FAILURE in 9m 27s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 13s

@arxcruz arxcruz force-pushed the feature/set-containers-module branch 4 times, most recently from 507f3c3 to 20386f6 Compare May 23, 2026 08:35
Comment thread roles/cifmw_setup/tasks/deploy_architecture.yml Outdated
Comment thread roles/cifmw_setup/tasks/deploy_architecture.yml
Comment thread roles/cifmw_setup/tasks/deploy_architecture.yml
Comment thread plugins/modules/set_containers.py Outdated
Comment thread plugins/modules/set_containers.py
Comment thread plugins/modules/set_containers.py Outdated
Comment thread plugins/modules/set_containers.py Outdated
Comment thread plugins/modules/set_containers.py Outdated
Comment thread tests/unit/modules/test_set_containers.py Outdated
@arxcruz arxcruz force-pushed the feature/set-containers-module branch from 20386f6 to 069b3b9 Compare May 26, 2026 09:21
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/f0e6dcf146e549c69f1b8afa58e43841

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 21m 40s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 25m 48s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 34m 55s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 02m 06s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 53s
✔️ adoption-standalone-to-crc-ceph-provider SUCCESS in 3h 08m 45s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 02s
cifmw-pod-pre-commit FAILURE in 8m 33s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 08s

@jokke-ilujo jokke-ilujo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some concerns of the dest_path file handling. Other than that looks quite good on the first pass.

Comment thread plugins/modules/set_containers.py Outdated
cr_yaml = yaml.dump(cr, default_flow_style=False, sort_keys=False)

existing = None
if os.path.exists(dest_path):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not guarantee That the dest_path is a file nor that we actually can read it (it might be for example existing folder returning true). os.path.isabs and os.path.is_file checks with graceful module failure could mitigate the risks here quite a bit.

kubeconfig=dict(type="path", default=None),
)

module = AnsibleModule(argument_spec=module_args, supports_check_mode=True)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using ansible.module_utils.common.arg_spec.ArgumentSpecValidator here could be very useful to ensure we actually have correct looking arguments provided before moving forward.

Comment thread plugins/modules/set_containers.py Outdated
existing = None
if os.path.exists(dest_path):
with open(dest_path, "r") as fh:
existing = fh.read()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably should check here that the "existing" somewhat looks like CR, start by yaml parsing "is this file content actually yaml", etc. To make sure we don't overwrite some random file in the filesystem with the new CR.

Comment thread plugins/modules/set_containers.py Outdated
set_containers.run_module()
mock_rm.assert_called_once_with("/tmp/set_containers.yml")
self.assertTrue(cm.exception.args[0]["changed"])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests for various invalid 'dest_path's (no write permissions, directory, empty string "", invalid content (say /etc/hosts for example))

@arxcruz arxcruz force-pushed the feature/set-containers-module branch from 069b3b9 to cc1fa8f Compare May 28, 2026 12:51
module.fail_json(
msg="dest_path '{}' contains invalid YAML: {}".format(dest_path, exc)
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it stands now should also except at least IsADirectoryError, PermissionError and IOError; or OSError if we don't want to be descriptive of the nature of the error of which caused the open() or read() to fail.

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/7b41f5ae7f4b4805ab5ac925152da08f

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 42m 22s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 32m 58s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 31m 10s
cifmw-crc-podified-edpm-baremetal-minor-update FAILURE in 44m 48s
✔️ cifmw-pod-zuul-files SUCCESS in 6m 22s
adoption-standalone-to-crc-ceph-provider POST_FAILURE in 3h 20m 51s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 12m 30s
cifmw-pod-pre-commit FAILURE in 10m 49s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 10s

@evallesp evallesp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general!

# cifmw_update_containers_watcher -> cifmw_set_containers_extra_images
# cifmw_update_containers_edpm_image_url -> cifmw_set_containers_images (name_prefix: "")
# cifmw_update_containers_ipa_image_url -> cifmw_set_containers_images (name_prefix: "")
registry: "{{ cifmw_set_containers_registry | default(cifmw_update_containers_registry | default(omit)) }}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(blocking) possible error: I think this should be: {{ cifmw_set_containers_registry | default(cifmw_update_containers_registry) | default(omit) }}

@arxcruz arxcruz force-pushed the feature/set-containers-module branch from cc1fa8f to ed106cb Compare May 29, 2026 14:54
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/90cfef65b528431e9cea48d47736c51f

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 27m 03s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 23m 44s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 17s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 02m 19s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 34s
✔️ adoption-standalone-to-crc-ceph-provider SUCCESS in 3h 13m 35s
✔️ noop SUCCESS in 0s
cifmw-pod-ansible-test FAILURE in 5m 59s
cifmw-pod-pre-commit FAILURE in 8m 15s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 10s

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been for over 15 days with no activity.
Remove stale label or comment or this will be closed in 7 days.

@github-actions github-actions Bot added the Stale label Jun 14, 2026
@arxcruz arxcruz force-pushed the feature/set-containers-module branch from ed106cb to 21faf2f Compare June 16, 2026 08:47
#!/usr/bin/python

# Copyright: (c) 2026, Red Hat
# GNU General Public License v3.0+ (see COPYING or

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it should be Apache Licence

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/30705ac73b8645f29d0b3f795ff0a09f

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 46m 51s
podified-multinode-edpm-deployment-crc FAILURE in 41m 10s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 33m 48s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 13m 12s
✔️ cifmw-pod-zuul-files SUCCESS in 6m 24s
✔️ adoption-standalone-to-crc-ceph-provider SUCCESS in 3h 15m 32s
✔️ noop SUCCESS in 0s
cifmw-pod-ansible-test FAILURE in 7m 58s
cifmw-pod-pre-commit FAILURE in 9m 12s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 19s

@github-actions github-actions Bot removed the Stale label Jun 17, 2026
@arxcruz arxcruz force-pushed the feature/set-containers-module branch from 21faf2f to 0490f77 Compare June 17, 2026 08:23
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/f38c771a80b4477aa97441355a1a247d

✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 31m 03s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 22m 27s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 40m 30s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 13m 08s
✔️ cifmw-pod-zuul-files SUCCESS in 6m 21s
✔️ adoption-standalone-to-crc-ceph-provider SUCCESS in 3h 16m 08s
✔️ noop SUCCESS in 0s
cifmw-pod-ansible-test FAILURE in 6m 24s
✔️ cifmw-pod-pre-commit SUCCESS in 9m 31s
✔️ cifmw-molecule-cifmw_setup SUCCESS in 2m 13s

Replace the update_containers role invocation in deploy_architecture with
the new cifmw.general.set_containers module. The module generates and
optionally applies an OpenStackVersion CR, accepting a dynamic images list
where each entry can override registry, org, name_prefix, and tag
individually. This removes the rigid Jinja2 template in favor of a Python
module that is easier to test, extend, and call from any playbook without
role-level variable coupling.

Key design points:
- _OPENSTACK_SUFFIXES table enables partial overrides (e.g. tag-only) for
  any standard OpenStack image field without repeating the suffix.
- Empty name_prefix is supported so images that ship without the
  "openstack-" prefix (EDPM, IPA) can be built from parts rather than
  requiring a pre-assembled full_registry URL.
- backends list handles cinderVolumeImages / manilaShareImages nested dicts.
- Unit tests cover _build_url, _resolve_image, _resolve_backend, _build_cr,
  and the full run_module flow including apply, absent, and validation paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Arx Cruz <arxcruz@redhat.com>
@arxcruz arxcruz force-pushed the feature/set-containers-module branch from 0490f77 to acdfc24 Compare June 17, 2026 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants