ceph-devstack start failing on Fedora 39

[amathuria]

(venv) aishwaryamathuria:teuthology$ ceph-devstack doctor
(venv) aishwaryamathuria:teuthology$ ceph-devstack pull
Pulling images...
(venv) aishwaryamathuria:teuthology$ ceph-devstack start
Creating containers...
Starting containers...
Traceback (most recent call last):
  File "/home/aishwaryamathuria/teuthology/venv/bin/ceph-devstack", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/aishwaryamathuria/teuthology/venv/lib64/python3.12/site-packages/ceph_devstack/cli.py", line 40, in main
    asyncio.run(run())
  File "/usr/lib64/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/aishwaryamathuria/teuthology/venv/lib64/python3.12/site-packages/ceph_devstack/cli.py", line 37, in run
    await obj.apply(args.command)
  File "/home/aishwaryamathuria/teuthology/venv/lib64/python3.12/site-packages/ceph_devstack/resources/ceph/__init__.py", line 167, in apply
    return await getattr(self, action)()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/aishwaryamathuria/teuthology/venv/lib64/python3.12/site-packages/ceph_devstack/resources/ceph/__init__.py", line 207, in start
    await kind(name=name).start()
  File "/home/aishwaryamathuria/teuthology/venv/lib64/python3.12/site-packages/ceph_devstack/resources/container.py", line 115, in start
    await self.cmd(
  File "/home/aishwaryamathuria/teuthology/venv/lib64/python3.12/site-packages/ceph_devstack/resources/__init__.py", line 71, in cmd
    raise CalledProcessError(cmd=args, returncode=returncode)
subprocess.CalledProcessError: Command '['podman', 'container', 'start', 'testnode_1']' returned non-zero exit status 125.

[RakshithaMH24]

I’d like to work on fixing this start failure on Fedora 39.
Could you please assign this issue to me if it’s still open?

[dnyaneshwaripachankar-droid]

Hi, I would like to work on this issue. Please assign it to me.

System had kernel.pid_max set below recommended value.
After setting kernel.pid_max=4194304, container startup issues reduced.

[Nikita-baghela07]

@amathuria
Common Causes of podman start … exit status 125

• Container already running → podman refuses to “start” it again.
• Container doesn’t exist → name mismatch or it was removed.
• Storage/network conflicts → leftover volumes, ports, or mounts from a previous run.
• SELinux / cgroups issues, especially on Fedora/Ubuntu with podman + rootless mode.
• Podman vs Docker mismatch → ceph‑devstack assumes podman is available and configured.

Quick Fix Checklist

Run podman ps -a → confirm container state.
If the container exists but is broken, then podman rm testnode_1 and restart devstack.
If podman itself is misconfigured, check podman info and fix cgroups/storage.
If you want to avoid podman issues, switch ceph‑devstack to Docker.