fix(packaging): include runtime setup scripts in Python wheel

[Takeshi Masukawa (takeaship)]

fix(packaging): include runtime setup scripts in Python wheel

TL;DR

The published Python wheel omits the runtime setup scripts that RuntimeManager expects, so apm runtime setup fails before running any installer. This change makes those scripts package resources, loads them with importlib.resources, and retains the existing PyInstaller layout. A regression test builds and installs the wheel into an isolated venv, then loads every supported setup script and the Unix token helper from that installed wheel.

Note

This is related to closed issue #56, but the Linux reproduction below shows that the underlying failure is wheel packaging rather than Windows support.

Problem (WHY)

• In a clean Linux Python 3.13 container, the official apm-cli==0.29.0 wheel reports RUNTIME_SCRIPT_ENTRIES=0 and TOKEN_HELPER_ENTRIES=0; apm runtime setup llm exits 1 with Script not found: setup-llm.sh.
• The supported install surface explicitly documents pip install apm-cli, while the runtime guide documents apm runtime setup llm. The wheel therefore needs the same runtime assets as the source and binary distributions.
• Issue #56 reported the same Failed to load embedded script setup-copilot.sh: Script not found: setup-copilot.sh symptom after uv tool install; it was closed after the discussion treated the failure as Windows-specific. The Linux result reproduces the same missing-resource path.

Approach (WHAT)

• Move the canonical assets under apm_cli.runtime, declare them as package data, and load them with importlib.resources from source or wheel installs.
• Keep the established PyInstaller _MEIPASS/scripts/... lookup and emitted layout; only point the build spec at the new canonical source location.
• Exercise the built artifact by installing the wheel outside the repository and loading every resource through the installed RuntimeManager.

Implementation (HOW)

Files	Change
src/apm_cli/runtime/scripts/**, pyproject.toml, src/apm_cli/runtime/manager.py	Move the unchanged assets into the package, include them as package data, and load them with importlib.resources.
.github/workflows/build-release.yml, tests/unit/scripts/test_build_release_workflow.py	Include the relocated helper in release artifacts and preserve its relative layout in isolated macOS validation; exercise both workflow preparation blocks.
build/apm.spec	Reads assets from the package tree but still emits scripts/runtime and scripts/github-token-helper.sh in the frozen application.
scripts/test-{integration,release-validation}.sh, tests/integration/{test_runtime_smoke,test_tls_r3_verify}.py, tests/utils/runtime_setup_codex.py, docs/src/content/docs/integrations/runtime-compatibility.md	Update source-tree references and contributor guidance to the canonical path.
tests/integration/test_runtime_wheel_scripts.py, existing runtime-manager tests	Add the installed-wheel regression and make existing resource assertions mandatory.

Diagrams

Legend: the dashed nodes are the package-resource path added by this PR; the frozen path keeps its current runtime layout.

flowchart LR
    A[src/apm_cli/runtime/scripts]
    B[setuptools package data]
    C[installed apm_cli.runtime]
    D{sys.frozen}
    E[MEIPASS scripts]
    F[importlib.resources]
    G[RuntimeManager]
    A --> B
    B --> C
    C --> F
    D -->|yes| E
    D -->|no| F
    E --> G
    F --> G
    classDef new stroke-dasharray: 5 5;
    class A,B,C,F new;

Loading

Trade-offs

• Relocate instead of duplicate. The assets live once under src/apm_cli/runtime/scripts; a second checked-in copy or a custom build-time copy hook would create drift or extra packaging logic.
• Keep the frozen branch. The PyInstaller lookup remains explicit instead of assuming identical importlib.resources behavior inside every frozen build, preserving the supported binary path and its existing filesystem layout.
• Platform scope is honest. Linux wheel and PyInstaller paths were exercised. Windows was not available; both PowerShell variants are included and loaded by the wheel regression, but a native Windows wheel/PyInstaller execution was not run.

Benefits

1. The wheel contains 10 runtime setup/common scripts and one Unix token helper instead of zero.
2. An installed-wheel RuntimeManager loads every script without access to the source checkout, and apm runtime setup llm completes instead of failing with Script not found.
3. Source execution and the existing PyInstaller scripts/... layout continue to use the same canonical assets.

Validation

Latest rebase (c7f5dd58), author-reported checks: all four targeted tests passed (7.15s), along with Ruff, pylint R0801, auth/architecture checks, and git diff --check. The broader test run was interrupted; full-suite results below refer to earlier revisions.

Official 0.29.0 wheel baseline

The earlier rootless Podman reproduction used docker.io/library/python:3.13, a fresh venv, no host mounts, and no host credentials:

python -m venv /tmp/apm-venv
. /tmp/apm-venv/bin/activate
python -m pip install --disable-pip-version-check --no-cache-dir -q apm-cli==0.29.0
apm --version
apm --help
apm runtime setup llm

RUNTIME_SCRIPT_ENTRIES=0
TOKEN_HELPER_ENTRIES=0
DIRECT_LOAD_ERROR=RuntimeError: Could not load setup script: setup-llm.sh
[x] Failed to load embedded script setup-llm.sh: Script not found: setup-llm.sh
[x] Error setting up llm: Could not load setup script: setup-llm.sh
RUNTIME_SETUP_EXIT=1

PR wheel in disposable Python 3.13 container

verify_dir=$(mktemp -d /tmp/apm-wheel-pr.XXXXXX)
uv build --wheel --out-dir "$verify_dir"
sha256sum "$verify_dir"/*.whl

podman create --rm --name apm-wheel-pr-verify-20260903 \
  docker.io/library/python:3.13 sh /tmp/apm-wheel-podman-verify.sh
podman cp "$verify_dir/apm_cli-0.29.0-py3-none-any.whl" \
  apm-wheel-pr-verify-20260903:/tmp/apm_cli-0.29.0-py3-none-any.whl
podman cp /tmp/apm-wheel-podman-verify.sh \
  apm-wheel-pr-verify-20260903:/tmp/apm-wheel-podman-verify.sh
podman inspect apm-wheel-pr-verify-20260903 \
  --format 'IMAGE={{.ImageName}} MOUNTS={{json .Mounts}} ENV={{json .Config.Env}}'
podman start -a apm-wheel-pr-verify-20260903

The copied /tmp/apm-wheel-podman-verify.sh ran these probes inside the container (each command's exit code is recorded below):

git --version
python -m venv /tmp/apm-venv
. /tmp/apm-venv/bin/activate
python -m pip install --disable-pip-version-check --no-cache-dir -q \
  /tmp/apm_cli-0.29.0-py3-none-any.whl
apm --version
apm --help
python - <<'PY'
import importlib.metadata
from apm_cli.runtime.manager import RuntimeManager
from apm_cli.runtime.registry import runtime_descriptors
files = [str(path) for path in importlib.metadata.files("apm-cli") or ()]
print([path for path in files if path.startswith("apm_cli/runtime/scripts/runtime/")])
print([path for path in files if path == "apm_cli/runtime/scripts/github-token-helper.sh"])
manager = RuntimeManager()
names = {f"{item.setup_script}{ext}" for item in runtime_descriptors() for ext in (".sh", ".ps1")}
names |= {"setup-common.sh", "setup-common.ps1"}
print({name: len(manager.get_embedded_script(name)) for name in sorted(names)})
print(len(manager.get_token_helper_script()))
PY
apm runtime setup llm
/root/.apm/runtimes/llm --version

Artifact and container results

SHA256=de465925470682e2eff7ec7450006e9b93ea250378f79ccb310f7268b2c002be
IMAGE=docker.io/library/python:3.13 MOUNTS=[]
git version 2.47.3
GIT_EXIT=0
VENV_EXIT=0
PIP_INSTALL_EXIT=0
Agent Package Manager (APM) CLI version 0.29.0
APM_VERSION_EXIT=0
APM_HELP_EXIT=0
MANAGER_MODULE=/tmp/apm-venv/lib/python3.13/site-packages/apm_cli/runtime/manager.py
RUNTIME_SCRIPT_ENTRIES=10
TOKEN_HELPER_ENTRIES=1
RESOURCE_PROBE_EXIT=0
RUNTIME_SETUP_LLM_EXIT=0
llm, version 0.33
INSTALLED_LLM_VERSION_EXIT=0

LOADED_SCRIPTS contained non-empty content for setup-{codex,copilot,gemini,llm}.{sh,ps1} and setup-common.{sh,ps1}; TOKEN_HELPER_LENGTH=4092. No host home was mounted, no host credentials were passed, all downloads and writes stayed inside the disposable container, and --rm removed it after exit.

Original automated checks (2026-09-03)

$ uv run --extra dev pytest tests/integration/test_runtime_wheel_scripts.py -n0 -q
1 passed in 9.37s
$ uv run pytest tests/unit tests/test_console.py -x
20849 passed, 3 skipped, 21 xfailed, 20 warnings in 402.44s (0:06:42)
$ uv run --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/
All checks passed!
$ uv run --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/
1774 files already formatted
$ uv run --frozen --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/
Your code has been rated at 10.00/10

The CI YAML I/O, file-length, path, auth, and architecture guards, plus git diff --check and touched-shell bash -n, all exited 0.

PyInstaller compatibility check

uv run --extra build pyinstaller --noconfirm --clean \
  --distpath /tmp/apm-pyinstaller-check-dist \
  --workpath /tmp/apm-pyinstaller-check-work build/apm.spec
test -x /tmp/apm-pyinstaller-check-dist/apm/apm
for asset in runtime/setup-codex.sh runtime/setup-common.ps1 github-token-helper.sh; do
  test -f "/tmp/apm-pyinstaller-check-dist/apm/_internal/scripts/$asset"
done
/tmp/apm-pyinstaller-check-dist/apm/apm --version

PYINSTALLER_BUILD_EXIT=0
ASSET_ASSERTIONS_EXIT=0
Agent Package Manager (APM) CLI version 0.29.0
APM_VERSION_EXIT=0

Revalidation after rebase (2026-09-06)

Rebased onto 9cb174b2; reviewed through 1fe622a1. Searches across open/closed Issues and PRs found no competing wheel fix; #56 remains closed. Independent review found a missed release-workflow helper path; it is now fixed and re-reviewed.
Rebased 0.29.1 verification passed: wheel CLI version/help, metadata and all-resource probes, full LLM setup, and a Linux PyInstaller build/version/help plus byte comparisons of all 11 frozen assets. A further Python 3.13 Podman run copied only the frozen distribution (no checkout or credentials) and completed apm runtime setup llm plus llm --version (0.34), exit 0. Native macOS/Windows execution remains untested.

New verification commands and results (historical logs above retained)

uv build --wheel --out-dir .git/apm-reverify-20260906/wheel
podman create --rm --name apm-wheel-reverify-20260906 docker.io/library/python:3.13 sh /tmp/container.sh
podman cp .git/apm-reverify-20260906/wheel/apm_cli-0.29.1-py3-none-any.whl apm-wheel-reverify-20260906:/tmp/apm_cli-0.29.1-py3-none-any.whl
podman cp .git/apm-reverify-20260906/container.sh apm-wheel-reverify-20260906:/tmp/container.sh
podman start -a apm-wheel-reverify-20260906

The copied script runs the same in-container probes shown above against 0.29.1, asserting 10 runtime entries, one helper, and nonempty resources. No mounts or host credentials; container removed on exit. Wheel SHA256: 8c6c947fef7006a3a4d5ee4c57a5ba6906ccad955246d7a27966de786fac0351.
All container commands exited 0, including apm runtime setup llm; installed llm, version 0.34. Metadata contained 10 runtime scripts and one helper; all loaded, with TOKEN_HELPER_LENGTH=4092.

• uv run pytest tests/unit tests/test_console.py -x: 21355 passed, 5 skipped, 21 xfailed, 21 warnings in 521.98s; exit 0.
• uv run --extra dev pytest tests/integration/test_runtime_wheel_scripts.py -n0 -q: 1 passed in 11.50s; exit 0. The same test, building unmodified upstream 9cb174b2 instead, fails with Could not load setup script: setup-copilot.sh (1 failed in 8.78s; exit 1).
• uv run --no-sync pytest tests/unit/scripts/test_build_release_workflow.py -n0 -q: 3 passed in 2.20s; exit 0. All 3 fail against the pre-fix workflow (exit 1).
• Runtime/resource/TLS/build-spec focused tests: 96 passed; exit 0. Full CI lint mirror, architecture guards, shell syntax, and Mermaid validation: exit 0.
• GitHub Actions require maintainer approval (action_required); local passes do not claim hosted CI is green.

Scenario Evidence

#	Scenario (user promise)	Principle(s)	Test(s) proving it	Type
1	Install the APM wheel and run runtime setup without a source checkout; every required script is available.	DevX (pragmatic as npm)	tests/integration/test_runtime_wheel_scripts.py::test_installed_wheel_loads_all_runtime_scripts (regression-trap for #56)	integration
2	Run APM from source or from the supported frozen binary without changing the setup-script behavior or layout.	DevX (pragmatic as npm), OSS / community-driven	tests/unit/test_runtime_manager.py::TestRuntimeManagerGetEmbeddedScript tests/unit/test_build_spec.py::TestSpecFileSyntax	unit
3	Validate release binaries outside a source checkout with the required helper available.	OSS / community-driven	tests/unit/scripts/test_build_release_workflow.py	unit

How to test

• Run uv run --extra dev pytest tests/integration/test_runtime_wheel_scripts.py tests/unit/scripts/test_build_release_workflow.py -n0 -q; expect four passes covering the installed wheel and release-validation workflow.
• Inspect that wheel; expect 10 files under apm_cli/runtime/scripts/runtime/ and one helper under apm_cli/runtime/scripts/, all loadable by RuntimeManager.
• On Linux, run apm runtime setup llm; expect execution to proceed beyond the former Script not found failure. A full network-backed LLM setup completed in the disposable verification above.

Warning

Untested scope: native Windows/macOS execution and network-backed Copilot, Codex, and Gemini installation. The regression loads their shell and PowerShell assets, and Linux PyInstaller was built and smoke-tested, but those external installers were not executed.

Python 3.10 limitation: a clean docker.io/library/python:3.10 venv installs the wheel, but apm --version exits 1 with ImportError: cannot import name 'Self' from 'typing' in unchanged models/dependency/provider_coordinates.py:5. Reproduced with both this PR's wheel and a wheel built from unmodified upstream 9cb174b2; this pre-existing startup failure prevents Python 3.10 end-to-end validation and is outside the packaging fix.

apm-spec-waiver: Packaging fix makes existing runtime setup assets available from installed wheels; no OpenAPM behavior changes.

AI assistance disclosure: OpenAI Codex helped implement, review, and validate this change; the author reviewed the diff and all reported command results.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Takeshi Masukawa (takeaship)]

@microsoft-github-policy-service agree

[Takeshi Masukawa (takeaship)]

Daniel Meppiel (@danielmeppiel)
Sergio Sisternes (@sergio-sisternes-epam)
Could you review this PR and approve the pending workflow runs?

[Copilot]

🔵 Needs a closer look

Newly packaged runtime helper/scripts contain non-ASCII status glyphs in output strings; these should be converted to ASCII-only prefixes before shipping them via the wheel.

Pull request overview

Fixes wheel packaging so apm runtime setup can load its embedded setup scripts when APM is installed from a published wheel, while preserving the existing PyInstaller _MEIPASS/scripts/... layout for frozen builds.

Changes:

• Moves runtime setup assets under apm_cli.runtime, declares them as package data, and switches RuntimeManager to importlib.resources lookups for non-frozen execution.
• Updates PyInstaller spec + release workflow isolated-validation prep to source/copy the helper from the new canonical location.
• Adds regression tests that (a) build+install a wheel into an isolated venv and verify all expected scripts load via RuntimeManager, and (b) verify the release workflow uploads/prepares the canonical helper path.

File summaries

File	Description
tests/utils/runtime_setup_codex.py	Updates test helper to reference the new canonical setup script location under src/apm_cli/runtime/scripts/runtime/.
tests/unit/test_runtime_manager.py	Simplifies script-loading test to assert package-resource loading works.
tests/unit/scripts/test_build_release_workflow.py	New tests asserting build-release workflow artifacts and macOS isolated validation prep preserve the helper layout.
tests/integration/test_tls_r3_verify.py	Updates TLS script assertions to read from the new packaged script path.
tests/integration/test_runtime_wheel_scripts.py	New regression test: build wheel, install outside repo, and verify all runtime scripts + token helper load from the installed wheel.
tests/integration/test_runtime_smoke.py	Updates smoke tests to use the new canonical script paths in-tree.
tests/integration/test_remaining_modules_coverage.py	Makes runtime script loading assertions mandatory (no longer “acceptable to be missing”).
tests/integration/test_integration_runtime_coverage.py	Updates common/token-helper tests to assert package-resource content is present.
src/apm_cli/runtime/scripts/runtime/setup-llm.sh	Adds/moves LLM runtime setup script into package resources.
src/apm_cli/runtime/scripts/runtime/setup-llm.ps1	Adds/moves Windows LLM setup script into package resources.
src/apm_cli/runtime/scripts/runtime/setup-gemini.sh	Adds/moves Gemini runtime setup script into package resources.
src/apm_cli/runtime/scripts/runtime/setup-gemini.ps1	Adds/moves Windows Gemini setup script into package resources.
src/apm_cli/runtime/scripts/runtime/setup-copilot.sh	Moves Copilot setup script into package resources (content unchanged in this hunk).
src/apm_cli/runtime/scripts/runtime/setup-copilot.ps1	Adds/moves Windows Copilot setup script into package resources.
src/apm_cli/runtime/scripts/runtime/setup-common.sh	Adds/moves common bash helpers into package resources.
src/apm_cli/runtime/scripts/runtime/setup-common.ps1	Adds/moves common PowerShell helpers into package resources.
src/apm_cli/runtime/scripts/runtime/setup-codex.sh	Adds/moves Codex setup script into package resources.
src/apm_cli/runtime/scripts/runtime/setup-codex.ps1	Adds/moves Windows Codex setup script into package resources.
src/apm_cli/runtime/scripts/github-token-helper.sh	Adds/moves token helper into package resources so release validation and runtime scripts can source it from the canonical location.
src/apm_cli/runtime/manager.py	Switches non-frozen script loading to importlib.resources.files(...) and keeps _MEIPASS behavior for frozen builds.
scripts/test-release-validation.sh	Updates sourcing path to the canonical packaged token helper.
scripts/test-integration.sh	Updates sourcing path and comment to reflect the canonical packaged token helper.
pyproject.toml	Declares apm_cli.runtime scripts as wheel package data.
docs/src/content/docs/integrations/runtime-compatibility.md	Updates docs to point contributors to the new canonical setup-script path.
build/apm.spec	Updates PyInstaller datas to pull scripts/helper from the new canonical source location while emitting the same frozen layout.
.github/workflows/build-release.yml	Updates artifact upload and macOS isolated release-validation prep to include/copy the helper from the new canonical location.

Review details

• Files reviewed: 15/26 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Takeshi Masukawa (takeaship)]

These characters already exist in the original scripts; this PR relocates them without changing their behavior. I'd prefer to address ASCII-only output separately to keep this fix focused on wheel packaging.