test_smus

Sandbox to validate a CI/CD-less TEST → PROD promotion model for SageMaker Unified Studio (SMUS) projects, without Jenkins and without hardcoding project-specific values into code.

The experiment

Two SMUS projects bind to one repo, one branch each:

SMUS project	branch
odin-datenexperten-test	test
odin-datenexperten-prod	prod

Promotion = git merge test → prod. No pipeline, no servers.

What we want to prove:

1. Branch-per-project binding — can two SMUS projects point at the same repo on different branches? (SMUS-console step: connect each project's code connection to this repo + its branch.)
2. Project-independent code — notebooks/scripts contain no project-specific values (no S3 paths, no account ids). Everything project-specific is resolved at runtime from config/environments.json, keyed by the current project.
3. Glue job creation from a notebook via boto3 — jobs are not versioned (they drift and aren't synced by SMUS). Instead deploy/create_glue_jobs.py (re)creates them per project from the versioned scripts, injecting the right per-project defaults (wheel S3 path, etc.).

Layout

config/environments.json   # per-project config (test + prod), the ONLY place paths live
src/config_loader.py       # resolves "which project am I" -> its config entry
notebooks/example_job.py    # project-independent job logic (no hardcoded paths)
deploy/create_glue_jobs.py  # boto3: create/update Glue jobs from scripts per project

The --additional-python-modules chicken-and-egg

%%configure must know the wheel S3 path before Spark starts, so it can't be read from a runtime config. We resolve it before runtime instead:

• Glue jobs: deploy/create_glue_jobs.py sets --additional-python-modules / the wheel path as a job default argument per project at creation time.
• Notebooks: derive the path from something known pre-Spark — the project's own bucket follows the convention amazon-sagemaker-<account>-<region>-<projectId>/shared/…, so the wheel URI is computable from the bucket the notebook already runs in, no runtime config needed.

This repo is a test harness, not production code.