Skip to content

feat: API Server - Pipeline runs - Mirror Pipeline Run Annotations when creating a new pipeline run#216

Merged
yuechao-qin merged 1 commit intomasterfrom
ycq/feat-mirror-pipeline-run-annotations
Apr 22, 2026
Merged

feat: API Server - Pipeline runs - Mirror Pipeline Run Annotations when creating a new pipeline run#216
yuechao-qin merged 1 commit intomasterfrom
ycq/feat-mirror-pipeline-run-annotations

Conversation

@yuechao-qin
Copy link
Copy Markdown
Collaborator

@yuechao-qin yuechao-qin commented Apr 19, 2026

TL;DR

Closes https://github.com/Shopify/oasis-backend/issues/221

User-provided annotations are now mirrored into the PipelineRunAnnotation table at pipeline run creation time, and set_annotation now applies truncation and None-to-empty-string coercion via a shared helper.

What changed?

  • Introduced _mirror_single_annotation, a helper that centralizes annotation write logic: guards against system-key prefix collisions, coerces None values to empty strings, truncates values to the VARCHAR max length, and upserts the row.
  • Introduced _mirror_pipeline_run_annotations, which iterates over a user-supplied annotations dict and delegates each entry to _mirror_single_annotation.
  • _mirror_system_annotations now accepts an optional annotations dict and calls _mirror_pipeline_run_annotations so user annotations are persisted alongside system annotations at creation time.
  • create passes annotations through to _mirror_system_annotations.
  • set_annotation now delegates to _mirror_single_annotation instead of directly constructing and merging a PipelineRunAnnotation, gaining truncation and coercion for free.
  • The previously documented known failure for set_annotation overflowing the VARCHAR column is resolved — long values are now silently truncated rather than raising an IntegrityError.

How to test?

Run the updated test suite:

uv run pytest tests/test_api_server_sql.py

Why make this change?

Previously, user-provided annotations passed to create were not written to the PipelineRunAnnotation table, making them invisible to filter queries. Additionally, set_annotation lacked the same truncation and coercion guards applied to system annotations, causing potential IntegrityError failures on long values. Centralizing annotation write logic into _mirror_single_annotation ensures consistent behavior across all annotation write paths.

Copy link
Copy Markdown
Collaborator Author

@yuechao-qin yuechao-qin marked this pull request as ready for review April 19, 2026 10:37
@yuechao-qin yuechao-qin requested a review from Ark-kun as a code owner April 19, 2026 10:37
Comment thread cloud_pipelines_backend/api_server_sql.py Outdated
@Ark-kun Ark-kun changed the title feat: Mirror Pipeline Run Annotations feat: API Server - Pipeline runs - Mirror Pipeline Run Annotations when creating a new pipeline run Apr 19, 2026
@yuechao-qin yuechao-qin force-pushed the ycq/feat-mirror-pipeline-run-annotations branch from 9a4ee75 to e804eeb Compare April 22, 2026 13:41
@yuechao-qin yuechao-qin merged commit c571775 into master Apr 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants