-
Notifications
You must be signed in to change notification settings - Fork 201
Description
Summary
- Context: The
Pullfunction inpkg/remote/pull.godownloads agent artifacts from OCI registries and stores them locally using a content store. - Bug: When pulling an artifact using a digest-based reference (e.g.,
myregistry.io/org/app@sha256:abc...), the code constructs an invalid local reference format. - Actual vs. expected: The code produces
org/app:sha256:abc...(missing registry host, wrong separator) instead of preserving the original referencemyregistry.io/org/app@sha256:abc.... - Impact: Pulling any artifact by digest fails with a parsing error from
crane.Save(), making digest-based references completely unusable.
Code with bug
localRef := ref.Context().RepositoryStr() + ":" + ref.Identifier() // <-- BUG 🔴 Creates malformed reference for digest-based refsThe bug is on line 32 of pkg/remote/pull.go. The code has two problems:
ref.Context().RepositoryStr()returns only the repository path (e.g.,org/app) without the registry host (e.g.,myregistry.io)- The hardcoded
:separator is incorrect for digest-based references, which should use@
Evidence
Example
Consider pulling an artifact by digest:
Input reference: myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
Step-by-step execution:
- Line 17:
name.ParseReference()successfully parses the digest-based reference - Line 32: The code constructs
localRef:ref.Context().RepositoryStr()returns"org/app"(missingmyregistry.io)ref.Identifier()returns"sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"- Result:
localRef = "org/app:sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
- Line 57: Calls
store.StoreArtifact(img, localRef)with the malformed reference - In
pkg/content/store.goline 78:crane.Save(img, reference, tarPath)attempts to parse the reference - Failure:
crane.Save()fails because"org/app:sha256:..."is not a valid OCI reference format (digest-based references must use@separator, not:)
Expected behavior: The code should use ref.String() which returns the complete, correctly-formatted reference: "myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
Failing test
Test script
package remote
import (
"testing"
"github.com/google/go-containerregistry/pkg/name"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestLocalRefConstruction tests that localRef is constructed correctly for both tag and digest references
func TestLocalRefConstruction(t *testing.T) {
tests := []struct {
name string
registryRef string
expectedLocalRef string
}{
{
name: "tag-based reference",
registryRef: "myregistry.io/org/app:latest",
expectedLocalRef: "myregistry.io/org/app:latest",
},
{
name: "digest-based reference",
registryRef: "myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
expectedLocalRef: "myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
},
{
name: "tag with registry port",
registryRef: "myregistry.io:5000/org/app:v1.0.0",
expectedLocalRef: "myregistry.io:5000/org/app:v1.0.0",
},
{
name: "digest with registry port",
registryRef: "myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234",
expectedLocalRef: "myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ref, err := name.ParseReference(tt.registryRef)
require.NoError(t, err)
// This is the buggy code from pull.go line 32
localRefBuggy := ref.Context().RepositoryStr() + ":" + ref.Identifier()
// Show what the code currently produces
t.Logf("Input: %s", tt.registryRef)
t.Logf("Expected: %s", tt.expectedLocalRef)
t.Logf("Current code: %s", localRefBuggy)
t.Logf("ref.String(): %s", ref.String())
// The bug: using RepositoryStr() loses the registry host
// and using ":" separator is wrong for digest-based references
assert.Equal(t, tt.expectedLocalRef, localRefBuggy,
"localRef should match the original reference format")
})
}
}Test output
=== RUN TestLocalRefConstruction
=== RUN TestLocalRefConstruction/tag-based_reference
pull_digest_test.go:49: Input: myregistry.io/org/app:latest
pull_digest_test.go:50: Expected: myregistry.io/org/app:latest
pull_digest_test.go:51: Current code: org/app:latest
pull_digest_test.go:52: ref.String(): myregistry.io/org/app:latest
pull_digest_test.go:56:
Error Trace: /home/user/cagent/pkg/remote/pull_digest_test.go:56
Error: Not equal:
expected: "myregistry.io/org/app:latest"
actual : "org/app:latest"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-myregistry.io/org/app:latest
+org/app:latest
Test: TestLocalRefConstruction/tag-based_reference
Messages: localRef should match the original reference format
=== RUN TestLocalRefConstruction/digest-based_reference
pull_digest_test.go:49: Input: myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
pull_digest_test.go:50: Expected: myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
pull_digest_test.go:51: Current code: org/app:sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
pull_digest_test.go:52: ref.String(): myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
pull_digest_test.go:56:
Error Trace: /home/user/cagent/pkg/remote/pull_digest_test.go:56
Error: Not equal:
expected: "myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
actual : "org/app:sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-myregistry.io/org/app@sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
+org/app:sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
Test: TestLocalRefConstruction/digest-based_reference
Messages: localRef should match the original reference format
=== RUN TestLocalRefConstruction/tag_with_registry_port
pull_digest_test.go:49: Input: myregistry.io:5000/org/app:v1.0.0
pull_digest_test.go:50: Expected: myregistry.io:5000/org/app:v1.0.0
pull_digest_test.go:51: Current code: org/app:v1.0.0
pull_digest_test.go:52: ref.String(): myregistry.io:5000/org/app:v1.0.0
pull_digest_test.go:56:
Error Trace: /home/user/cagent/pkg/remote/pull_digest_test.go:56
Error: Not equal:
expected: "myregistry.io:5000/org/app:v1.0.0"
actual : "org/app:v1.0.0"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-myregistry.io:5000/org/app:v1.0.0
+org/app:v1.0.0
Test: TestLocalRefConstruction/tag_with_registry_port
Messages: localRef should match the original reference format
=== RUN TestLocalRefConstruction/digest_with_registry_port
pull_digest_test.go:49: Input: myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234
pull_digest_test.go:50: Expected: myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234
pull_digest_test.go:51: Current code: org/app:sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234
pull_digest_test.go:52: ref.String(): myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234
pull_digest_test.go:56:
Error Trace: /home/user/cagent/pkg/remote/pull_digest_test.go:56
Error: Not equal:
expected: "myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234"
actual : "org/app:sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-myregistry.io:5000/org/app@sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234
+org/app:sha256:abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234
Test: TestLocalRefConstruction/digest_with_registry_port
Messages: localRef should match the original reference format
--- FAIL: TestLocalRefConstruction (0.00s)
--- FAIL: TestLocalRefConstruction/tag-based_reference (0.00s)
--- FAIL: TestLocalRefConstruction/digest-based_reference (0.00s)
--- FAIL: TestLocalRefConstruction/tag_with_registry_port (0.00s)
--- FAIL: TestLocalRefConstruction/digest_with_registry_port (0.00s)
FAIL
Full context
The Pull function is the primary mechanism for downloading agent artifacts from OCI registries. It's used in two main contexts:
- Command-line pulls: When users run
cagent pull docker.io/user/agent@sha256:...to download an agent artifact locally - Automatic pulls for OCI sources: When an agent configuration references an OCI artifact (via
pkg/config/sources.go), the system automatically pulls it before loading
The function performs these steps:
- Parses the registry reference
- Resolves the remote digest
- Checks if a local copy exists (using the malformed
localRef) - Pulls the image from the registry
- Stores it locally with the malformed reference
The bug occurs at step 3 and 5, where the code constructs an incorrect local reference. This causes the storage operation to fail at pkg/content/store.go:78 when crane.Save() attempts to parse the malformed reference.
The content store uses reference strings as lookup keys by hashing them (pkg/content/store.go:255-256). When the reference is malformed during storage, subsequent attempts to retrieve the artifact using the original (correct) reference will fail because the hashes don't match.
External documentation
type Reference interface {
Context() Repository
Identifier() string
Name() string
Scope(string) string
String() string
}
String returns the complete reference as a string, including registry, repository, and either tag or digest.
References can be by tag or digest:
- By tag: registry.example.com/repository/image:tag
- By digest: registry.example.com/repository/image@sha256:digest
The @ separator is required for digest-based references.
func Save(img v1.Image, src, path string) error
Save writes the v1.Image img as a tarball at path with tag src.
The src parameter must be a valid reference string.
Why has this bug gone undetected?
This bug has gone undetected because:
-
No digest-based reference testing: All existing tests in
pkg/remote/pull_test.gouse tag-based references (e.g.,test:latest,test-app:latest). No tests verify pulling by digest. -
Limited real-world usage: The feature appears to be relatively new (added in commit 64ade5e on July 10, 2025). Most users likely pull agents by tag rather than by digest.
-
Bug affects all reference types, but manifests differently:
- For digest-based references: Complete failure -
crane.Save()rejects the malformed reference formatorg/app:sha256:...immediately - For tag-based references: Partial failure - The reference
org/app:latestis technically valid, socrane.Save()succeeds, but the artifact is stored without the registry host. This causes subtle issues when trying to pull from registries with the same repository paths but different hosts (e.g.,registry1.io/org/app:latestvsregistry2.io/org/app:latestwould collide)
- For digest-based references: Complete failure -
-
Error may be attributed to network issues: When the pull command fails, users might assume it's a network problem or authentication issue rather than a bug in reference construction.
Recommended fix
Replace line 32 with:
localRef := ref.String() // <-- FIX 🟢 Preserves complete reference with correct formatThe ref.String() method returns the complete, correctly-formatted reference string that preserves:
- The registry host (e.g.,
myregistry.io) - The repository path (e.g.,
org/app) - The correct separator (
:for tags,@for digests) - The identifier (tag or digest)
This ensures that:
crane.Save()can successfully parse the reference- The artifact can be retrieved later using the same reference format
- Multiple registries with the same repository paths don't collide