checkout-data-sync

A Dockerized CLI that migrates commercetools Checkout resources between projects — for example when moving between cloud providers or regions.

• What it does
• Prerequisites
• Quick start
• Configuration
  • Option A — config file
  • Option B — environment variables
  • Required resource fields
• Running the tool
  • With Docker
  • Without Docker
• CLI reference
• Examples
• Sync behaviour
  • Applications
  • Payment integrations
  • Error handling
  • Sample output
• Scopes

What it does

Synchronizes the following Checkout resource types from a source project to a target project, in this order:

1. Applications
2. Payment Integrations

Applications are synced first because Payment Integrations reference them by ID. The tool builds a sourceAppID → targetAppID map during the application phase and uses it to rewrite references before touching payment integrations.

By default the tool runs in dry-run mode — it prints a plan of what would change without writing anything. Pass -f to execute the migration.

Prerequisites

• Docker (to run the published image) or Go 1.21+ (to build from source)
• A commercetools API client with Checkout permissions on both the source and target projects (see Scopes)
• Either a config.yml file or SOURCE_* / TARGET_* environment variables (see Configuration)

Quick start

# 1. Pull the image
docker pull gcr.io/ct-images/checkout/data-sync:latest

# 2. Create your config
cp config.example.yml config.yml
# …then edit config.yml with your project credentials

# 3. Dry-run to preview the plan
docker run --rm -v $(pwd)/config.yml:/app/config.yml \
  gcr.io/ct-images/checkout/data-sync:latest

# 4. Execute the migration
docker run --rm -v $(pwd)/config.yml:/app/config.yml \
  gcr.io/ct-images/checkout/data-sync:latest -f

Configuration

The tool reads its source/target credentials and deployment mapping from either a YAML config file (via -c) or environment variables. Pick one.

Option A — config file

Copy the example and fill in your credentials:

cp config.example.yml config.yml

source:
  project_key: "source-project-key"
  client_id: "sourceClientId"
  client_secret: "sourceClientSecret"
  auth_url: "https://auth.europe-west1.gcp.commercetools.com/oauth/token"
  checkout_api_url: "https://checkout.europe-west1.gcp.commercetools.com"
  scopes: "manage_project"

target:
  project_key: "target-project-key"
  client_id: "targetClientId"
  client_secret: "targetClientSecret"
  auth_url: "https://auth.eu-central-1.aws.commercetools.com/oauth/token"
  checkout_api_url: "https://checkout.eu-central-1.aws.commercetools.com"
  scopes: ""

# Map source connector deployment UUIDs to target deployment UUIDs.
# Required for every Payment Integration that has a connectorDeployment.
# Connector deployment IDs are environment-specific and will not resolve
# across cloud providers without an explicit mapping.
deployment_mapping:
  "source-deployment-uuid": "target-deployment-uuid"

Note: auth_url and checkout_api_url must not have a trailing slash.

Option B — environment variables

When -c is omitted, the tool reads configuration from the environment. Each YAML field maps to an env var prefixed with SOURCE_ or TARGET_:

YAML field	Env var (per project)	Required
project_key	SOURCE_PROJECT_KEY / TARGET_PROJECT_KEY	yes
client_id	SOURCE_CLIENT_ID / TARGET_CLIENT_ID	yes
client_secret	SOURCE_CLIENT_SECRET / TARGET_CLIENT_SECRET	yes
auth_url	SOURCE_AUTH_URL / TARGET_AUTH_URL	yes
checkout_api_url	SOURCE_CHECKOUT_API_URL / TARGET_CHECKOUT_API_URL	yes
scopes	SOURCE_SCOPES / TARGET_SCOPES	no

DEPLOYMENT_MAPPING is read as a JSON object:

export DEPLOYMENT_MAPPING='{"source-deployment-uuid":"target-deployment-uuid"}'

If any required variable is missing, the tool exits with an error.

Copy .env.example to .env and fill in the values:

cp .env.example .env

Note: when using docker run --env-file, Docker does not strip surrounding quotes — write DEPLOYMENT_MAPPING={"src":"tgt"} (no '…' wrapping). When source-ing the file in a shell, the single quotes shown in .env.example are correct.

Required resource fields

The following fields must be set on source resources for them to sync:

Resource	Required fields
Application	key
Payment Integration	key (derived from source ID + name if absent — see Sync behaviour)

Running the tool

With Docker

Pull the published image (recommended):

docker pull gcr.io/ct-images/checkout/data-sync:latest

Or build it locally from source:

docker build -t checkout-data-sync .

Run it. Mount your config.yml at /app/config.yml — credentials must never be baked into the image.

# Dry-run (default)
docker run --rm -v $(pwd)/config.yml:/app/config.yml checkout-data-sync

# Execute the migration
docker run --rm -v $(pwd)/config.yml:/app/config.yml checkout-data-sync -f

Or with environment variables:

docker run --rm --env-file .env checkout-data-sync -f

Without Docker

go build -o checkout-data-sync .

# Config-file mode
./checkout-data-sync -c config.yml        # dry-run
./checkout-data-sync -c config.yml -f     # execute

# Env-var mode (requires SOURCE_*, TARGET_* exported in the shell)
set -a; source .env; set +a
./checkout-data-sync                      # dry-run
./checkout-data-sync -f                   # execute

CLI reference

usage: checkout-data-sync
 -c, --config <path>   Path to config file. When omitted, configuration
                       is read from environment variables (see
                       Configuration → Option B).
 -f, --full            Execute the migration. Omit to perform a dry-run
                       instead (shows what would be created or updated,
                       without making any changes).

Examples

Dry-run with a config file, showing the planned changes:

docker run --rm -v $(pwd)/config.yml:/app/config.yml checkout-data-sync

=== DRY RUN — pass -f to execute ===

--- Applications ---
Found 2 application(s) in source project "source-project-key"

  [CREATE] application "my-application"
  [SKIP]   application "existing-app" — already up to date

Applications: 1 to create, 0 to update, 1 skipped/errors

--- Payment Integrations ---
Found 3 payment integration(s) in source project "source-project-key"

  [CREATE] payment integration "credit-card-via-adyen"
  [UPDATE] payment integration "paypal" (target id: abc123, version: 2)
           • action: setStatus
  [SKIP]   payment integration "apple-pay" — already up to date

Payment integrations: 1 to create, 1 to update, 1 skipped/errors

Custom config path:

docker run --rm \
  -v $(pwd)/staging.yml:/app/staging.yml \
  checkout-data-sync -c /app/staging.yml -f

Inline environment variables (instead of --env-file):

docker run --rm \
  -e SOURCE_PROJECT_KEY=src-project \
  -e SOURCE_CLIENT_ID=... \
  -e SOURCE_CLIENT_SECRET=... \
  -e SOURCE_AUTH_URL=https://auth.europe-west1.gcp.commercetools.com/oauth/token \
  -e SOURCE_CHECKOUT_API_URL=https://checkout.europe-west1.gcp.commercetools.com \
  -e TARGET_PROJECT_KEY=tgt-project \
  -e TARGET_CLIENT_ID=... \
  -e TARGET_CLIENT_SECRET=... \
  -e TARGET_AUTH_URL=https://auth.eu-central-1.aws.commercetools.com/oauth/token \
  -e TARGET_CHECKOUT_API_URL=https://checkout.eu-central-1.aws.commercetools.com \
  -e DEPLOYMENT_MAPPING='{"source-deployment-uuid":"target-deployment-uuid"}' \
  checkout-data-sync -f

Sync behaviour

Applications

Applications are matched between source and target by their key.

Condition	Action
Key not found in target	CREATE
Key found, no fields differ	SKIP
Key found, fields differ	UPDATE

Update actions used: setName, setStatus, setDescription, setApplicationLogo, setCountries, setAllowedOrigins, setPaymentsConfiguration, setDiscountsConfiguration.

Agreements within an application are synced by name:

Condition	Action
Agreement name not in target	addAgreement
Agreement name not in source	removeAgreement
Agreement exists but differs	setAgreementName / setAgreementType / setAgreementStatus / setAgreementText

Payment integrations

Payment integrations are matched against target resources using a two-step lookup:

1. Exact key match — looks up the source key in the target index.
2. Name fallback — if no key match, looks up by name. When a name match is found with a different key, the target key is considered stale and a setKey action is prepended to the update. This handles key renames in the source project.

Note: If multiple target payment integrations share the same name, the name index entry is removed to prevent ambiguous matching. A missing key match in that case results in a new resource being created.

Condition	Action
No match by key or name	CREATE
Key match, no fields differ	SKIP
Key match, fields differ	UPDATE
Name match, key differs	UPDATE with setKey prepended

Update actions used: setKey, setName, setStatus, setComponentType, setPredicate, setDisplayInfo, setSortingInfo, setAutomatedReversalConfiguration, setConnectorDeployment.

Key derivation for keyless integrations. When a payment integration has no key, one is derived from the combination of its source ID and name: {sanitised-id}-{sanitised-name}. Using the source ID as a prefix guarantees that two integrations sharing the same display name receive distinct keys.

Connector deployment mapping. Connector deployment IDs are environment-specific. Any payment integration that has a connectorDeployment must have its source deployment UUID listed in deployment_mapping. Without a mapping entry the integration is skipped with a clear error:

payment integration "credit-card-via-adyen": connectorDeployment "src-uuid" has no entry
in deployment_mapping — add it under deployment_mapping in config.yml and retry

Error handling

Errors on individual resources are non-fatal. The tool logs each failure, continues processing the remaining resources, and exits with a non-zero status when any error occurred. A single bad resource never blocks the rest of the migration.

Sample output

=== MIGRATING resources (source → target) ===

--- Applications ---
Found 1 application(s) in source project "hello-johnson-test"

applications   0% |                                        | (0/1, 0 res/hr) [0s:0s]  [CREATE] application "sample-johnson"
           → created (id: 7936a42c-8d88-4ba9-b135-d9ed1b01438d)

Applications: 1 to create, 0 to update, 0 skipped/errors

--- Payment Integrations ---
Found 2 payment integration(s) in source project "hello-johnson-test"

payment-integrations   0% |                                        | (0/2, 0 res/hr) [0s:0s]  WARN: payment integration "9d99323c-5f8c-43f0-af0a-a56543a0c04b" has no key; derived key "9d99323c-5f8c-43f0-af0a-a56543a0c04b-credit-card"
  [CREATE] payment integration "9d99323c-5f8c-43f0-af0a-a56543a0c04b-credit-card"
           → created (id: cab9fbf3-84bf-4be7-8689-51b80274956e)

Payment integrations: 1 to create, 0 to update, 1 skipped/errors

ERROR during payment-integration sync: payment integration errors:
payment integration "helo-jay": cannot resolve target app ID for source app "ff3947a3-f9a6-429f-9614-7d875ccc9411"

=== source HTTP metrics ===
  4 requests (3.7 req/s over 1.08s)
  2xx=3  4xx=1  5xx=0  other=0  net-errors=0
  latency p50=94ms  p95=170ms  p99=170ms

=== target HTTP metrics ===
  5 requests (5.2 req/s over 970ms)
  2xx=5  4xx=0  5xx=0  other=0  net-errors=0
  latency p50=127ms  p95=457ms  p99=457ms
Error: sync completed with errors (see output above)
sync completed with errors (see output above)
exit status 1

Scopes

For least-privilege access, use the following scopes instead of manage_project:

Operation	Source scope	Target scope
Read applications	view_checkout_applications:{projectKey}	—
Write applications	—	manage_checkout_applications:{projectKey}
Read payment integrations	view_checkout_payment_integrations:{projectKey}	—
Write payment integrations	—	manage_checkout_payment_integrations:{projectKey}

---

Maintained by: ogwurujohnson