Skip to content

Commit ec60f31

Browse files
peter-smith-phdquetzalliwrites
peter-smith-phd
and
quetzalliwrites
committed
Migrate CI pipeline docs to lstk (#879)
Co-authored-by: Quetzalli <hola@quetzalliwrites.com> Co-authored-by: Quetzalli <alejandra.olvera.novack@gmail.com>
1 parent 8b5ba36 commit ec60f31

9 files changed

Lines changed: 618 additions & 812 deletions

File tree

‎src/content/docs/aws/ci-pipelines/best-practices.mdx‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
---
2+
title: CI Best Practices
3+
description: Commands and general practices for running LocalStack in any CI system, from authentication and tool installation to seeding state and collecting logs.
4+
template: doc
5+
sidebar:
6+
order: 2
7+
---
8+
9+
import { Tabs, TabItem } from '@astrojs/starlight/components';
10+
11+
Every CI system has its own configuration syntax, runner model, and feature set.
12+
Consult your CI provider's own documentation for how to declare jobs, secrets, caches, and artifacts.
13+
This guide covers the parts that are the same everywhere, such as the LocalStack-specific commands you run, and the best practices for running a LocalStack job.
14+
15+
Whatever the provider, a CI job follows the same steps:
16+
17+
1. Expose your CI Auth Token to the job as `LOCALSTACK_AUTH_TOKEN`.
18+
2. Install `lstk` and any tools your tests need, such as the AWS CLI or Terraform.
19+
3. Configure and start the emulator with `lstk start`.
20+
4. Deploy your infrastructure, using an Infrastructure as Code tool.
21+
5. Alternatively, seed state from a snapshot with `lstk load`.
22+
6. Run your tests.
23+
7. Collect the emulator logs as a build artifact.
24+
25+
## Set your Auth Token
26+
27+
Every LocalStack CI run needs a [CI Auth Token](https://app.localstack.cloud/workspace/auth-tokens), rather than a personal Developer Auth Token.
28+
Store the token in your CI system as `LOCALSTACK_AUTH_TOKEN`.
29+
Every CI provider offers somewhere to keep sensitive values, and most distinguish secrets from plain environment variables.
30+
Secrets are masked in job logs and withheld from forked-repository builds.
31+
Never commit a token to your repository or paste it into a pipeline definition.
32+
33+
The `lstk` CLI tool automatically passes the `LOCALSTACK_AUTH_TOKEN` value into the emulator container when it starts.
34+
There is no need to invoke `lstk login`, which is only useful in an interactive session.
35+
36+
## Install the tools
37+
38+
Your job needs the `lstk` CLI, plus whichever AWS tooling your tests use.
39+
Many hosted runners already ship Docker, the AWS CLI, and Terraform, so check your runner image before adding an install step.
40+
41+
### `lstk`
42+
43+
[`lstk`](/aws/developer-tools/running-localstack/lstk/) is the recommended way to run and manage LocalStack.
44+
It is a single binary, so installing it in CI is quick with tools such as `npm` or `brew`.
45+
46+
<Tabs>
47+
<TabItem label="npm">
48+
49+
```bash
50+
npm install -g @localstack/lstk
51+
```
52+
53+
</TabItem>
54+
<TabItem label="Homebrew">
55+
56+
```bash
57+
brew install localstack/tap/lstk
58+
```
59+
60+
</TabItem>
61+
</Tabs>
62+
63+
See the [`lstk` installation guide](/aws/developer-tools/running-localstack/lstk/#installation) for all installation methods.
64+
`lstk` also needs a working Docker daemon on the runner, with access to a Docker socket so the emulator can spawn its own containers for services such as Lambda and ECS.
65+
66+
### AWS CLI
67+
68+
`lstk aws` proxies your host `aws` binary with the LocalStack endpoint, credentials, and region already configured, so the AWS CLI must be installed separately (if not already installed in your CI system).
69+
70+
Refer to the [AWS CLI installation instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for details, and to the [AWS CLI guide](/aws/connecting/aws-cli/) for using it against LocalStack.
71+
72+
### Terraform
73+
74+
`lstk terraform` drives the real `terraform` binary, so Terraform itself must be on the job's `PATH`.
75+
Install it with your provider's setup step where one exists (for example `hashicorp/setup-terraform` on GitHub Actions), or install it directly.
76+
77+
Refer to the [Terraform installation instructions](https://developer.hashicorp.com/terraform/install) for details, and to the [Terraform guide](/aws/connecting/infrastructure-as-code/terraform/) for using it against LocalStack.
78+
79+
## Configure the emulator
80+
81+
The `lstk` CLI tool uses a `config.toml` file to discover the required configuration parameters when starting the emulator.
82+
Commit a `.lstk/config.toml` to your repository, and both your developers and your CI jobs get the same emulator configuration, with no environment variables to duplicate across pipeline files.
83+
`lstk` picks up `./.lstk/config.toml` automatically when it is run from the root of your source tree:
84+
85+
```toml
86+
# .lstk/config.toml
87+
[[containers]]
88+
type = "aws" # Emulator type: "aws", "snowflake", or "azure"
89+
tag = "2026.4" # Pin the image tag for reproducible builds
90+
port = "4566"
91+
env = ["ci"] # Apply the [env.ci] profile below
92+
93+
[env.ci]
94+
DEBUG = "1"
95+
```
96+
97+
See the [configuration reference](/aws/developer-tools/running-localstack/lstk/#configuration) for every available field.
98+
99+
Keep a single `.lstk/config.toml` for local development and CI where you can.
100+
However, if a CI job needs different settings, pass an alternative file with `lstk --config ./ci/lstk.toml start`.
101+
102+
## Start the emulator
103+
104+
Start LocalStack with a single command:
105+
106+
```bash
107+
lstk start
108+
```
109+
110+
`lstk start` brings the LocalStack emulator all the way to a ready state.
111+
It pulls the container image if needed, validates your license, starts the container, and returns only once the emulator is ready, so there is no need for a separate wait or health-check step.
112+
If startup fails, the command exits with a non-zero return code, causing your CI job to fail.
113+
114+
For machine-readable output, add the global `--json` flag to any command.
115+
See [structured output](/aws/developer-tools/running-localstack/lstk/#structured-output) and [exit codes](/aws/developer-tools/running-localstack/lstk/#exit-codes) if your pipeline needs to inspect results programmatically.
116+
117+
## Seed state from Infrastructure as Code
118+
119+
Most CI pipelines create the resources their tests need by applying the same Infrastructure as Code they use for production.
120+
The `lstk` proxies automatically point those tools at the emulator, without any explicit configuration.
121+
122+
For example, with Terraform, run your usual commands through `lstk terraform` (or its `lstk tf` alias):
123+
124+
```bash
125+
lstk terraform init
126+
lstk terraform apply -auto-approve
127+
```
128+
129+
The [`lstk cdk`](/aws/connecting/infrastructure-as-code/aws-cdk/) and [`lstk sam`](/aws/connecting/infrastructure-as-code/aws-sam/) proxies work the same way, and [other IaC tools](/aws/connecting/infrastructure-as-code/) can target the emulator through its endpoint directly.
130+
131+
## Seed state from a snapshot
132+
133+
Rather than deploying your whole infrastructure on every run, you can seed the emulator from a [snapshot](/aws/developer-tools/snapshots/) captured earlier, either from a Cloud Pod or from a local snapshot file:
134+
135+
```bash
136+
# Load a Cloud Pod (requires LOCALSTACK_AUTH_TOKEN)
137+
lstk load pod:my-baseline
138+
139+
# Load a snapshot file produced by an earlier job
140+
lstk load ./baseline.snapshot
141+
```
142+
143+
`lstk load` starts the emulator first, if it is not already running, so it can replace a separate `lstk start` step.
144+
Alternatively, name the snapshot in your config, and `lstk start` loads it for you on every fresh start:
145+
146+
```toml
147+
[[containers]]
148+
type = "aws"
149+
port = "4566"
150+
snapshot = "pod:my-baseline"
151+
```
152+
153+
Override the configured snapshot for a single run with `lstk start --snapshot pod:other-baseline`, or skip auto-loading entirely with `lstk start --no-snapshot`.
154+
155+
To produce the snapshot in the first place, see [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/) and [saving snapshots locally](/aws/developer-tools/snapshots/saving-snapshots-locally/).
156+
157+
## Run your tests
158+
159+
Once the emulator is running, point your tooling at it.
160+
For Infrastructure as Code tools (such as Terraform), use the `lstk` proxy version of the tool, such as `lstk terraform`.
161+
162+
For test suites and SDK-based code, either set the endpoint and test credentials in the job's environment:
163+
164+
```bash
165+
export AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566
166+
export AWS_ACCESS_KEY_ID=test
167+
export AWS_SECRET_ACCESS_KEY=test
168+
```
169+
170+
Or create a `localstack` AWS profile and select it:
171+
172+
```bash
173+
lstk setup aws
174+
export AWS_PROFILE=localstack
175+
```
176+
177+
See [connecting to LocalStack](/aws/connecting/) for the full set of options.
178+
179+
## Collect logs
180+
181+
The emulator container disappears when the job ends, so consider exporting the logs before the test terminates, then store them as a build artifact:
182+
183+
```bash
184+
lstk logs --verbose > localstack.log
185+
```
186+
187+
Run this step even when the tests fail, so you capture the logs regardless of success or failure. To make failures easier to diagnose in the first place, set `DEBUG = "1"` in your CI environment profile.
188+
See [logging](/aws/customization/logging/) for the available log levels.

‎src/content/docs/aws/ci-pipelines/bitbucket.md‎

Lines changed: 23 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -3,27 +3,33 @@ title: BitBucket
33
description: Use LocalStack in BitBucket Pipelines.
44
template: doc
55
sidebar:
6-
order: 3
6+
order: 4
77
---
88

99
## Introduction
1010

1111
[BitBucket Pipeline](https://bitbucket.org/product/features/pipelines) is a CI/CD tool that allows you to build, test, and deploy your code directly from BitBucket.
1212
This guide will show you how to use LocalStack in BitBucket Pipelines.
1313

14+
BitBucket runs your build and the Docker daemon in separate containers, and does not support mounting volumes.
15+
This guide therefore starts the LocalStack container directly with `docker run`, so the pipeline controls the port mappings and the Docker connection itself, and then uses the [`lstk`](/aws/developer-tools/running-localstack/lstk/) tool proxies to interact with it.
16+
On CI systems without those constraints, `lstk` can manage the container lifecycle as well; see [CI Best Practices](/aws/ci-pipelines/best-practices/).
17+
1418
## Setting up the BitBucket Pipeline
1519

1620
When you want to integrate LocalStack into your job configuration, you just have to execute the following steps:
1721

1822
- Specify the Docker Socket to allow the LocalStack container to access the Docker daemon.
19-
- Export the `AWS_ENDPOINT_URL` environment variable to point to the LocalStack endpoint.
20-
- Install the `localstack` CLI and `awscli-local` to interact with LocalStack's emulated services.
23+
- Pass your CI Auth Token to the container, which is required to start the emulator.
24+
- Export the `LSTK_ENDPOINT_URL` environment variable to point `lstk` at the LocalStack endpoint.
25+
- Install the AWS CLI and `lstk` to interact with LocalStack's emulated services.
2126
- Start the LocalStack container in detached mode by specifying the Docker Socket and Docker Host.
27+
- Wait for the emulator to become ready before using it.
2228

2329
The following example BitBucket Pipeline configuration (`bitbucket-pipelines.yaml`) executes these steps, creates a new S3 bucket, and queries the list of S3 buckets:
2430

2531
```yaml showshowLineNumbers
26-
image: python:3.9
32+
image: node:22
2733

2834
definitions:
2935
services:
@@ -37,26 +43,25 @@ pipelines:
3743
services:
3844
- docker
3945
script:
40-
- export PYTHONPATH=$PYTHONPATH:$(pwd)
4146
- export DOCKER_SOCK=$DOCKER_HOST
42-
- export AWS_ENDPOINT_URL="http://localhost.localstack.cloud:4566"
43-
- env
47+
- export LSTK_ENDPOINT_URL="http://localhost.localstack.cloud:4566"
4448
- echo "${BITBUCKET_DOCKER_HOST_INTERNAL} localhost.localstack.cloud " >> /etc/hosts
45-
- pip install localstack awscli-local
49+
- apt-get update && apt-get install -y awscli
50+
- npm install -g @localstack/lstk
51+
- docker run -d --rm -p 4566:4566 -p 4510-4559:4510-4559 -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} -e DEBUG=1 -e DOCKER_SOCK=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 -e DOCKER_HOST=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 --name localstack-aws localstack/localstack-pro
4652
- |
47-
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
48-
unzip awscliv2.zip
49-
./aws/install
50-
- docker run -d --rm -p 4566:4566 -p 4510-4559:4510-4559 -e DOCKER_SOCK=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 -e DOCKER_HOST=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 --name localstack-main localstack/localstack
51-
- localstack wait -t 60
52-
- awslocal s3 mb s3://test-bucket
53-
- awslocal s3 ls
53+
for _ in $(seq 1 60); do
54+
curl -sf "${LSTK_ENDPOINT_URL}/_localstack/health" > /dev/null && break
55+
sleep 2
56+
done
57+
- lstk aws s3 mb s3://test-bucket
58+
- lstk aws s3 ls
5459
```
5560
5661
## Configuring a CI Auth Token
5762
58-
You can enable LocalStack for AWS by using the `localstack/localstack-pro` image and adding your CI Auth Token to the project's environment variables.
59-
The LocalStack container will automatically pick it up and activate the Pro features.
63+
For the configuration above to work, add your CI Auth Token to the project's environment variables.
64+
The LocalStack container will automatically pick it up and activate your LocalStack license.
6065
6166
Go to the [CI Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and copy your CI Auth Token.
6267
To add a CI Auth Token to your BitBucket Pipeline:
@@ -65,24 +70,7 @@ To add a CI Auth Token to your BitBucket Pipeline:
6570
- Select the **Settings** on the top navigation bar.
6671
- Select **Workspace settings** from the **Settings dropdown** menu.
6772
- On the left-hand menu, navigate to **Pipelines** and click on **Workspace variables**.
68-
- Add a new variable with the name `LOCALSTACK_AUTH_TOKEN` and the value of your CI Auth Token.
69-
70-
Navigate to your BitBucket Pipeline and add the following lines to the `bitbucket-pipelines.yaml` file:
71-
72-
```yaml showshowLineNumbers
73-
pipelines:
74-
default:
75-
- step:
76-
name: Test Localstack
77-
services:
78-
- docker
79-
script:
80-
...
81-
- export LOCALSTACK_AUTH_TOKEN=$LOCALSTACK_AUTH_TOKEN
82-
...
83-
- docker run -d --rm -p 4566:4566 -p 4510-4559:4510-4559 -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} -e DEBUG=1 -e LS_LOG=trace -e DOCKER_SOCK=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 -e DOCKER_HOST=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 --name localstack-main localstack/localstack-pro
84-
...
85-
```
73+
- Add a new variable with the name `LOCALSTACK_AUTH_TOKEN` and the value of your CI Auth Token, and mark it as **Secured**.
8674

8775
## Current Limitations
8876

0 commit comments

Comments
 (0)