Emulating Cloud Services in a Local Compose Stack

A service that stores uploads in S3, consumes jobs from SQS and sends email through SES cannot be started on a laptop without one of three compromises: pointing it at a shared cloud development account, stubbing the cloud SDK calls in code, or running local emulators. Shared accounts create cost, credential sprawl and cross-developer interference ("who deleted the test bucket?"). Code-level stubs hide the integration bugs the service is most likely to have. Emulators, run as ordinary Compose services, give each developer a private, disposable, offline copy of the cloud APIs the code actually calls. This topic belongs to containerized local environments with Docker Compose and shows how to add them without drifting away from the real services.

The approach rests on one rule: the application code must not know whether it is talking to the emulator or the cloud. The only difference is configuration — an endpoint URL and a set of throwaway credentials — injected through the same environment variables that production uses. When that holds, a test that passes locally exercises the same SDK calls, serialisation and error handling as production.

Emulators also change what onboarding looks like. Without them, a new engineer's first day includes requesting cloud access, waiting for an IAM role, configuring SSO profiles and learning which shared resources are safe to touch. With them, docker compose up brings the whole dependency surface up on the laptop with no credentials at all, and the cloud access request can wait until the engineer actually needs to deploy. That shift alone often removes a day from time-to-first-PR, which is why emulated services belong in the environment baseline rather than being an optional extra a few developers configure for themselves.

Prerequisites

• Docker Engine 24+ and Compose v2.20+, with enough VM memory for the emulators: LocalStack uses 300–600 MB depending on services enabled; MinIO and Mailpit under 100 MB each; the Firestore emulator (a JVM) around 500 MB.
• Cloud SDKs that support endpoint overrides. Every current AWS SDK honours AWS_ENDPOINT_URL (and service-specific variants such as AWS_ENDPOINT_URL_S3) since late 2023. Google Cloud client libraries honour PUBSUB_EMULATOR_HOST and FIRESTORE_EMULATOR_HOST.
• The AWS CLI v2 (2.13+) on the host or in a tools container, for seeding and inspecting resources.
• An inventory of cloud resources the service touches: bucket names, queue names, topics, subscriptions. Resource names must come from configuration so the local and cloud names can differ if needed.

Check the SDK and CLI versions once, because older versions silently ignore the endpoint variables and talk to real AWS:

#!/usr/bin/env bash
set -euo pipefail
aws --version
node -e "console.log('aws-sdk v3 client-s3', require('@aws-sdk/client-s3/package.json').version)" 2>/dev/null || true
python3 -c "import boto3; print('boto3', boto3.__version__)" 2>/dev/null || true

One endpoint variable instead of code branches

The worst pattern is if (process.env.NODE_ENV === 'development') { s3 = new S3({ endpoint: 'http://localhost:4566' }) }. It puts environment knowledge in code, it is easy to get wrong in one of many client constructors, and it never runs in production, so it rots. The AWS SDKs now read the endpoint from the environment, which removes the branch entirely:

services:
  api:
    build: ./api
    environment:
      AWS_REGION: us-east-1
      AWS_ACCESS_KEY_ID: test
      AWS_SECRET_ACCESS_KEY: test
      AWS_ENDPOINT_URL: http://localstack:4566
      UPLOADS_BUCKET: uploads-local
      JOBS_QUEUE_URL: http://localstack:4566/000000000000/jobs
    depends_on:
      localstack:
        condition: service_healthy

  localstack:
    image: localstack/localstack:3.7
    environment:
      SERVICES: s3,sqs,sns
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:4566/_localstack/health"]
      interval: 5s
      retries: 20

1. Remove every hard-coded endpoint from application code and rely on AWS_ENDPOINT_URL.
2. Use obviously fake credentials (test/test) so a misconfigured client fails against real AWS instead of silently succeeding with a developer's real keys.
3. Make resource names configuration, never literals, so local and cloud names can differ.

A drift diagnostic for this section greps for endpoint literals that crept back into code:

#!/usr/bin/env bash
set -euo pipefail
git grep -nE 'localhost:4566|localstack:4566|endpoint: *["'\'']http' -- 'src/**' && { echo "hard-coded endpoint found"; exit 1; } || echo "no hard-coded endpoints"

Service-specific overrides exist for the cases where one service must stay on the cloud while others are emulated. AWS_ENDPOINT_URL_S3=http://minio:9000 sends only S3 calls to MinIO while SQS still follows AWS_ENDPOINT_URL; setting AWS_ENDPOINT_URL_SES to an empty string is not the same as unsetting it, so remove the variable rather than blanking it. Keep the full set of endpoint variables for a service in one place — an x-aws-local extension field reused by every service that talks to AWS — so a new worker cannot be added with half the overrides and quietly reach a real account.

Credentials deserve the same discipline. The value test is accepted by LocalStack for any account and rejected by AWS, which is exactly the behaviour wanted: a misrouted request fails immediately with InvalidClientTokenId instead of creating a real resource on a developer's personal account. Never let the local stack inherit ~/.aws/credentials from the host through a volume mount or an AWS_PROFILE variable; that single convenience is how local test data ends up in production buckets.

The LocalStack S3 and SQS guide covers the per-service details, including path-style addressing for S3.

Choosing an emulator per service

There is no single emulator for everything, and picking the most complete one is not always right. LocalStack covers dozens of AWS services behind one endpoint and is the default for SQS, SNS, SES and DynamoDB. For S3 alone, MinIO is lighter, faster for large objects, has a proper web console and persists data in a normal volume. Email is best captured with Mailpit regardless of provider, because it gives a UI for inspecting what was sent. Google Cloud ships official emulators for Pub/Sub, Firestore, Datastore, Bigtable and Spanner through the gcloud SDK image.

A practical way to decide is to list, for each service, the three or four operations the application actually performs — PutObject with a presigned URL, ReceiveMessage with long polling, Publish with message attributes — and run exactly those against each candidate emulator. An emulator with a hundred services is no use if the one operation you depend on behaves differently, and a narrow emulator that implements your operations faithfully is often lighter and more reliable. Record the result next to the Compose file, because the reasoning ("MinIO for S3 because multipart uploads over 5 GB are tested locally") is what the next person needs when they consider consolidating.

Fidelity matters in the places emulators simplify: IAM policies are not enforced by default in LocalStack's community edition, S3 eventual-consistency behaviour is not reproduced, and emulated queues do not throttle. Keep a small set of integration tests that run against the real cloud in CI for anything that depends on those behaviours. The MinIO guide, the Mailpit guide and the GCP emulator guide cover each one.

Creating resources at startup

An emulator starts empty. If the bucket and queue do not exist, the application fails on first use with NoSuchBucket or AWS.SimpleQueueService.NonExistentQueue. Resources must be created every time the emulator starts, before the application needs them, and the creation must be idempotent. LocalStack runs any executable script in /etc/localstack/init/ready.d/ once its services are ready:

#!/usr/bin/env bash
set -euo pipefail
awslocal s3 mb s3://uploads-local 2>/dev/null || true
awslocal sqs create-queue --queue-name jobs-dlq >/dev/null
dlq_arn=$(awslocal sqs get-queue-attributes --queue-url http://localhost:4566/000000000000/jobs-dlq --attribute-names QueueArn --query Attributes.QueueArn --output text)
awslocal sqs create-queue --queue-name jobs \
  --attributes "{\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$dlq_arn\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\"}" >/dev/null
echo "localstack resources ready"

Idempotency is the property that makes this reliable. The script runs on every start, including restarts where the volume still holds yesterday's resources, so every command must succeed whether or not the resource exists. create-queue is naturally idempotent when the attributes match; mb is not, hence the || true. For anything more complex — a bucket policy, a notification configuration, an SNS subscription — prefer "put" operations that overwrite over "create" operations that fail on existence, and keep the script short enough that a failure message points at a single line.

Mount it with ./localstack/init:/etc/localstack/init/ready.d:ro and keep the healthcheck gated on the init having run. Better still, generate these resources from the same infrastructure-as-code that defines production — the LocalStack init hooks guide shows both approaches and how to fail the healthcheck until seeding completes.

Keeping emulated and real resources in parity

Emulated resources drift from real ones the same way configuration does. A queue in production gets a longer visibility timeout, a bucket gets versioning, a topic gets a new subscription — and the local init script is never updated. The cheapest parity check compares the resource definitions the application depends on, not every attribute:

#!/usr/bin/env bash
set -euo pipefail
local_timeout=$(docker compose exec -T localstack awslocal sqs get-queue-attributes \
  --queue-url http://localhost:4566/000000000000/jobs --attribute-names VisibilityTimeout \
  --query Attributes.VisibilityTimeout --output text)
tf_timeout=$(grep -A10 'resource "aws_sqs_queue" "jobs"' infra/queues.tf | awk -F'= *' '/visibility_timeout_seconds/{print $2}')
[ "$local_timeout" = "$tf_timeout" ] && echo "jobs visibility timeout in parity ($tf_timeout)" || { echo "drift: local=$local_timeout infra=$tf_timeout"; exit 1; }

Run checks like this in CI next to the rest of the consolidated CI parity checks. The attributes worth checking are those that change application behaviour: visibility timeouts, dead-letter redrive counts, FIFO settings, bucket CORS rules and message retention.

The strongest form of parity is to stop maintaining two definitions at all. Tools such as tflocal (a Terraform wrapper that points providers at LocalStack) and CDK's cdklocal apply the same infrastructure code to the emulator that production uses. The init hook then shrinks to a single tflocal apply -auto-approve against a small module containing only the resources the local stack needs. This costs a few seconds of startup time and removes the drift problem for every attribute, not just the ones someone remembered to compare. Teams that cannot adopt it for everything often start with queues, where visibility timeout and redrive mismatches cause the most confusing local-only bugs: a job that is processed twice locally because the emulated timeout is shorter than the handler's runtime looks exactly like an idempotency bug in the application.

Offline and fast feedback

Emulators also make the stack work offline and make tests fast. A test suite that creates and deletes real SQS queues spends seconds on each API call and can be throttled; against LocalStack the same calls take milliseconds. To keep that speed, reset state between test runs instead of restarting containers — delete and recreate the handful of resources, or purge queues — and let Compose profiles start emulators only for the services that need them.

#!/usr/bin/env bash
set -euo pipefail
docker compose exec -T localstack sh -c '
  awslocal sqs purge-queue --queue-url http://localhost:4566/000000000000/jobs
  awslocal s3 rm s3://uploads-local --recursive >/dev/null'
echo "emulator state reset"

Purging is much faster than recreating the container because the emulator's JVM or Python process, its loaded service plugins and its network are reused. On a typical stack a purge-and-empty takes well under a second, while a full container restart with init hooks takes ten to twenty — a difference that adds up over hundreds of test runs a week. Keep the reset script next to the init script so both change together when a resource is added.

Platform caveats

Apple Silicon (ARM64): LocalStack, MinIO and Mailpit publish arm64 images. The gcloud emulator image is multi-arch, but some older Firestore emulator versions bundled amd64-only binaries; pin a recent gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators tag.

macOS (Docker Desktop): LocalStack's Lambda support starts a container per function through the Docker socket. Mount /var/run/docker.sock into the LocalStack container, or Lambda invocations fail with Docker not available.

WSL2: host tools calling emulators through published ports use localhost; containers use service names. Keep both forms in configuration (AWS_ENDPOINT_URL differs between host scripts and containers) rather than using host.docker.internal from inside containers.

Rollback and recovery

Emulators hold only disposable state, so recovery is always the same: remove the emulator's volume and restart it, letting the init hook recreate resources.

#!/usr/bin/env bash
set -euo pipefail
docker compose rm -sfv localstack minio
docker volume ls -q --filter label=com.docker.compose.volume=minio-data | xargs -r docker volume rm
docker compose up -d --wait localstack minio

To stop using an emulator for one service, unset its endpoint variable and supply real credentials through the local secret workflow — no code changes are required, which is the point of the endpoint-variable design.

Frequently Asked Questions

Related

• Run S3 and SQS against LocalStack
• Create buckets and queues automatically on startup
• Stub third-party HTTP APIs the same way
• Gate startup on emulator health