Debugging Env Variable Leakage in Multi-Stage Docker Builds

Secrets passed as build ARG or ENV persist in image layers and docker history. Diagnose and stop env variable leakage across multi-stage Docker build stages.

A token you passed with --build-arg to install private dependencies shows up in docker history and in the final image's environment, even though you only used it in an earlier build stage. This is part of CI/CD pipeline parity checks under the broader environment sync, secrets and CI parity baseline.

Diagnostic

Inspect what actually persisted in the published image. Build args and ENV instructions are recorded in image metadata and remain readable by anyone who can pull the image.

#!/usr/bin/env bash
set -euo pipefail
docker build --build-arg NPM_TOKEN=npm_secret123 -t app:leaky .

# Build args and ENV show up in the layer history
docker history --no-trunc app:leaky | grep -i "token\|secret\|NPM"

# ENV values leak into the running container's environment
docker run --rm app:leaky env | grep -i "token\|secret"

Expected BAD output:

  ARG NPM_TOKEN=npm_secret123
  /bin/sh -c npm config set //registry.npmjs.org/:_authToken=npm_secret123
NPM_TOKEN=npm_secret123

The token is visible in two places: the build history and the final container environment.

Root cause

ARG and ENV values are baked into image layer metadata. A multi-stage build does not automatically scrub them — only the filesystem of intermediate stages is discarded, not the metadata of any stage you COPY --from or build FROM. An ENV set in the final stage persists into every container started from the image, and an ARG is recorded in docker history even if it was "only" used in stage one. Echoing a secret into a RUN command writes it into that layer's command string permanently.

Resolution

  1. Replace build-time secrets with BuildKit --secret mounts, which expose the value only to a single RUN and never write it to a layer.
# syntax=docker/dockerfile:1.7
FROM node:20-slim AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN="$(cat /run/secrets/npm_token)" \
    npm config set //registry.npmjs.org/:_authToken="$NPM_TOKEN" && \
    npm ci && \
    npm config delete //registry.npmjs.org/:_authToken

FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
CMD ["node", "server.js"]
  1. Build with the secret supplied from a file or env var, never as a build arg.
#!/usr/bin/env bash
set -euo pipefail
export DOCKER_BUILDKIT=1
printf '%s' "$NPM_TOKEN" | docker build \
  --secret id=npm_token,src=/dev/stdin \
  -t app:clean .
  1. Audit any remaining ENV lines. If a value is runtime configuration, inject it at docker run/Compose time instead of baking it in. Use ARG only for non-sensitive build inputs.

Expected output

$ docker history --no-trunc app:clean | grep -i "token\|secret"
$ docker run --rm app:clean env | grep -i "token\|secret"
$

Both greps return nothing — the secret never entered the image.

Prevention

  1. Add a CI step that fails if any secret-shaped string appears in image metadata:
#!/usr/bin/env bash
set -euo pipefail
if docker history --no-trunc "$IMAGE" | grep -Eiq 'token|secret|password|_key='; then
  echo "FAIL: potential secret in image history"; exit 1
fi
  1. Add # syntax=docker/dockerfile:1.7 and a pre-commit grep that rejects ARG .*TOKEN / ENV .*SECRET patterns in Dockerfile.

macOS (Docker Desktop): ensure BuildKit is the active builder (docker buildx ls); the legacy builder ignores --secret and silently falls back to no secret. Apple Silicon (ARM64): pin --platform linux/amd64 when the published image targets amd64 runners, or the leaked-history check runs against a different architecture's image.

Rollback

If a leaked image was already pushed, treat the secret as compromised: rotate it, then docker rmi the local copies and delete the pushed tags. Layer metadata cannot be edited after the fact — rebuild clean and re-push.