CI Parity Validation Reference

Drift rarely stays in one place. A floating base image, a missing env key, and a stale lockfile each break a build differently, but they all surface as the same symptom: green locally, red in CI. This reference consolidates the parity checks that otherwise live in three separate domains into one checklist and one runnable validation pass, so a developer or a CI job can confirm alignment in a single command. It extends the environment sync, secrets and CI parity baseline and deliberately reaches across all three of this site's domains: containerized local environments, environment sync, secrets and CI parity, and developer onboarding and friction mapping.

The cost of scattered checks is not that any one of them is hard — it is that no single person can hold all three in their head at review time. A backend engineer who pins image digests correctly may never touch the env schema; the person who owns the schema may not know a workflow silently behaves differently under a self-hosted runner. Each gap is invisible until a pull request that passed every local check fails the merge queue, and the failing log points at a symptom three layers removed from its cause. A consolidated gate collapses that surface into one exit code: either every domain agrees that local and CI describe the same environment, or the gate names the first domain that does not. The rest of this guide builds that gate from the ground up and explains exactly what each stage compares, why it drifts, and how to recover when a baseline is captured from a bad state.

Prerequisites

Before wiring the consolidated check, confirm each domain already emits a comparable signal. The gate does not invent new detection logic; it composes signals the three domains already produce, so if any one of them is missing you will get a false pass rather than a caught divergence.

• Docker Engine 24+ with docker compose v2 and docker buildx. The v2 plugin is required because docker compose config in v1 interpolated variables unconditionally, which made a hash of the merged file non-deterministic across machines with different shell environments.
• A Dockerfile that pins base images by digest, not floating tags — see the digest-pinning workflow in CI/CD pipeline parity checks. A FROM node:20 line resolves to a different layer set every time the upstream tag is re-pushed, so a digest is the only reference that two machines can prove they share.
• A machine-readable env schema (env-schema.json or a Zod module) as built in environment variable validation. Without a schema there is nothing to validate against, and "the app booted" is not the same claim as "every required variable is declared and typed".
• Committed lockfiles (package-lock.json, poetry.lock, etc.) and a one-command bootstrap target, as covered by the onboarding work in common local failure points.
• jq, yq, and act (nektos/act) on PATH for the runner-emulation step. Pin their versions in the same place you pin toolchain versions; a newer yq that changes its default output style will change a downstream hash and produce a spurious failure.

Verify the toolchain is present before the first run so a missing binary reports as a setup problem, not a parity failure:

#!/usr/bin/env bash
set -euo pipefail
for bin in docker jq yq act ajv git; do
  command -v "$bin" >/dev/null 2>&1 || { echo "missing required tool: $bin" >&2; exit 1; }
done
docker compose version >/dev/null || { echo "docker compose v2 not available" >&2; exit 1; }
echo "toolchain OK"

Section 1 - The consolidated parity checklist

Treat this as the canonical list. Each row maps a drift source to the command that detects it and the domain that owns the deeper fix. The value of writing it as a table is that it makes the ownership boundary explicit: when the gate fails on compose resolution, the on-call engineer does not debug the env schema, they open the containers runbook.

Each row is deliberately a comparison against a committed baseline, not a live introspection. Live introspection tells you what the current machine looks like; it cannot tell you whether that matches CI, because CI is not present at the moment you run the check locally. The only way one machine can assert parity with another without a network round trip is to compare both against a shared, version-controlled reference. That is why every stage in the script that follows reads an expected value from a committed file and diffs the current value against it.

The eight checks fall into three tiers of confidence. Digest and compose-hash comparisons are exact: two strings match or they do not, with no interpretation. Schema and required-variable checks are structural: they prove the shape of the environment is correct but say nothing about the values, which is the correct boundary because values are secrets and must never enter a baseline file. Lockfile and runner checks are behavioural: they prove the resolution and workflow-parse steps produce identical artifacts. Keeping these tiers distinct matters when a check fails, because the remediation differs — an exact-match failure is almost always "regenerate the baseline after an intentional bump", while a structural failure is almost always "someone added a variable and forgot the schema".

Section 2 - Container parity checks in depth

Container drift is the most common source of the green-local-red-CI pattern because the container is the one artifact that is supposed to be byte-identical everywhere, and any deviation from that promise is a bug by definition. Three signals capture almost all of it.

The image digest is the strongest single check on the page. A digest is the SHA-256 of the image manifest, so if two machines report the same digest they are provably running the same layers, the same entrypoint, and the same declared environment. The failure mode it catches is subtle: a Dockerfile that says FROM postgres:16 will pull a different digest the day the upstream 16 tag is re-published with a patch, and nothing in your repository changes to signal that. Capture the digest of the built image as a baseline and compare on every run.

#!/usr/bin/env bash
set -euo pipefail
# Record the built image digest as the committed baseline
docker image inspect app:local --format '{{index .RepoDigests 0}}' > .ci/image.digest
git add .ci/image.digest
echo "baseline digest: $(cat .ci/image.digest)"

Compose resolution drift is the second signal. docker compose config merges every file in the stack — the base compose.yaml, any compose.override.yaml, and files named on the command line — and prints the fully resolved result. If a teammate adds an override that mounts a local volume, or CI reads the files in a different order, the merged output changes even though no single file looks wrong in a diff. Hashing the resolved output with --no-interpolate gives a stable fingerprint that ignores runtime variable values but catches structural change.

#!/usr/bin/env bash
set -euo pipefail
# --no-interpolate keeps runtime values out of the fingerprint
docker compose config --no-interpolate | sha256sum | awk '{print $1}' > .ci/compose.sha
echo "compose fingerprint: $(cat .ci/compose.sha)"

Platform and architecture drift is the third. An Apple Silicon laptop builds linux/arm64 by default; a standard GitHub-hosted runner is linux/amd64. Most images run under emulation, so the mismatch is silent until a native dependency — a compiled crypto library, a platform-specific wheel — behaves differently or fails to load. Inspect the architecture of the built image and assert it matches the target CI platform explicitly rather than trusting the default.

#!/usr/bin/env bash
set -euo pipefail
ARCH="$(docker image inspect app:local --format '{{.Architecture}}')"
[ "$ARCH" = "amd64" ] || { echo "image is $ARCH, CI runner expects amd64" >&2; exit 1; }
echo "architecture OK: $ARCH"

Section 3 - Env and secret parity checks in depth

Environment drift splits cleanly into two questions that must never be conflated: is the shape of the environment correct, and are the values present. The shape question is safe to answer in a committed baseline; the value question is not, because values are secrets. Blurring the two is how secrets end up in a repository, so the gate answers each with a different mechanism.

Schema validation answers the shape question. Point ajv at the schema and a converted view of .env, and it reports any key that is undeclared, mistyped, or missing from the required list. This catches the classic onboarding failure where a new hire copies a stale .env.example, boots the app, and hits an undefined-variable crash deep in a request handler hours later.

#!/usr/bin/env bash
set -euo pipefail
# Validate declared shape without ever reading secret values into a baseline
ajv validate -s env-schema.json -d .env --strict-types \
  || { echo "env does not match schema" >&2; exit 1; }
echo "env schema OK"

The required-variables check answers a narrower question that schema validation alone can miss: a variable can be declared in the schema and still be empty at boot. Iterate the schema's required array and assert each key is present and non-empty in the actual environment. Note that this asserts presence, not correctness — it never compares a secret value against a stored copy, because no correct baseline contains a secret.

#!/usr/bin/env bash
set -euo pipefail
for key in $(jq -r '.required[]' env-schema.json); do
  grep -qE "^${key}=.+" .env || { echo "required var missing or empty: $key" >&2; exit 1; }
done
echo "required vars present"

Secret presence is the third env-domain check and the one most often skipped. When secrets are mounted from a vault or an external file rather than living in .env, the schema check passes and the app still fails at runtime because the mount is not wired. A test -s on each resolved secret path confirms the file exists and is non-empty without printing its contents, which keeps the check log-safe.

#!/usr/bin/env bash
set -euo pipefail
for path in /run/secrets/db_password /run/secrets/api_token; do
  test -s "$path" || { echo "secret not mounted or empty: $path" >&2; exit 1; }
done
echo "secrets present"

The discipline that makes all three checks safe is the same one that makes them useful: the baseline records structure, the environment supplies values, and the two are compared only for presence and shape. For a deeper treatment of failing fast on the missing-value case, see the entrypoint guard technique in block startup on missing env vars.

Section 4 - Onboarding and runner parity checks

The third domain covers the gap between "the code is correct" and "a fresh machine can reproduce the exact dependency graph and CI behaviour". Two checks carry most of the weight.

Lockfile integrity is the cheaper of the two and the more frequently violated. A lockfile records the exact transitive resolution of every dependency; if it drifts from what is committed, two machines install different code even though the manifest is identical. The check is a git diff --exit-code against the committed lockfile after a resolve step — if the resolve rewrote the lockfile, the working tree is dirty and the gate fails, telling you the committed lockfile is stale before CI ever runs.

#!/usr/bin/env bash
set -euo pipefail
git diff --exit-code -- package-lock.json >/dev/null 2>&1 \
  || { echo "lockfile drifted from committed state — commit the update" >&2; exit 1; }
echo "lockfile clean"

Runner emulation is the more powerful check. A GitHub Actions workflow can behave differently in CI than any local script because it runs under a specific runner image, with specific default environment variables and action versions. Running the workflow under act in dry-run mode (-n) parses and plans every job without executing it, which catches the class of failure where a workflow references a secret, matrix value, or action input that only resolves in the real runner.

#!/usr/bin/env bash
set -euo pipefail
act -n -W .github/workflows/ci.yml >/dev/null \
  || { echo "workflow fails to parse/plan under act" >&2; exit 1; }
echo "runner emulation OK"

Together these two prove that a fresh clone will resolve the same dependencies and that the CI workflow describing how to build them is internally consistent. For the broader runtime-parity picture — matching not just the runner but the staging environment — see match local container runtimes to staging.

Section 5 - Implementing the single validation entrypoint

The whole point is one entrypoint. The Makefile below collects the detection stages and exits non-zero on the first failure, so it works identically as a pre-push hook and as a CI gate. There is exactly one command a human or a pipeline needs to remember, and it is the same in both contexts.

1. Capture the expected image digest as a committed baseline file so both contexts compare against the same value.
2. Resolve the merged Compose configuration and hash it — a stable hash proves file order and overrides match.
3. Validate .env against the schema and assert required keys are present.
4. Confirm lockfiles are clean and the workflow parses under the emulated runner.

# Makefile — consolidated parity gate
.PHONY: verify-parity
verify-parity:
	@bash scripts/verify-parity.sh

.PHONY: parity-baseline
parity-baseline:
	@docker image inspect app:local --format '{{index .RepoDigests 0}}' > .ci/image.digest
	@docker compose config --no-interpolate | sha256sum | awk '{print $$1}' > .ci/compose.sha
	@echo "Baseline written to .ci/"

#!/usr/bin/env bash
# scripts/verify-parity.sh — runs every domain's check in one pass
set -euo pipefail

fail() { echo "PARITY FAIL: $1" >&2; exit 1; }

# 1. Container: image digest matches committed baseline
EXPECTED_DIGEST="$(cat .ci/image.digest)"
ACTUAL_DIGEST="$(docker image inspect app:local --format '{{index .RepoDigests 0}}')"
[ "$EXPECTED_DIGEST" = "$ACTUAL_DIGEST" ] || fail "image digest drift ($ACTUAL_DIGEST)"

# 2. Container: merged compose config hash matches baseline
EXPECTED_COMPOSE="$(cat .ci/compose.sha)"
ACTUAL_COMPOSE="$(docker compose config --no-interpolate | sha256sum | awk '{print $1}')"
[ "$EXPECTED_COMPOSE" = "$ACTUAL_COMPOSE" ] || fail "compose resolution drift"

# 3. Env/secrets: schema validation + required keys present
ajv validate -s env-schema.json -d .env --strict-types || fail "env schema invalid"
for key in $(jq -r '.required[]' env-schema.json); do
  grep -qE "^${key}=" .env || fail "required env var missing: $key"
done

# 4. Onboarding: lockfile clean + runner emulation parses
git diff --exit-code -- package-lock.json >/dev/null 2>&1 || fail "lockfile dirty"
act -n -W .github/workflows/ci.yml >/dev/null || fail "workflow does not parse under act"

echo "PARITY OK — containers, env/secrets, and runner all aligned"

The fail function centralizes the exit path so every stage reports with the same PARITY FAIL: prefix, which makes the log greppable and lets a wrapping tool key off a single string. Because set -euo pipefail is in force, any unhandled non-zero — an unreadable baseline file, a missing binary — also aborts the run rather than silently skipping a stage, which is the behaviour you want in a gate: a check that cannot run is not a check that passed.

Drift diagnostic

Run the gate in isolation and inspect which stage tripped. Because the script fails fast, the first PARITY FAIL: line names the owning domain directly, and the exit is non-zero so a wrapping hook can react:

#!/usr/bin/env bash
set -euo pipefail
make verify-parity || echo "Re-run individual checks above to localize the failing domain"

Order the stages cheapest-first — the digest comparison is a string equality that runs in milliseconds, while act -n shells out and parses YAML. Putting the fast, high-signal checks first means the common case (a stale digest after an upstream bump) fails in under a second, and the expensive runner emulation only runs when everything cheaper has already agreed.

Section 6 - Wiring the gate into CI

Mirror the local gate as a blocking job so a passing local run guarantees a passing CI run. The job does nothing the developer's pre-push hook does not; it simply runs the same entrypoint in the environment that matters for the merge decision.

# .github/workflows/parity.yml
name: Parity Gate
on: [pull_request]
jobs:
  parity:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build pinned image
        run: docker build -t app:local .
      - name: Restore baselines
        run: make parity-baseline
      - name: Run consolidated parity check
        run: make verify-parity

There is a deliberate subtlety in the CI job: it regenerates the baselines with make parity-baseline before running make verify-parity. In CI the point is not to catch drift against a stored file — CI is the source of truth — but to confirm that the committed baselines the developer pushed match what CI itself produces. If a developer commits a digest from a locally-built arm64 image, the CI job rebuilds on amd64, regenerates the baseline, and the digest comparison in verify-parity fails, surfacing the platform mismatch as a blocking failure instead of a silent divergence. Run the parity job as a required status check on the protected branch so it cannot be bypassed by a merge.

Section 7 - Measuring the gate's cost

A gate that developers route around is worse than no gate, because it manufactures false confidence. The single most effective way to keep a parity gate adopted is to keep it fast, so it is worth measuring where the wall-clock time goes and ordering the stages accordingly. The numbers below are representative wall-clock times for the four stages on a warm cache; treat them as a shape, not a promise, and measure your own.

The distribution is the argument for the fail-fast ordering in Section 5: three of the four stages complete in well under two seconds combined, and the one expensive stage is behavioural rather than exact-match. When drift is present, it is overwhelmingly in the exact-match tier — a re-pushed base tag, a new compose override — so the gate returns a verdict in under a second on the common failure and only pays the act cost when everything cheaper already passed. If the emulation step becomes a bottleneck as workflows grow, scope it to changed workflow files with a path filter rather than dropping it, so the coverage stays intact while the cost tracks the change surface.

macOS (Docker Desktop): image digests differ from Linux runners when an image lacks an arm64 manifest; build with --platform linux/amd64 before generating the baseline so the digest is comparable to CI. WSL2: keep the repo on the Linux filesystem (~/code, not /mnt/c) so git diff --exit-code on lockfiles is not tripped by line-ending rewrites. Apple Silicon (ARM64): install the aarch64 builds of jq, yq, and act; x86 binaries under emulation slow the gate enough to mask real failures behind timeouts.

Rollback and recovery

If the gate blocks work and you need to unblock while triaging, the correct move is almost never to disable a check — it is to confirm the current state is genuinely correct and regenerate the baselines from it:

#!/usr/bin/env bash
set -euo pipefail
# Regenerate baselines from the current verified state, then re-run.
make parity-baseline
make verify-parity

If a baseline was committed from a bad state — a digest from an arm64 build, a compose hash captured with an accidental override present — revert just the baseline files: git checkout HEAD~1 -- .ci/image.digest .ci/compose.sha, rebuild, and regenerate. Reverting only the baseline files keeps the rest of the branch intact while you re-establish a known-good reference. Never disable the gate in CI to merge — instead scope the failing check out with an explicit, reviewed skip comment so the gap is visible and shows up in the next audit. A silently disabled gate is indistinguishable from a passing one until the day it lets a real divergence through.

When the failure is in the behavioural tier — a dirty lockfile or an act parse error — recovery is different because there is no baseline to regenerate. A dirty lockfile means the committed resolution is stale: run the resolve, commit the updated lockfile, and re-run. An act parse failure means the workflow references something that does not resolve locally; fix the workflow or supply the missing input to act through an event file so the local plan matches the CI plan.

Frequently Asked Questions

Related

• Detect digest and seed drift in CI pipelines
• Block startup on missing env vars
• Reproduce CI-only failures locally with act
• Match local container runtimes to staging