Common Local Failure Points

A new contributor's first day stalls on the same five breakages, year after year: a port already in use, a poisoned dependency cache, an .env that drifted from .env.example, a seed script that races the database, and a base image built for the wrong CPU architecture. This guide gives each one a deterministic diagnosis, a copy-paste fix, and a drift check so engineers can self-serve instead of escalating. It is part of the broader work on onboarding architecture and friction mapping, and the most acute single-symptom cases — containers that exit immediately on startup and DNS resolution failures between local containers — get dedicated walkthroughs.

These five failures share a structural trait: none of them is a bug in the application code, and all of them surface as an ambiguous symptom — a container that never becomes healthy, a stack trace that names the wrong module, or a service that binds and then vanishes. Because the root cause lives in the environment rather than the codebase, a new engineer has no mental model to reach for, and the default response is to message a senior teammate. Every message costs both people twenty minutes of context switching. The goal of this guide is to convert each ambiguous symptom into a named condition with a reproducible check, so the first response is a command the contributor runs themselves rather than a question they escalate.

The pattern for each section is identical on purpose: reproduce the symptom with a command that shows the bad output, name the root cause in one paragraph, apply a fix that is safe to run more than once, then leave behind a drift check that a make doctor target or CI job can run continuously. Idempotency matters throughout — a fix that only works on a clean machine is useless to the contributor whose machine is already in a broken state.

Prerequisites

• Docker Engine 24+ with the Compose v2 plugin (docker compose version).
• jq, lsof, and pg_isready available on the host (brew install jq lsof postgresql on macOS; the Postgres client package on Linux).
• A repository with a docker-compose.yml, a committed lockfile, and a .env.example.

Confirm the toolchain in one pass before you start, because a missing binary produces the same "command not found" noise that the failures below produce and will send you chasing the wrong cause. Run for b in docker jq lsof pg_isready; do command -v "$b" >/dev/null || echo "missing: $b"; done and resolve any gaps first. All commands assume you run them from the repository root where docker-compose.yml lives, since Compose resolves relative paths and the .env file against the project directory rather than your shell's working directory.

Each drift check below is written to be composed into a single make doctor target so a contributor runs one command and receives a list of named conditions rather than a wall of raw output. Keep every check exit-code honest — return non-zero on a real problem and zero on a clean state — because that is what lets you chain them and what lets CI reuse the same script without modification. The remaining sections follow the diagnose, explain, fix, verify order so you can read only the one that matches your current symptom.

Port Allocation and Service Binding Conflicts

Hardcoded ports and implicit host bindings cause silent collisions that stop a service before it logs anything useful. The classic symptom is Error starting userland proxy: listen tcp4 0.0.0.0:3000: bind: address already in use, printed once and then swallowed by a restart loop. A second, quieter failure mode is the reverse: the container starts, binds to 0.0.0.0 on every interface, and a colleague on the same network reaches a database you thought was private to your laptop. Both problems come from leaving the bind address and port implicit. Before bringing the stack up, audit what is already listening.

1. List current listeners so you can spot the conflict:

   #!/usr/bin/env bash
   set -euo pipefail
   lsof -iTCP -sTCP:LISTEN -P -n

2. Bind explicitly to loopback and parameterize the port so two stacks never fight over 3000:

   # docker-compose.yml
   services:
     app:
       container_name: local-app-primary
       build: .
       ports:
         - "127.0.0.1:${APP_PORT:-3000}:3000"
       environment:
         APP_PORT: "${APP_PORT:-3000}"

3. Run the drift check and reset stale bindings if the runtime mapping diverges from intent:

   #!/usr/bin/env bash
   set -euo pipefail
   RUNTIME_PORTS=$(docker ps --format '{{.Ports}}' --filter "name=local-app-primary")
   EXPECTED_PORTS="127.0.0.1:${APP_PORT:-3000}->3000/tcp"
   if [ "$RUNTIME_PORTS" != "$EXPECTED_PORTS" ]; then
     echo "Port drift detected; recreating containers." >&2
     docker compose down --remove-orphans
     docker compose up -d
   fi

The mechanism behind the collision is worth internalizing because it explains why "just kill the process" is the wrong first move. When you publish a port, Docker's userland proxy (or the kernel's iptables DNAT rules on Linux) claims the host-side socket at container start. If another container from an abandoned stack still holds 3000, or a native process such as a previously launched next dev never exited, the bind fails atomically — nothing partial happens, so there is no half-started service to debug. The fix is to make the host port a variable with a sane default (${APP_PORT:-3000}) and to pin the bind address to 127.0.0.1, which both prevents accidental LAN exposure and lets a contributor run two feature branches side by side simply by exporting a different APP_PORT.

Once the mapping is explicit, the drift check earns its place in a health script. It compares the runtime port string reported by docker ps against the string you expect from the current APP_PORT, and recreates the containers only when they disagree. That guard keeps the check idempotent: running it on an already-correct stack is a no-op, so it is safe to wire into a pre-up hook. The full Compose-specific fix for bind: address already in use lives in fixing "port is already allocated" errors in Compose.

Dependency Resolution and Cache Poisoning

Unpinned transitive dependencies and a stale package cache produce non-deterministic builds: the same git clone yields a different node_modules on two laptops. The symptom is maddening because it is intermittent — the build passes for the engineer who wrote the code and fails for the one who checked it out an hour later, with an error that names a transitive package nobody added on purpose. Pin exact versions in lockfiles and validate integrity.

1. Install strictly from the lockfile and record its hash inside the dev container:

   // .devcontainer/devcontainer.json
   {
     "name": "Local Dev Environment",
     "postCreateCommand": "npm ci --prefer-offline && sha256sum package-lock.json",
     "mounts": [
       "source=npm-cache,target=/home/node/.npm,type=volume"
     ]
   }

2. Diff the resolved tree against a known-good baseline and fail on divergence:

   #!/usr/bin/env bash
   set -euo pipefail
   npm ls --all --json > local-tree.json
   if ! diff -q ci-baseline-tree.json local-tree.json; then
     echo "Dependency drift detected versus CI baseline." >&2
     exit 1
   fi

There are two distinct failure sources hiding under the label "cache poisoning," and separating them is the whole job. The first is resolution drift: an engineer ran npm install (which is free to update the lockfile) instead of npm ci (which refuses to), so a caret range like ^4.17.0 silently resolved to a newer patch on one machine. The fix is procedural — npm ci fails hard if package.json and package-lock.json disagree, which is exactly the behavior you want in a reproducible environment. The second source is genuine cache corruption: a partial download or an interrupted extraction left a truncated tarball in ~/.npm/_cacache, and every subsequent install replays the broken artifact. That one is fixed by clearing the offending entry with npm cache verify or, in the worst case, discarding the cache volume entirely and rebuilding from the registry.

Mounting the cache as a named volume rather than a bind mount is the detail most teams get wrong. A named volume lives inside the Docker VM, so its permissions and inode layout are stable across host operating systems; a bind mount to a host directory inherits the host's case sensitivity and line-ending quirks, which is how a cache that works on Linux poisons a build on macOS. Recording the lockfile's sha256sum in postCreateCommand gives you a one-line fingerprint you can compare across machines: if two contributors report different hashes, you know the drift is in the lockfile itself and not in the resolved tree. When a build wedges on a transitive conflict, trace it with detecting circular dependencies in local builds and the broader dependency tree visualization workflow.

Apple Silicon (ARM64): native modules (node-gyp, grpc) often fail to compile without architecture flags; set npm_config_arch=arm64 or --build-from-source in .npmrc. macOS (Docker Desktop): the :cached/:delegated mount consistency flags can mask a poisoned cache; mount lockfiles read-only and rebuild from a clean volume to confirm. WSL2: keep node_modules on the Linux filesystem; the /mnt/c translation layer corrupts symlinks. Set core.autocrlf=false.

Environment Variable and Secret Drift

When local .env falls behind .env.example, services boot with missing keys and fail at the first call that needs them. The failure is deferred, which is what makes it expensive: the container starts cleanly, passes its healthcheck, and only throws when the code path that reads STRIPE_SECRET_KEY or REDIS_URL finally executes — sometimes minutes into a manual test, sometimes only in a specific feature. By then the contributor has lost the causal thread and blames the feature, not the missing key. Enforce completeness before execution.

1. Add a pre-commit hook that rejects an incomplete .env:

   # .pre-commit-config.yaml
   repos:
     - repo: local
       hooks:
         - id: check-env
           name: Validate .env against schema
           entry: bash -c 'jq -e ".required | all(. as $k | env | has($k))" .env.schema.json'
           language: system
           files: '\.env$'

2. Surface the drift explicitly during diagnosis:

   #!/usr/bin/env bash
   set -euo pipefail
   if ! diff -y --suppress-common-lines .env.example .env; then
     echo "Environment variable drift detected; reconcile required keys." >&2
   fi

The reason .env.example drift is so persistent is that the two files have different owners and different update triggers. .env.example changes when a feature adds a new configuration key, and that change rides in on a normal pull request. The local .env, by contrast, is git-ignored by design — it holds secrets — so it never receives the update automatically. Nothing forces a contributor to re-diff after git pull, and the gap accumulates one key at a time until a boot fails. Making the check a pre-commit hook shifts the enforcement to the moment of change, and treating a JSON schema as the source of truth lets you distinguish a required key (a hard failure) from an optional one (a warning) instead of naively diffing every line.

The diff -y --suppress-common-lines variant is the diagnostic you run by hand when a service misbehaves and you suspect config, not code. It shows only the lines that differ, side by side, so the missing key is obvious at a glance. Wire the schema check into your health script and it becomes a gate rather than an afterthought. For schema-driven generation and validation that prevents this class entirely, see catching missing env vars before container startup.

WSL2: CRLF line endings make .env parsers fail silently; run dos2unix .env or set Git core.eol=lf. macOS (Docker Desktop): .env resolves relative to the Compose project root, but docker run resolves it against the working directory — always pass --env-file with an explicit path.

Database State and Seed Script Execution Failures

Non-idempotent init scripts and missing readiness gates cause race conditions where the app connects before the schema exists. The symptom is a first-boot-only crash: on a machine with an empty data volume the database container spends several seconds initializing, the app container starts in parallel, opens a connection during that window, gets FATAL: the database system is starting up or a relation does not exist error, and exits. On the second run the volume already holds an initialized cluster, the database is ready almost instantly, and the same stack comes up green — which is exactly why the failure is dismissed as a fluke instead of fixed. Gate startup on a health probe and make seeds deterministic.

1. Block the app until the database is healthy:

   # docker-compose.yml
   services:
     db:
       image: postgres:16-alpine
       environment:
         POSTGRES_PASSWORD: "${DB_PASS:-localdev}"
         POSTGRES_DB: app_db
       healthcheck:
         test: ["CMD-SHELL", "pg_isready -U postgres"]
         interval: 2s
         timeout: 5s
         retries: 5
     app:
       build: .
       depends_on:
         db:
           condition: service_healthy

2. Run an idempotent, readiness-gated seed:

   #!/usr/bin/env bash
   set -euo pipefail
   export PGPASSWORD="${POSTGRES_PASSWORD:-localdev}"
   until pg_isready -h db -p 5432 -U postgres; do
     echo "Waiting for PostgreSQL readiness..."
     sleep 1
   done
   psql -h db -U postgres -d app_db -f /docker-entrypoint-initdb.d/01-schema.sql
   psql -h db -U postgres -d app_db -f /docker-entrypoint-initdb.d/02-seed.sql

Two guarantees have to hold together, and dropping either one reopens the race. The first is ordering: depends_on with condition: service_healthy tells Compose not to start the app until the database's healthcheck passes, which is stronger than the default depends_on that only waits for the container process to exist. The pg_isready probe is the right check because it returns success only once Postgres is accepting connections, not merely once the process has forked. The second guarantee is idempotency of the seed itself: an init script that runs CREATE TABLE users (...) will crash on the second boot when the table already exists, so a seed that is safe to replay must use CREATE TABLE IF NOT EXISTS and INSERT ... ON CONFLICT DO NOTHING. Without that, the healthcheck gate merely moves the failure from "connected too early" to "seeded twice."

Note the subtle distinction between the healthcheck's own retry loop and the until pg_isready loop in the seed script. The healthcheck governs Compose's startup ordering; the until loop protects the seed against the narrow window between "container healthy" and "this particular client connects," which matters when the seed runs from a separate one-shot container rather than the app itself. Belt and suspenders is correct here because the cost of a false start is a corrupted first boot that the contributor has to down -v and repeat. Startup-order races between healthy and dependent services are covered in depth in resolving service startup order and healthcheck races.

WSL2: bind-mounted DB volumes suffer severe I/O latency; use named volumes or a path under ~/, never /mnt/c. Apple Silicon (ARM64): postgres:16-alpine is multi-arch, but x86-only extensions (some PostGIS builds) require platform: linux/amd64 and emulation.

Cross-Architecture Container Runtime Mismatch

Pulling an image built for the wrong architecture, or compiling a native extension for the wrong target, yields exec format error — the kernel refusing to run a binary whose machine code it does not recognize. The failure has spread since Apple Silicon became the default developer laptop: an image built and pushed from an amd64 CI runner runs under slow QEMU emulation on an arm64 host, or a teammate rebuilds a native module against arm64 and the resulting node_modules is unusable on the x86 CI box. Detect host architecture early and map it to a build platform.

1. Make the build architecture-aware:

   # Dockerfile
   ARG TARGETARCH=amd64
   FROM --platform=linux/${TARGETARCH} node:20-alpine
   RUN apk add --no-cache python3 make g++ gcc

2. Diagnose a mismatch and rebuild for the host:

   #!/usr/bin/env bash
   set -euo pipefail
   CONTAINER_ARCH=$(docker inspect --format='{{.Architecture}}' local-app-primary)
   HOST_ARCH=$(uname -m | sed 's/x86_64/amd64/; s/aarch64/arm64/')
   if [ "$CONTAINER_ARCH" != "$HOST_ARCH" ]; then
     echo "Architecture mismatch: container=$CONTAINER_ARCH host=$HOST_ARCH; rebuilding." >&2
     docker compose build --build-arg "TARGETARCH=$HOST_ARCH"
   fi

The performance cost of an unnoticed mismatch is easy to underestimate. A container running under emulation is not merely a little slower; a native-code workload such as a database import or a webpack build routinely runs three to five times slower under QEMU, which turns a thirty-second local build into a two-minute one and quietly erodes the day. Because emulation works — the binary runs, just slowly — nobody investigates until someone benchmarks it. That is why the drift check compares the image's recorded Architecture against the normalized host architecture and rebuilds only on a real mismatch: it makes the invisible slowdown visible and actionable. The sed normalization matters because uname -m and Docker disagree on spelling (x86_64 versus amd64, aarch64 versus arm64), and a naive string compare would report a mismatch on every machine.

The durable fix is multi-architecture base images plus a build-time TARGETARCH argument, so the same Dockerfile produces a native image on both a developer's ARM laptop and an x86 CI runner without emulation on either. Where a dependency genuinely lacks an ARM build, pin that one service to platform: linux/amd64 explicitly and accept the emulation for it alone, rather than forcing the whole stack onto a single architecture.

Apple Silicon (ARM64): Docker Desktop defaults to linux/arm64; emulating linux/amd64 via Rosetta 2/QEMU is slow, so prefer native multi-arch base images. WSL2: uname -m reports x86_64 regardless of the underlying CPU; use dpkg --print-architecture for accurate detection in scripts.

Rollback - Recovery

When a diagnostic fix leaves the stack in a worse state, return to a clean baseline in one move:

#!/usr/bin/env bash
set -euo pipefail
docker compose down -v --remove-orphans
docker network prune -f
git checkout -- docker-compose.yml .env.example
docker compose up -d --wait

This tears down containers and their volumes, clears dangling networks, restores tracked config, and brings a fresh stack up gated on health. The -v flag is deliberate and consequential: it deletes named volumes, which resets the database to an empty cluster and forces the seed to run again from scratch. That is the correct behavior during onboarding, where the data is disposable and a clean seed is the goal, but you would never run this against a stack holding local work you care about — snapshot or pg_dump first if there is anything to keep. The closing --wait makes the command self-verifying: it blocks until every service with a healthcheck reports healthy, so a successful exit code is genuine confirmation that the reset worked rather than merely that the containers were created.

Keep this block in a make reset target so contributors reach for a named, reviewed command instead of improvising docker invocations under pressure. A recovery path that is one memorable command long is one a new engineer will actually use.

Taken together, the five checks and this reset form a small closed loop: make doctor names the condition, the matching section fixes it, and make reset returns to a known-good baseline when a fix goes sideways. Wiring the same scripts into CI closes the last gap, because a drift that a contributor could introduce locally is then caught on the pull request instead of on the next person's first day. The measurable payoff is fewer escalations and a shorter time to first productive commit, which is the entire point of treating environment setup as automation rather than tribal knowledge.

Frequently Asked Questions

Related

• Diagnose containers that exit immediately on startup
• Resolve DNS resolution failures between local containers
• Cut setup friction for junior engineers
• Build an onboarding health-check script