Diagnosing Native Module Build Failures on ARM64

Fix npm, pip and gem installs that fail on Apple Silicon and arm64 containers: missing prebuilt binaries, node-gyp toolchains and wrong-arch node_modules.

npm ci inside an arm64 container stops with gyp ERR! stack Error: not found: make or prebuild-install warn install No prebuilt binaries found (target=20.17.0 runtime=node arch=arm64 libc=musl platform=linux); pip install spends eight minutes compiling grpcio from source and then fails with error: command 'gcc' failed: No such file or directory; or the container starts and crashes with Error: /app/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node: invalid ELF header. These all come from native extensions — packages with compiled code — meeting an architecture or libc they were not built for. This page diagnoses which case you have and fixes each, as part of cross-platform container builds.

The failures cluster on Apple Silicon laptops and arm64 CI runners, but the root causes are the same on any architecture change.

Diagnostic

Identify the platform the install runs on, the libc, and whether a prebuilt binary exists for that combination:

#!/usr/bin/env bash
set -euo pipefail
docker compose run --rm --no-deps api sh -c '
  echo "arch: $(uname -m)"
  (ldd --version 2>&1 | head -1) || echo "libc: musl (no glibc ldd)"
  node -p "process.versions.node + \" \" + process.arch + \" \" + (process.report.getReport().header.glibcVersionRuntime || \"musl\")"
  file node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node 2>/dev/null || true
  python3 -c "import sysconfig; print(sysconfig.get_platform())" 2>/dev/null || true
'

Expected bad output for the wrong-arch case:

arch: aarch64
libc: musl (no glibc ldd)
20.17.0 arm64 musl
node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node: ELF 64-bit LSB shared object, x86-64, dynamically linked

The container is arm64 with musl (Alpine), but the compiled module inside node_modules is x86-64 — it was installed on the host or an amd64 runner and copied or bind-mounted in.

Which Native Module Failure Is This? Decision diagram classifying native module failures by their error message. Which Native Module Failure Is This? What does the error say? invalid ELF header modules built on other arch no prebuilt binaries falls back to compiling gcc or make not found no build toolchain
Each error class has a different fix; identify it before changing the Dockerfile.

Root cause

Native packages ship compiled code in one of two ways. Most publish prebuilt binaries per platform — linux-x64-glibc, linux-arm64-glibc, linux-arm64-musl, darwin-arm64 — and download the matching one at install time. When no prebuilt exists for the exact combination, the installer falls back to compiling from source, which needs a C/C++ toolchain, Python (for node-gyp) and the library's headers; minimal images lack all of these, so the build fails. Prebuilt coverage is best for linux-x64-glibc and weakest for linux-arm64-musl, which is why Alpine on arm64 is the most common failure. The third case is not an install failure at all: node_modules or a virtualenv built on the host (macOS arm64) or on an amd64 runner is copied or bind-mounted into a Linux arm64 container, and the loader rejects binaries for the wrong OS or architecture with invalid ELF header or wrong ELF class.

Resolution

  1. Install dependencies inside the image, never from the host. Exclude host dependency directories from the build context and keep them off the bind mount:
node_modules
**/node_modules
.venv
__pycache__

Save as .dockerignore, and in Compose shadow the directory with a volume so the host's copy is never used:

services:
  api:
    build: ./api
    volumes:
      - ./api:/app
      - api-node-modules:/app/node_modules
volumes:
  api-node-modules:
  1. Prefer a glibc base with good prebuilt coverage over Alpine when native modules are involved; -slim Debian images are a small size increase for far fewer source builds:
# syntax=docker/dockerfile:1.7
FROM node:20-bookworm-slim AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
FROM node:20-bookworm-slim
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
CMD ["node", "server.js"]
  1. When a source build is unavoidable, install the toolchain in a build stage only, so the runtime image stays small:
# syntax=docker/dockerfile:1.7
FROM python:3.12-slim AS build
RUN apt-get update && apt-get install -y --no-install-recommends build-essential libpq-dev && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip wheel --wheel-dir /wheels -r requirements.txt
FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends libpq5 && rm -rf /var/lib/apt/lists/*
COPY --from=build /wheels /wheels
RUN pip install --no-index --find-links=/wheels /wheels/*
  1. Rebuild dependency volumes after switching architecture, because a named volume created by an amd64 container keeps amd64 binaries:
#!/usr/bin/env bash
set -euo pipefail
docker compose down
docker volume rm "$(basename "$PWD")_api-node-modules" || true
docker compose build --no-cache api
docker compose up -d api
Where Native Binaries Should Come From Flow showing dependencies installed inside the target image and kept separate from the host. Where Native Binaries Should Come From lockfile copied into build install in image target arch named volume not host dir runtime loads matching ELF
The binary must be built or downloaded on the same OS, architecture and libc it runs on.

Expected output

$ docker compose run --rm --no-deps api sh -c 'uname -m; file node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node'
aarch64
node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node: ELF 64-bit LSB shared object, ARM aarch64, dynamically linked
$ docker compose logs api | tail -1
api-1  | server listening on :3000

The native module matches the container's architecture, and the service starts. On a Debian-based image, npm ci downloads prebuilt arm64 binaries in seconds instead of compiling.

A quick way to confirm nothing else is lurking is to list every native binary in the dependency tree and check its architecture in one pass: find node_modules -name '*.node' -exec file {} + inside the container should print only ARM aarch64 lines on arm64 and only x86-64 lines on amd64. Any mismatch points to a package that was vendored, cached or copied from elsewhere, and is worth fixing before it causes an intermittent crash in a code path tests rarely hit.

Prevention

  1. Build and test on both architectures in CI with a matrix or buildx --platform linux/amd64,linux/arm64, so an unsupported native dependency fails the pull request that introduces it.

  2. Check new dependencies for prebuilt coverage before adding them: npm view <pkg> binary or the package's release assets for Node, and the wheel list on PyPI for Python (look for manylinux_2_28_aarch64 or musllinux tags).

  3. Never bind-mount host dependency directories into Linux containers; lint Compose files for mounts that include node_modules or .venv from the host.

Prebuilt Coverage by Platform Table of how often popular native packages publish prebuilt binaries per platform. Prebuilt Coverage by Platform Platform Node prebuilt Python wheels Source build risk linux x64 glibc very common very common low linux arm64 glibc common common moderate linux arm64 musl sparse sparse high darwin arm64 host common common low
glibc targets have far better prebuilt coverage than musl, especially on arm64.

Platform caveats

Apple Silicon (ARM64): host-side npm install produces darwin-arm64 binaries, which never work in Linux containers of any architecture. Keep installs inside containers even when the CPU architecture matches.

macOS (Docker Desktop): with Rosetta enabled, running --platform linux/amd64 images works, but installing inside them produces amd64 binaries that then fail in native arm64 containers sharing the same volume.

WSL2: host-side installs in WSL produce linux-x64-glibc binaries, which work in amd64 Debian containers by coincidence and fail on Alpine; the same rule applies — install inside the container.

Rollback

Revert the base image and Compose changes and rebuild; dependency volumes must be recreated whenever the base changes:

#!/usr/bin/env bash
set -euo pipefail
git checkout HEAD~1 -- Dockerfile compose.yaml .dockerignore
docker compose down -v
docker compose up -d --build

Frequently Asked Questions

What does invalid ELF header mean?

The loader found a compiled file for a different OS or architecture — typically node_modules installed on macOS or on an amd64 machine and mounted into an arm64 Linux container. Reinstall dependencies inside the container on the target platform.

Why does Alpine cause more native build failures?

Alpine uses musl libc, and many native packages publish prebuilt binaries only for glibc. Without a matching prebuilt, the installer compiles from source, which needs a toolchain Alpine images do not include.

Should we install build tools in the runtime image?

No. Install them in a build stage, produce wheels or compiled modules there, and copy only the results into a slim runtime stage.

How do I check if a package has arm64 binaries?

For Python, look for aarch64 wheels on the package's PyPI files page. For Node, check the package's GitHub release assets or its binary field, and try npm ci in an arm64 container.