Debugging Tests That Time Out Only in CI

Find why tests pass locally but hit Exceeded timeout of 5000 ms in CI: CPU and memory limits, parallel workers, cold services, DNS and missing healthchecks.

The integration suite passes in forty seconds on a laptop and fails in CI with Error: Exceeded timeout of 5000 ms for a test or thrown: "Exceeded timeout of 5000 ms for a hook" — in different tests each run. Retrying sometimes helps, which makes it tempting to raise every timeout to 60 seconds and move on. But CI-only timeouts nearly always have a specific environmental cause: fewer CPUs than the test runner assumes, services that are started but not ready, DNS lookups to hosts that do not resolve on the runner, or memory pressure. This page measures which one applies and fixes it, as part of CI/CD pipeline parity checks.

The aim is to make CI as predictable as the laptop — not to hide slowness behind larger timeouts.

Diagnostic

Record the runner's resources and the test runner's parallelism, and time the slowest tests:

#!/usr/bin/env bash
set -euo pipefail
echo "cpus: $(nproc)  mem: $(free -g | awk '/Mem/{print $2}') GiB  load: $(cut -d' ' -f1-3 /proc/loadavg)"
node -e "console.log('os.availableParallelism', require('os').availableParallelism())"
docker compose ps --format '{{.Service}} {{.State}} {{.Health}}'
npx vitest run --reporter=json --outputFile=/tmp/results.json || true
jq -r '.testResults[].assertionResults[] | select(.duration > 2000) | "\(.duration)ms  \(.fullName)"' /tmp/results.json | sort -rn | head -5

Expected bad output on a hosted runner:

cpus: 4  mem: 15 GiB  load: 11.84 9.02 5.13
os.availableParallelism 4
db running starting
cache running
5012ms  orders > creates order with discount
4988ms  orders > setup hook
3120ms  checkout > applies coupon

The load average is nearly three times the CPU count, the database is still starting, and the slowest tests are the first ones to touch it.

Slowest Test Durations, Laptop vs CI Bar chart comparing durations of the slowest tests on a laptop and on a CI runner. Slowest Test Durations, Laptop vs CI laptop, first db test 0.4 s CI, first db test 5.0 s laptop, coupon test 0.2 s CI, coupon test 3.1 s
The first database-touching tests absorb both CPU contention and service warm-up in CI.

Root cause

Four differences explain most CI-only timeouts. CPU: test runners size their worker pools from the host's CPU count or default to aggressive parallelism, and CI runners have fewer cores than developer laptops — often shared with Docker services on the same machine — so each worker runs several times slower. Readiness: CI starts services and tests in quick succession; locally the stack has usually been running for hours. A depends_on without a health condition, or tests that start before docker compose up finishes, send the first queries to a database that is still initialising. Network: tests that call external hosts, or resolve names that exist only on a developer's network, wait for DNS or TCP timeouts in CI. Memory: with many workers plus services, the runner swaps, and everything slows unpredictably. Raising timeouts treats all four as one and leaves the suite slow and flaky.

Resolution

  1. Size the worker pool to the runner. Limit parallelism explicitly in CI rather than letting the runner guess:
#!/usr/bin/env bash
set -euo pipefail
workers=$(( $(nproc) > 2 ? $(nproc) - 1 : 1 ))
npx vitest run --pool=threads --poolOptions.threads.maxThreads="$workers"

For Jest, --maxWorkers=50%; for pytest-xdist, -n $(nproc) minus the cores the services need.

  1. Wait for real readiness before tests start. Use healthchecks and --wait so tests never race service start-up:
services:
  db:
    image: postgres:16.4
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d shop && psql -U postgres -d shop -c 'select 1' >/dev/null"]
      interval: 2s
      retries: 60
#!/usr/bin/env bash
set -euo pipefail
docker compose up -d --wait db cache
npm run db:migrate
npx vitest run
  1. Remove network dependencies from tests. Find outbound calls and replace them with local stubs; block the network during tests to catch new ones:
#!/usr/bin/env bash
set -euo pipefail
docker compose run --rm --network none api-tests npx vitest run 2>&1 | grep -E 'ENOTFOUND|EAI_AGAIN|ECONNREFUSED' | sort | uniq -c || echo "no network calls from tests"

The external API mocking topic covers the stubs.

  1. Give services memory limits and check swap. Constrain service containers so they cannot starve test workers, and fail fast if the runner swaps:
#!/usr/bin/env bash
set -euo pipefail
swap_used=$(free -m | awk '/Swap/{print $3}')
[ "$swap_used" -lt 200 ] || echo "warning: runner is swapping (${swap_used} MiB); reduce workers or service memory"
  1. Only then set explicit, justified timeouts for the few tests that are legitimately slow, next to the test with a comment explaining why.
Classifying a CI-Only Timeout Decision diagram mapping CI timeout symptoms to CPU, readiness, network or memory causes. Classifying a CI-Only Timeout What does the diagnostic show? load well above CPUs cap worker count service still starting healthcheck and --wait ENOTFOUND in logs stub external calls
Measure load, service health and network calls before changing any timeout.

Expected output

cpus: 4  mem: 15 GiB  load: 3.21 2.87 2.10
db running healthy
cache running healthy
 Test Files  38 passed (38)
      Tests  412 passed (412)
   Duration  71.42s

Load stays below the CPU count, services are healthy before the first test, no test exceeds its timeout, and the suite takes about 70 seconds — slower than a laptop, but consistently.

Consistency is the real improvement. A suite that takes 70 seconds every run is easy to reason about; one that takes 40 seconds most of the time and times out every fifth run erodes trust in CI, trains people to press "re-run", and hides genuine regressions among the noise. Recording the suite duration per run and alerting when it jumps keeps it that way.

Prevention

  1. Record test durations in CI and fail or warn when a test's duration grows by several times its median, which catches new slow tests before they become flaky.

  2. Run the suite locally under CI-like limits when investigating: docker run --cpus=4 --memory=8g reproduces the resource shape of a hosted runner.

  3. Ban sleep in tests in favour of waiting on conditions; sleeps tuned on a fast laptop are exactly the ones that break on a slow runner.

Raising Timeouts vs Fixing the Cause Comparison of increasing test timeouts against addressing CPU, readiness and network causes. Raising Timeouts vs Fixing the Cause raise timeouts to 60 s fix the cause still flaky under load stable under load failures take a minute failures stay fast network waits remain tests fully offline root cause unknown cause measured
Larger timeouts hide the problem and make every run slower; fixing the cause restores determinism.

Platform caveats

Hosted runners: CPU counts differ between runner types (2 to 4 vCPUs is common on free tiers) and are shared with Docker services. Measure with nproc in the job rather than assuming.

macOS runners: Docker is not available on hosted macOS runners by default; integration tests that need services should run on Linux runners.

Apple Silicon (ARM64) laptops: local runs on M-series machines are often several times faster than CI; use --cpus limits locally to approximate CI when reproducing timeouts.

Rollback

Revert worker limits or healthcheck changes if they cause regressions; they are independent:

#!/usr/bin/env bash
set -euo pipefail
git checkout HEAD~1 -- vitest.config.ts compose.yaml .github/workflows/test.yml

Frequently Asked Questions

Why do tests time out only in CI?

Usually because CI runners have fewer CPUs, services have only just started, tests call hosts that do not resolve there, or the runner is under memory pressure. Measure load, service health and network calls to find which.

Should I just increase the test timeout?

Only for tests that are legitimately slow, with a comment explaining why. Raising timeouts globally hides the cause, keeps the suite flaky under load and makes failures slower to report.

How many test workers should CI use?

Roughly the runner's CPU count minus what the services need — often nproc - 1. More workers than cores increases contention and makes every test slower.

How can I reproduce CI timing on my laptop?

Run the suite in a container with CPU and memory limits similar to the runner (--cpus=4 --memory=8g) and a freshly started service stack, rather than one that has been running for hours.