Keeping API Mocks in Sync With Provider Contracts

Every local and CI test passes, the release goes out, and the first real call to the partner fails with TypeError: Cannot read properties of undefined (reading 'amount') because the partner renamed amount to amount_cents two months ago and the mock still returns the old shape. The mock was accurate when written; nothing ever checked it again. This page adds the three checks that keep HTTP stand-ins honest — specification validation, scheduled sandbox comparison, and consumer-driven contracts for internal services — as part of mocking external APIs and third-party services.

A mock is a claim about another system's behaviour. Like any claim that drives decisions, it needs a way to be proven wrong.

Diagnostic

Find how old each set of mocks is and whether anything compares it with reality:

#!/usr/bin/env bash
set -euo pipefail
for d in mocks/*/; do
  last=$(git log -1 --format=%cs -- "$d")
  printf '%-24s last changed %s\n' "$(basename "$d")" "$last"
done
ls specs/ 2>/dev/null || echo "no pinned specifications"
grep -rlE 'prism proxy|pact|contract' .github/workflows 2>/dev/null || echo "no contract checks in CI"

Expected bad output:

partner                  last changed 2026-02-11
payments                 last changed 2026-06-30
shipping                 last changed 2025-11-04
no pinned specifications
no contract checks in CI

Mocks up to ten months old, no specification to validate them against, and no job that would notice a difference.

Root cause

Mocks are written from the provider's behaviour at one point in time and then change only when someone edits them. Providers change on their own schedule: renamed fields, new required parameters, new enum values, different error bodies, deprecated endpoints. Nothing in a normal test run compares the two, because the tests only talk to the mock — that is the point of the mock. So drift accumulates silently, and the tests become confirmation that the code works against a provider that no longer exists. The fix is structural: every stand-in needs at least one automated check against the real contract, placed where it runs often enough to catch changes before a release. For external providers that is the published specification plus the sandbox; for internal services owned by another team, it is a consumer-driven contract verified in the provider's own pipeline.

Resolution

1. Pin the provider's specification in the repository and validate stubs against it on every pull request. Running the client's integration tests through Prism's validation proxy, in front of the WireMock stubs, fails on any stub response the specification does not allow:

#!/usr/bin/env bash
set -euo pipefail
curl -fsSL https://raw.githubusercontent.com/stripe/openapi/v1234/openapi/spec3.yaml -o specs/payments.yaml
git diff --stat specs/payments.yaml
docker compose up -d --wait payments-stub
docker run -d --rm --name prism-check --network "$(basename "$PWD")_default" -p 4012:4010 \
  -v "$PWD/specs:/specs:ro" stoplight/prism:5.12.0 proxy -h 0.0.0.0 --errors /specs/payments.yaml http://payments-stub:8080
PAYMENTS_BASE_URL=http://localhost:4012 npm run test:integration -- --grep payments
docker logs prism-check 2>&1 | grep -q 'violation' && { echo "stub responses violate the spec"; docker rm -f prism-check; exit 1; }
docker rm -f prism-check

Replace the tag in the URL with the specification version you have reviewed; updating it is a deliberate pull request.

2. Compare with the sandbox on a schedule. A small smoke suite runs against the real sandbox weekly and compares the structure of responses with the corresponding stubs:

#!/usr/bin/env bash
set -euo pipefail
shape() { jq -S 'paths(scalars) | map(if type == "number" then "[]" else . end) | join(".")' | sort -u; }
curl -fsS -H "Authorization: Bearer ${PARTNER_SANDBOX_KEY:?}" https://sandbox.partner.example/v2/quotes/sample | shape > /tmp/real.txt
jq '.response.jsonBody' mocks/partner/mappings/quote-success.json | shape > /tmp/mock.txt
diff -u /tmp/mock.txt /tmp/real.txt && echo "quote shape in sync" || { echo "quote shape drifted"; exit 1; }

Structural comparison — field paths, not values — tolerates changing IDs and amounts while catching renamed, added and removed fields.

3. Use consumer-driven contracts for internal services. The consuming team writes a Pact test that records its expectations; the providing team verifies them in its own pipeline, so the provider cannot merge a change that breaks a known consumer:

import { PactV3, MatchersV3 } from '@pact-foundation/pact';
import { getQuote } from '../src/shipping-client.js';

const provider = new PactV3({ consumer: 'checkout', provider: 'shipping' });

test('quote for a German address', async () => {
  provider
    .uponReceiving('a quote request for DE')
    .withRequest({ method: 'POST', path: '/v2/quotes', body: { destination: { country: 'DE' } } })
    .willRespondWith({ status: 200, body: { quote_id: MatchersV3.uuid(), amount_cents: MatchersV3.integer(1299), currency: 'EUR' } });
  await provider.executeTest(async (mock) => {
    const quote = await getQuote(mock.url, { country: 'DE' });
    expect(quote.amountCents).toBe(1299);
  });
});

The generated pact file is published to a broker; the shipping team's CI runs pact-provider-verifier against its service on every change.

4. Surface drift as work, not noise. Scheduled checks open an issue or a pull request with the diff rather than failing an unrelated build, so the owner can update the stub and the client together.

Expected output

$ ./scripts/contract-check.sh
payments: stubs conform to specs/payments.yaml (v1234)
partner: quote shape in sync
partner: shipment shape drifted
--- /tmp/mock.txt
+++ /tmp/real.txt
-"status"
+"state"
+"state_reason"
opened issue #482: partner shipment response renamed status -> state

Conforming stubs pass quietly, and the one real drift produces a precise diff and an issue — months before it would have surfaced in production.

The issue carries everything the owner needs: which mock, which field paths changed, and the date the change was first detected. Fixing it means updating the stub, adjusting the client to read the new field, and adding a test for the transition period if the provider sends both shapes for a while. Handled this way, a provider change is a routine ticket rather than an incident.

Prevention

1. Require an owner and a check for every mock directory. A mocks/*/CONTRACT file naming the specification, sandbox or pact that validates it makes gaps visible in review.

2. Pin specifications and update them deliberately, reading the diff for renamed or newly required fields.

3. Include contract results in the consolidated parity report alongside environment and image checks, as described in the CI parity validation reference.

Platform caveats

CI secrets: sandbox keys for scheduled checks belong in the CI secret store, scoped to the scheduled workflow, never on laptops. See managing local secrets without committing to git.

Apple Silicon (ARM64): the Pact JavaScript library ships native binaries for arm64 macOS; older Pact versions required Rosetta. Upgrade to Pact JS 12+ on M-series Macs.

Rate limits: scheduled sandbox checks should call each endpoint once and cache the response for all comparisons in the run, to stay well within provider rate limits.

Rollback

Contract checks are additive CI jobs. Disable a noisy check by removing its workflow while its root cause is investigated, rather than deleting the mocks:

#!/usr/bin/env bash
set -euo pipefail
gh workflow disable contract-check.yml
gh workflow list | grep contract

Frequently Asked Questions

Related

• Generate mocks directly from the contract
• Refresh recorded traffic on a schedule
• Automate parity checks between local and staging