Splitting a Large Stack With Compose include

The monorepo's compose.yaml has grown to 900 lines and 23 services; every team edits it, merge conflicts are weekly, and adding a service means scrolling past everyone else's. Attempts to split it with multiple -f flags break in confusing ways: service "payments" refers to undefined network "payments-net", relative paths like ./payments/Dockerfile resolve from the wrong directory, and one team's .env overrides another's. Compose's include directive (Compose 2.20+) was built for exactly this: each included file keeps its own directory, relative paths and env file, and the result is one project. This page splits a large stack with it, as part of Compose profiles and targeted environments.

include is different from passing several -f files: -f merges files as if they were one (all paths relative to the first file), while include imports each file as a self-contained unit.

Diagnostic

Measure the current file and check that Compose supports include:

#!/usr/bin/env bash
set -euo pipefail
docker compose version --short
wc -l compose.yaml
docker compose config --services | wc -l
git log --since=90.days --format=%an -- compose.yaml | sort -u | wc -l | xargs echo "distinct authors in 90 days:"
git log --since=90.days --merges --format=%s | grep -ci conflict | xargs echo "merge commits mentioning conflicts:"

Expected output that signals a split is overdue:

2.29.2
912 compose.yaml
23
distinct authors in 90 days: 14
merge commits mentioning conflicts: 9

One file, 23 services, fourteen authors, and regular conflicts. Compose is new enough for include.

Root cause

A single Compose file is a shared hot spot: every service change touches it, so unrelated teams conflict. Splitting with multiple -f files looks like a fix but merges everything into one model whose working directory is the first file's directory, so build: ./Dockerfile in payments/compose.yaml resolves relative to the repository root and fails. Networks, volumes and env files declared in one fragment are expected to exist in the merged whole, and one fragment's .env interpolation leaks into others. include avoids all of that by loading each file as its own Compose application — its own project directory for relative paths, its own env_file for interpolation — and then combining the resulting services into one project. Services from different includes can still depend on each other by name.

There is also an organisational cause worth naming. A single file with no owner becomes a commons: everyone adds to it, nobody removes anything, and dead services linger because deleting someone else's definition feels risky. Splitting by team gives every service a clear owner through the directory it lives in, which makes cleanup possible again. In practice, splitting a large file usually reveals a few services nobody claims — candidates for deletion that the single file had been hiding for months.

Before splitting, it helps to see how the services depend on each other, since the fragments should follow those boundaries: services that start together and talk mostly to each other belong in the same fragment, and shared infrastructure that many depend on belongs in its own. A generated dependency graph makes those clusters of services visible at a glance and shows which cross-team edges will remain after the split.

Resolution

1. Give each team a Compose file next to its code, with paths relative to that directory:

services:
  payments:
    build:
      context: .
      dockerfile: Dockerfile
    env_file: .env.defaults
    environment:
      DATABASE_URL: postgres://postgres:postgres@db:5432/payments
    depends_on:
      db:
        condition: service_healthy
  payments-worker:
    build:
      context: .
      target: worker
    depends_on:
      - payments

Save as payments/compose.yaml. The db service it depends on is defined in a shared file.

2. Keep shared infrastructure in one file that every team relies on:

services:
  db:
    image: postgres:16.4
    environment:
      POSTGRES_PASSWORD: postgres
    volumes:
      - db-data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "postgres"]
      interval: 3s
  cache:
    image: redis:7.4
volumes:
  db-data:

Save as infra/compose.yaml.

3. Include everything from the root file, which becomes a short index:

include:
  - infra/compose.yaml
  - path: payments/compose.yaml
    env_file: payments/.env
  - path: catalog/compose.yaml
    env_file: catalog/.env
  - web/compose.yaml

Each path may carry its own env_file for variable interpolation, so one team's .env cannot change another team's services.

4. Check the merged result and name conflicts. Two includes defining the same service name is an error by design, which surfaces conflicts immediately instead of silently merging:

#!/usr/bin/env bash
set -euo pipefail
docker compose config --services | sort
docker compose config --format json | jq -r '.services | to_entries[] | "\(.key)\t\(.value.build.context // .value.image)"'
docker compose up -d --wait db payments

5. Add CODEOWNERS so each team owns its fragment and the root file changes rarely:

/compose.yaml            @acme/platform
/infra/compose.yaml      @acme/platform
/payments/compose.yaml   @acme/payments
/catalog/compose.yaml    @acme/catalog
/web/compose.yaml        @acme/web

Expected output

$ docker compose config --services | sort
cache
catalog
catalog-indexer
db
payments
payments-worker
web
$ docker compose config --format json | jq -r '.services.payments.build.context'
/home/dev/src/shop/payments
$ wc -l compose.yaml
7 compose.yaml

All services appear in one project, each build context resolves relative to its own directory, and the root file is seven lines.

Day-to-day commands do not change: docker compose up, logs, ps and profiles work across the whole project exactly as before, because include produces one project rather than several. What changes is the pull-request flow. A payments change touches only payments/compose.yaml, is reviewed by the payments team, and cannot conflict with a catalog change made the same afternoon.

Prevention

1. Fail CI on a large root file. A check that the root compose.yaml contains only include: entries keeps services from creeping back into it.

2. Validate each fragment on its own where possible (docker compose -f payments/compose.yaml config with the shared file included), so a team's change is checked in isolation.

3. Keep cross-team dependencies explicit with depends_on conditions on shared services, not on another team's internal services, so fragments stay loosely coupled.

Platform caveats

Compose version: include requires Compose 2.20 or later. Pin the minimum in the toolchain file; older versions fail with additional properties 'include' not allowed.

macOS and WSL2: relative bind mounts inside fragments resolve from each fragment's directory; keep the repository in a fast filesystem (not /mnt/c on WSL2) as the number of mounts grows.

Apple Silicon (ARM64): nothing specific; per-fragment platform settings apply only to that fragment's services.

Rollback

Flatten back to one file by rendering the merged configuration, which produces a single valid Compose file:

#!/usr/bin/env bash
set -euo pipefail
docker compose config > compose.flat.yaml
mv compose.flat.yaml compose.yaml
git rm -q infra/compose.yaml payments/compose.yaml catalog/compose.yaml web/compose.yaml

The rendered file contains absolute paths; replace them with relative ones before committing.

Frequently Asked Questions

Related

• Layer local and CI overrides on the combined stack
• Share one Compose file across repositories
• Map service dependencies before splitting