Time-to-First-PR Metrics

If you cannot measure onboarding, you cannot tell whether last quarter's tooling change helped or quietly made things worse. Time-to-first-PR (TTFPR) is the durable proxy: the wall-clock interval from a contributor's first clone to their first merged, human-authored pull request. This guide instruments those boundaries deterministically, filters out bots and CI noise, and reports the distribution per cohort so outliers surface as actionable friction. It is part of onboarding architecture and friction mapping; the distributed-team measurement details live in how to measure developer onboarding time in distributed teams, and the regression-hunting playbook in fixing time-to-first-PR regressions after a dependency upgrade.

TTFPR is attractive precisely because it is hard to game and cheap to capture. It does not require self-reported survey data, it does not depend on a project manager remembering to close a ticket, and it aligns with an event a contributor already produces — a merged change. The cost of getting it wrong, however, is that a poorly instrumented metric produces confident numbers that are silently misleading: a delta measured against a desynchronized clock, a median contaminated by Dependabot merges, or a p90 computed over a cohort of two people. The rest of this guide treats each of those failure modes as an engineering problem with a runnable fix, not a dashboard preference.

Prerequisites

• A metrics endpoint or time-series database that accepts JSON over HTTP (Prometheus with a Pushgateway, VictoriaMetrics, or an InfluxDB line-protocol receiver all work).
• Read access to the version-control API (GitHub/GitLab) for merge timestamps, scoped to a token with repo:read and pull_requests:read.
• NTP synchronization on every host that emits a timestamp (timedatectl status), because every delta you compute is a subtraction of two clocks you do not control.
• jq 1.6+ and the gh CLI (or glab) available on the aggregation host, used below to reconcile telemetry against the authoritative merge log.

Before writing a single collector, agree on the exact definition of the two boundaries. Ambiguity here is the most common source of numbers that two teams cannot reconcile. In this guide the start is the first successful git clone (or the first git checkout of a fresh worktree) recorded for a given contributor identity, and the end is the merged_at timestamp of that contributor's first pull request that was authored by a human, approved, and merged into the default branch. Reverts, bot commits, and PRs that were closed without merging never count as an end event. Writing that definition down — and encoding it in the filter below — is what makes the metric portable across teams.

Instrumenting the Onboarding Pipeline

Accurate TTFPR starts with deterministic timestamp capture at the repository boundary: the start is the first successful clone, the end is the merge of the first approved, human-authored PR. Capturing the start event locally is deliberate. You could infer a start from the first push, but pushes happen after the contributor has already fought through provisioning, so a push-based start systematically undercounts the friction you are trying to see. Emitting the event from a post-checkout hook or a make bootstrap target captures the moment work truly begins.

1. Emit a milestone event with a UTC timestamp:

   #!/usr/bin/env bash
   set -euo pipefail
   EVENT_TYPE="${1:?usage: record-event <clone|first-pr>}"
   TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
   REPO_URL=$(git remote get-url origin 2>/dev/null || echo unknown)
   curl -fsS -X POST https://metrics.internal/api/v1/onboarding \
     -H 'Content-Type: application/json' \
     -d "{\"event\":\"${EVENT_TYPE}\",\"ts\":\"${TIMESTAMP}\",\"repo\":\"${REPO_URL}\"}"

2. Validate clock sync before trusting any delta:

   #!/usr/bin/env bash
   set -euo pipefail
   timedatectl status | grep -q 'System clock synchronized: yes' \
     || { echo "Clock not synchronized; deltas unreliable." >&2; exit 1; }

Always emit timestamps in UTC (the -u flag above is not optional). A contributor in UTC+9 whose laptop reports local time will otherwise appear to finish before they started, and a naive aggregator will either drop the record or clamp it to zero — both of which quietly shrink your median. Store the raw event with its source timezone offset if you have it, but compute all deltas in UTC. To make the start event fire automatically rather than relying on a human to run the script, wire it into a Git hook so the first checkout of the repository records itself:

#!/usr/bin/env bash
# .git/hooks/post-checkout — installed by `make bootstrap`
set -euo pipefail
FLAG="$(git rev-parse --git-dir)/.ttfpr-clone-recorded"
[ -f "$FLAG" ] && exit 0        # only the first checkout counts as a clone
record-event clone && touch "$FLAG"

The idempotency flag matters: post-checkout fires on every branch switch, so without the guard you would overwrite the start timestamp every time the contributor changes branches, resetting the interval to near zero on the day they finally open their PR. The guard file lives inside .git, so it is never committed and is naturally per-clone. The diagram below shows the two authoritative boundaries and the noise the aggregation stage must discard between them.

WSL2: Windows host time desyncs after sleep/hibernate; run wsl --shutdown or enable systemd-timesyncd. Apple Silicon (ARM64): some SBCs lack a battery-backed RTC and drift on cold boot; enable hardware RTC fallback. macOS (Docker Desktop): if the agent runs in a container, mount /etc/localtime read-only so the container timezone matches the host.

Standardizing Local Environments via Devcontainers

Version-mismatch delays vanish when every contributor provisions the same pinned runtime. Prune conflicting packages using dependency tree visualization during provisioning. The single largest hidden term in most teams' TTFPR is the "works on my machine" tail: a contributor spends two days chasing a Node or Python version mismatch that never appears in the median because it only hits new hires on unusual hardware. Pinning the toolchain by digest collapses that tail, and because the pin is content-addressed, drift becomes a detectable event rather than a slow slide.

1. Pin the image and warm caches before the IDE attaches:

   // .devcontainer/devcontainer.json
   {
     "image": "ghcr.io/org/dev-base@sha256:0000000000000000000000000000000000000000000000000000000000000000",
     "features": {
       "ghcr.io/devcontainers/features/docker-in-docker:2": {}
     },
     "postCreateCommand": "npm ci --prefer-offline && pip install -r requirements.txt",
     "customizations": {
       "vscode": {
         "extensions": ["ms-python.python", "dbaeumer.vscode-eslint"]
       }
     }
   }

2. Fail provisioning PRs when install exit codes diverge from the baseline:

   #!/usr/bin/env bash
   set -euo pipefail
   devcontainer build --no-cache --workspace-folder . || { echo "provision baseline broken" >&2; exit 1; }

Pinning by @sha256: digest rather than a floating tag like :latest is what makes the environment reproducible across the weeks between a contributor's clone and their first PR. A floating tag means two people who cloned on different days provisioned different environments, and any TTFPR difference between them is confounded by that drift. When you do need to move the pin forward, do it in a reviewed PR so the change is timestamped and attributable — that same commit becomes the marker you correlate against if the metric regresses, which is exactly the workflow covered in fixing time-to-first-PR regressions after a dependency upgrade. The postCreateCommand warms the dependency cache before the editor attaches, so the contributor's first interactive minute is spent reading code rather than watching npm ci resolve a lockfile.

WSL2: keep the repo on the Linux filesystem; /mnt/c degrades npm ci and pip install by 40–60%. macOS (Docker Desktop): raise VM memory to ≥8GB so parallel dependency resolution does not OOM. Apple Silicon (ARM64): set "platform": "linux/amd64" only when an upstream image lacks an arm64 manifest.

Orchestrating Multi-Service Stacks

A new contributor's stack must come up on first try. Gate the app on healthchecks and seed mock data automatically so there is no manual migration step. Every manual step in the startup path is a fork in the contributor's experience: some will run the migration, some will not, and the ones who forget spend an afternoon debugging an empty database instead of writing their first patch. Encoding startup order and seeding into the Compose file removes that variance, and variance is precisely what widens the gap between your median and your p90.

1. Block startup until dependencies are healthy:

   # docker-compose.yml
   services:
     db:
       image: postgres:16-alpine
       healthcheck:
         test: ["CMD-SHELL", "pg_isready -U dev"]
         interval: 5s
         retries: 5
       volumes:
         - ./seed:/docker-entrypoint-initdb.d
     app:
       build: .
       depends_on:
         db:
           condition: service_healthy
       ports:
         - "3000:3000"

2. Detect Compose drift against the CI baseline:

   #!/usr/bin/env bash
   set -euo pipefail
   LOCAL_HASH=$(docker compose config | sha256sum | awk '{print $1}')
   [ "$LOCAL_HASH" = "$(cat .cache/compose.sha256)" ] && echo "compose parity OK" || echo "compose drift" >&2

The condition: service_healthy gate is the difference between a stack that comes up reliably and one that fails intermittently on slower hardware. Without it, app starts the instant the db container exists, races the database's own startup, and crashes on the first connection — a failure that new contributors on lower-spec laptops hit far more often than the maintainers who wrote the Compose file on fast machines. The docker compose config hash is a cheap, deterministic drift probe: it canonicalizes the fully-resolved configuration (interpolating variables and merging overrides) so a stale local override or an uncommitted port change shows up as a hash mismatch before it costs anyone an afternoon. The stages below map the ordered startup path a first-run stack must complete before the contributor can write any code.

Port collisions and localhost resolution issues that inflate setup time are catalogued in common local failure points.

macOS (Docker Desktop): use 127.0.0.1 explicitly in app configs to bypass Docker Desktop's DNS proxy caching. WSL2: if localhost:3000 fails to bind on the Windows host, add a portproxy rule. Apple Silicon (ARM64): confirm base images are multi-arch so healthcheck polling is not slowed by emulation.

Aggregating and Reporting TTFPR Data

Raw events are noise until you filter non-humans and report a distribution per cohort. A single average is the wrong summary for onboarding data because the distribution is right-skewed: most contributors group near a floor, and a few stragglers with hardware or permissions problems form a heavy right tail. The mean is dragged around by that tail and hides the very cases you want to fix, while the median tells you what a typical new hire experiences and the p90 tells you how bad the worst-supported tenth of them have it. Report both, always per cohort, never as one global number.

1. Drop bot and CI sources at ingestion:

   # prometheus.yml
   scrape_configs:
     - job_name: onboarding_telemetry
       metrics_path: /metrics
       static_configs:
         - targets: ["localhost:9090"]
       metric_relabel_configs:
         - source_labels: [job]
           regex: "ci-runner|dependabot"
           action: drop

2. Reconcile telemetry against the VCS API and dedupe merges:

   #!/usr/bin/env bash
   set -euo pipefail
   gh api "repos/$ORG/$REPO/pulls?state=closed&per_page=100" \
     | jq -r '.[] | select(.merged_at != null and (.user.type == "User")) | "\(.user.login) \(.merged_at)"' \
     | sort -u

The select(... .user.type == "User") predicate is the load-bearing filter: GitHub tags Dependabot, Renovate, and app integrations with .user.type == "Bot", and a single unfiltered Dependabot merge on a contributor's first day would record a near-zero TTFPR and pull the whole cohort's median down. Reconciling against the VCS API rather than trusting local telemetry alone also catches the case where a contributor's first-pr event never fired — a hook that failed silently, a fork that pushed directly — because the authoritative merged_at from the API becomes the end timestamp of record. Once you have the clean per-contributor deltas, compute the median and p90 with a small deterministic pass rather than eyeballing a chart:

#!/usr/bin/env bash
set -euo pipefail
# deltas.txt: one integer number of hours per contributor, one per line
sort -n deltas.txt | awk '
  { v[NR] = $1 }
  END {
    n = NR
    med = (n % 2) ? v[(n+1)/2] : (v[n/2] + v[n/2+1]) / 2
    p90 = v[int(0.9 * n + 0.5)]
    printf "n=%d  median=%.1fh  p90=%.1fh\n", n, med, p90
  }'

Store the raw per-contributor deltas alongside the summarized percentiles rather than discarding them after each report. Percentile definitions drift — someone will eventually argue for p95, or want the interval measured in business hours rather than wall-clock — and if you only kept the rolled-up numbers you cannot recompute history under the new definition. A flat table of (contributor, cohort, platform, delta_hours, merged_at) rows costs almost nothing to keep and lets every past quarter be re-derived when the question changes. It also makes the metric auditable: when a stakeholder disputes a figure, you replay the exact rows that produced it instead of defending a number whose provenance you no longer have.

Guard the report against tiny cohorts. A p90 computed over five people is a single person's bad week presented as a trend; suppress percentiles below a minimum n (eight is a reasonable floor) and roll small cohorts up into a broader bucket until they cross it. The chart below shows a representative distribution: the median is a modest, stable floor while the p90 exposes the tail that pinning and healthchecks are meant to compress.

Cross-region clock and latency normalization is detailed in how to measure developer onboarding time in distributed teams.

macOS / Apple Silicon (ARM64): run the TSDB on native arm64 binaries; emulated storage layers add write amplification that artificially inflates ingest lag. WSL2: set [automount] options = "metadata,uid=1000,gid=1000" so Prometheus can write bind-mounted volumes.

Cohorting and Attribution

A global TTFPR number answers no useful question, because it blends populations whose friction has different causes. Split the metric along the axes that actually change the onboarding path: hire type (full-time versus contractor versus open-source drive-by), primary platform (Linux, macOS, WSL2, ARM64), and the repository or service they landed in. Each split is a hypothesis about where friction lives, and a per-cohort p90 that diverges from the rest is a pointer at the subsystem to fix. Choose cohorting keys you can derive from data you already have — the contributor's platform from the provisioning event, the team from an HR feed or the CODEOWNERS file, the repository from the event payload — so the split costs nothing at report time.

The decision of which summary to trust for a given cohort follows a simple rule you can encode once and stop arguing about. If the cohort is smaller than your minimum sample size, report nothing and roll it up; if it clears the floor, report the median as the headline and the p90 as the alarm. The path below captures that choice.

Attribution closes the loop. When a cohort's p90 jumps, correlate the change window against the reviewed events you already timestamp — the devcontainer digest bump, a Compose change, a dependency lockfile update — and you usually have your cause before you open a single support ticket. This is why every environment change belongs in a PR: an unreviewed, untimestamped change to the base image is a regression you can see in the metric but never explain. Feed the same clean event stream into an onboarding health-check script so contributors self-diagnose the common failures before they ever count against the tail.

Platform Caveats

Onboarding metrics are only as trustworthy as the least-synchronized clock and the slowest filesystem in the fleet, so the platform-specific hazards deserve a consolidated pass rather than only the inline notes above.

WSL2: the Windows host clock and the Linux guest clock diverge after every sleep cycle, and the divergence is silent — no error, just a skewed delta. Prefer systemd-timesyncd inside the distro and treat any negative interval as a clock fault to be discarded, not a data point to be clamped to zero. macOS (Docker Desktop): the file-sharing layer adds latency to every bind mount, so provisioning steps that hammer many small files (npm ci, pip install) run slower than on Linux and inflate the provisioning term for Mac cohorts specifically. Cache aggressively in postCreateCommand and compare Mac and Linux cohorts separately before concluding the tooling is slow. Apple Silicon (ARM64): an image without an arm64 manifest runs under emulation, and emulated builds can be several times slower — enough to move a whole cohort's median. Confirm multi-arch manifests before pinning, and never compare an emulated cohort against a native one without labeling the difference.

Rollback and Recovery

If a telemetry change corrupts the metrics stream, stop emitting, purge the bad window, and restore the prior collector config:

#!/usr/bin/env bash
set -euo pipefail
docker compose stop telemetry-agent
curl -fsS -X POST "https://metrics.internal/api/v1/onboarding/purge?since=$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ)"
git checkout -- prometheus.yml
docker compose up -d telemetry-agent

Purge by time window rather than dropping the whole series: a targeted since deletes only the corrupted interval and preserves the history you need to prove a trend. After restoring, re-run the VCS reconciliation pass for the affected window so any end events that arrived during the outage are backfilled from the authoritative merged_at timestamps rather than lost. Keep the previous prometheus.yml in version control precisely so git checkout -- prometheus.yml is a one-line recovery; a collector config edited live and never committed is the change you cannot roll back when it turns out to have dropped a legitimate cohort.

Frequently Asked Questions

Related

• Measure developer onboarding time in distributed teams
• Fix time-to-first-PR regressions after a dependency upgrade
• Build an onboarding health-check script
• Visualize a dependency tree to prune provisioning conflicts