Scanning CI Logs for Accidentally Printed Secrets

A debugging step added env | sort to a workflow "temporarily" six months ago, and every run since has printed the database URL with its password — masking did not catch it because the secret is stored as DB_PASSWORD but appears in the log inside postgres://app:hunter2@db:5432/shop. Another job runs with set -x, which echoes a curl -H "Authorization: Bearer ..." line where the token was read from a file rather than a masked variable. CI platforms mask registered secret values in logs, but only exact matches of values they know about. This page finds what slipped through in existing logs and stops new leaks, as part of secret scanning and leak prevention.

Logs are usually readable by everyone with repository access and are retained for months, which makes them a quiet but wide exposure.

Diagnostic

Download recent workflow logs and scan them with a secret scanner, then look for the usual culprits in workflow files:

#!/usr/bin/env bash
set -euo pipefail
mkdir -p /tmp/ci-logs && cd /tmp/ci-logs
for id in $(gh run list --limit 30 --json databaseId -q '.[].databaseId'); do
  gh run view "$id" --log > "run-$id.log" 2>/dev/null || true
done
gitleaks detect --no-git --source /tmp/ci-logs --redact --report-format json --report-path /tmp/ci-leaks.json || true
jq -r '.[] | "\(.RuleID)\t\(.File)"' /tmp/ci-leaks.json | sort | uniq -c | sort -rn | head
cd "$OLDPWD"
grep -nE '^\s+(- )?run: .*(env\s*(\||$)|printenv|set -x|--verbose|-v )' .github/workflows/*.yml | head

Expected bad output:

     30 postgres-connection-string	run-10821...log
      4 generic-api-key	run-10797...log
.github/workflows/test.yml:41:        run: env | sort
.github/workflows/deploy.yml:18:        run: set -x && ./scripts/deploy.sh

Every one of the last 30 runs contains a database connection string with an embedded password, and four contain an API token — from an environment dump and a traced deploy script.

Root cause

CI masking works by string replacement: the platform knows the values of registered secrets and replaces exact occurrences in log output with ***. Anything that changes the string defeats it. A password embedded in a connection string is still masked only if the password itself is registered and appears verbatim — URL-encoding a @ or / in it changes the bytes. Base64-encoded credentials (common in Authorization: Basic headers and Kubernetes secrets) are different strings entirely. Values derived at runtime — tokens fetched from a vault, read from a file, generated by an OIDC exchange — were never registered, so nothing masks them unless the job registers them explicitly. Shell tracing (set -x) and verbose flags print full command lines, and environment dumps print every variable at once. Each is harmless in local debugging and a leak in CI, where logs are shared and retained.

The parity angle is easy to miss. Debugging habits that are fine on a laptop — dumping the environment to check a variable, turning on tracing to see why a script fails — are exactly the habits developers bring to CI when a job fails only there. The laptop's terminal disappears when it closes; the CI log is kept for ninety days and visible to everyone with read access to the repository, including contractors and integrations. Treating CI logs as shared, persistent output — closer to a public document than a private terminal — is the mental shift that prevents most of these leaks, and the lint and scheduled scan below enforce it for the moments when that shift is forgotten.

Resolution

1. Rotate everything the scan found, first. A secret printed in a retained log is exposed to everyone who can read logs; deleting the log afterwards does not undo earlier reads.

2. Delete the affected logs once rotation is done:

#!/usr/bin/env bash
set -euo pipefail
jq -r '.[].File' /tmp/ci-leaks.json | sed -E 's/run-([0-9]+)\.log/\1/' | sort -u | while read -r id; do
  gh api -X DELETE "repos/{owner}/{repo}/actions/runs/$id/logs" && echo "deleted logs for run $id"
done

3. Register runtime-derived secrets as masks as soon as they are obtained, and mask derived forms too:

#!/usr/bin/env bash
set -euo pipefail
token=$(vault kv get -field=token secret/ci/deploy)
echo "::add-mask::$token"
echo "::add-mask::$(printf '%s' "$token" | base64 | tr -d '\n')"
echo "::add-mask::$(jq -rn --arg v "$token" '$v|@uri')"
echo "DEPLOY_TOKEN=$token" >> "$GITHUB_ENV"

4. Remove environment dumps and tracing from workflows, replacing them with targeted, safe diagnostics:

      - name: Show non-secret configuration
        run: |
          echo "node: $(node --version)"
          echo "db host: ${DATABASE_URL#*@}"
          env | grep -E '^(NODE_ENV|CI|GITHUB_REF_NAME)=' | sort

${DATABASE_URL#*@} prints the host part after the credentials; the grep allowlist prints only named, non-secret variables.

5. Scan logs continuously with a scheduled job that runs the same gitleaks scan over the last day's runs and opens an issue on findings.

Expected output

$ gitleaks detect --no-git --source /tmp/ci-logs-new --redact --report-format json --report-path /tmp/new.json; jq length /tmp/new.json
    ○
    │╲
    │ ○
    ○ ░
    ░    gitleaks
INF scan completed in 812ms
INF no leaks found
0
$ grep -cE 'run: .*(env\s*(\||$)|set -x)' .github/workflows/*.yml | awk -F: '{s+=$2} END {print s}'
0

New runs contain no secrets, workflows no longer dump the environment or trace commands, and runtime tokens appear as *** in logs.

The scheduled scan keeps it that way. Most regressions are a debugging line added under pressure — an env dump while chasing a CI-only failure, a --verbose flag on a deploy tool — and the daily scan turns them into an issue within a day rather than months of accumulated exposure. The finding points at the exact run and rule, so the fix is quick.

Prevention

1. Lint workflows for env, printenv, set -x and verbose flags in run: steps and require justification in review.

2. Mask derived values whenever a job computes or fetches a credential, including encoded forms, before any command could print it.

3. Reduce log retention for repositories that handle sensitive data (repository settings → Actions → artifact and log retention), limiting how long a missed leak stays readable.

Platform caveats

GitHub Actions: ::add-mask:: applies from the moment it is printed; anything logged earlier in the job stays visible. Mask immediately after obtaining a value.

GitLab CI: masked variables must meet format rules (length, character set); values that do not qualify are silently unmasked. Check the variable's "masked" status after saving.

Self-hosted runners: logs may also be written on the runner's disk; include runner log directories in the scan and clean-up.

Rollback

The prevention changes are workflow edits and can be reverted individually; rotated secrets and deleted logs cannot, and should not, be restored:

#!/usr/bin/env bash
set -euo pipefail
git checkout HEAD~1 -- .github/workflows/test.yml

Frequently Asked Questions

Related

• Tune secret scanner false positives with allowlists
• Scan Docker images for embedded secrets
• Debug env variable leakage in multi-stage Docker builds