CI/CD Pipeline Review Checklist

Pick the strategy in use today, not the one in the wiki from 2021. Trunk-based with short-lived branches works well when the team is shipping daily and feature flags are mature; Gitflow fits scheduled-release shops with a long QA tail. The reviewer should be able to point to a recent PR that exemplifies the strategy.

Required status checks should include the full PR pipeline (lint, unit, integration, SAST). Required reviewer count should be at least 1 from CODEOWNERS. Force-push and branch-deletion should be disabled. Admins-can-bypass should be off — the most common drift is an admin override that nobody remembers turning on for a hotfix six months ago.

Every directory with production code should map to a team, not an individual. Spot-check three recent PRs and confirm the auto-requested reviewers were the right people. Stale ownership is the most common reason reviews stall.

Capture the current line-coverage percentage from the CI output (Jest, pytest-cov, JaCoCo, etc.). The number itself matters less than the trend — coverage that's been silently sliding for two quarters is the signal to dig into. Confirm the build fails when coverage drops below the floor; coverage-as-warning gets ignored.

Integration suite should run on every PR, not just nightly. If the suite takes longer than 15 minutes, that's a separate problem worth a ticket — but skipping it on PRs is how regressions reach main.

Flakes that have been quarantined for more than 30 days are either dead code waiting to be deleted or real bugs being ignored. Either resolve them or assign owners with a deadline. The cultural failure mode is 'just rerun it' becoming permission to ignore real regressions.

Confirm Semgrep / CodeQL / SonarQube run as required PR checks, not nightly batches. New findings should fail the PR with the offending line annotated; existing findings can be tracked as backlog. Lint config (ESLint, RuboCop, golangci-lint) lives in-repo so contributors run it locally.

Use Syft, CycloneDX, or the GitHub-native SBOM action to emit an SPDX or CycloneDX file per build artifact. Increasingly required for federal contracts (EO 14028) and a precondition for fast triage when the next Log4Shell-class CVE drops.

Capture the strategy actually in use for the primary service — not the one in the architecture diagram. Rolling is the default for stateless services on Kubernetes; canary requires a traffic-splitting layer (Istio, ArgoRollouts, LaunchDarkly server-side); blue-green requires duplicate infrastructure and a cutover step.

The canary needs its own dashboard with error rate, p99 latency, and saturation isolated to the canary cohort — comparing canary vs. baseline. If the dashboard lumps canary metrics with the rest of production, the canary signal is invisible and the strategy is theater.

The rollback procedure documented in the runbook is fiction until someone has run it end-to-end. Pick a recent release in staging, redeploy it, then roll it back. Common failures: the previous container image was pruned, the migration wasn't reversible, the rollback script depends on a credential that rotated. File a ticket for every gap; do not declare success until the drill works clean.

Every production service needs latency, traffic, errors, and saturation visible at a glance. Datadog, Grafana, or New Relic — the tool matters less than coverage. Spot-check that a new on-call engineer could open the dashboard for any service and read the current health in under 30 seconds.

Trigger a test page through PagerDuty / Opsgenie and confirm it reaches the on-call engineer's phone within two minutes. Common failure: rotation references a Slack channel that was renamed, or an escalation policy that points to someone who left six months ago.

Capture the compliance frameworks the pipeline must satisfy. SOC 2 touches change management, access reviews, and vulnerability management. HIPAA adds encryption in transit/at rest and a BAA with infrastructure providers. PCI scope depends on whether you store PAN.

Export a sample of merged PRs from the past quarter showing approver, ticket reference, and CI status. Vanta / Drata / Secureframe should be wired to GitHub for automated collection — confirm the integration is healthy and a recent sample looks right. Manual evidence collection at audit time is the slow path.

Pull the Dependabot / Snyk / Renovate queue. Critical and high CVEs in production deps should have a ticket and a deadline; transitive low-severity findings can be batched. The failure mode is a queue of 80+ ignored notifications hiding the next Log4Shell.

The runbook in Confluence / Notion / Backstage should match what this review found — current branching strategy, test gates, deployment strategy, paging routes. Diagrams older than 12 months need a refresh. A new engineer should be able to read the runbook and ship their first change without shoulder-tapping a senior.