Software Update Checklist

Read the upstream changelog and CHANGELOG.md entries for every version between current and target. Flag deprecations, config changes, and required migrations. For semver major bumps, expect breaking changes; for minor/patch, watch for security fixes you need to call out to support.

Verify the previous container image is still in ECR/GCR and not subject to lifecycle pruning. If a DB migration is included, confirm it is reversible — or document the forward-fix plan. Untested rollbacks are the most common reason a release night turns into an outage.

Check that runtime versions (Node, Python, JVM, Postgres), Kubernetes API versions, and Terraform provider constraints meet the new release's minimums. A surprise minimum-Postgres bump in a point release has eaten more than one release window.

Trigger CI on the release-candidate tag (e.g., v2024.45.0-rc.1). Investigate any flaky tests rather than re-running blindly — a habit of "just rerun it" hides real regressions.

Deploy the RC to staging and run Playwright/Cypress suites against it. Cover the critical user paths: signup, auth, checkout, primary CRUD flows. Staging should mirror prod config — environment drift breaks this gate.

Manual exploratory pass by QA on the changed surfaces. The release-notes review in the prep phase determines what gets exercised here. Any blocking defects gate the release.

Check PagerDuty / Incident.io for open incidents. Don't ship into an active production fire — even an unrelated SEV2 will mask new symptoms introduced by the deploy.

Post in #engineering and #customer-support: deploy window, scope of changes, named release captain, named on-call. Update the status page if the change is user-visible or carries downtime risk.

Run the migration ahead of the application deploy so the new schema is live before code that depends on it. Use CREATE INDEX CONCURRENTLY for index work, batched backfills with sleeps, and watch replication lag throughout. Avoid ADD COLUMN ... DEFAULT on large tables — split into add-column, backfill, set-default.

Route 5% of traffic to the new version via the canary deployment in ArgoCD / the load balancer. Hold for 10 minutes minimum and watch error rate, p99 latency, and saturation on the canary pods specifically — not just aggregate dashboards.

Step through 25% → 50% → 100% with a few minutes between increments to let metrics stabilize. Backend goes fully out before frontend so the frontend can rely on new API contracts.

Push the new frontend artifact to CloudFront / Vercel / Netlify and invalidate the CDN cache. Confirm the new asset hashes are being served before declaring the deploy complete.

Execute the synthetic user journey against production: signup, login, primary action, logout. A green canary plus a green smoke test is the gate for declaring the deploy successful.

Latency (p50, p95, p99), traffic, errors, saturation. Compare to the same time-of-day in the previous week, not just to the prior hour. A 10% error-rate bump can hide in absolute numbers if traffic is also up.

Filter Sentry / Bugsnag to the new release tag. New error fingerprints, even at low volume, are the early signal — investigate before they become a spike.

Promote the RC tag to the final release tag (e.g., v2024.45.0). Update the public changelog and post the release summary to #engineering with the deployed sha.

Reflect any new operational behavior in the service runbook: new env vars, new alerts, new dashboards, deprecated endpoints. Stale runbooks are a SOC 2 finding and an on-call tax.

Attach the approved PR list, QA sign-off, deploy log, and rollback evidence (if any) to the change ticket in Jira / Linear. Vanta / Drata pulls from this for SOC 2 change-management evidence.