Deployment Strategies

Why Deployment Strategy Matters

In a monolith, deploys were rare and risky. In microservices, they’re hourly and routine — which only works if the strategy itself contains the blame. The conversation has shifted from “how do we ship safely?” to “how do we ship continuously without anyone noticing?” That second question is what this tutorial answers.

What a Deployment Strategy Actually Controls

Every strategy is a different set of trade-offs across four axes. The job is to know which axis matters most for the service you’re shipping.

The rest of this tutorial walks each strategy in production-flavored detail — not the textbook version, the version that survives an on-call rotation.

Recreate & Rolling Deploys

Recreate tears down all the old pods, then starts the new ones. There is downtime by design — usually seconds, sometimes minutes. Use it for stateful workloads that cannot run two versions side-by-side (e.g., a single-writer batch job, or a schema migration that’s incompatible with the old code). Never use it for a customer-facing service.

Rolling replaces pods a few at a time. Kubernetes adds maxSurge new pods, waits for them to become ready, then removes maxUnavailable old ones, and repeats. The two knobs control how aggressive you are.

# A production-shaped rolling deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
  labels:
    app: orders-api
spec:
  replicas: 12
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%        # at most 3 extra pods during the roll
      maxUnavailable: 0     # never drop capacity below replicas
  minReadySeconds: 15      # let metrics & warm-up settle before counting healthy
  progressDeadlineSeconds: 600  # auto-rollback if not done in 10 min
  selector:
    matchLabels:
      app: orders-api
  template:
    metadata:
      labels:
        app: orders-api
    spec:
      containers:
      - name: orders-api
        image: registry.example.com/orders-api:v2.4.1
        readinessProbe:
          httpGet: { path: /healthz/ready, port: 8080 }
          initialDelaySeconds: 5
          periodSeconds: 3
          failureThreshold: 2
        livenessProbe:
          httpGet: { path: /healthz/live, port: 8080 }
          initialDelaySeconds: 30
          periodSeconds: 10
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 10 && /app/drain"]

Blue-Green Deployment

The classic shape: blue is serving 100% of traffic on v1.4. You stand up green on v1.5, run smoke tests against it directly (no real traffic), then switch the load balancer’s target group from blue to green. New requests hit green; in-flight requests on blue drain. Blue stays warm for at least one cycle (often a full day) so you can flip back instantly if something rotted in the first hour.

On Kubernetes, blue-green is just two Deployments and a Service whose selector you change:

# Two Deployments, distinguished by version label
apiVersion: apps/v1
kind: Deployment
metadata: { name: checkout-blue }
spec:
  replicas: 10
  selector: { matchLabels: { app: checkout, slot: blue } }
  template:
    metadata: { labels: { app: checkout, slot: blue, version: v1.4 } }
    spec: { containers: [{ name: app, image: checkout:v1.4 }] }
---
apiVersion: apps/v1
kind: Deployment
metadata: { name: checkout-green }
spec:
  replicas: 10
  selector: { matchLabels: { app: checkout, slot: green } }
  template:
    metadata: { labels: { app: checkout, slot: green, version: v1.5 } }
    spec: { containers: [{ name: app, image: checkout:v1.5 }] }
---
# The Service is the switch. Change `slot: blue` to `slot: green` to cut over.
apiVersion: v1
kind: Service
metadata: { name: checkout }
spec:
  selector:
    app: checkout
    slot: blue           # <-- one-line cutover
  ports: [{ port: 80, targetPort: 8080 }]

Blue-green’s killer feature is time to rollback. The killer constraint is no partial validation — you go from 0% to 100% on the new version with one button. If green has a bug, every customer sees it. That’s why production teams that can run canary usually do, and reserve blue-green for releases where coupling forces an atomic switch.

Canary Deployment

A canary deployment is named after the bird in the coal mine. The new version sees real traffic but only enough to die quietly if it’s going to die. Tooling has caught up: Argo Rollouts, Flagger, LaunchDarkly, Linkerd’s SMI traffic split, and managed offerings from AWS CodeDeploy all do canary as a first-class concept. You shouldn’t hand-roll it in 2026.

Argo Rollouts canary spec

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payments-api
spec:
  replicas: 20
  strategy:
    canary:
      canaryService: payments-canary    # gets canary traffic only
      stableService: payments-stable    # gets the rest
      trafficRouting:
        istio:
          virtualService:
            name: payments-vs
            routes: [primary]
      steps:
      - setWeight: 5                  # 5% canary
      - pause: { duration: 10m }
      - analysis:
          templates: [{ templateName: success-rate-and-latency }]
      - setWeight: 25
      - pause: { duration: 15m }
      - analysis:
          templates: [{ templateName: success-rate-and-latency }]
      - setWeight: 50
      - pause: { duration: 30m }
      - setWeight: 100
  selector: { matchLabels: { app: payments-api } }
  template:
    metadata: { labels: { app: payments-api } }
    spec:
      containers:
      - name: app
        image: registry.example.com/payments-api:v3.7.0
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata: { name: success-rate-and-latency }
spec:
  metrics:
  - name: success-rate
    interval: 1m
    successCondition: result[0] >= 0.995
    failureLimit: 2            # 2 bad samples = automatic rollback
    provider:
      prometheus:
        address: http://prometheus:9090
        query: |
          sum(rate(http_requests_total{service="payments-api",code!~"5.."}[2m]))
          /
          sum(rate(http_requests_total{service="payments-api"}[2m]))
  - name: p99-latency
    successCondition: result[0] <= 300     # ms
    provider:
      prometheus:
        address: http://prometheus:9090
        query: |
          histogram_quantile(0.99,
            sum by (le) (rate(http_request_duration_ms_bucket{service="payments-api"}[2m])))

Flagger Canary CR (alternative)

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: orders
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders
  service:
    port: 80
    targetPort: 8080
    gateways: [public-gateway]
    hosts: [orders.example.com]
  analysis:
    interval: 1m
    threshold: 5            # consecutive failed checks before rollback
    maxWeight: 50           # cap canary at 50% before promoting
    stepWeight: 5           # +5% per successful interval
    metrics:
    - name: request-success-rate
      thresholdRange: { min: 99 }      # >= 99% non-5xx
      interval: 1m
    - name: request-duration
      thresholdRange: { max: 500 }     # P99 ms
      interval: 1m
    webhooks:
    - name: load-test
      url: http://flagger-loadtester.test/
      timeout: 5s
      metadata:
        cmd: "hey -z 1m -q 10 -c 2 http://orders-canary/"

Istio VirtualService for traffic splitting

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: payments-vs
spec:
  hosts: [payments]
  http:
  - name: primary
    route:
    - destination: { host: payments, subset: stable }
      weight: 95
    - destination: { host: payments, subset: canary }
      weight: 5          # Argo Rollouts updates this on each step
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: payments-dr
spec:
  host: payments
  subsets:
  - name: stable
    labels: { app: payments-api, role: stable }
  - name: canary
    labels: { app: payments-api, role: canary }

Shadow / Mirror Traffic

Shadow traffic is the highest-confidence, highest-cost strategy. You’re running a second copy of the service that does the work and throws the answer away — or, more usefully, persists it for offline diff against the production answer. Istio supports this natively:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata: { name: search-vs }
spec:
  hosts: [search]
  http:
  - route:
    - destination: { host: search, subset: stable }
      weight: 100
    mirror:
      host: search
      subset: candidate           # new version sees a copy of every request
    mirrorPercentage:
      value: 100.0                # mirror everything — or set to 10.0 for cost

Shadow traffic shines when paired with response diffing. Twitter’s Diffy and LinkedIn’s internal tooling compare old vs. new responses field-by-field, flagging cases where the candidate disagrees with stable. You catch correctness regressions you would never notice from success-rate metrics alone.

Feature Flags vs. Deployment Strategies

The two axes:

Together they enable dark launches — ship the code, leave the flag off, then enable it for 1% of users next Tuesday with no deploy. And they enable ring-based rollouts, the model Microsoft has used for Windows and Office for over a decade:

Progressive Delivery and SLO-Driven Promotion

Progressive delivery is what canary becomes when you take it seriously. The promotion logic is code. The metrics are SLOs. The rollback is automated. The human is a reviewer of choices made by the controller, not the operator pressing buttons.

What makes a good promotion metric:

• Customer-visible. Success rate at the edge, not at the database. Latency at the user, not at the load balancer.
• Statistically valid at canary scale. A 1% canary on a service doing 100 rps gives you 1 rps of canary traffic — not enough signal to detect a P99 regression in 5 minutes. Either pump up the canary weight, lengthen the wait, or pick metrics with bigger denominators.
• Compared to the baseline, not an absolute threshold. “P99 must be under 300 ms” is brittle — legitimate traffic shifts move it. “P99 must be no more than 10% worse than the stable subset” survives reality.
• Symmetric. If the canary is better than baseline by 50%, that’s also suspicious — it usually means the canary isn’t actually getting the requests you think it is.

# A good Argo Rollouts AnalysisRun compares canary to stable, not to a constant.
metrics:
- name: success-rate-vs-baseline
  successCondition: |
    result[0] >= 0.99 * baseline[0]
  failureCondition: |
    result[0] < 0.95 * baseline[0]
  provider:
    prometheus:
      address: http://prometheus:9090
      query: |
        sum(rate(http_requests_total{service="payments",role="canary",code!~"5.."}[2m]))
        /
        sum(rate(http_requests_total{service="payments",role="canary"}[2m]))
  baseline:
    provider:
      prometheus:
        query: |
          sum(rate(http_requests_total{service="payments",role="stable",code!~"5.."}[2m]))
          /
          sum(rate(http_requests_total{service="payments",role="stable"}[2m]))

Real-World Examples

Netflix — Spinnaker pipelines. Netflix open-sourced Spinnaker after building it for their own needs. A typical Spinnaker pipeline bakes an AMI, runs it through judgment-day testing, deploys to a canary cluster alongside a baseline cluster (same code, same traffic split, no new version — the baseline exists to detect noise), runs Kayenta canary analysis comparing canary vs. baseline metrics, and either promotes or rolls back automatically. The deploy itself is 50+ stages long, and humans rarely touch it once it starts.

Facebook (Meta) — mobile rings. Facebook’s mobile apps roll through alpha (employees), beta (opted-in users via Play Store/TestFlight), and production (general availability). Each ring is days to weeks long. A regression caught in alpha never reaches a billion users. The same binary moves through all three; what changes is the cohort the build is published to.

Google — canary analysis service. Google’s SRE book describes a canary analysis service that statistically compares canary and control populations across hundreds of metrics, accounts for natural variance, and emits a single “promote / hold / rollback” verdict. The output is so trusted that it gates deploys with no human review by default. Kayenta (open-sourced via Spinnaker) is the publicly available analog.

Amazon — deployment automation philosophy. Amazon’s public talks describe a culture where the deployment system is owned harder than the application. Every change goes through a pipeline (CodePipeline / CodeDeploy) that does region-by-region, AZ-by-AZ, cell-by-cell rollout, with bake times between stages and automated rollback on metric regression. The blast radius of any single deploy is bounded to one cell of one AZ in one region. A bad deploy can hurt <0.1% of customers before the system catches it.

Helm + Argo Rollouts — the open-source default. For most teams not at FAANG scale, the working stack is Helm to template the manifests, Argo CD to apply them, and Argo Rollouts (or Flagger) to drive the canary. Plus Prometheus for metrics and a service mesh (Istio or Linkerd) for traffic splitting. Every piece is open source. None of it requires a custom team of platform engineers.

Best Practices