Deployment Pipeline Design

Architecture patterns for multi-stage CI/CD pipelines with approval gates, deployment strategies, and environment promotion workflows.

Purpose

Design robust, secure deployment pipelines that balance speed with safety through proper stage organization, automated quality gates, and progressive delivery strategies. This skill covers both the structural design of pipeline architecture and the operational patterns for reliable production deployments.

Input / Output

What You Provide

• Application type: Language/runtime, containerized or bare-metal, monolith or microservices
• Deployment target: Kubernetes, ECS, VMs, serverless, or platform-as-a-service
• Environment topology: Number of environments (dev/staging/prod), region layout, air-gap requirements
• Rollout requirements: Acceptable downtime, rollback SLA, traffic splitting needs, canary vs blue-green preference
• Gate constraints: Approval teams, required test coverage thresholds, compliance scans (SAST, DAST, SCA)
• Monitoring stack: Prometheus, Datadog, CloudWatch, or other metrics sources used for automated promotion decisions

What This Skill Produces

• Pipeline configuration: Stage definitions, job dependencies, parallelism, and caching strategy
• Deployment strategy: Chosen rollout pattern with annotated configuration (canary weights, blue-green switchover, rolling parameters)
• Health check setup: Shallow vs deep readiness probes, post-deployment smoke test scripts
• Gate definitions: Automated metric thresholds and manual approval workflows
• Rollback plan: Automated rollback triggers and manual runbook steps

When to Use

• Design CI/CD architecture for a new service or platform migration
• Implement deployment gates between environments
• Configure multi-environment pipelines with mandatory security scanning
• Establish progressive delivery with canary or blue-green strategies
• Debug pipelines where stages succeed but production behavior is wrong
• Reduce mean time to recovery by automating rollback on metric degradation

Pipeline Stages

Standard Pipeline Flow

┌─────────┐   ┌──────┐   ┌─────────┐   ┌────────┐   ┌──────────┐
│  Build  │ → │ Test │ → │ Staging │ → │ Approve│ → │Production│
└─────────┘   └──────┘   └─────────┘   └────────┘   └──────────┘

Detailed Stage Breakdown

1. Source - Code checkout, dependency graph resolution
2. Build - Compile, package, containerize, sign artifacts
3. Test - Unit, integration, SAST/SCA security scans
4. Staging Deploy - Deploy to staging environment with smoke tests
5. Integration Tests - E2E, contract tests, performance baselines
6. Approval Gate - Manual or automated metric-based gate
7. Production Deploy - Canary, blue-green, or rolling strategy
8. Verification - Deep health checks, synthetic monitoring
9. Rollback - Automated rollback on failure signals

Approval Gate Patterns

Pattern 1: Manual Approval (GitHub Actions)

production-deploy:
  needs: staging-deploy
  environment:
    name: production
    url: https://app.example.com
  runs-on: ubuntu-latest
  steps:
    - name: Deploy to production
      run: kubectl apply -f k8s/production/

Environment protection rules in GitHub enforce required reviewers before this job starts. Configure reviewers at Settings → Environments → production → Required reviewers.

Pattern 2: Time-Based Approval (GitLab CI)

deploy:production:
  stage: deploy
  script:
    - deploy.sh production
  environment:
    name: production
  when: delayed
  start_in: 30 minutes
  only:
    - main

Pattern 3: Multi-Approver (Azure Pipelines)

stages:
  - stage: Production
    dependsOn: Staging
    jobs:
      - deployment: Deploy
        environment:
          name: production
          resourceType: Kubernetes
        strategy:
          runOnce:
            preDeploy:
              steps:
                - task: ManualValidation@0
                  inputs:
                    notifyUsers: "[email protected]"
                    instructions: "Review staging metrics before approving"

Pattern 4: Automated Metric Gate

Use an AnalysisTemplate (Argo Rollouts) or a custom gate script to block promotion when error rates exceed a threshold:

# Argo Rollouts AnalysisTemplate — blocks canary promotion automatically
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  metrics:
  - name: success-rate
    interval: 60s
    successCondition: "result[0] >= 0.95"
    failureCondition: "result[0] < 0.90"
    inconclusiveLimit: 3
    provider:
      prometheus:
        address: http://prometheus:9090
        query: |
          sum(rate(http_requests_total{status!~"5..",job="my-app"}[2m]))
          / sum(rate(http_requests_total{job="my-app"}[2m]))

Deployment Strategies

Decision Table

1. Rolling Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2         # at most 12 pods during rollout
      maxUnavailable: 1   # at least 9 pods always serving

Characteristics: gradual rollout, zero downtime, easy rollback, best for most applications.

2. Blue-Green Deployment

# Switch traffic from blue to green
kubectl apply -f k8s/green-deployment.yaml
kubectl rollout status deployment/my-app-green

# Flip the service selector
kubectl patch service my-app -p '{"spec":{"selector":{"version":"green"}}}'

# Rollback instantly if needed
kubectl patch service my-app -p '{"spec":{"selector":{"version":"blue"}}}'

Characteristics: instant switchover, easy rollback, doubles infrastructure cost temporarily, good for high-risk deployments with long warm-up times.

3. Canary Deployment (Argo Rollouts)

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: my-app
spec:
  replicas: 10
  strategy:
    canary:
      analysis:
        templates:
          - templateName: success-rate
        startingStep: 2
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - setWeight: 25
        - pause: { duration: 5m }
        - setWeight: 50
        - pause: { duration: 10m }
        - setWeight: 100

Characteristics: gradual traffic shift, real-user metric validation, automated promotion or rollback, requires Argo Rollouts or a service mesh.

4. Feature Flags

from flagsmith import Flagsmith

flagsmith = Flagsmith(environment_key="API_KEY")

if flagsmith.has_feature("new_checkout_flow"):
    process_checkout_v2()
else:
    process_checkout_v1()

Characteristics: deploy without releasing, A/B testing, instant rollback per user segment, granular control independent of deployment.

Pipeline Orchestration

Multi-Stage Pipeline Example (GitHub Actions)

name: Production Pipeline

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      image: ${{ steps.build.outputs.image }}
    steps:
      - uses: actions/checkout@v4
      - name: Build and push Docker image
        id: build
        run: |