Service Mesh

Hard 25 min read

Service Mesh Architecture

A service mesh is a dedicated infrastructure layer for handling service-to-service communication

Think of it as a smart network layer that sits between your microservices, handling traffic routing, security, and observability without changing application code.

Core Components

Data Plane

Proxy sidecars handling actual network traffic between services

Control Plane

Configuration and policy management for the entire mesh

Observability

Metrics, logs, and distributed tracing across all services

Security

mTLS, authentication, and authorization between services

Why Service Mesh?

As microservice architectures grow, managing communication between services becomes increasingly complex. A service mesh solves these challenges:

traffic-splitting.yaml 
# Traffic splitting example
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews
spec:
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 75
    - destination:
        host: reviews
        subset: v2
      weight: 25
auth-policy.yaml 
# Authorization policy
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: httpbin
spec:
  selector:
    matchLabels:
      app: httpbin
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/default/sa/sleep"]
    to:
    - operation:
        methods: ["GET"]
        paths: ["/info*"]

Istio Service Mesh

Istio is the most feature-rich service mesh

Istio uses Envoy proxies as sidecars and provides comprehensive traffic management, security, and observability features. It is the industry standard for complex deployments.

Installation and Setup

istio-install.sh 
# Download and install Istio
curl -L https://istio.io/downloadIstio | sh -
cd istio-*
export PATH=$PWD/bin:$PATH

# Install Istio with demo profile
istioctl install --set profile=demo -y

# Enable sidecar injection
kubectl label namespace default istio-injection=enabled

# Verify installation
kubectl get pods -n istio-system
istioctl verify-install

Traffic Management

destination-rule.yaml 
# Destination Rule
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: productpage
spec:
  host: productpage
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 100
        h2UpgradePolicy: UPGRADE
    loadBalancer:
      simple: LEAST_REQUEST
  subsets:
  - name: v1
    labels:
      version: v1
    trafficPolicy:
      connectionPool:
        tcp:
          maxConnections: 10

Circuit Breaking

circuit-breaker.yaml 
# Circuit breaker configuration
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: httpbin
spec:
  host: httpbin
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 1
      http:
        http1MaxPendingRequests: 1
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutiveErrors: 1
      interval: 1s
      baseEjectionTime: 3m
      maxEjectionPercent: 100
      minHealthPercent: 0

Advanced Patterns

canary-deploy.yaml 
# Canary deployment with Istio
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: helloworld
spec:
  hosts:
  - helloworld
  http:
  - match:
    - headers:
        canary:
          exact: "true"
    route:
    - destination:
        host: helloworld
        subset: v2
  - route:
    - destination:
        host: helloworld
        subset: v1
      weight: 90
    - destination:
        host: helloworld
        subset: v2
      weight: 10
content-routing.yaml 
# Content-based routing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        cookie:
          regex: "^(.*?;)?(user=jason)(;.*)?$"
    route:
    - destination:
        host: reviews
        subset: v3
  - match:
    - queryParams:
        test:
          exact: "true"
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

Linkerd Service Mesh

Linkerd is the lightweight alternative

Built with a Rust micro-proxy, Linkerd uses far fewer resources than Istio (~50MB vs ~500MB per sidecar) while providing automatic mTLS, observability, and reliability features out of the box.

Installation

linkerd-install.sh 
# Install Linkerd CLI
curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install | sh

# Add to PATH
export PATH=$PATH:$HOME/.linkerd2/bin

# Validate cluster
linkerd check --pre

# Install control plane
linkerd install --crds | kubectl apply -f -
linkerd install | kubectl apply -f -

# Verify installation
linkerd check

# Install viz extension
linkerd viz install | kubectl apply -f -

Linkerd Features

linkerd-mtls.sh 
# Inject Linkerd proxy
kubectl get deploy -o yaml | \
  linkerd inject - | \
  kubectl apply -f -

# Check mTLS status
linkerd viz edges deployment

# View TLS certificates
linkerd identity cert
traffic-split.yaml 
# Traffic split with SMI
apiVersion: split.smi-spec.io/v1alpha1
kind: TrafficSplit
metadata:
  name: web-svc-split
spec:
  service: web-svc
  backends:
  - service: web-svc-v1
    weight: 500
  - service: web-svc-v2
    weight: 500

Service Profiles

service-profile.yaml 
# Service profile for webapp
apiVersion: linkerd.io/v1alpha2
kind: ServiceProfile
metadata:
  name: webapp.default.svc.cluster.local
  namespace: default
spec:
  routes:
  - name: GET
    condition:
      method: GET
      pathRegex: "/"
    responseClasses:
    - condition:
        status:
          min: 200
          max: 299
      isFailure: false
  - name: POST
    condition:
      method: POST
      pathRegex: "/api"
    timeout: 30s
    retryBudget:
      retryRatio: 0.2
      minRetriesPerSecond: 10
      ttl: 10s
  retryBudget:
    retryRatio: 0.2
    minRetriesPerSecond: 10
    ttl: 10s

Traffic Management Patterns

Load Balancing Strategies

round-robin.yaml 
# Istio round robin
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service
spec:
  host: my-service
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
consistent-hash.yaml 
# Session affinity
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service
spec:
  host: my-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpCookie:
          name: session-cookie
          ttl: 3600s

Resilience Patterns

retry-policy.yaml 
# Retry policy
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - timeout: 10s
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: gateway-error,connect-failure,refused-stream
      retryRemoteLocalities: true
    route:
    - destination:
        host: ratings
        subset: v1
fault-injection.yaml 
# Inject faults for testing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: ratings
spec:
  hosts:
  - ratings
  http:
  - fault:
      delay:
        percentage:
          value: 10
        fixedDelay: 5s
      abort:
        percentage:
          value: 10
        httpStatus: 500
    route:
    - destination:
        host: ratings
        subset: v1

Rate Limiting

rate-limit.yaml 
# Envoy rate limiting
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: rate-limit
spec:
  workloadSelector:
    labels:
      app: productpage
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.http.local_ratelimit
        typed_config:
          "@type": type.googleapis.com/udpa.type.v1.TypedStruct
          type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
          value:
            stat_prefix: http_local_rate_limiter
            token_bucket:
              max_tokens: 100
              tokens_per_fill: 100
              fill_interval: 60s

Observability

Observability is one of the biggest wins of a service mesh

Without changing a single line of application code, you get distributed tracing, metrics, and service topology maps across your entire cluster.

Distributed Tracing

tracing-config.yaml 
# Enable tracing in Istio
apiVersion: v1
data:
  mesh: |
    defaultConfig:
      proxyStatsMatcher:
        inclusionRegexps:
        - ".*outlier_detection.*"
        - ".*osconfig.*"
      tracing:
        zipkin:
          address: zipkin.istio-system:9411
        sampling: 100.0
kind: ConfigMap
metadata:
  name: istio
  namespace: istio-system
deploy-jaeger.sh 
# Deploy Jaeger for distributed tracing
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.14/samples/addons/jaeger.yaml

# Access Jaeger UI
istioctl dashboard jaeger

Metrics and Monitoring

custom-metrics.yaml 
# Prometheus metrics
apiVersion: v1
kind: Service
metadata:
  name: prometheus
  namespace: istio-system
spec:
  selector:
    app: prometheus
  ports:
  - name: http
    port: 9090

---
# Custom metrics
apiVersion: telemetry.istio.io/v1alpha1
kind: Telemetry
metadata:
  name: custom-metrics
spec:
  metrics:
  - providers:
    - name: prometheus
    dimensions:
      request_protocol: request.protocol | "unknown"
      response_code: response.code | 200
    name: request_total
    unit: REQUEST
    value: "1"

Service Mesh Dashboards

dashboards.sh 
# Install Kiali
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.14/samples/addons/kiali.yaml

# Access Kiali dashboard
istioctl dashboard kiali

# Grafana dashboards
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.14/samples/addons/grafana.yaml
istioctl dashboard grafana

Service Mesh Security

Mutual TLS (mTLS)

strict-mtls.yaml 
# Enable strict mTLS cluster-wide
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT

---
# Per-namespace mTLS
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT
external-certs.yaml 
# External certificate
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: external-mtls
spec:
  selector:
    matchLabels:
      app: external-service
  mtls:
    mode: STRICT
    # Use external certs
    caCertificates: /etc/ssl/certs/ca-cert.pem

Authorization Policies

jwt-auth.yaml 
# RBAC with JWT
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: jwt-auth
  namespace: default
spec:
  selector:
    matchLabels:
      app: httpbin
  jwtRules:
  - issuer: "testing@secure.istio.io"
    jwksUri: "https://raw.githubusercontent.com/istio/istio/release-1.14/security/tools/jwt/samples/jwks.json"

---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: require-jwt
  namespace: default
spec:
  selector:
    matchLabels:
      app: httpbin
  action: ALLOW
  rules:
  - from:
    - source:
        requestPrincipals: ["testing@secure.istio.io/testing@secure.istio.io"]
    when:
    - key: request.auth.claims[groups]
      values: ["group1"]

Security Best Practices

  • Always enable mTLS in production
  • Use namespace isolation
  • Implement defense in depth
  • Regular certificate rotation
  • Audit all authorization policies

Service Mesh Comparison

Feature Istio Linkerd Consul Connect AWS App Mesh
Architecture Envoy-based Rust micro-proxy Envoy/Native Envoy-based
Performance High overhead Low overhead Medium overhead Medium overhead
Complexity High Low Medium Medium
Multi-cluster Excellent Good Excellent Good (AWS)
Protocol Support HTTP, gRPC, TCP HTTP, gRPC HTTP, gRPC, TCP HTTP, gRPC, TCP
mTLS Automatic Automatic Automatic Manual config
Observability Comprehensive Built-in dashboard Good CloudWatch/X-Ray
Resource Usage ~500MB per sidecar ~50MB per sidecar ~100MB per sidecar ~200MB per sidecar

When to Use Each

Istio

Complex traffic management, multi-cluster deployments, advanced security requirements, rich observability features

Linkerd

Simple lightweight mesh, low resource overhead critical, getting started with service mesh, focus on reliability

Consul Connect

Multi-cloud deployments, existing Consul users, service discovery focus, VM and container workloads

AWS App Mesh

AWS-native deployments, EKS integration, managed service preference, CloudWatch integration

Migration Strategies

Adopting Service Mesh

Gradual Rollout Strategy

Start small and expand. A phased approach minimizes risk and gives your team time to learn the mesh tooling.

gradual-rollout.yaml 
# 1. Start with observability only
kubectl label namespace dev istio-injection=enabled

# 2. Enable permissive mTLS
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
spec:
  mtls:
    mode: PERMISSIVE

# 3. Gradually add traffic management
# 4. Enable strict mTLS when ready
# 5. Implement authorization policies

Common Challenges

Migration Pitfalls

  • Performance Impact: Test sidecar overhead in staging
  • Debugging Complexity: Prepare team with training
  • Configuration Drift: Use GitOps for consistency
  • Protocol Support: Verify all protocols work
  • Egress Traffic: Plan external service access

Practice Exercises

Easy Install Istio and Deploy a Sample App

Install Istio with the demo profile, enable sidecar injection, and deploy the Bookinfo sample application.

Use istioctl install --set profile=demo and then label the namespace for injection before deploying.

# Install Istio
istioctl install --set profile=demo -y

# Enable injection
kubectl label namespace default istio-injection=enabled

# Deploy Bookinfo
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

# Verify pods have 2 containers (app + sidecar)
kubectl get pods

Medium Configure Canary Deployment with Traffic Splitting

Create a VirtualService that sends 90% of traffic to v1 and 10% to v2 of a service, with header-based routing for testers.

Use a VirtualService with a match block for header-based routing and weight fields for percentage splitting.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-app
spec:
  hosts:
  - my-app
  http:
  - match:
    - headers:
        x-canary:
          exact: "true"
    route:
    - destination:
        host: my-app
        subset: v2
  - route:
    - destination:
        host: my-app
        subset: v1
      weight: 90
    - destination:
        host: my-app
        subset: v2
      weight: 10

Medium Implement Circuit Breaking

Create a DestinationRule with circuit breaking that limits connections to 10, pending requests to 5, and ejects unhealthy hosts after 3 consecutive errors.

Use connectionPool for connection limits and outlierDetection for ejection configuration in a DestinationRule.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: my-service
spec:
  host: my-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 10
      http:
        http1MaxPendingRequests: 5
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutiveErrors: 3
      interval: 30s
      baseEjectionTime: 1m
      maxEjectionPercent: 50

Hard Set Up End-to-End mTLS with JWT Authorization

Configure strict mTLS for a namespace, add JWT-based request authentication, and create an authorization policy that only allows requests with a valid JWT from a specific issuer.

You need three resources: PeerAuthentication for mTLS, RequestAuthentication for JWT validation, and AuthorizationPolicy for access control.

# 1. Strict mTLS
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT
---
# 2. JWT Authentication
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: jwt-auth
  namespace: production
spec:
  jwtRules:
  - issuer: "https://auth.example.com"
    jwksUri: "https://auth.example.com/.well-known/jwks.json"
---
# 3. Authorization Policy
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: require-jwt
  namespace: production
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        requestPrincipals: ["https://auth.example.com/*"]

Pro Tip

Start with Linkerd if you want a quick win with minimal overhead. Move to Istio when you need advanced traffic management features like header-based routing, fault injection, or multi-cluster support.

Related Topics

Services & Networking Ingress Controllers Security Monitoring & Logging Patterns