API Gateway

Why API Gateways Matter

The pain a gateway solves is mostly invisible until you’ve felt it. Picture a 30-service backend with no edge layer. The mobile team needs to call 12 of them to render a home screen. Each one has its own DNS name, its own auth header conventions, its own way of paginating, its own preferred error format. The mobile app ships with a list of 12 hostnames hardcoded into a config — and the day you split orders into orders and order-history, every installed copy of the mobile app needs an update.

That’s the before. The after is one host, one auth scheme, one error envelope, one rate limit, one observability story. Every service-shaped change is invisible to clients because the gateway absorbs it.

What you avoid by not having one

• Auth-per-service: Each team rolls their own JWT validation. One of them gets the signature check wrong. You don’t find out until the post-mortem.
• CORS forever: Browsers preflight every cross-origin call. Multiply that across services. Now multiply it across environments.
• Client coupling to topology: Splitting a service requires coordinating a release with every consumer. Backwards compatibility becomes a treadmill.
• Inconsistent observability: Some services log JSON, some log text. Latency histograms are bucketed differently. Tracing is patchy.
• No throttle: A misbehaving client can hammer one service into the ground because nothing at the edge says “slow down.”

What an API Gateway Does

The gateway is doing a small number of things, repeatedly, very fast. None of these are exotic on their own — the value is that they all live in one place and ship as one configuration.

Gateway vs BFF (Backend for Frontend)

The shared API gateway and the BFF are not opposites — they’re two points on the same spectrum, and many production systems use both.

A common production pattern: a thin shared gateway at the very edge handles auth, TLS termination, and global rate limits. Behind it sit per-client BFFs that handle aggregation and shaping. Mobile traffic goes edge → mobile-bff → services; web traffic goes edge → web-bff → services. Partners often get their own — with different rate limits and a stricter contract.

Routing Strategies

Routing is the gateway’s simplest job and the one you’ll touch most often. Five flavors cover almost everything:

A Kong declarative route

# kong.yaml — declarative config, version-controlled
_format_version: "3.0"

services:
  - name: orders-service
    url: http://orders.svc.cluster.local:8080
    routes:
      - name: orders-route
        paths:
          - /orders
        strip_path: false
        methods: [GET, POST, PUT, DELETE]
    plugins:
      - name: jwt
      - name: rate-limiting
        config:
          minute: 600
          policy: redis

  - name: payments-service
    url: http://payments.svc.cluster.local:8080
    routes:
      - name: payments-route
        paths:
          - /payments
    plugins:
      - name: jwt
      - name: rate-limiting
        config:
          minute: 60     # tighter limit on payment writes
          policy: redis

Notice the configuration is declarative — no code, no imperative steps. The gateway is the executor; this file is the truth. That’s the property you want: the gateway behaves the same way wherever you run it from this file.

An Envoy route fragment

# envoy.yaml route_config — weighted routing for a canary
route_config:
  name: api_routes
  virtual_hosts:
    - name: api
      domains: ["api.example.com"]
      routes:
        - match:
            prefix: "/orders"
          route:
            weighted_clusters:
              clusters:
                - name: orders_v1
                  weight: 95
                - name: orders_v2
                  weight: 5          # 5% canary
            timeout: 2s
            retry_policy:
              retry_on: "5xx,reset,connect-failure"
              num_retries: 2
              per_try_timeout: 800ms
        - match:
            prefix: "/payments"
            headers:
              - name: "x-tenant"
                exact_match: "acme"
          route:
            cluster: payments_acme       # tenant-isolated upstream
        - match:
            prefix: "/payments"
          route:
            cluster: payments_default

Cross-Cutting Concerns at the Edge

Authentication

Two flavors dominate at the edge:

• JWT validation — the gateway holds the public key (or a JWKS URL), checks the signature, validates exp / iss / aud, and forwards claims to the upstream as headers.
• OAuth introspection — the gateway calls the auth server’s /introspect endpoint per request (or per cache window) to learn whether a token is still valid. Slower but supports revocation.

# Kong JWT plugin — validate signatures, then forward claims
plugins:
  - name: jwt
    config:
      key_claim_name: iss
      claims_to_verify:
        - exp                           # reject expired
      maximum_expiration: 3600           # reject > 1h tokens
      header_names: ["Authorization"]
      uri_param_names: []                # no tokens in query strings
      cookie_names: []
      run_on_preflight: false            # skip OPTIONS
  - name: request-transformer
    config:
      add:
        headers:
          - "X-User-Id:$(jwt.claim.sub)"
          - "X-Tenant:$(jwt.claim.tenant)"
      remove:
        headers:
          - "Authorization"             # don’t leak tokens to upstream

That last bit matters. The internal service should trust the headers the gateway forwards (because nothing else can reach it) and never see the raw bearer token. Token handling stays at the edge; auth claims propagate inward as plain identifiers.

Rate limiting

Use a distributed store — Redis is the standard — so a multi-instance gateway counts requests across all replicas. A local in-memory limiter on each instance silently allows N× your intended limit, where N is the replica count.

CORS, body limits, schema validation

• CORS — one allowlist at the gateway, not per-service. Preflight responses cached aggressively.
• Body size limits — a hard cap (e.g. 1 MB) at the edge defends every upstream from oversized bodies.
• Schema validation — reject requests that don’t match the OpenAPI/JSON Schema before they ever touch your service. Cheap to do at the edge; valuable to remove from every service.

Aggregation and Fan-Out

A simple aggregation handler in pseudocode:

// Express-style BFF aggregation for the mobile home screen
app.get('/home', async (req, res) => {
    const userId = req.user.id;

    // Fan out in parallel; each leg has its own timeout + breaker
    const [profile, orders, recs, balance] = await Promise.allSettled([
        userClient.getProfile(userId),       // required
        ordersClient.recent(userId, 5),       // required
        recsClient.forUser(userId),           // optional — ok to fail
        walletClient.balance(userId),         // optional — ok to fail
    ]);

    if (profile.status === 'rejected' || orders.status === 'rejected') {
        return res.status(503).json({ error: 'home_unavailable' });
    }

    res.json({
        profile: profile.value,
        orders:  orders.value,
        recs:    recs.status    === 'fulfilled' ? recs.value    : null,
        balance: balance.status === 'fulfilled' ? balance.value : null,
    });
});

GraphQL as an alternative

GraphQL gateways — Apollo Gateway, GraphQL federation, GitHub’s public GraphQL API — turn aggregation into a first-class concern. The client sends one query specifying exactly the fields it wants; the gateway plans the resolver calls, fans out, and stitches the response. The win: clients can reshape responses without backend changes. The cost: a query planner is now in your hot path, and an over-eager client query can fan out into a denial-of-service against your own services. Guard with query cost analysis and persisted queries.

Gateway Tools Comparison

An AWS API Gateway resource

# SAM/CloudFormation snippet — HTTP API with JWT authorizer
Resources:
  Api:
    Type: AWS::Serverless::HttpApi
    Properties:
      Auth:
        DefaultAuthorizer: JwtAuth
        Authorizers:
          JwtAuth:
            JwtConfiguration:
              issuer: https://auth.example.com/
              audience:
                - api.example.com
            IdentitySource: "$request.header.Authorization"
      RouteSettings:
        "POST /orders":
          ThrottlingBurstLimit: 100
          ThrottlingRateLimit:  50
        "GET /catalog/{proxy+}":
          ThrottlingBurstLimit: 5000
          ThrottlingRateLimit:  2000

  OrdersFn:
    Type: AWS::Serverless::Function
    Properties:
      Handler: orders.handler
      Events:
        CreateOrder:
          Type: HttpApi
          Properties:
            ApiId: !Ref Api
            Path: /orders
            Method: POST

Operational Concerns

The single-point-of-failure problem

A gateway is by definition a fan-in point. The standard mitigations:

• N+1 replicas behind a load balancer — never run a single instance.
• Multi-AZ — replicas in at least two availability zones, with health checks that fail traffic over.
• Stateless — the gateway should hold no per-request state in memory; rate-limit counters and session caches live in Redis.
• Capacity headroom — size for 2–3× peak. The day you need it, you really need it.
• Bypass plan — for the rare disaster scenario, document how an internal team can call services directly. You hope to never use it.

Version skew with downstreams

The gateway and its upstream services rarely deploy at exactly the same instant. A change to a route prefix or a header convention has to land in both. The pattern that works: change is additive on both sides, deploy each side in arbitrary order, then remove the old behavior in a follow-up release. Never make a destructive change in one place that requires the other side to deploy at the same moment.

The gateway-as-monolith risk

Observability for the edge itself

• requests_per_second by route — baseline traffic shape.
• latency_p50/p95/p99 by route — the gateway adds 1–5 ms of overhead. Watch for drift.
• upstream_error_rate by service — isolates whether a 5xx came from the gateway or behind it.
• auth_failures — spikes mean misconfigured clients or an attack.
• rate_limited_total — spikes mean a client (or attacker) hit a quota.
• active_connections — sudden climbs mean upstreams are slow and the queue is filling.

Real-World Examples

Netflix Zuul → Spring Cloud Gateway. Netflix built Zuul as the front door to their cloud edge. Zuul 1 was synchronous and blocking; Zuul 2 introduced a reactive, non-blocking model. The patterns Netflix codified — dynamic routing, request and response filters, integrated circuit breakers (with Hystrix originally) — are now industry-standard. The community has largely migrated to Spring Cloud Gateway, which is the spiritual successor on the JVM.

Amazon API Gateway. AWS’s managed offering ties API routes directly to Lambda, Step Functions, or VPC services. Pay-per-request pricing, built-in throttling, IAM-based auth, JWT authorizers, and WebSocket support. The trade-off is the typical managed-service trade: you trade flexibility for not having to operate it.

GitHub’s GraphQL API. GitHub publishes a single GraphQL endpoint backing essentially their entire product surface. The GraphQL gateway plans queries across many internal services and assembles a single response. Cost analysis and rate limits are computed in “points” — a complex query consumes more budget than a simple one. This is the GraphQL-as-gateway pattern at industrial scale.

Twitter’s Finagle. Twitter’s Finagle library blurs the gateway/RPC boundary — it provides routing, load balancing, retries, circuit breakers, and observability as composable filters around any service-to-service call. Many of the patterns now standard in API gateways were validated at Twitter scale through Finagle first.

Kong at Cisco / Yahoo / Expedia. Kong’s public reference list reads like a who’s-who of large web platforms. The common shape: a Kong fleet behind a cloud load balancer, plugins for JWT and rate limiting, declarative config in Git, multi-region for residency.

Best Practices