API Gateway: What It Does and When to Add One

As the number of services and integrations grows, so does the number of things every API must do: check credentials, enforce limits, log requests and route traffic to the right backend. An API gateway takes those shared jobs out of individual services and handles them at a single entry point. It can simplify security and operations considerably. It can also add latency, cost and a new point of failure if it is introduced too early or asked to do too much. This guide explains what an API gateway does, how it differs from integration middleware and when the extra layer is worth it.

What an API gateway does

The microservices.io description of the API gateway pattern calls it the single entry point for all clients. Some requests are simply proxied or routed to the appropriate service, while others are fanned out to several services. Common gateway responsibilities:

• Routing: sending each request to the right backend service based on path, host or headers
• Authentication: validating API keys, tokens or certificates before requests reach services
• Rate limiting and throttling: protecting backends from overload and enforcing usage plans
• Request and response transformation: adjusting headers, formats or paths
• Aggregation: combining responses from several services into one
• Observability: logging, metrics and tracing for every request
• TLS termination: handling HTTPS at the edge

Managed gateways typically implement throttling with well-known algorithms. Amazon's documentation on API Gateway throttling, for example, explains that it uses the token bucket algorithm and that clients may receive 429 Too Many Requests responses when limits are exceeded.

API gateway vs integration middleware

The two are often confused because both sit between systems. The difference is direction and purpose:

• An API gateway sits in front of your APIs and manages inbound traffic from clients. It is mostly about access, protection and routing.
• Integration middleware sits between business systems and moves data among them. It is mostly about transformation, orchestration and reliable delivery.

A request from a mobile app to your orders service goes through a gateway. A flow that copies new orders from your e-commerce platform to your ERP runs in middleware. Our guide to API integration middleware covers that second role in detail. Larger organisations often have both.

When an API gateway is worth adding

Signs that a gateway would help:

• Several services expose APIs, each implementing its own authentication and rate limiting slightly differently
• External partners or customers call your APIs and need keys, quotas and usage reporting
• You need a stable public URL structure while backend services are reorganised
• Security reviews keep finding inconsistent controls across services
• You want consistent logging and metrics for all API traffic

When you can skip it

A gateway may be unnecessary when:

• You have one or two services, and a load balancer or reverse proxy already handles TLS and routing
• All callers are internal and trusted, and services already share an authentication library
• Your cloud platform's load balancer covers the few features you need

Adding a gateway to a small system brings extra hops, configuration and cost without much benefit.

Backends for frontends

A variation of the pattern is the backend for frontend, where each type of client, such as a web app, mobile app or partner API, gets its own gateway tailored to its needs. The microservices.io pattern page covers this variation alongside the general gateway. It helps when clients need very different response shapes, but multiplies the number of components to run.

Design principles for an API gateway

• Keep business logic out. The gateway should route, protect and observe, not calculate prices or apply discounts.
• Keep aggregation light. Simple combination of responses is fine; complex orchestration belongs in a service.
• Make it highly available. Every request passes through it, so run several instances and monitor them closely.
• Version its configuration. Store routes and policies in source control and deploy changes through the same review process as code.
• Pass identity downstream. Forward verified identity information so services can make authorisation decisions without re-checking credentials.
• Set timeouts. The gateway should not wait forever on a slow backend.

Our guide to microservices communication patterns explains how gateways fit alongside service-to-service calls and messaging.

Worked example: what goes where

The table below shows a hypothetical allocation of responsibilities for a company exposing order and customer APIs to partners while syncing data internally. It is illustrative, not a prescription.

ResponsibilityAPI gatewayServicesIntegration middlewarePartner API keys and quotasYesNoNoTLS terminationYesNoNoRequest routing by pathYesNoNoBusiness validation of ordersNoYesNoPricing and discount rulesNoYesNoCopying orders to the ERPNoNoYesField mapping between systemsNoNoYesRequest logging and metricsYesYes, service-levelYes, per message

In this example, each layer has a clear job. The gateway never touches business rules, and the middleware never faces external partners directly.

Common API gateway mistakes

• Putting business logic in gateway scripts, where it is hard to test and easy to forget
• Running a single gateway instance, creating a single point of failure
• Setting rate limits without telling clients, so they discover limits through errors
• Forgetting that the gateway's own limits and timeouts can become the bottleneck
• Bypass routes that let some traffic skip the gateway's security controls

Choosing an API gateway product

Options range from managed cloud gateways to open source gateways you run yourself and gateway features built into service meshes or ingress controllers. Compare them on:

• The authentication methods you need, such as API keys, OAuth tokens or mutual TLS
• How rate limits and quotas are defined and reported to clients
• Latency added per request under your expected load
• How configuration is stored, reviewed and deployed
• Logging and metrics integration with your existing monitoring
• Pricing model, especially per-request charges at high volume

Test the shortlisted options with a realistic traffic sample before committing, because added latency and per-request costs are easy to underestimate.

Plan the rollout

Introduce a gateway gradually. Start with one API or one group of clients, route traffic through the gateway while keeping the existing path available, compare behaviour and latency, then move further APIs once the first is stable. Communicate any changes in URLs, keys or limits to clients well in advance.

Frequently Asked Questions

What is an API gateway?

It is a single entry point in front of your APIs that routes requests to backend services and handles shared concerns such as authentication, rate limiting, logging and TLS.

What is the difference between an API gateway and middleware?

A gateway manages inbound traffic to your APIs. Integration middleware moves and transforms data between business systems. They solve different problems and often coexist.

Do small systems need an API gateway?

Often not. With one or two internal services, a load balancer or reverse proxy may provide enough routing and TLS handling without a separate gateway.

Should business logic run in the API gateway?

No. Keep the gateway focused on routing, protection and observability, and keep business rules in services where they can be tested and maintained.

Put the right layer in front of your APIs

We assess whether an API gateway fits your architecture and design gateways and integration layers with clear responsibilities. See our CRM and API integration services, or talk to us about API gateways.

About the Author

Manav Sharma is the founder of Insyrge and a CRM solutions architect. He works across Zoho, Salesforce, HubSpot, Odoo and more, designing the CRM setups, integrations and automations that keep sales and operations teams working from the same data.